Learnings
Outcomes
We have access to a complete and accurate steelwork assessment approach at scale. This will accelerate decision making regarding tower maintenance, as well as create a new pool of data for future forecasting using the automated techniques developed in the previous project, VICAP 2. The AI decisions made during the steelwork assessments have already been validated through human peer review.
In addition, it will reduce the need for climbing surveys which will in turn reduce costs and improve safety.
The work and learnings from the Fittings Wear Measurement will push what’s possible in terms of defect analysis from image surveys and pave the way for accurate defect assessment of various different defect scenarios. A follow-on project is currently being scoped to extend the work to include all OHL fittings.
Internal tower surveys, paired with the bar-by-bar processing pipeline, will provide a complete view of the tower from all angles, as well as introduce new angles that allow more inspection and defect detection. This is subject to the development and testing of drone flight procedures by the drone operations team within NGET.
The final TRL of the project is:
Level 8: Actual technology completed and qualified through tests and demonstrations
The technology has been proven to work in its final form and under expected conditions. Activities include developmental testing and evaluation of whether it will meet operational requirements.
Success Criteria
The following three bullet-pointed criteria were met:
- The localised algorithm can assess the individual steel bars
This was a complex problem to solve that has multiple different facets. We already had a good foundation on steelwork and corrosion detection from VICAP2, but the limitation was that all detections remained exclusively in image-space. Whilst we did some work to localise the detections on specific tower sections, this was still far from a bar-by-bar localisation. As such we needed a method to accurately take the image-space detections and translate them to tower-space.
To do this we hypothesised that reconstructing the real-life tower in 3D space as a digital twin will allow us to bridge the gap between image and tower space. We have, on paper, all the requirements for this: a 3D model of a (single) tower, images with GPS co-ordinates, altitude and camera metadata. With these, not only can we represent the tower in a digital space, we can also replicate the camera(s) that took each image in the same space.
From there, we can employ a technique called “texture projection” which applies an image onto the tower as a texture, through the camera’s perspective. Repeating this for each camera/image will result in a 3D model that is completely textured. Since the 3D model is split into individual bars, we can then interrogate each bar mesh about the amount of texture it has (which informs us of the amount of corrosion on that bar).
Following the successful prototype, we set out to prove that the technique works on more complex geometry such as that of a transmission tower. We opted to build a very basic 3D model instead of obtaining an open-source model from the internet because this way we had more control over the specific requirements of the pipeline.
The geometry has to be composed of individual, uniquely named bars, where each bar is subdivided into smaller segments. This is crucial for applying the projected texture (corrosion) accurately, though a balance is required since a very high level of subdivision results in a significant performance hit. Similarly, an overall geometry with a high polygon count also impacts processing performance.
At this stage real data was yet to be collected, so we created synthetic images using the 3D model. This was great because it allowed us to control the exact amount of corrosion on each bar which we could then use as the ground-truth when measuring the accuracy of the pipeline.
We applied rust to 12 bars, rendered several images and produced the corrosion overlays that simulate the results of the Deepsteel pipeline. We input this data into our projection pipeline alongside a texture-less 3D model of the same tower. The results showed an impressive accuracy, with a mean error of just 0.8% when measuring the corrosion on a bar as a ratio of its surface area.
The 3D reconstruction worked although 100% alignment was not achieved. Portions of the tower in each image were aligned but in some cases the image and the 3D model did not line up correctly. We identified several issues with the 3D model provided to us for this specific tower:
1. Model scale doesn’t match real-life tower
2. Model not completely symmetrical
3. Real-life tower has different base compared to 3D model
We cleaned up a portion of the 3D model (the peak); this included simplifying the mesh to improve processing performance, uniquely segmenting each bar and subdividing each bar’s mesh. We also separated non-bar meshes (such as braces and bolts).
With additional experiments and controlled surveys, we also observed that the positional data for the drone carried an inherent and significant error which was hard to correct for even with high-accuracy GPS devices such as Emlid. Correcting for this error is extremely difficult but important, since it heavily impacts the projection pipeline. We proposed a couple of alternative workflows which would resolve this in one way or another.
Having identified issues with the 3D model, we turned our attention to LIDAR point clouds. The issue with 3D models is that you need exact digital twins for each tower, which is not something that is available at this time. LIDAR on the other hand is collected with a drone and results in an exact 1:1 copy of the tower. The downside is that this point cloud does not carry any notion of unique bars; it is a single fused “geometry”.
We created a prototype pipeline that would ingest the images from the survey and the point cloud. The first step was to prove that we could identify unique bars from the point cloud. Using a RANSAC algorithm, we had partial success identifying the individual bars on a subsection of the tower: the peak.
We found that the detection accuracy was heavily dependent on the quality of the point cloud, measured as point density. The pipeline showed potential but multiple hurdles remained: reliable bar segmentation/identification and corrosion projection.
In parallel, we developed an alternative workflow which involved a human-in-the-loop. Human-assisted Bar Segmentation (HABS) is a pipeline that offloads the most complex part of the process, the identification of unique bars, to a human. The pipeline consists of an intuitive interface that allows users to point and click individual bars, which would then be automatically segmented using a state-of-the-art computer vision model.
We processed images of 3 separate towers/surveys, focusing on the peak only. We found that there was an optimal number of images to be processed: feeding too many images of the same bar worsens the results. We found that around ~2 images per tower face was considered the optimal number. With all of the learnings from the previous experiments combined, we used the initial pipeline with a human-in-the-loop to provide assisted alignment. The interface shows the user each image and overlays the 3D model based on our scene reconstruction algorithm. The initial parameters tend to be good but not perfect, so the user simply needs to re-position the 3D model slightly until it fits. Sometimes minor adjustments to the scale/distance are required.
- The localised algorithm can assess wear on the fittings
Measuring fittings in real-life requires a climbing survey, which is costly, dangerous and time consuming. In this work package we aimed to improve the current process and develop a pipeline which can reliably measure the shackle which holds the insulator string. These measurements can be checked against the ideal reference for the fitting, providing an indication of the amount of wear on the metal.
To achieve this we proposed an image-level measurement interface which uses a known reference object in the image. We developed a user-centric interface where, for a number of images, the user must identify the reference object in the image and measure along its known axis. For this reference object we chose the mounting bracket since it was present in the candidate tower, with good proximity to the target object and its easy to measure geometry.
Once the anchor has been identified, the user then has to measure the shackle “union”. Once submitted for a number of images, the background processing pipeline uses the known dimensions of the reference object and the user input to calculate the conversion factor from pixels to millimetres. The algorithm considers all images that were submitted. Finally, the user is shown the measured distance and whether it falls within the tolerance levels for this fitting.
Additionally, we pass the input images to an image-to-3D generative model, which approximates the 3D mesh of the fitting. Whilst this is not used in the final measurement yet, it is planned to investigate how we can leverage this for a more accurate assessment.
It was difficult to obtain recent ground truth measurements of a shackle which corresponded to a tower in the candidate list. Therefore we were limited to testing the accuracy using synthetic data (with a correctly proportioned 3D model a similar fitting). For these tests we used two reference dimensions, and when using the minimum value from both measurements we obtained an error of ±9% on average (across 10 images).
On real data, assuming the shackle we are measuring is in a near perfect condition, we consistently obtain measurements that are within ±2mm of the expected 60mm value.
We have good ideas about how to improve this pipeline using a mixture of computer vision techniques such as depth mapping, key-point detection for automation and the aforementioned generative models.
- A complete 360-degree steel tower can be provided from flight data of the internal steel structure.
We developed a synthetic environment where we could control variables more easily. Building on the work developed in WP1, we placed a digital camera inside the peak of the 3D model and textured the model with a basic corroded steel texture and applied a realistic background to the scene using an off the shelf HDRI.
The idea was to test whether the existing Deepsteel models would struggle with the novel view. Since they have never been trained on imagery from inside the tower, this was considered a likely outcome. However, the nature of synthetic imagery is itself a factor which can impact the model performance.
The digital camera replicates our target camera of choice for the real-life setup, using known sensor size, depth of field, etc. The steel segmentation model struggled to parse the novel view well, which had downstream effects on the entire pipeline.
We experimented with different tile sizes which showed some improvement but were still far from the usual performance of the model. Following this we replaced this model for a basic depth map with some thresholding to replicate good steel segmentation results. This achieved much better results and showed us that the corrosion detection was still working well.
We used video footage from a drone flying inside a tower to analyse performance on real-world data. Although frames taken from a video are not near the level of resolution and overall image quality we seek, this was still a useful exercise. The Deepsteel pipeline showed great results on real data. We conducted experiments with and without tiling and both showed pros and cons. Ultimately, we will have to fine-tune Deepsteel to deal with this novel view, but it is likely that significantly less data would be needed than originally thought.
Total NIA Expenditure on project
Total External Costs: £ 348,300.00
Total Indirect Costs: £ 45,000.00
Total Internal Costs: £9,444.02
Grand Total: £402,477.02
Lessons Learnt
1) Creating a development environment that allows you to control different variables is extremely valuable in the initial stages of such projects
2) Creating a simulated digital environment is similarly valuable and allows you to progress rapidly
3) Image metadata can reside outside of EXIF, namely Extensible Metadata Platform files (XMP)
4) High accuracy Real-time Kinetic Global Positioning System ( RTK GPS) positioning is not as accurate as it sounds on paper.
5) 3D models need to be as accurate as possible
6) 3D models may need specific adjustment to their mesh geometry to make them work with this approach
7) Controlled test flights, similarly to a development environment, are crucial when learning what works and what doesn’t
8) A balance needs to be struck between ideal scenario and BAU practices; not all data or equipment can be available at all times
Overall, these lessons highlight the need to validate data quality, modelling assumptions and data capture methods early in delivery. Future project should prioritise this upfront, alongside ensuring solutions are designed with practical BAU deployment constraints in mind to reduce re-work and increase the likelihood of successful implementation at scale.
Dissemination
Innovation Zero Conference – London April 2025
EIS Conference – Glasgow November 2025
NGET Executives Innovation workshop – Warwick November 2025
UHVNET Conference – Birmingham May 2026
