Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling
"> Figure 1
<p>Simulation of the Highline Canal near Payson, Utah.</p> "> Figure 2
<p>Closeup views of anomaly locations. (<b>a</b>) Canal slump; (<b>b</b>) industrial piping; (<b>c</b>) power line; (<b>d</b>) railway.</p> "> Figure 3
<p>Illustration of single track (<b>above</b>) and double track (<b>below</b>) UAV flight paths.</p> "> Figure 4
<p>Dome projection inserted to account for unknown anomaly shape. (<b>a</b>) Terrain without dome projection; (<b>b</b>) terrain with dome projection.</p> "> Figure 5
<p>An added dome projection helps guide the algorithm to capture additional oblique imagery of an anomaly with unknown geometry. (<b>a</b>) Selected camera positions with no added dome; (<b>b</b>) selected camera positions with added dome.</p> "> Figure 6
<p>Angle ranges for histogram, where <math display="inline"> <semantics> <mi>α</mi> </semantics> </math> is the camera angle of view.</p> "> Figure 7
<p>Example of an optimized camera view plan and flight path for a generic anomaly with and without the added dome projection. Yellow circles represent image locations; green triangles are corresponding image targets. (<b>a</b>) Optimized flight plan with dome projection; (<b>b</b>) optimized flight plan without dome projection; (<b>c</b>) top down view of (<b>a</b>); (<b>d</b>) top down view of (<b>b</b>).</p> "> Figure 8
<p>Examples of a ground truth model and a reconstructed model for the piping anomaly location. (<b>a</b>) Ground truth model geometry exported from Terragen; (<b>b</b>) model reconstructed from images in Agisoft Photoscan.</p> "> Figure 9
<p>Illustration of the quadratic fitting technique used to find the distance between the reference cloud (black) and the compared cloud (blue).</p> "> Figure 10
<p>Flight path simulated along the Highline Canal section.</p> "> Figure 11
<p>Top view of flight path simulated along the Highline Canal section.</p> "> Figure 12
<p>Average accuracy of 3D models from test cases.</p> "> Figure 13
<p>Qualitative comparison of canal models: (<b>a</b>) original geometry; (<b>b</b>) reconstruction from single path; (<b>c</b>) reconstruction from double path. Note the disjoint at the canal surface in the double path case.</p> "> Figure 14
<p>Cross-section view of canal from a double path model. Again, the model is seen to be disconnected at the canal surface.</p> "> Figure 15
<p>Flight time comparison for a 100-mile flight.</p> "> Figure 16
<p>Data quantity comparison for a 100-mile flight.</p> ">
Abstract
:1. Introduction
1.1. Related Work in Linear Feature Monitoring
1.2. Optimized View Planning
1.3. Simulation
1.4. Novel Contributions and Paper Overview
- It is shown that when a reliable anomaly detection system becomes available, the proposed method will be capable of generating detailed 3D models of the areas of interest while avoiding the often unwieldy amounts of data produced by repeatedly creating 3D models of the entire structure in routine inspections.
- The monitoring system incorporates on demand optimized view planning, taking advantage of the onboard processing capabilities of unmanned aircraft to maximize information gain through in-flight re-planning.
- The potential benefits of this method are demonstrated and quantified in simulation, motivating further work on the supporting automatic detection technologies.
2. Methods
2.1. Simulated Test Scene
2.2. Standard Path Planning
2.3. Optimized 3D Flight Path Planning
- Receive detected anomaly location
- Obtain a point-cloud
- Add dome projection at the point of interest
- Convert the point-cloud terrain into a triangle mesh
- Place an aerial camera location on the normal line of every triangle
- Remove underground and blocked images
- Select the best images from the group based on a set of value heuristics
- Find the shortest path through all of these points
2.4. 3D Modeling
2.5. 3D Accuracy Testing
2.6. Simulated System Implementation
3. Results
3.1. 3D Accuracy Testing Results
3.2. Flight Time Results
3.3. Data Quantity Results
4. Discussion
5. Conclusions
Acknowledgments
Author Contributions
Conflicts of Interest
References
- Matikainen, L.; Lehtomäki, M.; Ahokas, E.; Hyyppä, J.; Karjalainen, M.; Jaakkola, A.; Kukko, A.; Heinonen, T. Remote sensing methods for power line corridor surveys. ISPRS J. Photogramm. Remote Sens. 2016, 119, 10–31. [Google Scholar] [CrossRef]
- Goncalves, J.; Henriques, R. UAV photogrammetry for topographic monitoring of coastal areas. ISPRS J. Photogramm. Remote Sens. 2015, 104, 101–111. [Google Scholar] [CrossRef]
- Martin, R.A.; Hall, A.; Brinton, C.; Franke, K.; Hedengren, J.D. Privacy Aware Mission Planning and Video Masking for UAV Systems. In Proceedings of the AIAA Infotech @ Aerospace, San Diego, CA, USA, 4–8 January 2016. [Google Scholar]
- Sankarasrinivasan, S.; Balasubramanian, E.; Karthik, K.; Chandrasekar, U.; Gupta, R. Health Monitoring of Civil Structures with Integrated UAV and Image Processing System. Procedia Comput. Sci. 2015, 54, 508–515. [Google Scholar] [CrossRef]
- Pajares, G. Overview and current status of remote sensing applications based on unmanned aerial vehicles (UAVs). Photogramm. Eng. Remote Sens. 2015, 81, 281–329. [Google Scholar] [CrossRef]
- Ruggles, S.; Clark, J.; Franke, K.W.; Wolfe, D.; Reimschiissel, B.; Martin, R.A.; Okeson, T.J.; Hedengren, J.D. Comparison of SfM computer vision point clouds of a landslide derived from multiple small UAV platforms and sensors to a TLS-based model. J. Unmanned Veh. Syst. 2016, 4, 1–20. [Google Scholar] [CrossRef]
- Palmer, L.M.; Franke, K.W.; Abraham Martin, R.; Sines, B.E.; Rollins, K.M.; Hedengren, J.D. Application and Accuracy of Structure from Motion Computer Vision Models with Full-Scale Geotechnical Field Tests. In Proceedings of the IFCEE, San Antonio, TX, USA, March 17–21 2015; pp. 2432–2441. [Google Scholar]
- Franke, K.W.; Rollins, K.M.; Ledezma, C.; Hedengren, J.D.; Wolfe, D.; Ruggles, S.; Bender, C.; Reimschiissel, B. Reconnaissance of Two Liquefaction Sites Using Small Unmanned Aerial Vehicles and Structure from Motion Computer Vision Following the April 1, 2014 Chile Earthquake. J. Geotech. Geoenviron. Eng. 2016. [Google Scholar] [CrossRef]
- Colomina, I.; Molina, P. Unmanned aerial systems for photogrammetry and remote sensing: A review. ISPRS J. Photogramm. Remote Sens. 2014, 92, 79–97. [Google Scholar] [CrossRef]
- Westoby, M.; Brasington, J.; Glasser, N.; Hambrey, M.; Reynolds, J. ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications. Geomorphology 2012, 179, 300–314. [Google Scholar] [CrossRef]
- Saadatseresht, M.; Hashempour, A.; Hasanlou, M. UAV Photogrammetry: A Practical Solution for Challenging Mapping Projects. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2015, 40, 619. [Google Scholar] [CrossRef]
- Fytsilis, A.L.; Prokos, A.; Koutroumbas, K.D.; Michail, D.; Kontoes, C.C. A methodology for near real-time change detection between Unmanned Aerial Vehicle and wide area satellite images. ISPRS J. Photogramm. Remote Sens. 2016, 119, 165–186. [Google Scholar] [CrossRef]
- Ruf, B.; Schuchert, T. Towards real-time change detection in videos based on existing 3D models. Proc. SPIE 2016. [Google Scholar] [CrossRef]
- Shi, J.; Wang, J.; Xu, Y. Object-based change detection using georeferenced UAV images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci 2011, 38, 177–182. [Google Scholar] [CrossRef]
- Sun, L.; Castagno, J.; Hedengren, J.; Beard, R. Parameter estimation for towed cable systems using moving horizon estimation. IEEE Trans. Aerosp. Electron. Syst. 2015, 51, 1432–1446. [Google Scholar] [CrossRef]
- Kamate, S.; Yilmazer, N. Application of Object Detection and Tracking Techniques for Unmanned Aerial Vehicles. Procedia Comput. Sci. 2015, 61, 436–441. [Google Scholar] [CrossRef]
- Gleason, J.; Nefian, A.V.; Bouyssounousse, X.; Fong, T.; Bebis, G. Vehicle detection from aerial imagery. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; Volume 2, pp. 2065–2070. [Google Scholar]
- Krizhevsky, A.; Sutskever, I.; Hinton, G.E. Imagenet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems; The MIT Press: Cambridge, MA, USA, 2012; pp. 1097–1105. [Google Scholar]
- Máthé, K.; Buşoniu, L. Vision and control for UAVs: A survey of general methods and of inexpensive platforms for infrastructure inspection. Sensors 2015, 15, 14887–14916. [Google Scholar] [CrossRef] [PubMed]
- Hausamann, D.; Zirnig, W.; Schreier, G.; Strobl, P. Monitoring of gas pipelines—A civil UAV application. Aircr. Eng. Aerosp. Technol. 2005, 77, 352–360. [Google Scholar] [CrossRef]
- Zhang, J.; Liu, L.; Wang, B.; Chen, X.; Wang, Q.; Zheng, T. High speed automatic power line detection and tracking for a UAV-based inspection. In Proceedings of the 2012 International Conference on Industrial Control and Electronics Engineering (ICICEE 2012), Xi’an, China, 23–25 August 2012; pp. 266–269. [Google Scholar]
- Rathinam, S.; Kim, Z.W.; Sengupta, R. Vision-based monitoring of locally linear structures using an unmanned aerial vehicle 1. J. Infrastruct. Syst. 2008, 14, 52–63. [Google Scholar] [CrossRef]
- Zhou, H.; Kong, H.; Wei, L.; Creighton, D.; Nahavandi, S. Efficient road detection and tracking for unmanned aerial vehicle. IEEE Trans. Intell. Transp. Syst. 2015, 16, 297–309. [Google Scholar] [CrossRef]
- Chen, S.; Li, Y.; Kwok, N.M. Active vision in robotic systems: A survey of recent developments. Int. J. Robot. Res. 2011, 30, 1343–1377. [Google Scholar] [CrossRef]
- Zhao, J.; Cheung, S.C.S. Optimal visual sensor planning. In Proceedings of the IEEE International Symposium on Circuits and Systems, Taipei, Taiwan, 24–27 May 2009; pp. 165–168. [Google Scholar]
- Chen, S.Y.; Li, Y.F. Automatic Sensor Placement for Model-Based Robot Vision. IEEE Trans. Syst. Man Cybern. Part B Cybern. 2004, 34, 393–408. [Google Scholar] [CrossRef]
- Siebert, S.; Teizer, J. Mobile 3D mapping for surveying earthwork projects using an Unmanned Aerial Vehicle (UAV) system. Autom. Constr. 2014, 41, 1–14. [Google Scholar] [CrossRef]
- Van Leeuwen, J.; Leeuwen, J. Handbook of Theoretical Computer Science: Algorithms and Complexity; Elsevier: Amsterdam, The Netherlands, 1994. [Google Scholar]
- Saadat, S.M.; Samadzadegan, F. Multi-Objective Optimization of Vision Metrology Camera Placement Based on Pareto Front Concept by NSGA-II Method. J. Fac. Eng. 2008, 42, 313–326. [Google Scholar]
- Papachristos, C.; Alexis, K.; Carrillo, L.R.G.; Tzes, A. Distributed infrastructure inspection path planning for aerial robotics subject to time constraints. In Proceedings of the 2016 International Conference on Unmanned Aircraft Systems (ICUAS), Arlington, VA, USA, 7–10 June 2016; pp. 406–412. [Google Scholar]
- Hoppe, C.; Wendel, A.; Zollmann, S.; Pirker, K.; Irschara, A.; Bischof, H.; Kluckner, S. Photogrammetric camera network design for micro aerial vehicles. In Proceedings of the Computer Vision Winter Workshop (CVWW), Mala Nedelja, Slovenia, 1–3 February 2012; Volume 8, pp. 1–3. [Google Scholar]
- Sun, L.; Hedengren, J.D.; Beard, R.W. Optimal Trajectory Generation using Model Predictive Control for Aerially Towed Cable Systems. J. Guid. Control Dyn. 2014, 37, 525–539. [Google Scholar] [CrossRef]
- Piatti, E.J.; Lerma, J.L. Virtual Worlds for Photogrammetric Image-Based Simulation and Learning. Photogramm. Rec. 2013, 28, 27–42. [Google Scholar] [CrossRef]
- James, M.R.; Robson, S. Mitigating systematic error in topographic models derived from UAV and ground-based image networks. Earth Surf. Process. Landf. 2014, 39, 1413–1420. [Google Scholar] [CrossRef]
- Saur, G.; Bartelsen, J. Experimental application of simulation tools for evaluating UAV video change detection. SPIE Secur. Def. Int. Soc. Opt. Photonics 2015. [Google Scholar] [CrossRef]
- Martin, R.; Rojas, I.; Franke, K.; Hedengren, J. Evolutionary View Planning for Optimized UAV Terrain Modeling in a Simulated Environment. Remote Sens. 2015, 8, 26. [Google Scholar] [CrossRef]
- Pix4D. Designing the Images Acquisition Plan. Available online: https://support.pix4d.com/hc/en-us/articles/202557459-Step-1-Before-Starting-a-Project-1-Designing-the-Images-Acquisition-Plan-a-Selecting-the-Images-Acquisition-Plan-Type (accessed on 15 June 2016).
- Scott, W.R. Model-based view planning. Mach. Vis. Appl. 2009, 20, 47–69. [Google Scholar] [CrossRef]
- Chekuri, C.; Clarkson, K.L.; Har-Peled, S. On the Set Multi-cover Problem in Geometric Settings. In Proceedings of the Twenty-Fifth Annual Symposium on Computational Geometry, Aarhus, Denmark, 8–10 June 2009; pp. 341–350. [Google Scholar]
- Christofides, N. Worst-Case Analysis of a New Heuristic for the Travelling Salesman Problem; Defense Technical Information Center: Fort Belvoir, VA, USA, 1976. [Google Scholar]
- Agisoft. Agisoft PhotoScan Professional Edition. 2014. Available online: http://www.agisoft.ru/products/photoscan (accessed on 15 June 2016).
- CloudCompare. 2016. Available online: http://cloudcompare.org/ (accessed on 12 July 2016).
- Wu, C. Towards Linear-time Incremental Structure from Motion. In Proceedings of the 2013 International Conference on 3D Vision, Seattle, WA, USA, 29 June–1 July 2013; pp. 127–134. [Google Scholar]
- Pieterse, V.; Black, P.E. Big-O Notation; National Institute of Standards and Technology: Gaithersburg, MD, USA, 2016. [Google Scholar]
- Bruggemann, T.S.; Ford, J.J.; Walker, R.A. Control of Aircraft for Inspection of Linear Infrastructure. IEEE Trans. Control Syst. Technol. 2011, 19, 1397–1409. [Google Scholar] [CrossRef]
- Sharma, H.; Dutta, T.; Adithya, V.; Balamuralidhar, P. A Real-Time Framework for Detection of Long Linear Infrastructural Objects in Aerial Imagery; Springer: Cham, Switzerland, 2015; pp. 71–81. [Google Scholar]
- Abayomi, E.; Ayo, D.; Omowunmi, O. Above-Ground Pipeline Monitoring and Surveillance Drone Reactive To Attacks. In Proceedings of the 3rd International Conference on African Development Issues, Ota, Nigeria, 9–11 May 2016. [Google Scholar]
- Bircher, A.; Kamel, M.; Alexis, K.; Burri, M.; Oettershagen, P.; Omari, S.; Mantel, T.; Siegwart, R. Three-dimensional coverage path planning via viewpoint resampling and tour optimization for aerial robots. Auton. Robot. 2016, 40, 1059–1078. [Google Scholar] [CrossRef]
- Xue, Q.; Cheng, P.; Cheng, N. Offline path planni ng and online replanning of UAVs in complex terrain. In Proceedings of the 2014 IEEE Chinese Guidance, Navigation and Control Conference, Yantai, China, 8–10 August 2014; pp. 2287–2292. [Google Scholar]
- Oleynikova, H.; Burri, M.; Taylor, Z.; Nieto, J.; Siegwart, R.; Galceran, E. Continuous-time trajectory optimization for online UAV replanning. In Proceedings of the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Deajeon, Korea, 9–14 October 2016; pp. 5332–5339. [Google Scholar]
Setting | Value |
---|---|
Sensor Width (mm) | 23.5 |
Focal Length (mm) | 35 |
Image Width (pixels) | 6000 |
Image Height (pixels) | 4000 |
Case # | Speed (mph) | Overlap (%) | # of Images | Elevation (m) | GSD (cm) |
---|---|---|---|---|---|
1 | 20 | 75 | 193 | 84 | 0.94 |
2 | 60 | 75 | 65 | 252 | 2.8 |
3 | 20 | 90 | 193 | 210 | 2.3 |
4 | 60 | 90 | 65 | 630 | 7.1 |
Case # | Speed (mph) | Overlap (%) | # of Images | Elevation (m) | GSD (cm) |
---|---|---|---|---|---|
5 | 20 | 75 | 392 | 84 | 0.94 |
6 | 60 | 75 | 137 | 252 | 2.8 |
7 | 20 | 90 | 403 | 210 | 2.3 |
8 | 60 | 90 | 158 | 630 | 7.1 |
Christophides (m) | Optimal (m) | Difference (%) |
---|---|---|
1270.2 | 1202.9 | 5.59% |
998.6 | 918.8 | 8.69% |
851.4 | 800.8 | 6.31% |
678.8 | 634.6 | 6.96% |
205.1 | 189.4 | 8.31% |
Anomaly | # of Images | Average Image Elevation (m) |
---|---|---|
Power Line | 14 | 90.2 |
Road Disc | 14 | 91.0 |
Piping | 58 | 86.0 |
Railway | 19 | 85.4 |
Canal Slump | 23 | 84.8 |
Setting | Value |
---|---|
Photo alignment | High |
Pair preselection | Generic |
Key point limit | 100,000,000 |
Tie point limit | 10,000 |
Dense cloud quality | High |
Depth filtering | Mild |
Site | Case 1 | Case 2 | Case 3 | Case 4 | Case 5 | Case 6 | Case 7 | Case 8 | Opt |
---|---|---|---|---|---|---|---|---|---|
Power | 0.15 | 0.14 | 0.12 | 0.22 | 0.51 | 0.19 | 0.48 | 0.19 | 0.16 |
Disc | 0.16 | 0.33 | 0.12 | 0.51 | 0.41 | 0.15 | 0.46 | 0.52 | 0.08 |
Pipe | N/A | 0.19 | 0.19 | 0.57 | 0.26 | 0.21 | 0.4 | 0.61 | 0.09 |
Rail | 0.09 | 0.26 | 0.15 | 0.28 | 0.11 | 0.21 | 0.29 | 0.42 | 0.10 |
Slump | 0.06 | 0.31 | 0.34 | 0.68 | 0.13 | 0.31 | 0.39 | 0.69 | 0.12 |
Average | 0.12 | 0.25 | 0.18 | 0.45 | 0.28 | 0.21 | 0.40 | 0.49 | 0.11 |
Site | Case 1 | Case 2 | Case 3 | Case 4 | Case 5 | Case 6 | Case 7 | Case 8 | Opt |
---|---|---|---|---|---|---|---|---|---|
Power | 0.18 | 0.15 | 0.12 | 0.27 | 0.37 | 0.2 | 0.35 | 0.23 | 0.15 |
Disc | 0.18 | 0.44 | 0.22 | 0.79 | 0.29 | 0.19 | 0.39 | 0.72 | 0.11 |
Pipe | N/A | 0.03 | 0.35 | 0.90 | 0.27 | 0.34 | 0.33 | 0.81 | 0.18 |
Rail | 0.02 | 0.29 | 0.26 | 0.28 | 0.11 | 0.25 | 0.27 | 0.50 | 0.20 |
Slump | 0.09 | 0.41 | 0.48 | 0.48 | 0.16 | 0.39 | 0.49 | 0.53 | 0.19 |
Average | 0.12 | 0.26 | 0.29 | 0.54 | 0.24 | 0.27 | 0.37 | 0.56 | 0.17 |
Parameter | Value |
---|---|
Miles | 100 |
HD Frames Per Mile (FPM) | 193 |
Desired Frame Overlap (FO) | 90% |
Camera One Megapixels (M1) | 2.1 |
Average Images Per Anomaly (IPA) | 25 |
Camera Two Megapixels (M2) | 24 |
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Martin, R.A.; Blackburn, L.; Pulsipher, J.; Franke, K.; Hedengren, J.D. Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling. Remote Sens. 2017, 9, 434. https://doi.org/10.3390/rs9050434
Martin RA, Blackburn L, Pulsipher J, Franke K, Hedengren JD. Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling. Remote Sensing. 2017; 9(5):434. https://doi.org/10.3390/rs9050434
Chicago/Turabian StyleMartin, R. Abraham, Landen Blackburn, Joshua Pulsipher, Kevin Franke, and John D. Hedengren. 2017. "Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling" Remote Sensing 9, no. 5: 434. https://doi.org/10.3390/rs9050434
APA StyleMartin, R. A., Blackburn, L., Pulsipher, J., Franke, K., & Hedengren, J. D. (2017). Potential Benefits of Combining Anomaly Detection with View Planning for UAV Infrastructure Modeling. Remote Sensing, 9(5), 434. https://doi.org/10.3390/rs9050434