Automated Transverse Crack Mapping System with Optical Sensors and Big Data Analytics
<p>Overall process of the image-based data processing pipeline.</p> "> Figure 2
<p>Multi-view camera system mounted on a road vehicle.</p> "> Figure 3
<p>Parameters that control the accuracy of crack measurement.</p> "> Figure 4
<p>Aggregation of BoW-based similarity score for sequence matching.</p> "> Figure 5
<p>Image sequences and an adjacency graph with sets of edges EL and ET.</p> "> Figure 6
<p>Image-Based localization and mapping.</p> "> Figure 7
<p>Hierarchy of crack pixels and crack segments.</p> "> Figure 8
<p>Detection of a crack pixel using an orientation histogram.</p> "> Figure 9
<p>Crack segment generation.</p> "> Figure 10
<p>Crack width measurement.</p> "> Figure 11
<p>Images collected through our multi-view camera system. Each row represents one frame with four images and consists of two wide field-of-view images, <span class="html-italic">I</span><sub>0</sub> and <span class="html-italic">I</span><sub>1</sub>, and two high-resolution images <span class="html-italic">I</span><sub>0</sub><sup>h</sup>, and <span class="html-italic">I</span><sub>1</sub><sup>h</sup>.</p> "> Figure 12
<p>The reconstructed camera pose of all camera sequences and reconstructed 3D points (left image: zoom-out, right image: zoom-in).</p> "> Figure 13
<p>Global map with stitched images of the top surface of the entire bridge deck.</p> "> Figure 14
<p>Detected crack pixels and generated crack segments: (<b>a</b>) input image, (<b>b</b>) detected crack pixels: blue colored circles represent crack pixels in a different direction than the green colored circles of crack pixels in one dominating orientation. The size of circle represents scale of the crack pixel, (<b>c</b>) represents initialized and extended crack segments, and (<b>d</b>) represents linked crack segments.</p> "> Figure 15
<p>Cracks detected on the top surface of a bridge deck.</p> "> Figure 16
<p>Global map of detected cracks on the top surface of the entire bridge deck.</p> "> Figure 17
<p>Examples of cases that did not indicate the crack properly.</p> ">
Abstract
:1. Introduction
1.1. Motivation
1.2. Related Work
1.3. Knowledge Gap
- The data acquisition speed for such systems is dependent on the speed of optical sensors and the moving platform. The ideal system should be able to acquire data in a fast and simple way without obstructing traffic. Most of the previous studies use either GPS, odometer sensor data or LiDAR data for localization tasks. However, there are locations where GPS signals are not always available for example, in an urban environment. GPS also does not provide pose information. In addition, odometer sensor data can provide relative positions along the direction of traffic but not in the transverse direction perpendicular to the direction of traffic. If one would like to generate a stitched (global) map of an entire bridge deck in an urban area, an alternative localization strategy may be required.
- 2.
- Many studies focus on developing crack detection algorithms and demonstrate their methods only for a local region. Implementing these algorithms to a larger scale and for the entire structural member is challenging and rarely demonstrated in these research studies.
- 3.
- Most of the crack detection algorithms are relying only on line segments and their intensity. This will limit the information to whether there is a crack or not and not necessarily capturing the shape of a long crack shown in the entire member that a structural engineer would be interested in.
- 4.
- Previous research focuses on detecting cracks and not necessarily on making measurements of crack widths and spacings to create a database for future maintenance.
1.4. Research Significance
- Proposed an alternative localization strategy using multi-view image sequences and used Bag-of-Words (BoW) representation of images to complete the localization without GPS and odometer data.
- Created a hierarchy of crack pixels and crack segments to detect cracks, and used a circular histogram to capture the orientation of cracks which is not relying on finding line segments.
- Demonstrated that this algorithm can be implemented for an entire bridge deck member rather than a local region of interest.
- Created ontologies and schemas for generating databases and providing crack measurement data that can be used for future maintenance and decision making additional to the crack detection.
- Demonstrated that the big data image pipeline for conducting crack investigations can be used as a reference pipeline/framework for other applications where large amounts of images are used for data collection, data analysis (detection of deficiencies; in this study, cracks), and decision making based on the database provided.
2. Proposed Methodology
3. Experimental Setup
3.1. Multi-View Sequence Acquisition
3.2. Image-Based Localization and Mapping
3.3. Crack Detection
3.4. Crack Measurements
4. Experimental Results
4.1. Data Collection
4.2. Data Analytics and Visualization
4.3. Crack Database
4.4. Results and Discussion
5. Conclusions
Author Contributions
Funding
Conflicts of Interest
References
- Prasanna, P.; Dana, K.J.; Gucunski, N.; Basily, B.B.; La, H.M.; Lim, R.S.; Parvardeh, H. Automated Crack Detection on Concrete Bridges. IEEE Trans. Autom. Sci. Eng. 2014, 13, 1–9. [Google Scholar] [CrossRef]
- Yeum, C.M.; Dyke, S.J. Vision-Based Automated Crack Detection for Bridge Inspection. Comput. Civ. Infrastruct. Eng. 2015, 30, 759–770. [Google Scholar] [CrossRef]
- Palacios, F.M.; Gucunski, N.; Kee, S.-H.; Van Nguyen, L. Data analysis and visualization for the bridge deck inspection and evaluation robotic system. Vis. Eng. 2015, 3, 59. [Google Scholar]
- Yamaguchi, T.; Hashimoto, S. Fast crack detection method for large-size concrete surface images using percolation-based image processing. Mach. Vis. Appl. 2009, 21, 797–809. [Google Scholar] [CrossRef]
- Chen, F.-C.; Jahanshahi, M.R. NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion. IEEE Trans. Ind. Electron. 2017, 65, 4392–4400. [Google Scholar] [CrossRef]
- Cha, Y.-J.; Choi, W.; Suh, G.; Mahmoudkhani, S.; Büyüköztürk, O. Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types. Comput. Civ. Infrastruct. Eng. 2017, 33, 731–747. [Google Scholar] [CrossRef]
- Liu, Z.; Cao, Y.; Wang, Y.; Wang, W. Computer vision-based concrete crack detection using U-net fully convolutional networks. Autom. Constr. 2019, 104, 129–139. [Google Scholar] [CrossRef]
- Alipour, M.; Harris, D.K. Increasing the robustness of material-specific deep learning models for crack detection across different materials. Eng. Struct. 2020, 206, 110157. [Google Scholar] [CrossRef]
- Mohan, A.; Poobal, S. Crack detection using image processing: A critical review and analysis. Alex. Eng. J. 2018, 57, 787–798. [Google Scholar] [CrossRef]
- Gandhi, R.; Sim, C. Bridge Health Schema. Available online: https://github.com/BridgingBigData/bridgehealthschema (accessed on 26 March 2020).
- Rosten, E.; Drummond, T. Machine Learning for High-Speed Corner Detection. Adv. Visual Comput. 2006, 3951, 430–443. [Google Scholar]
- Calonder, M.; Lepetit, V.; Strecha, C.; Fua, P. BRIEF: Binary Robust Independent Elementary Features. In Proceedings of the 11th European Conference on Computer Vision (ECCV), Heraklion, Greece, 5–11 September 2010; pp. 778–792. [Google Scholar]
- Rublee, E.; Rabaud, V.; Konolige, K.; Bradski, G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 2564–2571. [Google Scholar]
- Hartley, R.; Zisserman, A. Multiple View Geometry in Computer Vision; Cambridge University Press: Cambridge, UK, 2004. [Google Scholar]
- Galvez-López, D.; Tardos, J. Bags of Binary Words for Fast Place Recognition in Image Sequences. IEEE Trans. Robot. 2012, 28, 1188–1197. [Google Scholar] [CrossRef]
- Lepetit, V.; Moreno-Noguer, F.; Fua, P. EPnP: An Accurate O(n) Solution to the PnP Problem. Int. J. Comput. Vis. 2008, 81, 155–166. [Google Scholar] [CrossRef] [Green Version]
- Raúl, M.-A.; Montiel, J.M.M.; Tardos, J.D. ORB-SLAM: A Versatile and Accurate Monocular SLAM System. IEEE Trans. Robot. 2015, 31, 1147–1163. [Google Scholar]
- Kümmerle, R.; Grisetti, G.; Strasdat, H.; Konolige, K.; Burgard, W. G2o: A General Framework for Graph Optimization. In Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 3607–3613. [Google Scholar]
- Levin, A.; Lischinski, D.; Weiss, Y. A Closed-Form Solution to Natural Image Matting. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 30, 228–242. [Google Scholar] [CrossRef] [PubMed] [Green Version]
© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Won, K.; Sim, C. Automated Transverse Crack Mapping System with Optical Sensors and Big Data Analytics. Sensors 2020, 20, 1838. https://doi.org/10.3390/s20071838
Won K, Sim C. Automated Transverse Crack Mapping System with Optical Sensors and Big Data Analytics. Sensors. 2020; 20(7):1838. https://doi.org/10.3390/s20071838
Chicago/Turabian StyleWon, Kwanghee, and Chungwook Sim. 2020. "Automated Transverse Crack Mapping System with Optical Sensors and Big Data Analytics" Sensors 20, no. 7: 1838. https://doi.org/10.3390/s20071838