CN115937138A - Bridge crack identification and detection system and method based on unmanned aerial vehicle - Google Patents
Bridge crack identification and detection system and method based on unmanned aerial vehicle Download PDFInfo
- Publication number
- CN115937138A CN115937138A CN202211557263.3A CN202211557263A CN115937138A CN 115937138 A CN115937138 A CN 115937138A CN 202211557263 A CN202211557263 A CN 202211557263A CN 115937138 A CN115937138 A CN 115937138A
- Authority
- CN
- China
- Prior art keywords
- crack
- image
- aerial vehicle
- unmanned aerial
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 86
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims abstract description 32
- 238000004458 analytical method Methods 0.000 claims abstract description 27
- 238000011161 development Methods 0.000 claims abstract description 16
- 238000013461 design Methods 0.000 claims abstract description 12
- 238000013527 convolutional neural network Methods 0.000 claims abstract description 8
- 238000001914 filtration Methods 0.000 claims description 27
- 230000008569 process Effects 0.000 claims description 22
- 238000000605 extraction Methods 0.000 claims description 19
- 238000005070 sampling Methods 0.000 claims description 6
- 230000005540 biological transmission Effects 0.000 claims description 4
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000009432 framing Methods 0.000 claims description 3
- 230000004927 fusion Effects 0.000 claims description 3
- 230000002068 genetic effect Effects 0.000 claims description 3
- 238000005286 illumination Methods 0.000 claims description 3
- 238000012549 training Methods 0.000 claims description 3
- 230000002401 inhibitory effect Effects 0.000 claims description 2
- 230000003321 amplification Effects 0.000 claims 1
- 238000003199 nucleic acid amplification method Methods 0.000 claims 1
- 238000012423 maintenance Methods 0.000 description 5
- 230000006870 function Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 201000010099 disease Diseases 0.000 description 3
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000006073 displacement reaction Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000002035 prolonged effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000284 extract Substances 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 230000007847 structural defect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Image Analysis (AREA)
Abstract
The invention discloses a bridge crack identification and detection system and method based on an unmanned aerial vehicle, wherein the detection system comprises the unmanned aerial vehicle, an unmanned aerial vehicle route design and route planning unit, an unmanned aerial vehicle safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit and a crack information data output unit; when the system is used, a reasonable flight route can be formulated according to different bridge crack identification and detection tasks and identification and detection positions; meanwhile, when flying in a sunny area, the unmanned aerial vehicle can absorb solar energy and improve the working time of the unmanned aerial vehicle; the method has the advantages that the method can identify and detect the cracks in the image through the convolutional neural network, can calculate width parameter information of the cracks, predicts future development trend of the cracks through microscopic observation, finally generates a 3D (three-dimensional) model view of the bridge, comprehensively displays position information and actual parameter information of the cracks, and maintains the cracks in a targeted manner, and has the characteristics of convenience in use, comprehensive detection and identification and high efficiency.
Description
Technical Field
The invention relates to the technical field of bridge detection, in particular to a bridge crack identification and detection system and method based on an unmanned aerial vehicle.
Background
Bridge construction is one of national important infrastructures, and bridge engineering is life line engineering for harmonious development of the relationship society and the economy; the rapid development of bridge construction, huge capital investment and obvious effect in the economic society make people pay more and more attention to the safety and durability of bridges; meanwhile, bridges in China face the same problems as bridges in numerous international countries, namely a large number of bridges enter the old stage, some bridges have various diseases, but operate with diseases all the year round, and huge main application potential safety hazards are hidden;
in the use process of the existing bridge, reports of bridge damage caused by the influence of cracks on engineering quality are frequently rare, and nearly half of bridges in China enter a disease exposure period, so that the regular identification, detection, judgment and maintenance of the bridges are crucial to traffic safety;
the traditional bridge detection mainly depends on manpower and related detection equipment for direct detection, and has the defects of low efficiency, no detection of inaccessible parts, even danger brought to detection personnel and the like; in recent years, aiming at the detection requirement of the inaccessible part of the conventional bridge detection and the advantages of rapid development, miniaturization, flexibility, rapidness and the like of an unmanned aerial vehicle technology, an accurate identification detection method for researching the structural defects of the inaccessible part of the conventional bridge detection by taking the unmanned aerial vehicle as a carrier and combining large data analysis, a deep learning algorithm and convolutional neural network model analysis is developed;
based on the method, a bridge crack identification and detection system and a bridge crack identification and detection method based on the unmanned aerial vehicle are designed, and a necessary detection means is provided for solving the problems in the prior art.
Disclosure of Invention
Aiming at the existing problems, the invention aims to provide a bridge crack identification and detection system and method based on an unmanned aerial vehicle, and the system can formulate a reasonable flight line of the unmanned aerial vehicle for different specific bridge crack identification and detection tasks and identification and detection positions; meanwhile, when flying in a sunny area, the solar cell panel carried by the unmanned aerial vehicle can absorb solar energy, so that the working time of the unmanned aerial vehicle is prolonged; the method has the advantages that the method is convenient to use, comprehensive in detection and recognition and high in efficiency, and has the characteristics of being convenient to use, comprehensive in detection and recognition and high in efficiency.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
a bridge crack identification and detection system based on an unmanned aerial vehicle comprises the unmanned aerial vehicle, an unmanned aerial vehicle air route design and air route planning unit, an unmanned aerial vehicle safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit and a crack information data output unit;
the unmanned aerial vehicle route design and planning unit is used for generating an unmanned aerial vehicle flight route according to the bridge detection task, the bridge detection part and the unmanned aerial vehicle parameter information;
the unmanned aerial vehicle safe flight unit is used for ensuring the flight safety of the unmanned aerial vehicle according to the unmanned aerial vehicle obstacle avoidance module and the energy module, and simultaneously acquiring the position information of the unmanned aerial vehicle according to the GPS positioning module to determine the position of a crack;
the image acquisition and processing unit is used for acquiring a picture of a crack on the surface of the bridge and extracting crack characteristics;
the crack identification and analysis unit is used for detecting and identifying cracks on the PC end frame by frame according to the crack video information, calculating crack width parameter information and predicting crack development trend;
the crack information data output unit is used for deriving effective crack information pictures, intensively storing actual crack parameter information and position information in a crack information database, establishing a 3D (three-dimensional) bridge model view, and simultaneously extracting crack parameter information from the crack information database and displaying the crack parameter information at the node position corresponding to the bridge.
Preferably, the image acquisition and processing unit comprises a camera anti-shake module, a light complementing module, an industrial camera module, an image denoising module and a feature extraction module,
the camera anti-shake module is used for reducing video image blurring caused by external environment interference;
the light complementing module is used for ensuring sufficient light when the bridge collects the picture towards the shade and balancing the shooting brightness;
the industrial camera module is used for collecting high-resolution video recordings;
the image denoising module is used for denoising the acquired crack image information;
the characteristic extraction module is used for extracting characteristic information of the collected crack image information.
Preferably, the crack identification and analysis unit comprises a crack identification module, a crack analysis module, a crack prediction module and a PC terminal,
the crack identification module is used for identifying and detecting crack image information acquired by the industrial camera at the PC end according to the convolutional neural network;
the crack analysis module is used for calculating the width of the crack through an MATLAB algorithm program according to the crack image obtained by identification;
the crack prediction module is used for predicting the future development trend of crack growth trends at the fine part of the crack observed in an amplified manner by using an MATLAB program according to the recognized crack images.
A bridge crack identification and detection system based on an unmanned aerial vehicle carries out detection, and the detection method comprises the following steps
Step 1: fracture information collection process
Step 1.1, the unmanned aerial vehicle collects crack information of the surface of the bridge through a carried industrial camera according to a planned flight path, and all video videos of the collected cracks are stored in an SD card and remotely transmitted to a PC (personal computer) end;
and 2, step: data transmission process
Step 2.1, after receiving the video of the crack acquired by the unmanned aerial vehicle, the PC end firstly removes noise points in the image by using an image denoising module;
2.2, after the image is denoised, extracting crack feature information from the collected crack image by using a feature extraction module by means of an SIFT algorithm;
2.3, after the characteristic extraction module extracts crack characteristic information from the collected crack image by means of an SIFT algorithm, identifying and detecting the crack image information by using a crack identification module;
step 2.4, calculating the width of the crack by using a crack analysis module;
step 2.5, utilizing a crack prediction module to amplify the crack, observing the crack growth trend and predicting the possible future development trend;
and step 3: a 3D stereoscopic view is generated.
Preferably, in step S2.1, the image denoising module utilizes an optimized genetic algorithm to denoise and remove noise in the image, and the specific process includes
Step 2.11 according to the video image, firstly setting nonlinear filtering de-noising data epsilon s (m, n) is
Step 2.12 obtaining improved filtering and de-noising data according to the nonlinear filtering and de-noising data as
Step 2.13, according to the nonlinear filtering denoising data and the improved filtering denoising data, obtaining the denoised image data
ε(m,n)＝ε s (m,n)·ε l (m,n)(3)
Step 2.14, calculating the pixel value of the image after filtering processing as:
in the formulae (1) to (4): epsilon s (m, n) non-linearly filtering the de-noised data; m and n are image position information needing denoising processing; s is the neighborhood value; λ -filter coefficient; x, y-image edge information; a is a specific weight; epsilon l (m, n) -improved filtering the denoised data; l-improving neighborhood value; g (x, y) -image pixel values; b, improving the weight; ε (m, n) -denoised image data; f (x, y) -pixel values of the image after filtering processing; v (x, y) -other neighborhood values of the image center point;
step 2.15, when f (x, y) = g (x, y), a denoised image is obtained.
Preferably, in step S2.2, the process of extracting crack feature information from the acquired crack image by using the feature extraction module with the aid of the SIFT algorithm, improving the features of interest, and suppressing the features of no interest includes
(1) Potential key points of the cracks are extracted by utilizing the Gaussian differential function, the key points are screened again under different scales, the fact that the points with the cracks extracted have scale invariance is guaranteed, and the points are not affected by factors such as illumination, proportion and rotation;
(2) Calculating through different space operators to determine the direction of the crack characteristic points;
(3) And matching the crack characteristic points to obtain a corresponding relation.
Preferably, in step S2.3, the fracture identification module performs identification detection on the fracture image information according to the convolutional neural network, and the specific process includes
Step 2.31, training a YOLO crack detection algorithm model according to the existing crack data set;
step 2.32, performing framing processing on the crack video acquired by the unmanned aerial vehicle, and inputting the processed crack video into a crack data set to be detected;
2.33, firstly, carrying out crack feature extraction including operations such as convolution, up-sampling, down-sampling, feature fusion and the like on a crack data set to be detected to obtain high-level features;
step 2.34, the high-level features are divided into a convolution module 1 and a convolution module 2 through one convolution, the convolution module 1 obtains a convolution module 11 through convolution operations of two 3*3 and one 1*1, the convolution module 2 obtains a convolution module 22 through convolution operations of two 3*3 and two 1*1, and finally the convolution modules 11 and 22 are stacked to obtain a new feature level;
step 2.35, carrying out region division on the new characteristic layer, and confusing field information through a gradient inversion layer to enable the model to learn the invariant characteristics of the crack field;
step 2.36, target category and position information are finally predicted, crack target features are learned through reverse propagation of a loss function, and meanwhile, frame regression is carried out to obtain the position of a crack detection frame;
and 2.37, sending the detection and identification result of the crack to a crack analysis module for further processing.
Preferably, in step S2.4, the crack analysis module is configured to calculate the crack width through an algorithm program according to the crack image obtained by identification, and the specific process includes
Step 2.41 Gray processing of crack image
Y＝0.301×R+0.586×G+0.113×B(5)
Y in the formula (5) represents the final brightness value of the gray image, R, G, B respectively corresponds to the color values of red, green and blue three channels in the color image, and the range of Y, R, G, B is 0-255;
step 2.42 crack image binarization processing
In the formula (6), (x, y) is a pixel coordinate point, T is a threshold value, and P (x, y) is a binarized pixel value;
step 2.43, calculating and counting the percentage of the number of pixels with the gray scale value of 255 in the binary image to the total number of pixels, and multiplying the percentage by the area calculation of the image to calculate the area of the crack:
in the formula (7), S 0 The area of the crack is shown as S, the area of the picture is shown as N, the number of crack pixel points is shown as N, and the number of total pixel points is shown as N;
step 2.44, the crack width can be obtained by calculating the ratio of the crack area to the crack length and then multiplying the ratio by the unit pixel width:
in the formula (8), a is a unit pixel width, L is a slit length, and W is a slit width.
Preferably, the process of predicting the crack by using the crack prediction module in step S2.5 is to use the MATLAB program to enlarge the crack to observe the crack growth trend at a fine part and predict the possible future development trend.
Preferably, the process of generating the 3D stereoscopic view image in step S3 includes
And establishing a 3D model view of the bridge through SolidWorks modeling software, and simultaneously calling crack information from a crack information database to be embedded in the corresponding node position of the bridge to obtain the 3D model view of the bridge.
The beneficial effects of the invention are: the invention discloses a bridge crack identification and detection system and method based on an unmanned aerial vehicle, and compared with the prior art, the improvement of the invention is as follows:
1. the bridge crack identification and detection system based on the unmanned aerial vehicle is designed, and when the bridge crack identification and detection system is used, a reasonable unmanned aerial vehicle flight line can be designed according to a bridge crack identification and detection task, a bridge to-be-detected part and unmanned aerial vehicle parameters; carrying an industrial camera by means of a rotary wing type unmanned aerial vehicle to carry out aerial operation according to a specified flight path, and recording and acquiring crack information by the industrial camera in the whole process; when the unmanned aerial vehicle flies in a sunny area, the solar cell panel carried by the unmanned aerial vehicle can absorb solar energy, so that the working time of the unmanned aerial vehicle is prolonged; when flying to the shadow area, the light complementing module carries out balanced light complementing on the area with insufficient brightness to achieve uniform brightness; the acquired image is subjected to image denoising and feature extraction processing, so that the subsequent crack identification detection precision is improved; the system identifies and detects the cracks in the image through a convolutional neural network, calculates width parameter information of the cracks by adopting an algorithm program, predicts the development trend of the cracks through a deep learning algorithm, and finally generates a 3D model view of the bridge through modeling software to comprehensively display position information and actual parameter information of the cracks, so that the cracks can be maintained in a targeted manner;
2. the invention designs a detection method of a bridge crack identification and detection system based on an unmanned aerial vehicle, and the method can design a reasonable flight path when in use, and fully exerts the advantages of flexibility, high efficiency and controllability of the unmanned aerial vehicle; according to the fact that the unmanned aerial vehicle plays different roles when the unmanned aerial vehicle works in different sunny and cloudy flying areas, when the unmanned aerial vehicle flies in the sunny area, solar energy is fully absorbed by the solar cell panel and is distributed to the unmanned aerial vehicle battery module for supplying power, when the unmanned aerial vehicle flies in the cloudy area, in order to ensure that collected images are clear and visible, the module with insufficient light automatically adjusts brightness according to the brightness degree to keep the brightness of the images balanced and uniform in brightness and darkness; a convolution neural network is adopted at the PC end to detect and identify cracks of the video frame by frame, so that the phenomena of missing detection and false detection are reduced; calculating width parameter information of the crack according to the crack parameters of the video picture by using an algorithm program, and quantitatively describing the crack danger degree; the development trend of the cracks is predicted by adopting a deep learning algorithm, so that subsequent targeted maintenance is facilitated; and finally, crack parameter information is embedded at the position of the corresponding node of the 3D model view of the bridge, so that browsing, checking and maintenance are facilitated, and the method has the advantages of convenience in use, comprehensive detection and identification and high efficiency.
Drawings
Fig. 1 is a work flow chart of the bridge crack identification and detection system and method based on the unmanned aerial vehicle.
Fig. 2 is a system block diagram of the bridge crack recognition and detection system and method based on the unmanned aerial vehicle.
Wherein: 1. the system comprises an unmanned aerial vehicle, 2, an obstacle avoidance module, 3, an unmanned aerial vehicle battery module, 4, a solar panel, 5, an SD card, 6, a flight time sensor, 7, a GPS positioning module, 8, a camera anti-shake module, 9, a light complementing module, 10, an industrial camera module, 11, a crack, 12, a bridge, 13, an image denoising module, 14, a feature extraction module, 15, a crack identification module, 16, a crack analysis module, 17, a crack prediction module, 18, a crack information database, 19, a PC end, 20, a crack information unit, 21.3D stereoscopic model view, 22, a crack information acquisition process, 23, a data transmission process and 24, and a 3D stereoscopic view generation process.
Detailed Description
In order to make those skilled in the art better understand the technical solution of the present invention, the following further describes the technical solution of the present invention with reference to the drawings and the embodiments.
Example 1: referring to the attached drawings 1-2, the bridge crack identification and detection system based on the unmanned aerial vehicle comprises the unmanned aerial vehicle 1, an unmanned aerial vehicle air route design and air route planning unit, an unmanned aerial vehicle safe flight unit, an image acquisition and processing unit, a crack identification and analysis unit and a crack information data output unit, wherein the unmanned aerial vehicle air route design and air route planning unit, the unmanned aerial vehicle safe flight unit, the image acquisition and processing unit, the crack identification and analysis unit and the crack information data output unit are arranged on the unmanned aerial vehicle 1, and the crack identification and detection system comprises a first detection unit, a second detection unit, a crack detection unit and a crack detection unit
The unmanned aerial vehicle route design and planning unit is used for generating a reasonable unmanned aerial vehicle flight route according to a specific bridge detection task, a bridge detection part and unmanned aerial vehicle parameter information; when the unmanned aerial vehicle flight path design and planning unit is used, during the flight of the unmanned aerial vehicle 1, the unmanned aerial vehicle obstacle avoidance module 2 on the unmanned aerial vehicle calculates the distance between the unmanned aerial vehicle 1 and the bridge 12 through the flight time sensor 6 to plan the flight path of the unmanned aerial vehicle, meanwhile, the flight time sensor 6 emits infrared rays with certain frequency to the bridge 12, calculates the distance according to the phase difference between the reflected signal and the source signal, plans a course route and prevents the collision phenomenon of the unmanned aerial vehicle;
(1) Designing a flight path of the unmanned aerial vehicle preliminarily according to a bridge structure design drawing and flight parameter information of the unmanned aerial vehicle 1;
(2) The air route planning is used for optimizing the air route design according to the specific geographic position and the located environmental information of the bridge and the weather condition during working, dividing the air route into different flight areas according to the sunny side and the shady side of the bridge, and planning the final flight air route of the unmanned aerial vehicle;
the unmanned aerial vehicle safe flight unit comprises an energy module and a GPS positioning module 7, is used for ensuring the flight safety of the unmanned aerial vehicle according to the unmanned aerial vehicle obstacle avoidance module 2 and the energy module, and is used for acquiring the position information of the unmanned aerial vehicle according to the GPS positioning module 7 so as to determine the position of a bridge where a crack is located, wherein the energy module comprises a battery module 3 and a solar panel 4, and when the solar panel 4 works in a sunny area, solar energy is fully absorbed and supplied to the battery module 3 of the unmanned aerial vehicle, and electric equipment of the unmanned aerial vehicle 1 is redistributed; the GPS positioning module 7 is used for acquiring the flight position of the unmanned aerial vehicle 1 in the air in real time so as to determine the actual position of the bridge 12 where the crack is located;
the image acquisition and processing unit comprises a camera anti-shake module 8, a light complementing module 9, an industrial camera module 10, an image denoising module 13 and a feature extraction module 14, and is used for ensuring that the acquired picture of the surface crack 11 of the bridge 12 is clear and visible according to the industrial camera module 10, the camera anti-shake module 8 and the light complementing module 9 when in use, and improving crack detection precision according to the image denoising module 13 and the feature extraction module 14; when a gyroscope in a camera anti-shake module 8 embedded in an industrial camera 10 detects tiny shake, a signal is transmitted to an industrial camera microprocessor and a displacement required to be compensated is calculated, and then compensation is carried out according to the shake direction and the displacement of the camera through a compensation lens group; when the unmanned aerial vehicle 1 works in a shadow area and a camera in the industrial camera 10 detects that the brightness of a shooting environment is reduced, an instruction is sent to the light complementing module 9, and the brightness of the light of the near light group and the high light group on the light complementing module ensures the brightness balance of the collected images; the described
The crack identification and analysis unit comprises a crack identification module 15, a crack analysis module 16, a crack prediction module 17 and a PC (personal computer) end 19, when the crack identification and analysis unit is used, the crack identification module 15 is used for detecting and identifying cracks on the PC end 19 frame by frame according to crack video information, the crack analysis module 16 is used for calculating width parameter information of the crack 11 according to an MATLAB (matrix laboratory) program algorithm, and the crack prediction module 17 is used for predicting the development trend of the possible crack 11;
the crack information data output unit comprises a crack information database 18 and a crack information unit 20 and is used for deriving an effective crack information picture according to a video, storing actual crack parameter information and position information in the crack information unit 20 in the crack information database 18 in a centralized manner, establishing a 3D (three-dimensional) model view 21 of the bridge through modeling software, extracting crack parameter information from the crack information database 18 and displaying the crack parameter information at the position of a node corresponding to the bridge, and facilitating the follow-up maintenance work.
Preferably, the image acquisition and processing unit further comprises an SD card 5, and the SD card 5 is used for storing all video videos of the crack 11 acquired by the unmanned aerial vehicle 1 and remotely transmitting the videos to the PC terminal 19 through a data transmission process 23.
The detection method and the detection process of the bridge crack identification and detection system based on the unmanned aerial vehicle comprise
Step 1: fracture information acquisition process 22
Step 1.1, the unmanned aerial vehicle 1 collects information of cracks 11 on the surface of a bridge 12 through a carried industrial camera 8 according to a planned flight route, stores all video videos of the collected cracks 11 in an SD card 5 and remotely transmits the videos to a PC end 19;
step 2: data transfer process 23
Step 2.1, after receiving the video of the crack 11 acquired by the unmanned aerial vehicle 1, the PC end 19 utilizes the image denoising module 13 to denoise and remove noise points in the image by adopting an optimized genetic algorithm to improve the image quality, wherein the denoising process comprises
Step 2.11, according to the video image, firstly setting nonlinear filtering denoising data epsilon s (m, n) is
Step 2.12 obtaining improved filtering and denoising data from the nonlinear filtering and denoising data as
Step 2.13 obtaining the denoised image data according to the nonlinear filtering denoising data and the improved filtering denoising data
ε(m,n)＝ε s (m,n)·ε l (m,n)(3)
Step 2.14 further calculates the pixel value of the image after filtering processing as:
in the formulae (1) to (4): epsilon s (m, n) non-linearly filtering the de-noised data; m and n are image position information needing denoising processing; s is the neighborhood value; λ -filter coefficient; x, y-image edge information; a-specific weight; epsilon l (m, n) -improved filtering the denoised data; l-improving neighborhood value; g (x, y) -image pixel values; b, improving the weight; ε (m, n) -denoised image data; f (x, y) -pixel values of the image after filtering processing; v (x, y) -other neighborhood values of the image center point;
therefore, when f (x, y) = g (x, y), the image denoising effect is strong.
Step 2.2, after the image is denoised, extracting crack feature information from the collected crack image by using a SIFT algorithm by using the feature extraction module 14, improving the interested features and inhibiting the uninteresting features:
(1) Potential key points of the cracks are extracted by utilizing the Gaussian differential function, the key points are screened again under different scales, the fact that the points with the cracks extracted have scale invariance is guaranteed, and the points are not affected by factors such as illumination, proportion and rotation;
(2) Determining the direction of the crack characteristic points, and calculating and obtaining the crack characteristic points through different space operators;
(3) Matching the crack characteristic points to obtain a corresponding relation;
step 2.3, after the crack feature information of the collected crack image is extracted by the feature extraction module 14 by means of SIFT algorithm, the crack image information collected by the industrial camera is identified and detected by the crack identification module 15 according to the convolutional neural network, and the method comprises the following steps
Step 2.31, training a YOLO crack detection algorithm model according to the existing crack data set;
step 2.32, performing framing processing on the crack video acquired by the unmanned aerial vehicle, and inputting the processed crack video into a crack data set to be detected;
2.33, firstly, carrying out crack feature extraction including operations such as convolution, up-sampling, down-sampling, feature fusion and the like on a crack data set to be detected to obtain high-level features;
step 2.34, the high-level features are divided into a convolution module 1 and a convolution module 2 through one convolution, the convolution module 1 obtains a convolution module 11 through the convolution operation of two 3*3 and one 1*1, the convolution module 2 obtains a convolution module 22 through the convolution operation of two 3*3 and two 1*1, and finally, the convolution modules 11 and 22 are stacked to obtain a new feature layer;
step 2.35, carrying out region division on the new characteristic layer, and confusing field information through a Gradient Reversal Layer (GRL) to enable the model to learn the unchanged characteristics of the crack field;
step 2.36, target category and position information are finally predicted, crack target features are learned through back propagation of a loss function, and meanwhile, frame regression is conducted to obtain the position of a crack detection frame;
2.37, sending the detection and identification result of the crack to a crack analysis module 16 for further processing;
the crack analysis module 16 in step 2.4 is configured to calculate the crack width through an algorithm program according to the crack image obtained by identification, and includes the following steps:
step 2.41 fracture image graying treatment
Y＝0.301×R+0.586×G+0.113×B(5)
Y in the formula (5) represents the final brightness value of the gray image, R, G, B respectively corresponds to the color values of red, green and blue three channels in the color image, and the range of Y, R, G, B is 0-255;
step 2.42 crack image binarization processing
In the formula (6), (x, y) is a pixel coordinate point, T is a threshold value, and P (x, y) is a binarized pixel value;
step 2.43, calculating and counting the percentage of the number of pixels with the gray scale value of 255 in the binary image to the total number of pixels, and multiplying the percentage by the area calculation of the image to calculate the crack area, wherein the specific formula is as follows:
in the formula (7), S 0 The area of the crack is shown as S, the area of the picture is shown as N, the number of crack pixel points is shown as N, and the number of total pixel points is shown as N;
step 2.44, the crack width can be obtained by calculating the ratio of the crack area to the crack length and then multiplying the ratio by the unit pixel width:
in the formula (8), a is the width of a unit pixel, L is the length of a crack, and W is the width of the crack;
step 2.5, the crack prediction module 17 is used for amplifying the cracks according to the identified crack images and observing the crack growth trend of the fine part by using an MATLAB program and predicting the possible future development trend;
and step 3: generating 3D stereoscopic views Process 24
The foregoing shows and describes the general principles, essential features, and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, which are described in the specification and illustrated only to illustrate the principle of the present invention, but that various changes and modifications may be made therein without departing from the spirit and scope of the present invention, which fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (10)
1. The utility model provides a bridge crack discernment detecting system based on unmanned aerial vehicle which characterized in that: the unmanned aerial vehicle safety flight system comprises an unmanned aerial vehicle, an unmanned aerial vehicle route design and route planning unit, an unmanned aerial vehicle safety flight unit, an image acquisition and processing unit, a crack identification and analysis unit and a crack information data output unit;
the unmanned aerial vehicle route designing and planning unit is used for generating an unmanned aerial vehicle flight route according to the bridge detection task, the bridge detection part and the unmanned aerial vehicle parameter information;
the unmanned aerial vehicle safe flight unit is used for ensuring the flight safety of the unmanned aerial vehicle according to the unmanned aerial vehicle obstacle avoidance module and the energy module, and simultaneously acquiring the position information of the unmanned aerial vehicle according to the GPS positioning module to determine the position of a crack;
the image acquisition and processing unit is used for acquiring a crack picture on the surface of the bridge and extracting crack characteristics;
the crack identification and analysis unit is used for detecting and identifying cracks on the PC end frame by frame according to the crack video information, calculating the crack width parameter information and predicting the crack development trend;
the crack information data output unit is used for deriving effective crack information pictures, intensively storing actual crack parameter information and position information in a crack information database, establishing a 3D (three-dimensional) bridge model view, and simultaneously extracting crack parameter information from the crack information database and displaying the crack parameter information at the node position corresponding to the bridge.
2. The bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 1, wherein: the image acquisition and processing unit comprises a camera anti-shake module, a light complementing module, an industrial camera module, an image de-noising module and a feature extraction module,
the camera anti-shake module is used for reducing video image blurring caused by external environment interference;
the light complementing module is used for ensuring sufficient light when the bridge collects the picture towards the shade and balancing the shooting brightness;
the industrial camera module is used for collecting high-resolution video;
the image denoising module is used for denoising the acquired crack image information;
the characteristic extraction module is used for extracting characteristic information of the collected crack image information.
3. The unmanned aerial vehicle-based bridge crack identification and detection system of claim 1, wherein: the crack identification and analysis unit comprises a crack identification module, a crack analysis module, a crack prediction module and a PC end,
the crack identification module is used for identifying and detecting crack image information acquired by the industrial camera at the PC end according to the convolutional neural network;
the crack analysis module is used for calculating the width of the crack through an MATLAB algorithm program according to the crack image obtained by identification;
the crack prediction module is used for predicting the future development trend of crack growth trends at the fine part of crack amplification observation by using an MATLAB program according to the recognized crack images.
4. A method for detecting by using the unmanned aerial vehicle-based bridge crack identification and detection system according to claim 1, wherein the method comprises the following steps: the detection method comprises
Step 1: fracture information collection process
Step 1.1, the unmanned aerial vehicle collects crack information on the surface of a bridge through a carried industrial camera according to a planned flight route, and stores all collected video videos of the crack in an SD card and remotely transmits the video videos to a PC (personal computer) end;
step 2: data transmission process
Step 2.1, after receiving the video of the crack acquired by the unmanned aerial vehicle, the PC end firstly removes noise points in the image by using an image denoising module;
2.2, after denoising processing is carried out on the image, extracting crack feature information from the collected crack image by using a feature extraction module through an SIFT algorithm;
2.3, after extracting crack characteristic information from the acquired crack image by a characteristic extraction module by means of an SIFT algorithm, identifying and detecting the crack image information by using a crack identification module;
step 2.4, calculating the width of the crack by using a crack analysis module;
step 2.5, utilizing a crack prediction module to amplify the crack, observing the crack growth trend and predicting the possible future development trend;
and step 3: a 3D stereoscopic view is generated.
5. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 4, wherein: in step S2.1, the image denoising module utilizes an optimized genetic algorithm to denoise and remove noise points in the image, and the specific process comprises
Step 2.11 according to the video image, firstly setting nonlinear filtering de-noising data epsilon s (m, n) is
Step 2.12 obtaining improved filtering and de-noising data according to the nonlinear filtering and de-noising data as
Step 2.13 obtaining the denoised image data according to the nonlinear filtering denoising data and the improved filtering denoising data
ε(m,n)＝ε s (m,n)·ε l (m,n) (3)
Step 2.14, calculating the pixel value of the image after filtering processing as:
in the formulae (1) to (4): epsilon s (m, n) nonlinearly filtering the denoised data; m and n are image position information needing denoising processing; s is the neighborhood value; λ -filter coefficient; x, y-image edge information; a-specific weight; epsilon l (m, n) -improved filtering the denoised data; l-improving the neighborhood value; g (x, y) -image pixel values; b, improving the weight; ε (m, n) -denoised image data; f (x, y) -pixel values of the image after filtering processing; v (x, y) -other neighborhood values of the image center point;
step 2.15, when f (x, y) = g (x, y), a denoised image is obtained.
6. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 4, wherein: in step S2.2, the process of extracting crack feature information from the collected crack image by using a feature extraction module and by means of an SIFT algorithm, improving interesting features and inhibiting uninteresting features comprises
(1) Potential key points of the cracks are extracted by utilizing the Gaussian differential function, the key points are screened again under different scales, the fact that the points with the cracks extracted have scale invariance is guaranteed, and the points are not affected by factors such as illumination, proportion and rotation;
(2) Calculating through different space operators to determine the direction of the crack characteristic points;
(3) And matching the crack characteristic points to obtain a corresponding relation.
7. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 4, wherein: in step S2.3, the crack recognition module performs recognition and detection on the crack image information according to the convolutional neural network, and the specific process includes
Step 2.31, training a YOLO crack detection algorithm model according to the existing crack data set;
step 2.32, performing framing processing on the crack video acquired by the unmanned aerial vehicle, and inputting the processed crack video into a crack data set to be detected;
2.33, firstly, carrying out crack feature extraction including operations such as convolution, up-sampling, down-sampling, feature fusion and the like on a crack data set to be detected to obtain high-level features;
step 2.34, the high-level features are divided into a convolution module 1 and a convolution module 2 through one convolution, the convolution module 1 obtains a convolution module 11 through the convolution operation of two 3*3 and one 1*1, the convolution module 2 obtains a convolution module 22 through the convolution operation of two 3*3 and two 1*1, and finally, the convolution modules 11 and 22 are stacked to obtain a new feature layer;
step 2.35, carrying out region division on the new characteristic layer, and confusing field information through a gradient inversion layer to enable the model to learn the invariant characteristics of the crack field;
step 2.36, target category and position information are finally predicted, crack target features are learned through back propagation of a loss function, and meanwhile, frame regression is conducted to obtain the position of a crack detection frame;
and 2.37, sending the detection and identification result of the crack to a crack analysis module for further processing.
8. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 7, wherein: in step S2.4, the crack analysis module is configured to calculate the crack width through an algorithm program according to the crack image obtained by the recognition, and the specific process includes
Step 2.41 fracture image graying treatment
Y＝0.301×R+0.586×G+0.113×B(5)
Y in the formula (5) represents the final brightness value of the gray image, R, G, B respectively corresponds to the color values of red, green and blue three channels in the color image, and the range of Y, R, G, B is 0-255;
step 2.42 crack image binarization processing
In the formula (6), (x, y) is a pixel coordinate point, T is a threshold value, and P (x, y) is a binarized pixel value;
step 2.43, calculating and counting the percentage of the number of the pixels with the gray value of 255 in the binary image to the total number of the pixels, and multiplying the percentage by the area calculation of the picture to calculate the crack area:
in the formula (7), S 0 The area of the crack is shown as S, the area of the picture is shown as N, the number of crack pixel points is shown as N, and the number of total pixel points is shown as N;
step 2.44, the crack width can be obtained by calculating the ratio of the crack area to the crack length and then multiplying the ratio by the width of a unit pixel:
in the formula (8), a is a unit pixel width, L is a slit length, and W is a slit width.
9. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 4, wherein: the process of predicting the crack by using the crack prediction module described in the step S2.5 is to enlarge the crack by using the MATLAB program to observe the crack growth trend at the fine part and predict the possible future development trend.
10. The detection method of the bridge crack identification and detection system based on the unmanned aerial vehicle as claimed in claim 4, wherein: and S3, the process of generating the 3D stereoscopic view comprises the steps of establishing a 3D model view of the bridge through SolidWorks modeling software, and simultaneously calling crack information from the crack information database to be embedded in the corresponding node position of the bridge where the crack information is located to obtain the 3D stereoscopic view of the bridge.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557263.3A CN115937138A (en) | 2022-12-06 | 2022-12-06 | Bridge crack identification and detection system and method based on unmanned aerial vehicle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211557263.3A CN115937138A (en) | 2022-12-06 | 2022-12-06 | Bridge crack identification and detection system and method based on unmanned aerial vehicle |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115937138A true CN115937138A (en) | 2023-04-07 |
Family
ID=86655498
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211557263.3A Pending CN115937138A (en) | 2022-12-06 | 2022-12-06 | Bridge crack identification and detection system and method based on unmanned aerial vehicle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115937138A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117928539A (en) * | 2023-12-08 | 2024-04-26 | 淮阴工学院 | Bridge mass surveying method based on four-rotor unmanned aerial vehicle |
| CN118090742A (en) * | 2024-04-18 | 2024-05-28 | 江西赣粤高速公路股份有限公司 | Image acquisition method for detecting hinge joint damage of prefabricated bridge |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108051450A (en) * | 2017-11-01 | 2018-05-18 | 中铁大桥科学研究院有限公司 | A kind of bridge health monitoring system and method based on unmanned plane |
| CN109521019A (en) * | 2018-11-09 | 2019-03-26 | 华南理工大学 | A kind of bridge bottom crack detection method based on unmanned plane vision |
| CN110378879A (en) * | 2019-06-26 | 2019-10-25 | 杭州电子科技大学 | A kind of Bridge Crack detection method |
| CN112597812A (en) * | 2020-12-03 | 2021-04-02 | 西安格威西联科技有限公司 | Finger vein identification method and system based on convolutional neural network and SIFT algorithm |
-
2022
- 2022-12-06 CN CN202211557263.3A patent/CN115937138A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108051450A (en) * | 2017-11-01 | 2018-05-18 | 中铁大桥科学研究院有限公司 | A kind of bridge health monitoring system and method based on unmanned plane |
| CN109521019A (en) * | 2018-11-09 | 2019-03-26 | 华南理工大学 | A kind of bridge bottom crack detection method based on unmanned plane vision |
| CN110378879A (en) * | 2019-06-26 | 2019-10-25 | 杭州电子科技大学 | A kind of Bridge Crack detection method |
| CN112597812A (en) * | 2020-12-03 | 2021-04-02 | 西安格威西联科技有限公司 | Finger vein identification method and system based on convolutional neural network and SIFT algorithm |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117928539A (en) * | 2023-12-08 | 2024-04-26 | 淮阴工学院 | Bridge mass surveying method based on four-rotor unmanned aerial vehicle |
| CN118090742A (en) * | 2024-04-18 | 2024-05-28 | 江西赣粤高速公路股份有限公司 | Image acquisition method for detecting hinge joint damage of prefabricated bridge |
| CN118090742B (en) * | 2024-04-18 | 2024-07-30 | 江西赣粤高速公路股份有限公司 | Image acquisition method for detecting hinge joint damage of prefabricated bridge |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111476781B (en) | Concrete crack identification method and device based on video semantic segmentation technology | |
| CN106356757B (en) | A kind of power circuit unmanned plane method for inspecting based on human-eye visual characteristic | |
| Xu et al. | BrIM and UAS for bridge inspections and management | |
| Dong et al. | A deep-learning-based multiple defect detection method for tunnel lining damages | |
| Jianya et al. | A review of multi-temporal remote sensing data change detection algorithms | |
| CN107808133B (en) | Unmanned aerial vehicle line patrol-based oil and gas pipeline safety monitoring method and system and software memory | |
| CN110889327B (en) | Intelligent detection method for sewage outlet around water area based on thermal infrared image | |
| CN110910350B (en) | Nut loosening detection method for wind power tower cylinder | |
| Liu et al. | A night pavement crack detection method based on image‐to‐image translation | |
| CN114677601B (en) | Dam crack detection method based on unmanned aerial vehicle inspection and combined with deep learning | |
| CN114240868A (en) | Unmanned aerial vehicle-based inspection analysis system and method | |
| CN112396635B (en) | Multi-target detection method based on multiple devices in complex environment | |
| CN110991430B (en) | Ground feature identification and coverage rate calculation method and system based on remote sensing image | |
| CN114998251B (en) | Air multi-vision platform ground anomaly detection method based on federal learning | |
| CN111899219A (en) | Image identification method and system for power transmission line machine patrol | |
| CN115937138A (en) | Bridge crack identification and detection system and method based on unmanned aerial vehicle | |
| CN111105398A (en) | Transmission line component crack detection method based on visible light image data | |
| CN115115954A (en) | Intelligent identification method for pine nematode disease area color-changing standing trees based on unmanned aerial vehicle remote sensing | |
| CN115272826A (en) | Image identification method, device and system based on convolutional neural network | |
| CN113033386B (en) | High-resolution remote sensing image-based transmission line channel hidden danger identification method and system | |
| CN114494845A (en) | Artificial intelligence hidden danger troubleshooting system and method for construction project site | |
| Jiang et al. | Coral detection, ranging, and assessment (CDRA) algorithm-based automatic estimation of coral reef coverage | |
| CN112101168A (en) | Satellite and unmanned aerial vehicle linkage-based commonweal litigation auxiliary evidence obtaining system and method | |
| CN115761265A (en) | Method and device for extracting substation equipment in laser radar point cloud | |
| CN116109813A (en) | Anchor hole drilling identification method, system, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |