CN112699736A - Bridge bearing fault identification method based on space attention - Google Patents
Bridge bearing fault identification method based on space attention Download PDFInfo
- Publication number
- CN112699736A CN112699736A CN202011442501.7A CN202011442501A CN112699736A CN 112699736 A CN112699736 A CN 112699736A CN 202011442501 A CN202011442501 A CN 202011442501A CN 112699736 A CN112699736 A CN 112699736A
- Authority
- CN
- China
- Prior art keywords
- image
- bridge
- spatial
- sample image
- spatial attention
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 33
- 201000010099 disease Diseases 0.000 claims abstract description 20
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims abstract description 20
- 238000013528 artificial neural network Methods 0.000 claims abstract description 18
- 238000012549 training Methods 0.000 claims abstract description 10
- 238000011478 gradient descent method Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000011176 pooling Methods 0.000 claims description 3
- 230000006870 function Effects 0.000 abstract description 8
- 238000013527 convolutional neural network Methods 0.000 abstract description 4
- 238000003062 neural network model Methods 0.000 abstract 2
- 238000002372 labelling Methods 0.000 abstract 1
- 238000001514 detection method Methods 0.000 description 5
- 238000012512 characterization method Methods 0.000 description 3
- 238000005336 cracking Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 230000032683 aging Effects 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013136 deep learning model Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009440 infrastructure construction Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007774 longterm Effects 0.000 description 1
- 230000015654 memory Effects 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000010008 shearing Methods 0.000 description 1
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Image Analysis (AREA)
- Image Processing (AREA)
Abstract
本发明提供了一种基于空间注意力的桥梁支座病害识别方法，包括以下步骤：获取桥梁支座图像数据，通过人工标注的方法赋予标签，标签包含正常支座及桥梁服役的过程中可能出现的各类支座病害。构建带有空间注意力机制的神经网络模型，其中空间注意力机制通过一个小型的神经网络生成4个注意力坐标值，根据这4个坐标值筛选出图像中有价值的区域，并通过格点生成函数和双线性插值方法放缩至指定的大小；把空间注意力机制的输出作为卷积神经网络的输入进行训练，得到具有预测支座病害的神经网络模型。本发明的注意力模型能够让网络模型自动的提取出桥梁支座图像中有价值的区域进行学习，相较于传统卷积神经网络模型，能有效提高支座病害的识别精度。
The present invention provides a method for identifying diseases of bridge supports based on spatial attention, which includes the following steps: acquiring image data of bridge supports, and assigning labels by manual labeling. The labels include normal supports and possible occurrences during service of the bridge. of various bearing diseases. Construct a neural network model with a spatial attention mechanism, in which the spatial attention mechanism generates 4 attention coordinate values through a small neural network, screen out the valuable areas in the image according to these 4 coordinate values, and pass the grid points The generating function and bilinear interpolation method are scaled to the specified size; the output of the spatial attention mechanism is used as the input of the convolutional neural network for training, and the neural network model with predicting bearing disease is obtained. The attention model of the present invention can allow the network model to automatically extract valuable areas in the bridge support image for learning, and compared with the traditional convolutional neural network model, the identification accuracy of the support disease can be effectively improved.
Description
Technical Field
The invention relates to a bridge bearing fault identification method based on space attention, and belongs to the technical field of civil engineering and artificial intelligence interaction.
Background
With the rapid development of infrastructure construction, the civil engineering industry develops rapidly, and after a large number of bridges are constructed, the bridge support is an important stressed component of the bridge, and diseases such as aging, cracking and the like can occur in the long-term service process, so that the normal use function of the bridge support is influenced. People always rely on means such as daily and periodic inspection, sampling and temporary inspection and the like to acquire relevant information of the structure. However, the existing bridge appearance detection mainly depends on manual detection, the method is low in efficiency, long in time consumption and high in cost, the manual detection method is greatly influenced by factors such as environment, professional technical literacy of detection personnel and the like, and the detection result is uncertain.
The image processing technology based on deep learning is rapidly developed and widely applied to various industries, however, a deep learning model usually needs a large amount of labeled data to be trained so as to achieve higher precision. In actual engineering, some disease image data are difficult to obtain, and the support is used as a connecting member of a bridge, so that other members and some background information are included in the image acquisition engineering, which is difficult to avoid, and the identification result of the model is influenced. Therefore, a method for improving the identification precision of the model under the conditions of limited data set and complex scene is urgently needed.
Disclosure of Invention
The invention aims to solve the technical problem of providing a bridge support fault identification method based on space attention, which is used for improving the training efficiency and the precision of a neural network for identifying bridge support faults.
The invention adopts the following technical scheme for solving the technical problems: the invention designs a bridge support fault identification method based on space attention, which is characterized in that a target neural network based on a space attention system for identifying bridge support faults is trained and obtained by applying the following steps A to E based on a plurality of sample images respectively only containing one bridge support; the target neural network comprises a space coordinate generation network and a classification network; then, sequentially applying the space coordinates to generate a network and a classification network aiming at a target image to be detected, so as to obtain a classification result of the bridge bearing contained in the target image;
a, respectively associating each sample image with a preset classification label type of a bridge support contained in the sample image to construct a bridge support sample image database;
b, defining a minimum rectangular area where a bridge support is located in a sample image as a spatial attention area, and constructing a spatial coordinate generation network by taking the sample image as input and four spatial coordinates of the spatial attention area in the sample image as output based on a bridge support sample image database, wherein the four spatial coordinates of the spatial attention area are xmin,ymin,xmax,ymaxWherein x ismin,yminThe abscissa and ordinate, x, of the upper left corner of the spatial attention areamax,ymaxThe horizontal coordinate and the vertical coordinate of the right lower corner point of the spatial attention area are shown;
step C, aiming at each sample image, according to four space coordinates of a space attention area in the sample image, applying a lattice point generating function to obtain coordinates of each lattice point under the corresponding preset proportion grid division in the space attention area in the sample image, and then entering the step D;
step D, aiming at each sample image, obtaining the pixel value of each lattice point in a space attention area in the sample image through a preset interpolation method, and obtaining a processed image which corresponds to each sample image and is based on an attention mechanism;
and E, constructing a classification network which takes the pictures of the sample images based on the attention mechanism as input and the classes of the support images in the corresponding sample images as output based on the labels in the bridge support sample image database and the pictures of the sample images based on the attention mechanism, taking the difference between the output classes of the classification network and the label classes in the step A as a target of network optimization, and performing end-to-end iterative training on the spatial coordinates to generate a network and the classification network to obtain a target neural network.
As a preferred technical scheme of the invention, in the step a, the different states of the bridge support included in the constructed bridge support sample image database include both a normal support state and states of various support diseases preset in the service process of the bridge.
In a preferred embodiment of the present invention, in step a, the resolution of the image in the constructed bridge support sample image database is above 800 × 600.
As a preferred technical solution of the present invention, the spatial coordinate generation network in step B includes a convolutional layer, a pooling layer, and a full connection layer.
As a preferred embodiment of the present invention, in step C, the lattice point generating function generates image lattice points according to a proportional relationship between 4 image coordinates generated by the attention mechanism and the original image, and the lattice point generating function is described as follows:
wherein, x'ijIs the corresponding abscissa, y 'of the grid point i, j in the original image'ijIs the corresponding ordinate of the grid point i, j in the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, xijIs the abscissa, y, of the grid point i, j in the preset grid point imageijIs the ordinate of the grid point i, j in the preset grid point image.
As a preferred embodiment of the present invention, in step D, the preset interpolation method is a differentiable interpolation method to meet the requirement of error back propagation.
As a preferred technical solution of the present invention, in step E, the classification network is a neural network that satisfies the preset specified feature characterization capability.
As a preferred technical solution of the present invention, the neural network with the preset designated feature characterization capability is VGG or ResNet.
As a preferred technical solution of the present invention, in step E, a gradient descent method is selected for iterative training of the target neural network.
Compared with the prior art, the bridge bearing disease identification method based on space attention has the following technical effects:
according to the method, under a limited data set, a target neural network consisting of a spatial attention mechanism is obtained through a combined training spatial coordinate generation network and a classification network, aiming at a bridge support image to be detected, an interested area in the bridge support image is firstly identified, and then whether the bridge support is damaged or not and the type of the damage are obtained through the classification network.
Drawings
FIG. 1 is a flow chart of the present invention;
FIG. 2 is a schematic diagram of a spatial coordinate generation network in accordance with the present invention;
FIG. 3 is a schematic flow chart of the method for obtaining attention area according to the present invention;
fig. 4 is a schematic diagram of a VGG-16 classification network used in the present invention.
Detailed Description
The following description will explain embodiments of the present invention in further detail with reference to the accompanying drawings.
The hardware conditions for training the convolutional neural network of the present embodiment are: the amazon AWS cloud computing service is enabled to configure an amazon EC2P2.xlarge instance, the instance is configured with 1 GPU, 4 vCPUs, 61GB random access memories, the system adopts an ubuntu system, the programming language adopts python, and the deep learning platform uses pytorch.
As shown in the flowchart of fig. 1, in the method for identifying a bridge bearer fault based on spatial attention implemented in this embodiment, 5000 sample images with a resolution of more than 800 × 600 are selected, each sample image respectively includes different states of the bridge bearer, including a normal bearer state and states of various bearer faults preset in the bridge service process, where the bridge bearer states in the sample images in this embodiment include three types of a normal bearer, a cracked bearer, and a shear deformation bearer.
Based on the sample image, the following steps A to E are applied, and a target neural network which is used for identifying the bridge support diseases and is based on a space attention mechanism is trained and obtained; the target neural network comprises a space coordinate generation network and a classification network.
And A, respectively associating each sample image with a preset classification label type of the bridge support contained in the sample image to construct a bridge support sample image database, wherein the classification label in the embodiment contains three types of bridge supports, namely a normal support, a cracking support and a shearing deformation support.
Step B, as shown in FIG. 2, defining a minimum rectangular area where a bridge support is located in a sample image as a spatial attention area, constructing a spatial coordinate generation network based on a bridge support sample image database, wherein the spatial coordinate generation network takes the sample image as input and four spatial coordinates of the spatial attention area in the sample image as output, and comprises a convolution layer, a pooling layer and a full-connection layer, wherein the four spatial coordinates of the spatial attention area are xmin,ymin,xmax,ymaxWherein x ismin,yminThe abscissa and ordinate, x, of the upper left corner of the spatial attention areamax,ymaxThe abscissa and the ordinate of the lower right corner of the spatial attention area.
And step C, as shown in FIG. 3, applying a grid point generating function according to the four spatial coordinates of the spatial attention area in the sample image respectively for each sample image, and obtaining the coordinates of each grid point under the corresponding preset proportion grid division in the spatial attention area in the sample image.
The lattice point generating function generates image lattice points according to the proportional relation between 4 image coordinates generated by the attention mechanism and the original image, and the lattice point generating function is described as follows:
wherein, x'ijIs the corresponding abscissa, y 'of the grid point i, j in the original image'ijIs the corresponding ordinate of the grid point i, j in the original image, w 'is the width of the preset grid point image, h' is the height of the corresponding preset grid point image, xijIs the abscissa, y, of the grid point i, j in the preset grid point imageijIs the ordinate of the grid point i, j in the preset grid point image.
And D, aiming at each sample image, obtaining the pixel value of each lattice point in a spatial attention area in the sample image by a differentiable interpolation method meeting the requirement of error back propagation, wherein a bilinear interpolation method is specifically selected in the embodiment, and obtaining a processed image corresponding to each sample image based on an attention mechanism.
Step e, as shown in fig. 4, based on the tags in the bridge support sample image database and the attention-based pictures of each sample image, constructing a classification network that uses the attention-based pictures of each sample image as input and uses the category of the support image in the corresponding sample image as output and satisfies the preset specified feature characterization capability, specifically selecting a VGG network in a classical convolutional neural network in the present embodiment, then using the difference between the output category of the classification network and the category marked in step a as a target of network optimization, and performing joint training on the space coordinate generation network and the classification network by using a gradient descent method to obtain a target neural network.
And then, sequentially applying the space coordinates to generate a network and a classification network aiming at the target image to be detected, so as to obtain a classification result of the bridge bearing included in the target image.
The embodiments of the present invention have been described in detail with reference to the drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the gist of the present invention.
Claims (9)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011442501.7A CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011442501.7A CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112699736A true CN112699736A (en) | 2021-04-23 |
| CN112699736B CN112699736B (en) | 2024-06-07 |
Family
ID=75507773
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011442501.7A Active CN112699736B (en) | 2020-12-08 | 2020-12-08 | Bridge bearing disease identification method based on spatial attention |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112699736B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297959A (en) * | 2021-05-24 | 2021-08-24 | 南京邮电大学 | Target tracking method and system based on corner attention twin network |
| CN118820723A (en) * | 2024-08-01 | 2024-10-22 | 云南省公路科学技术研究院 | Tunnel lining disease detection system and method based on deep learning |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108288269A (en) * | 2018-01-24 | 2018-07-17 | 东南大学 | Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks |
| CN108363961A (en) * | 2018-01-24 | 2018-08-03 | 东南大学 | Bridge pad disease recognition method based on transfer learning between convolutional neural networks |
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN110070008A (en) * | 2019-04-04 | 2019-07-30 | 中设设计集团股份有限公司 | Bridge disease identification method adopting unmanned aerial vehicle image |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN111562220A (en) * | 2020-06-02 | 2020-08-21 | 吉林大学 | Rapid and intelligent detection method for bridge diseases |
| WO2020199538A1 (en) * | 2019-04-04 | 2020-10-08 | 中设设计集团股份有限公司 | Bridge key component disease early-warning system and method based on image monitoring data |
-
2020
- 2020-12-08 CN CN202011442501.7A patent/CN112699736B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018214195A1 (en) * | 2017-05-25 | 2018-11-29 | 中国矿业大学 | Remote sensing imaging bridge detection method based on convolutional neural network |
| CN108288269A (en) * | 2018-01-24 | 2018-07-17 | 东南大学 | Bridge pad disease automatic identifying method based on unmanned plane and convolutional neural networks |
| CN108363961A (en) * | 2018-01-24 | 2018-08-03 | 东南大学 | Bridge pad disease recognition method based on transfer learning between convolutional neural networks |
| CN109784347A (en) * | 2018-12-17 | 2019-05-21 | 西北工业大学 | Image classification method based on multiple dimensioned dense convolutional neural networks and spectrum attention mechanism |
| CN110070008A (en) * | 2019-04-04 | 2019-07-30 | 中设设计集团股份有限公司 | Bridge disease identification method adopting unmanned aerial vehicle image |
| WO2020199538A1 (en) * | 2019-04-04 | 2020-10-08 | 中设设计集团股份有限公司 | Bridge key component disease early-warning system and method based on image monitoring data |
| CN110222701A (en) * | 2019-06-11 | 2019-09-10 | 北京新桥技术发展有限公司 | A kind of bridge defect automatic identifying method |
| CN111562220A (en) * | 2020-06-02 | 2020-08-21 | 吉林大学 | Rapid and intelligent detection method for bridge diseases |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113297959A (en) * | 2021-05-24 | 2021-08-24 | 南京邮电大学 | Target tracking method and system based on corner attention twin network |
| CN118820723A (en) * | 2024-08-01 | 2024-10-22 | 云南省公路科学技术研究院 | Tunnel lining disease detection system and method based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112699736B (en) | 2024-06-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Karaaslan et al. | Artificial intelligence assisted infrastructure assessment using mixed reality systems | |
| Yin et al. | Automated semantic segmentation of industrial point clouds using ResPointNet++ | |
| CN111476781B (en) | Concrete crack identification method and device based on video semantic segmentation technology | |
| Xu et al. | Recognition of rust grade and rust ratio of steel structures based on ensembled convolutional neural network | |
| CN105825511A (en) | Image background definition detection method based on deep learning | |
| WO2020238256A1 (en) | Weak segmentation-based damage detection method and device | |
| CN109886947A (en) | High-voltage wire defect detection method based on region convolutional neural network | |
| CN113177456B (en) | Remote sensing target detection method based on single-stage full convolution network and multi-feature fusion | |
| TWI672637B (en) | Patern recognition method of autoantibody immunofluorescence image | |
| CN112699736A (en) | Bridge bearing fault identification method based on space attention | |
| CN112488990B (en) | Bridge bearing disease identification method based on attention regularization mechanism | |
| Shi et al. | Development of large-scale synthetic 3D point cloud datasets for vision-based bridge structural condition assessment | |
| Wang et al. | Instance segmentation of soft‐story buildings from street‐view images with semiautomatic annotation | |
| Huang et al. | Deep residual network training for reinforced concrete defects intelligent classifier | |
| CN112767351A (en) | Transformer equipment defect detection method based on sensitive position dependence analysis | |
| Shinozaki et al. | Detection of deterioration of furnace walls using large-scale point-clouds | |
| Chu et al. | A rendering‐based lightweight network for segmentation of high‐resolution crack images | |
| CN110660068A (en) | Semi-automatic brain region segmentation method for three-dimensional cell construction image | |
| CN116777865A (en) | Underwater crack identification method, system, device and storage medium | |
| CN115564954A (en) | Chromosome image segmentation method | |
| CN114612420A (en) | A method for identifying visible light defects in transmission lines based on YOLO algorithm | |
| Kang et al. | Automatic detection and classification process for concrete defects in deteriorating buildings based on image data | |
| Ngo et al. | Designing image processing tools for testing concrete bridges by a drone based on deep learning | |
| Bao et al. | Research on the detection and analysis of students’ classroom behavioral features based on deep cnns | |
| Mykolaichuk et al. | Image Segmentation in Loss Detection for Conservation and Restoration of Old Paintings |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province Applicant after: Jiangxi Academy of Transportation Sciences Co.,Ltd. Applicant after: SOUTHEAST University Address before: 330200 no.809 Jinsha Avenue, Xiaolan economic and Technological Development Zone, Nanchang City, Jiangxi Province Applicant before: Jiangxi Transportation Research Institute Applicant before: SOUTHEAST University |
|
| GR01 | Patent grant |