CN112329771B - Deep learning-based building material sample identification method - Google Patents

Abstract

The invention provides a building material sample recognition method based on deep learning, which comprises a model training stage and a sample recognition stage, wherein the model training stage comprises the steps of manufacturing a building material sample data set and constructing a multi-scale information fusion convolutional neural network for sample recognition; enhancing the data of the sample data set to obtain the best model performance; the sample identification stage comprises the steps of inputting processed building material sample images into a model, carrying out feature extraction to generate an optimal size feature map, correcting the generated feature map to be used as an ROI, transmitting the ROI into ROI pooling layers according to different scales, mapping the ROI into the same size proposal, projecting the image to generate proposal feature map on an original building material sample image, carrying out BBOX and CLS branch processing and other steps to generate a building material detection frame with accurate positions, and identifying the material performance state of the sample. The method carries out information extraction through the multi-scale feature map, well learns target feature information with different scales, has good identification performance and universality, and has wide application prospect in the field of construction engineering.

Description

Deep learning-based building material sample identification method

Technical Field

The invention belongs to the field of construction engineering, and particularly relates to a building material sample identification method based on deep learning.

Background

With the increase of data information processing demands and the rapid development of artificial intelligence technology, people begin to try to identify building material samples by using a method based on machine learning or deep learning, such as a clustering neural network, a support vector machine, a wavelet transformation neural network and other shallow network algorithms. However, the shallow network algorithm needs various complex algorithms to extract and determine sample identification characteristic information from echo information; the computational complexity and the consumption of computational resources are high and therefore the versatility is low. A convolutional neural network is one of important models in the field of deep learning, and its network structure has high invariance to image data having characteristics of translation, inversion, affine transformation, and the like, and therefore, in recent years, the convolutional neural network has been widely used in various fields in computer vision and has achieved excellent results. However, in the identification process of the traditional single linear convolutional neural network, only the last layer of output is aimed at, namely: the single-scale feature map is used for information extraction, so that target feature information with different scales cannot be well learned obviously; it is therefore difficult to achieve good recognition performance in complex building material scenarios.

Disclosure of Invention

In order to solve the problems in the prior art, the invention provides a building material sample identification method based on deep learning.

The aim of the invention is achieved by the following technical scheme:

a building material sample recognition method based on deep learning, which comprises a model training stage and a sample recognition stage,

The model training phase comprises the following steps:

S1, collecting and marking building material samples, and manufacturing a building material sample data set; the data set comprises building material samples of each category and is divided into a training set, a testing set and an evaluation set;

s2, constructing a multi-scale information fusion convolutional neural network for building material sample identification;

S3, carrying out data enhancement on the sample data set in the S1: to obtain optimal model performance;

The sample identification phase comprises the following steps:

s1, inputting the processed building material sample image into a model, pre-training through imagenet, removing Resnet on the top layer, extracting features, and generating a feature map with the optimal size;

S2, the feature images generated in the S1 are respectively processed by RPN2,3,4 and 5to generate candidate anchors with different sizes; the anchor area is set, and all the anchors generated by the RPN uniformly adopt 1: 1. 1: 2. 2:1, generating a plurality of candidate anchors, and then screening out the most complete anchor containing the target by utilizing a two-class function and bounding box regerssion function according to the RPN and real labeling, and correcting the most complete anchor to be used as the ROI;

S3, using the feature image layers output by different residual convolution modules as the input of ROI pooling layers, outputting the feature image by a deep convolution module by using the large-scale ROI, and using the ROI leave as a criterion output by a corresponding layer convolution module:

Wherein w and h are the length and width of the ROI area, and K ₀ is the reference leave;

S4, transmitting the ROI generated in the S3 into ROI pooling layers, uniformly mapping the multi-scale ROI into proposal with the same size by ROI pooling, and projecting the multi-scale ROI on an original building material sample image to generate proposal feature map, so that subsequent BBOX and CLS branches are convenient to process;

S5, calculating the category to which each sample belongs through the full connection layer and softmax, and outputting the highest category probability as a confidence coefficient by the CLS branch pair proposal feature map;

And S6, BBOX, correcting the proposal area by utilizing the bounding box regression function, generating a building material detection frame with more accurate positions, and identifying the material performance state of the building material sample.

Preferably, the model training phase S3 comprises the steps of:

s31, building material sample images under different scales and scenes are constructed by utilizing a combination of a plurality of data enhancement methods, existing data are expanded to simulate a complex identification scene, the learning of the model on detail characteristic information is improved, and the universality of the model is enhanced;

s32, setting initial weight as pre-training weight on imagenet, setting initial learning rate as 0.001, learning rate attenuation index as 0.1, batch_size as 16, and inputting image size;

s33, on a loss function, the RPN series module adopts two kinds of loss and regression loss; the CLS branch adopts multi-classification loss, and the BBOX branch adopts regression loss;

s34, training on a training set and a testing set by adopting an SGD random gradient descent optimizer until the model performance reaches the best.

Preferably, the method for enhancing data in S31 includes:

S311, random Erasing algorithm:

(3) IRE (Image-aware Random Erasing), randomly selecting an occlusion position on the whole target Image;

(4) ORE (Object-aware Random Erasing), randomly selecting an occlusion position within the Object's bounding-box region;

(3) Combining both IRE and ORE;

s312, HIDE AND SEEK algorithm:

Dividing the picture into S-S grids, hiding each grid according to probability, hiding different grid groups for each batch of the same picture in training;

S313, grid Mask algorithm:

in order to avoid the problem that the complete target is deleted or the context information is deleted because the over-deleted area exists in the step S311 and the step S312; setting four parameters of x, y, r and d through a Grid Mask:

Wherein r is the size of mask, M is the reserved pixel number, H, W is the image size; x and y are the area coordinates randomly generated on the image; the value of the non-shielding area is 1, the value of the shielding area is 0, a mask with the same resolution as the original image is generated, and then the mask is multiplied with the original image to obtain an image;

s314, mixup algorithm

The method comprises the steps of carrying out mixed enhancement on images, and mixing the images among different classes; the algorithm can be summarized as follows:

Where x ₁、x₂ is the pixel of the different image and λ is the weight; Is the output pixel after the mixed class;

s315, cutmix algorithm

A portion of the region is randomly cropped and region pixel values of other data in the training set are randomly populated.

Preferably, the step of generating the best feature map after the sample is extracted in the sample identifying stage S1 includes:

S11, marking the last plurality of residual convolution modules in Resnet as { C ₁,C₂,C₃,…C_n }, and respectively extracting output characteristic graphs of the residual modules as { P ₁,P₂,P₃,…,P_n };

S12, performing 2 times nearest neighbor up-sampling on the deepest feature map P ₅;

S13, extracting an output characteristic diagram P _n-1 of a residual convolution module C _n-1 adjacent to the C _n, and performing 1*1 convolution dimension reduction processing;

S14, fusing the feature map P _n by a pixel value adding method of the corresponding part;

S15, reducing an aliasing effect caused by up-sampling through 3*3 convolution of the fusion feature map;

S16, iterating the processes of S11 to S15 until the optimal size characteristic diagram is generated.

The beneficial effects of the invention are as follows: the method of the invention extracts information through the multi-scale feature map, well learns the target feature information with different scales, has good identification performance and universality, and has wide application prospect in the field of construction engineering.

Detailed Description

The technical proposal of the invention is specifically described below by combining the embodiment, the invention discloses a building material sample recognition method based on deep learning, which comprises a model training stage and a sample recognition stage,

The model training phase comprises the following steps:

S1, collecting and marking building material samples, and manufacturing a building material sample data set; the data set comprises building material samples of each category and is divided into a training set, a testing set and an evaluation set;

s2, constructing a multi-scale information fusion convolutional neural network for building material sample identification;

S3, carrying out data enhancement on the sample data set in the S1 to obtain the optimal model performance;

in particular, the method comprises the steps of,

S31, building material sample images under different scales and scenes are constructed by utilizing a combination of a plurality of data enhancement methods, existing data are expanded to simulate a complex identification scene, the learning of the model on detail characteristic information is improved, and the universality of the model is enhanced;

S32, setting initial weight as pre-training weight on imagenet, setting initial learning rate to be 0.001, learning rate attenuation index to be 0.1, batch_size to be 16, and input image size to be 224 x 224.

S33, on a loss function, the RPN series module adopts two kinds of loss and regression loss; the CLS branch uses multiple classification losses and the BBOX branch uses regression losses.

And S34, training 20 epochs on the training set and the testing set by adopting an SGD random gradient descent optimizer in the training until the performance of the model is optimal.

Wherein, the enhancing method in S31 includes the following steps:

S311, random Erasing algorithm:

(5) IRE, randomly selecting a shielding position on the whole target image;

(6) ORE, randomly selecting a shielding position in a binding-box area of the target;

(3) Combining both IRE and ORE;

s312, HIDE AND SEEK algorithm:

the picture is segmented into S-S grids, each grid is hidden by adopting a certain probability, the same picture is hidden in training, and different grid groups are hidden in each batch;

S313, grid Mask algorithm:

in order to avoid the problem that the complete target is deleted or the context information is deleted because the excessive deleting area exists in the step S31 and the step S32; setting four parameters of x, y, r and d through a Grid Mask:

Wherein r is the size of mask, M is the reserved pixel number, H, W is the image size; x and y are the area coordinates randomly generated on the image; the value of the non-shielding area is 1, the value of the shielding area is 0, a mask with the same resolution as the original image is generated, and then the mask is multiplied with the original image to obtain an image;

s314, mixup algorithm

The method comprises the steps of carrying out mixed enhancement on images, and mixing the images among different classes; the algorithm can be summarized as follows:

Where x ₁、x₂ is the pixel of the different image, lambda is the weight, Is the output pixel after the mixed class;

s315, cutmix algorithm

A portion of the region is randomly cropped and region pixel values of other data in the training set are randomly populated.

The sample identification phase comprises the following steps:

s1, inputting the processed building material sample image into a model, and extracting features through Resnet of pre-training and top layer removal on imagenet.

S2, marking the last 5 residual convolution modules in Resnet as { C ₁,C₂,C₃,C₄,C₅ }, and respectively extracting output characteristic graphs of the 5 residual modules as { P ₁,P₂,P₃,P₄,P₅ }; generating a feature map of an optimal size;

the step of generating the feature map comprises the following steps:

S21, performing 2 times nearest neighbor up-sampling on the deepest feature map P ₅;

S22, extracting an output characteristic diagram P ₄ of a residual convolution module C4 adjacent to the C5, and performing convolution dimension reduction processing of 1*1;

S23, fusing the pixel values of the corresponding parts with the feature map P ₅ by a pixel value adding method of the corresponding parts;

s24, reducing an aliasing effect brought by up-sampling through 3*3 convolution of the fusion feature map;

s25, iterating the processes of S11 to S15 until a feature map with the optimal size is generated;

S3, generating candidate anchors with different sizes through RPN2,3,4 and 5 respectively; anchor areas are respectively set to 32 x 32, 64 x 64, 128 x 128, 256 x 256, and all the RPNs are generated by uniformly adopting 1: 1. 1: 2. 2:1, generating a plurality of candidate anchors, and then screening out the most complete anchor containing the target by utilizing a two-class function and bounding box regerssion function according to the RPN and real labeling, and correcting the most complete anchor to be used as the ROI;

S4, using the feature image layers output by different residual convolution modules as the input of ROI pooling layers, outputting the feature image by a deep convolution module by using the large-scale ROI, and using the ROI leave as a criterion output by a certain layer convolution module: Wherein w and h are the length and width of the ROI region, K ₀ is the reference leave, and the small-scale ROI is set to be 5 by using the output characteristic diagram of the depth shallow convolution module, and the size of the representative characteristic diagram P ₅ is set.

S5, transmitting the ROI generated in the S4 into ROI pooling layers, uniformly mapping the multi-scale ROI into proposal with the size of 7*7 by ROI pooling, and projecting the multi-scale ROI on an original building material sample image to generate proposal feature map, so that subsequent BBOX and CLS branches are convenient to process;

S6, calculating the category to which each sample belongs through the full connection layer and softmax, and outputting the highest category probability as a confidence coefficient by the CLS branch pair proposal feature map;

And S7, BBOX, correcting the proposal area by utilizing the bounding box regression function, generating a building material detection frame with more accurate positions, and identifying the material performance state of the building material sample.

There are, of course, many specific embodiments of the invention, not set forth herein. All technical solutions formed by equivalent substitution or equivalent transformation fall within the scope of the invention claimed.

Claims (2)

1. A building material sample identification method based on deep learning is characterized in that: comprises a model training stage and a sample recognition stage,

The model training phase comprises the following steps:

S1, collecting and marking building material samples, and manufacturing a building material sample data set; the data set comprises building material samples of each category and is divided into a training set, a testing set and an evaluation set;

s2, constructing a multi-scale information fusion convolutional neural network for building material sample identification;

S3, carrying out data enhancement on the sample data set in the S1: to obtain optimal model performance;

The sample identification phase comprises the following steps:

s1, inputting the processed building material sample image into a model, pre-training through imagenet, removing Resnet on the top layer, extracting features, and generating a feature map with the optimal size;

S2, the feature images generated in the S1 are respectively processed by RPN2,3,4 and 5to generate candidate anchors with different sizes; the anchor area is set, and all the anchors generated by the RPN uniformly adopt 1: 1. 1: 2. 2:1, generating a plurality of candidate anchors, and then screening out the most complete anchor containing the target by utilizing a two-class function and bounding box regerssion function according to the RPN and real labeling, and correcting the most complete anchor to be used as the ROI;

S3, using the feature image layers output by different residual convolution modules as the input of ROI pooling layers, outputting the feature image by a deep convolution module by using the large-scale ROI, and using the ROI leave as a criterion output by a corresponding layer convolution module:

Wherein w and h are the length and width of the ROI area, and K ₀ is the reference leave;

S4, transmitting the ROI generated in the S3 into ROI pooling layers, uniformly mapping the multi-scale ROI into proposal with the same size by ROI pooling, and projecting the multi-scale ROI on an original building material sample image to generate proposal feature map, so that subsequent BBOX and CLS branches are convenient to process;

S5, calculating the category to which each sample belongs through the full connection layer and softmax, and outputting the highest category probability as a confidence coefficient by the CLS branch pair proposal feature map;

s6, BBOX, correcting a proposal area by utilizing a bounding box regression function, generating a building material detection frame with more accurate positions, and identifying the material performance state of a building material sample;

The model training stage S3 comprises the steps of:

s31, building material sample images under different scales and scenes are constructed by utilizing a combination of a plurality of data enhancement methods, existing data are expanded to simulate a complex identification scene, the learning of the model on detail characteristic information is improved, and the universality of the model is enhanced;

s32, setting initial weight as pre-training weight on imagenet, setting initial learning rate as 0.001, learning rate attenuation index as 0.1, batch_size as 16, and inputting image size;

s33, on a loss function, the RPN series module adopts two kinds of loss and regression loss; the CLS branch adopts multi-classification loss, and the BBOX branch adopts regression loss;

s34, training on a training set and a testing set by adopting an SGD random gradient descent optimizer in training until the performance of the model is optimal;

The step of generating the optimal feature map after the sample is extracted in the sample recognition stage S1 comprises the following steps:

S11, marking the last plurality of residual convolution modules in Resnet as { C ₁,C₂,C₃,…C_n }, and respectively extracting output characteristic graphs of the residual modules as { P ₁,P₂,P₃,…,P_n };

S12, performing 2 times nearest neighbor up-sampling on the deepest feature map P ₅;

S13, extracting an output characteristic diagram P _n-1 of a residual convolution module C _n-1 adjacent to the C _n, and performing 1*1 convolution dimension reduction processing;

S14, fusing the feature map P _n by a pixel value adding method of the corresponding part;

S15, reducing an aliasing effect caused by up-sampling through 3*3 convolution of the fusion feature map;

S16, iterating the processes of S11 to S15 until the optimal size characteristic diagram is generated.

2. A method for identifying a sample of building material based on deep learning as claimed in claim 1, wherein: the method for enhancing the data in the S31 comprises the following steps:

S311, random Erasing algorithm:

(1) IRE, randomly selecting a shielding position on the whole target image;

(2) ORE, randomly selecting a shielding position in a binding-box area of the target;

(3) Combining both IRE and ORE;

s312, HIDE AND SEEK algorithm:

Dividing the picture into S-S grids, hiding each grid according to probability, hiding different grid groups for each batch of the same picture in training;

S313, grid Mask algorithm:

To avoid the problem that the complete target is deleted or the context information is lost due to the existence of the over-deleted region in S311 and S312; setting four parameters of x, y, r and d through a Grid Mask:

Wherein r is the size of mask, M is the reserved pixel number, H, W is the image size; x and y are the area coordinates randomly generated on the image; the value of the non-shielding area is 1, the value of the shielding area is 0, a mask with the same resolution as the original image is generated, and then the mask is multiplied with the original image to obtain an image;

s314, mixup algorithm

The method comprises the steps of carrying out mixed enhancement on images, and mixing the images among different classes; the algorithm is as follows:

Where x ₁、x₂ is the pixel of the different image and λ is the weight; Is the output pixel after the mixed class;

s315, cutmix algorithm

A portion of the region is randomly cropped and region pixel values of other data in the training set are randomly populated.