CN112633327A - Staged metal surface defect detection method, system, medium, equipment and application - Google Patents
Staged metal surface defect detection method, system, medium, equipment and application Download PDFInfo
- Publication number
- CN112633327A CN112633327A CN202011395512.4A CN202011395512A CN112633327A CN 112633327 A CN112633327 A CN 112633327A CN 202011395512 A CN202011395512 A CN 202011395512A CN 112633327 A CN112633327 A CN 112633327A
- Authority
- CN
- China
- Prior art keywords
- defect
- layer
- areas
- training
- feature
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Biophysics (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Health & Medical Sciences (AREA)
- Quality & Reliability (AREA)
- Image Processing (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of metal defect detection, and discloses a method, a system, a medium, equipment and application for detecting a defect on a metal surface in stages, wherein the characteristics of a kth-1 layer and a kth layer of an image are extracted by utilizing a VGG pre-training model; taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T; taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set; and sending the feature vectors into the SVM, and training to obtain the classifier. The method avoids the difficulties of non-robust and time-consuming manual design characteristics; meanwhile, the method utilizes deep abundant semantic features to perform coarse positioning, utilizes shallow abundant position features to perform fine positioning, and improves the detection efficiency under the condition of ensuring the precision.
Description
Technical Field
The invention belongs to the technical field of metal defect detection, and particularly relates to a method, a system, a medium, equipment and application for detecting a defect on a metal surface in stages.
Background
At present: china is a large steel production country, and the quality of China is more and more valued by people while the demand is gradually expanded. Due to the influence of factors such as production equipment, steel plate incoming materials, processing technology, manual technology, external environment and the like, the surface of the steel plate is easy to have defects, the appearance, corrosion resistance, abrasion resistance, fatigue limit and other properties of the plate are influenced, and even the normal use of the plate is influenced. The steel plate is mainly characterized in that the steel plate is pressed into foreign matters, which are generally divided into metal inclusions, non-metal inclusions and mixed inclusions, and the steel plate has the following defect type (1) surface inclusions, and is characterized in that the metal surface has punctiform, flaky or strip tissues, and the position and the color have no specific rules. (2) The bubbles and the round bulges with different sizes on the metal surface can leave irregular gaps after cracking, thus influencing the appearance. (3) Cracks, cracks with different depths, lengths, widths and shapes are formed on the surface of the metal, serious decarburization phenomena and partial inclusions are formed at the edges, the physical properties of the metal are seriously damaged, and even serious safety accidents can be caused.
In the early stage of industrial production, the requirements on products are not very high, and a manual visual inspection method is generally adopted. The manual spot inspection method has been gradually replaced because the detection speed and detection range are very limited, the subjective influence of people is large, and no determined standard exists. With the gradual development of machine vision, vision-based solutions are increasing, and mainly divided into (1) a traditional image method based on artificial features, which represents a detection method of hog (histogram of oriented graphics) features + svm (support vector machine) proposed by Dalal in 2005, and the method is originally widely applied to pedestrian detection. Later, researchers put forward improved HOG characteristics and other artificial characteristics such as LBP texture characteristics and the like, so that the algorithm can adjust a corresponding characteristic extraction method according to a scene, and the generalization capability is enhanced to a certain extent. Considerable effort is still required by researchers to find suitable ways of feature extraction. (2) Compared with a computer vision algorithm for manually extracting the feature descriptors, the target detection algorithm based on the convolutional neural network has better effect without any doubt. Such as the relatively mature single-stage target detection network YOLOv3, the two-stage target detection network fast-rcnn, and so forth. However, the target detection network based on the convolutional neural network generally needs a large amount of training data, and an overfitting phenomenon easily occurs under a small data set. However, defects in an industrial scene often have low probability and high acquisition cost, and a large amount of defect data is difficult to collect.
Through the above analysis, the problems and defects of the prior art are as follows:
(1) the existing manual sampling inspection mode has low efficiency and is easily influenced by subjective factors of people to cause false inspection.
(2) The traditional design feature mode is too limited, is often related to a specific scene, requires a great deal of effort of researchers to debug and continuously optimize, and has weak generalization capability, so that a great amount of false detection is caused.
(3) The existing deep learning target detection method needs a large amount of sample data, is based on the current data training, causes an overfitting phenomenon, has poor performance on a verification data set, and has high hardware cost and low operation efficiency of deep learning.
The difficulty in solving the above problems and defects is: how to select a proper feature extraction mode and how to improve the detection efficiency.
The significance of solving the problems and the defects is as follows: the metal defect detection can greatly improve the yield of steel products or other metal original products, save a large amount of labor cost and promote the progress of the metal manufacturing field.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method, a system, a medium, equipment and application for detecting the defects of a staged metal surface.
The invention is realized in such a way that a staged metal surface defect detection method comprises the following steps:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
Further, the step of taking out the defect regions in the kth layer of features of all the training samples, and the step of determining the minimum feature value as the threshold T specifically includes: extracting the depth feature of the k-1 layer of each image in the data set by using a VGG network and recording the depth feature as Fk-1And the depth of the k-th layer is characterized by Fk(ii) a Only the feature extraction part is reserved by adopting a VGG16 network structure; the whole network consists of 13 convolutional layers and 5 pooling layers, and each time the size of a feature map of one pooling layer is reduced to 1/4 of the upper layer, the deeper the layer number, the richer the semantic features, and the shallower the layer number, the richer the position information; features were extracted using the VGG pre-training model from Imagenet, let k be 3.
Further, Fk-1And FkAll are three-dimensional matrixes, and the sum of each channel is used for obtaining a two-dimensional matrix which is respectively recorded as F'k-1And F'k(ii) a The conversion formula is as follows:
wherein, N is the number of the characteristic channels of the k-1 th layer, M is the number of the characteristic channels of the k-1 th layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively.
Further, defect areas in k-th layer features of all training samples are taken out, and the minimum feature value is determined as a threshold value T in F'kExtracting local characteristic graphs of all defect regions in the training set, and obtaining a minimum characteristic value in all the local characteristic graphs, namely a threshold value T;
1) the coordinates of the defect at the upper left corner and the lower right corner in the original drawing are represented by [ (x1, y1), (x2, y2) ], and the defect position in the feature map is represented by [ (x1', y1'), (x2', y2') ], and the correspondence relationship is as follows:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of selected feature layers, [ ] represents rounding down.
2) And storing the minimum value of each defect position in the feature map into a set S, and recording the minimum value in the final set S as a threshold value T.
Further, the step of taking out defect regions in the (k-1) th layer features of all training samples by using an n × n sliding window, expanding the defect regions into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background regions with the number equivalent to that of the positive samples, expanding the background regions into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set specifically includes: in F'k-1Extracting local characteristic graphs of all defect areas in the training set to form a positive sample, extracting local characteristic graphs of partial background areas to form a negative sample, and sending the negative sample to the SVM to obtain a detector;
1) selecting n x n sliding window s1, sliding on the original image, wherein the sliding step length is n/2, and if s1 and the IOU of the defect position are greater than 0.7, the sliding window is considered as a normal sample; if the IOU value is less than 0.3, the sliding window is considered negative. The two regions IOU are defined as follows:
2) find out that sample is F'k-1The feature diagram is expanded into a one-dimensional vector according to rows, samples with the same number as that of the positive sample set are randomly sampled from the negative sample set to form a new negative sample set, and the new negative sample set and the new positive sample set are sent to an SVM (support vector machine) for training to obtain a classifier model;
3) parameter setting of the trainer:
kernel function kernel: linear kernel
Penalty term C: 1.7
Size of the sliding window: 32
Sliding step length: 16.
further, the staged metal surface defect detection method further comprises the following steps: in the testing stage, the depth feature of the k-1 layer of each image in the data set is extracted and recorded as F by repeatedly utilizing the VGG networkk-1And the depth of the k-th layer is characterized by FkAnd Fk-1And FkAre three-dimensional matrixes, and the sum of each channel obtains the characteristic F 'of the k-1 th layer of the two-dimensional matrix'k-1And feature F 'of the k-th layer'k(ii) a Defining a two-dimensional matrix with the same size as the kth layer as D, the matrix D reflecting the salient region of the sample:
d's salient region is mapped to a region D ' on the k-1 layer feature, in a feature map F 'k-1In the D' area, extracting features by using a sliding window of n x n, expanding the features into a one-dimensional vector, sending the one-dimensional vector into a trained SVM discriminator, and judging whether the sliding window has defects or not; and marking all the sliding windows with defects, and combining the areas with excessive overlapping areas to finish the metal defect detection task.
It is a further object of the invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
It is another object of the present invention to provide a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
Another objective of the present invention is to provide a metal defect detection information data processing terminal, which is used for implementing the staged metal surface defect detection method.
Another object of the present invention is to provide a staged metal surface defect detecting system for implementing the staged metal surface defect detecting method, the staged metal surface defect detecting system including:
the image feature extraction module is used for extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
the defect region extraction module is used for extracting defect regions in the kth layer of features of all the training samples, and the minimum feature value of the defect regions is set as a threshold value T;
the sample set forming module is used for taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows and recording the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas with the number equivalent to that of the positive samples and expanding the background areas into one-dimensional vectors according to rows and recording the one-dimensional vectors as a negative sample set;
and the classifier obtaining module is used for sending the feature vectors into the SVM and training to obtain the classifier.
By combining all the technical schemes, the invention has the advantages and positive effects that: the invention discloses a staged metal surface defect detection method based on VGG depth characteristics and SVM, which is used for solving the problem of difficulty in characteristic selection in the existing defect detection method. According to the invention, the VGG pre-training model is adopted to extract the depth characteristics of the target, so that the difficulties of non-robustness and time consumption of manual design characteristics are avoided; meanwhile, the method utilizes deep abundant semantic features to perform coarse positioning, utilizes shallow abundant position features to perform fine positioning, and improves the detection efficiency under the condition of ensuring the precision. An SVM algorithm introduced into machine learning is used for classifying input features, and the robustness is higher than that of a neural network under the condition of a small sample.
Compared with the prior art, the invention has the following advantages:
(1) the invention introduces the VGG network to extract the target depth feature without manually designing the feature.
(2) The method adopts the SVM based on the structure risk minimization theory as the defect classifier, accurately classifies the input feature vectors, improves the detection precision of the model, and reduces the overfitting risk.
(3) The invention utilizes the staged detection method of deep-layer characteristic coarse positioning and shallow-layer characteristic fine positioning to improve the detection efficiency.
Table 1 comparison of the process of the invention with the prior art:
drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments of the present application will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained from the drawings without creative efforts.
FIG. 1 is a flow chart of a staged metal surface defect detection method according to an embodiment of the present invention.
FIG. 2 is a schematic structural diagram of a staged metal surface defect detection system provided in an embodiment of the present invention;
in fig. 2: 1. an image feature extraction module; 2. a defective region extraction module; 3. a sample set forming module; 4. and a classifier obtaining module.
Fig. 3 is a flowchart of an implementation of the staged metal surface defect detection method according to the embodiment of the present invention.
Fig. 4 is a schematic diagram of a VGG feature extraction structure provided in the embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
In view of the problems in the prior art, the present invention provides a method, a system, a medium, a device and an application for detecting defects on a metal surface in stages, and the present invention is described in detail below with reference to the accompanying drawings.
As shown in FIG. 1, the staged metal surface defect detection method provided by the invention comprises the following steps:
s101: extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
s102: taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
s103: taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
s104: and sending the feature vectors into the SVM, and training to obtain the classifier.
One skilled in the art can also use other steps to implement the staged metal surface defect detection method provided by the present invention, and the staged metal surface defect detection method provided by the present invention in fig. 1 is only one specific example.
As shown in fig. 2, the staged metal surface defect detecting system provided by the present invention comprises:
the image feature extraction module 1 is used for extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
the defect region extraction module 2 is used for extracting defect regions in the kth layer of features of all the training samples, and setting the minimum feature value of the defect regions as a threshold value T;
the sample set forming module 3 is used for taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows and recording the vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows and recording the vectors as a negative sample set;
and the classifier obtaining module 4 is used for sending the feature vectors into the SVM and training to obtain the classifier.
The technical solution of the present invention is further described below with reference to the accompanying drawings.
As shown in fig. 3, the staged metal surface defect detection method provided by the embodiment of the present invention includes the following steps: the method specifically comprises the following steps:
step one, acquiring a certain amount of metal image data with surface defects.
Step two, extracting the depth feature of the k-1 layer of each image in the data set by using a VGG network and recording the depth feature as Fk-1And the depth of the k-th layer is characterized by Fk(ii) a With the VGG16 network structure, only the feature extraction part is retained, as shown in fig. 4. The whole network consists of 13 convolutional layers and 5 pooling layers, and each time the size of a feature map of one pooling layer is reduced to 1/4 of the upper layer, the deeper the layer number, the richer the semantic features, and the shallower the layer number, the richer the position information. The present invention extracts features using the VGG pre-training model from Imagenet, let k be 3.
Step three, Fk-1And FkAll are three-dimensional matrixes, and the sum of each channel is used for obtaining a two-dimensional matrix which is respectively recorded as F'k-1And F'k(ii) a The conversion formula is as follows:
wherein, N is the number of the characteristic channels of the k-1 th layer, M is the number of the characteristic channels of the k-1 th layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively.
Step four, in F'kExtracting local characteristic graphs of all defect regions in the training set, and obtaining a minimum characteristic value in all the local characteristic graphs, namely a threshold value T;
1) the coordinates of the defect at the upper left corner and the lower right corner in the original drawing are represented by [ (x1, y1), (x2, y2) ], and the defect position in the feature map is represented by [ (x1', y1'), (x2', y2') ], and the correspondence relationship is as follows:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of selected feature layers, [ ] represents rounding down.
2) And storing the minimum value of each defect position in the feature map into a set S, and recording the minimum value in the final set S as a threshold value T.
Step five, in F'k-1And extracting local characteristic graphs of all defect areas in the training set to form a positive sample, extracting local characteristic graphs of part of background areas to form a negative sample, and sending the negative sample to the SVM to obtain the detector.
1) Selecting n x n sliding window s1, sliding on the original image, wherein the sliding step length is n/2, and if s1 and the IOU of the defect position are greater than 0.7, the sliding window is considered as a normal sample; if the IOU value is less than 0.3, the sliding window is considered negative. The two regions IOU are defined as follows:
2) find out that sample is F'k-1And the feature maps in (1) are expanded into one-dimensional vectors according to rows, and the dimensions of the vectors are naturally consistent because all the samples are generated by sliding windows of the same size. Generally, the number of positive examples is much smaller than that of negative examples, and the model obtained by unbalanced data training has poor detection performance. Therefore, samples with the same quantity as that of the positive sample set are randomly sampled from the negative sample set to form a new negative sample set, and the new negative sample set and the new positive sample set are sent to the SVM for training to obtain the classifier model.
3) Parameter setting of the trainer:
kernel function kernel: linear kernel
Penalty term C: 1.7
Size of the sliding window: 32
Sliding step length: 16.
step six, in the testing stage, the step 2 and the step 3 are repeated to obtain the characteristic F 'of the k-1 layer'k-1And feature F 'of the k-th layer'k. Defining a two-dimensional matrix with the same size as the kth layer as D, where the matrix D reflects a significant region of the sample, i.e. a region where there may be defects, so as to reduce the search range of the sliding window:
step seven, the salient region of step six is mapped to a region D' on the k-1 layer feature. In characteristic diagram F'k-1In the region D', the characteristics are extracted by using a sliding window of n x n, the characteristics are expanded into one-dimensional vectors, and the one-dimensional vectors are sent to an SVM discriminator trained in the step 5, so that whether the sliding window has defects or not is judged. All defective sliding windows are marked and the areas with excessive overlap can be merged. Thus, the metal defect detection task is completed.
The technical effects of the present invention will be described in detail with reference to simulations.
1. Simulation conditions
The invention uses Python language to complete the simulation experiment of the invention on the PC with CPU being Intel (R) core (TM) i5-4590, CPU3.30GHz, RAM 16.00GB and Windows 10 operating system.
2. Content of simulation experiment
The training data of the invention is 200 single-channel metal images with defects, and the test data is 50 images with defects and 50 images without defects. In the SVM training stage, positive and negative samples are divided by taking a sliding window as a basic unit, the samples are divided according to the mode of the step five, the positive samples (defects) are 276 samples, the number of the negative samples is far larger than that of the positive samples, and 276 samples are randomly extracted from the positive samples to serve as a negative sample set. After training, the model performance was tested. To verify the effectiveness of the present invention, a comparative experiment was designed and compared with the method of the present invention, and the results are shown in table 1, except for YOLOv3, which is based on a sliding window detection method.
Table 1 comparison of the process of the invention with the prior art:
according to the results, the network-based feature extraction method (VGG) has higher detection precision than the traditional feature extraction methods (HOG, LBP), but consumes more time. The invention greatly reduces the area to be detected by using a strategy of staged detection, thereby reducing the detection time and improving the efficiency; the effect of the coarse positioning mode is obvious, the detection precision is higher than that of a mode that all areas are checked, and the fact that certain interference exists in the areas except the coarse positioning mode is shown. Overall, the present invention significantly improves the accuracy of metal defect detection.
It should be noted that the embodiments of the present invention can be realized by hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or specially designed hardware. Those skilled in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such code being provided on a carrier medium such as a disk, CD-or DVD-ROM, programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier, for example. The apparatus and its modules of the present invention may be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of hardware circuits and software, e.g., firmware.
The above description is only for the purpose of illustrating the present invention and the appended claims are not to be construed as limiting the scope of the invention, which is intended to cover all modifications, equivalents and improvements that are within the spirit and scope of the invention as defined by the appended claims.
Claims (10)
1. A staged metal surface defect detection method is characterized by comprising the following steps:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
2. The method of claim 1, wherein the step of extracting defect regions in the kth layer of features of all training samples and the step of determining the minimum feature value as the threshold T specifically comprises: extracting the depth feature of the k-1 layer of each image in the data set by using a VGG network and recording the depth feature as Fk-1And the depth of the k-th layer is characterized by Fk(ii) a Only the feature extraction part is reserved by adopting a VGG16 network structure; the whole network consists of 13 convolutional layers and 5 pooling layers, and each time the size of a feature map of one pooling layer is reduced to 1/4 of the upper layer, the deeper the layer number, the richer the semantic features, and the shallower the layer number, the richer the position information; features were extracted using the VGG pre-training model from Imagenet, let k be 3.
3. The staged metal surface defect detection method of claim 2, wherein Fk-1And FkAll are three-dimensional matrixes, and the sum of each channel is used for obtaining a two-dimensional matrix which is respectively recorded as F'k-1And F'k(ii) a The conversion formula is as follows:
wherein, N is the number of the characteristic channels of the k-1 th layer, M is the number of the characteristic channels of the k-1 th layer, and x and y are indexes of the first two dimensions of the characteristic diagram respectively.
4. The phased method of metal surface defect detection according to claim 1, wherein said extracting all training samples for absence from the kth layer featureSetting the minimum characteristic value of the trap area as a threshold value T at F'kExtracting local characteristic graphs of all defect regions in the training set, and obtaining a minimum characteristic value in all the local characteristic graphs, namely a threshold value T;
1) the coordinates of the defect at the upper left corner and the lower right corner in the original drawing are represented by [ (x1, y1), (x2, y2) ], and the defect position in the feature map is represented by [ (x1', y1'), (x2', y2') ], and the correspondence relationship is as follows:
n'＝[n/2^k] n∈[x1,y1,x2,y2]；
k represents the number of selected feature layers, [ ] represents rounding down;
2) and storing the minimum value of each defect position in the feature map into a set S, and recording the minimum value in the final set S as a threshold value T.
5. The method according to claim 1, wherein the step of extracting the defect regions in the k-1 th layer of features of all training samples by using a sliding window of n × n, expanding the defect regions into one-dimensional vectors according to rows and recording the vectors as a positive sample set, and extracting background regions with the number equivalent to that of the positive samples and expanding the background regions into one-dimensional vectors according to rows and recording the vectors as a negative sample set specifically comprises: in F'k-1Extracting local characteristic graphs of all defect areas in the training set to form a positive sample, extracting local characteristic graphs of partial background areas to form a negative sample, and sending the negative sample to the SVM to obtain a detector;
1) selecting n x n sliding window s1, sliding on the original image, wherein the sliding step length is n/2, and if s1 and the IOU of the defect position are greater than 0.7, the sliding window is considered as a normal sample; if the IOU value is less than 0.3, the sliding window is considered as a negative example, and the definition of the IOU of two areas is as follows:
2) find out that sample is F'k-1The characteristic diagram is expanded into a one-dimensional vector according to rows, samples with the same number as that of the positive sample set are randomly sampled from the negative sample set to form a new negative sample set, and the new negative sample set and the new positive sample set are sent to the computerTraining in an SVM to obtain a classifier model;
3) parameter setting of the trainer:
kernel function kernel: linear kernel
Penalty term C: 1.7
Size of the sliding window: 32
Sliding step length: 16.
6. the phased metal surface defect detection method of claim 1, further comprising: in the testing stage, the depth feature of the k-1 layer of each image in the data set is extracted and recorded as F by repeatedly utilizing the VGG networkk-1And the depth of the k-th layer is characterized by Fk,，Fk-1And FkAll are three-dimensional matrixes, and the sum of each channel is used for obtaining a two-dimensional matrix which is respectively recorded as F'k-1And F'k(ii) a Defining a two-dimensional matrix of the same size as the kth layer as D, the matrix D reflecting the saliency areas of the sample:
d's salient region is mapped to a region D ' on the k-1 layer feature, in a feature map F 'k-1In the D' area, extracting features by using a sliding window of n x n, expanding the features into a one-dimensional vector, sending the one-dimensional vector into a trained SVM discriminator, and judging whether the sliding window has defects or not; and marking all the sliding windows with defects, and combining the areas with excessive overlapping areas to finish the metal defect detection task.
7. A computer device, characterized in that the computer device comprises a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to carry out the steps of:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
8. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of:
extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
taking out defect areas in the kth layer of features of all training samples, and setting the minimum feature value of the defect areas as a threshold value T;
taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows, recording the one-dimensional vectors as a positive sample set, simultaneously taking out background areas with the number equivalent to that of the positive samples, expanding the background areas into one-dimensional vectors according to rows, and recording the one-dimensional vectors as a negative sample set;
and sending the feature vectors into the SVM, and training to obtain the classifier.
9. A metal defect detection information data processing terminal, characterized in that the metal defect detection information data processing terminal is used for realizing the staged metal surface defect detection method as claimed in any one of claims 1 to 6.
10. A staged metal surface defect detection system for implementing the staged metal surface defect detection method of any one of claims 1 to 6, wherein the staged metal surface defect detection system comprises:
the image feature extraction module is used for extracting the kth-1 and kth layer features of the image by using a VGG pre-training model;
the defect region extraction module is used for extracting defect regions in the kth layer of features of all the training samples, and the minimum feature value of the defect regions is set as a threshold value T;
the sample set forming module is used for taking out defect areas in the k-1 layer characteristics of all training samples by using an n x n sliding window, expanding the defect areas into one-dimensional vectors according to rows and recording the one-dimensional vectors as a positive sample set, and simultaneously taking out background areas with the number equivalent to that of the positive samples and expanding the background areas into one-dimensional vectors according to rows and recording the one-dimensional vectors as a negative sample set;
and the classifier obtaining module is used for sending the feature vectors into the SVM and training to obtain the classifier.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395512.4A CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395512.4A CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112633327A true CN112633327A (en) | 2021-04-09 |
| CN112633327B CN112633327B (en) | 2023-06-30 |
Family
ID=75307530
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011395512.4A Active CN112633327B (en) | 2020-12-02 | 2020-12-02 | Staged metal surface defect detection method, system, medium, equipment and application |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112633327B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554631A (en) * | 2021-07-30 | 2021-10-26 | 西安电子科技大学 | Chip surface defect detection method based on improved network |
| CN114332495A (en) * | 2021-12-22 | 2022-04-12 | 携程旅游信息技术（上海）有限公司 | Image feature quantization method, system, device and storage medium |
Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103996209A (en) * | 2014-05-21 | 2014-08-20 | 北京航空航天大学 | Infrared vessel object segmentation method based on salient region detection |
| RU2570592C1 (en) * | 2014-10-27 | 2015-12-10 | Алексей Николаевич Кузьмин | Method of detecting and analysing acoustic emission signals |
| CN105574527A (en) * | 2015-12-14 | 2016-05-11 | 北京工业大学 | Quick object detection method based on local feature learning |
| US20170255832A1 (en) * | 2016-03-02 | 2017-09-07 | Mitsubishi Electric Research Laboratories, Inc. | Method and System for Detecting Actions in Videos |
| CN107491762A (en) * | 2017-08-23 | 2017-12-19 | 珠海安联锐视科技股份有限公司 | A kind of pedestrian detection method |
| CN108304873A (en) * | 2018-01-30 | 2018-07-20 | 深圳市国脉畅行科技股份有限公司 | Object detection method based on high-resolution optical satellite remote-sensing image and its system |
| US20190073568A1 (en) * | 2017-09-06 | 2019-03-07 | Kla-Tencor Corporation | Unified neural network for defect detection and classification |
| CN109685030A (en) * | 2018-12-29 | 2019-04-26 | 哈尔滨理工大学 | A kind of mug rim of a cup defects detection classification method based on convolutional neural networks |
| CN109934147A (en) * | 2019-03-05 | 2019-06-25 | 北京联合大学 | Object detection method, system and device based on deep neural network |
| CN110084131A (en) * | 2019-04-03 | 2019-08-02 | 华南理工大学 | A kind of semi-supervised pedestrian detection method based on depth convolutional network |
| CN110276754A (en) * | 2019-06-21 | 2019-09-24 | 厦门大学 | A kind of detection method of surface flaw, terminal device and storage medium |
| CN110766643A (en) * | 2019-10-28 | 2020-02-07 | 电子科技大学 | Microaneurysm detection method facing fundus images |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| EP3633605A1 (en) * | 2017-05-22 | 2020-04-08 | Canon Kabushiki Kaisha | Information processing device, information processing method, and program |
| CN111209848A (en) * | 2020-01-03 | 2020-05-29 | 北京工业大学 | Real-time fall detection method based on deep learning |
| US20200175352A1 (en) * | 2017-03-14 | 2020-06-04 | University Of Manitoba | Structure defect detection using machine learning algorithms |
| WO2020206861A1 (en) * | 2019-04-08 | 2020-10-15 | 江西理工大学 | Yolo v3-based detection method for key object at transportation junction |
-
2020
- 2020-12-02 CN CN202011395512.4A patent/CN112633327B/en active Active
Patent Citations (17)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103996209A (en) * | 2014-05-21 | 2014-08-20 | 北京航空航天大学 | Infrared vessel object segmentation method based on salient region detection |
| RU2570592C1 (en) * | 2014-10-27 | 2015-12-10 | Алексей Николаевич Кузьмин | Method of detecting and analysing acoustic emission signals |
| CN105574527A (en) * | 2015-12-14 | 2016-05-11 | 北京工业大学 | Quick object detection method based on local feature learning |
| US20170255832A1 (en) * | 2016-03-02 | 2017-09-07 | Mitsubishi Electric Research Laboratories, Inc. | Method and System for Detecting Actions in Videos |
| US20200175352A1 (en) * | 2017-03-14 | 2020-06-04 | University Of Manitoba | Structure defect detection using machine learning algorithms |
| EP3633605A1 (en) * | 2017-05-22 | 2020-04-08 | Canon Kabushiki Kaisha | Information processing device, information processing method, and program |
| CN107491762A (en) * | 2017-08-23 | 2017-12-19 | 珠海安联锐视科技股份有限公司 | A kind of pedestrian detection method |
| US20190073568A1 (en) * | 2017-09-06 | 2019-03-07 | Kla-Tencor Corporation | Unified neural network for defect detection and classification |
| CN108304873A (en) * | 2018-01-30 | 2018-07-20 | 深圳市国脉畅行科技股份有限公司 | Object detection method based on high-resolution optical satellite remote-sensing image and its system |
| CN109685030A (en) * | 2018-12-29 | 2019-04-26 | 哈尔滨理工大学 | A kind of mug rim of a cup defects detection classification method based on convolutional neural networks |
| CN109934147A (en) * | 2019-03-05 | 2019-06-25 | 北京联合大学 | Object detection method, system and device based on deep neural network |
| CN110084131A (en) * | 2019-04-03 | 2019-08-02 | 华南理工大学 | A kind of semi-supervised pedestrian detection method based on depth convolutional network |
| WO2020206861A1 (en) * | 2019-04-08 | 2020-10-15 | 江西理工大学 | Yolo v3-based detection method for key object at transportation junction |
| CN110276754A (en) * | 2019-06-21 | 2019-09-24 | 厦门大学 | A kind of detection method of surface flaw, terminal device and storage medium |
| CN110796643A (en) * | 2019-10-18 | 2020-02-14 | 四川大学 | Rail fastener defect detection method and system |
| CN110766643A (en) * | 2019-10-28 | 2020-02-07 | 电子科技大学 | Microaneurysm detection method facing fundus images |
| CN111209848A (en) * | 2020-01-03 | 2020-05-29 | 北京工业大学 | Real-time fall detection method based on deep learning |
Non-Patent Citations (4)
| Title |
|---|
| CHEN SHOUHONG 等: "基于多特征的SVM多分类PCB焊点缺陷检测方法", 《激光杂志》 * |
| HONGLING CHEN 等: "Face Recognition Algorithm Based on VGG Network Model and SVM", 《JOURNAL OF PHYSICS: CONFERENCE SERIES》 * |
| NING LV 等: "Clothing Images Attributes Classification Based on Deep Neural Network", 《2019 IEEE INTERNATIONAL CONFERENCE ON SMART INTERNET OF THINGS (SMARTIOT)》 * |
| 杜超等: "改进的VGG网络的二极管玻壳图像缺陷检测", 《图学学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113554631A (en) * | 2021-07-30 | 2021-10-26 | 西安电子科技大学 | Chip surface defect detection method based on improved network |
| CN113554631B (en) * | 2021-07-30 | 2024-02-20 | 西安电子科技大学 | Chip surface defect detection method based on improved network |
| CN114332495A (en) * | 2021-12-22 | 2022-04-12 | 携程旅游信息技术（上海）有限公司 | Image feature quantization method, system, device and storage medium |
| CN114332495B (en) * | 2021-12-22 | 2024-10-15 | 携程旅游信息技术（上海）有限公司 | Image feature quantization method, system, device and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112633327B (en) | 2023-06-30 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108765412B (en) | Strip steel surface defect classification method | |
| CN111753828B (en) | Natural scene horizontal character detection method based on deep convolutional neural network | |
| CN109190752A (en) | The image, semantic dividing method of global characteristics and local feature based on deep learning | |
| CN111027443B (en) | Bill text detection method based on multitask deep learning | |
| CN106557740B (en) | The recognition methods of oil depot target in a kind of remote sensing images | |
| CN111896540A (en) | Water quality on-line monitoring system based on block chain | |
| CN113298809B (en) | Composite material ultrasonic image defect detection method based on deep learning and superpixel segmentation | |
| CN110334594A (en) | A kind of object detection method based on batch again YOLO algorithm of standardization processing | |
| CN112749673A (en) | Method and device for intelligently extracting stock of oil storage tank based on remote sensing image | |
| CN110751619A (en) | Insulator defect detection method | |
| CN113487600B (en) | Feature enhancement scale self-adaptive perception ship detection method | |
| CN112949408A (en) | Real-time identification method and system for target fish passing through fish channel | |
| CN112633327B (en) | Staged metal surface defect detection method, system, medium, equipment and application | |
| CN111161237A (en) | Fruit and vegetable surface quality detection method, storage medium and sorting device thereof | |
| CN110659374A (en) | Method for searching images by images based on neural network extraction of vehicle characteristic values and attributes | |
| CN102708367A (en) | Image identification method based on target contour features | |
| CN109507193A (en) | A kind of fabric defects detection method based on local contrast enhancing and binary pattern | |
| CN114429577B (en) | Flag detection method, system and equipment based on high confidence labeling strategy | |
| Gui et al. | A fast caption detection method for low quality video images | |
| US20230386023A1 (en) | Method for detecting medical images, electronic device, and storage medium | |
| Dong | A pixel-wise framework based on convolutional neural network for surface defect detection | |
| Xie et al. | Defect detection of printed circuit board based on small target recognition network | |
| Yan et al. | A Method for the Intelligent Localization and Recognition of sporopollen fossil Based on Convolutional Neural Networks—A Case Study of Sandstone and Mudstone Reservoirs in the Bohai Sea Region | |
| CN113496159A (en) | Multi-scale convolution and dynamic weight cost function smoke target segmentation method | |
| CN111259974A (en) | Surface defect positioning and classifying method for small-sample flexible IC substrate |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |