An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms
<p>Flowchart of establishing the contour template.</p> "> Figure 2
<p>Calculation flowchart of adaptive pyramid layers.</p> "> Figure 3
<p>Extraction of template image features sub-process.</p> "> Figure 4
<p>The calculation process of block-level strong binary detection mode.</p> "> Figure 5
<p>Flowchart of contour matching algorithm.</p> "> Figure 6
<p>Flowchart of three-level matching process of pyramid image.</p> "> Figure 7
<p>Flowchart of number matching of contour points in integral graph of first-level.</p> "> Figure 8
<p>Flowchart of block level strong binary detection pattern matching of second-level.</p> "> Figure 9
<p>Flowchart of contour similarity matching of third-level.</p> "> Figure 10
<p>Selection process of the best candidate points.</p> "> Figure 11
<p>Product false detection caused by interference source occlusion.</p> ">
Abstract
:1. Introduction
2. Design of Contour Template Building Algorithm
2.1. Image Preprocessing
- (1)
- Grayscale processing: the three-channel color image is converted into a gray single-channel image, that is, the three-channel color image composed of R, G and B components is converted into a single-channel gray value image composed within the 0~255 range.
- (2)
- Grayscale stretching processing: grayscale stretching can improve the darker areas of an image or the lighter areas of the image, making the contour boundaries of the image more pronounced.
- (3)
- Gaussian blur processing: Gaussian blur processing eliminates the noise in the image, avoids the edge points formed by noise affecting the establishment of the contour template, and then affects the final matching similarity.
- (4)
- Image sharpening processing: The Laplacian operator is used to sharpen the image. If the pixel value of the target point is lower than the average value of the pixels in its neighborhood, the pixel value of the point should be further reduced. If the pixel gray value of this point is higher than the average gray value of other points in its neighborhood, the gray value of this point needs to be further improved.
2.2. Gaussian Image Pyramid and Adaptive Parameter Calculation
2.2.1. Establishing Gaussian Image Pyramid
2.2.2. Adaptive Pyramid Layers
2.3. Adaptive Angle Step
2.4. Extracting Image Template Features
2.4.1. Extracting Edge Contour Information
2.4.2. Calculating the Block-Level Strong Binary Detection Mode
3. Design of Contour Matching Algorithm
3.1. Number Matching of Contour Points in Integral Graph of First-Level
3.2. Block Level Strong Binary Detection Pattern Matching of Second-Level
- (1)
- Obtain the block-level strong binary detection mode of the image to be matched in the sliding window.
- (2)
- Read the block-level strong binary detection mode of each angle template image, and traverse the template block-level strong binary detection mode, corresponding to all rotation angle steps.
- (3)
- The binary detection mode of the sliding window is compared with the binary detection mode of each angle of the template image. If the number of inconsistent blocks is less than the threshold value Tblock, the second-level matching passes and the third-level matching stage is entered. Otherwise, compare the binary detection mode of the next angle. It should be noted that Tblock is the maximum allowed number of block inconsistencies. If Tblock is selected as a value that is too small, it is sensitive to noise. If the Tblock selection is too large, there will be too many non-candidate points that need to be further matched accurately. Here Tblock = total number of blocks in sliding window *0.2.
- (4)
- Complete the comparison of the block-level strong binary detection modes of all angle templates through traversal, and slide the window to the next position.
- (5)
- Traverse the entire image to be matched, compare all candidate points, and select the best candidate points.
3.3. Contour Similarity Matching of Third-Level
3.4. Selection of the Best Candidate Points
4. Experimental Test and Result Analysis
4.1. Function Test of Template Establishment
- (1)
- Uniform lighting conditions
- (2)
- Nonlinear illumination
4.2. Function Test of Contour Matching
- (1)
- Good product detection under uniform lighting conditions: Under uniform lighting conditions (light intensity is E = 50, 100, 150 lux), the images of good products at any angle, any position and with external disturbances are collected, matched with the template, and the qualified products are marked as ‘OK’. It is expected that a good product image to be matched can normally match the template image (such as any template image given in Table 4, Table 5 and Table 6) under uniform lighting conditions.
- (2)
- Good product detection under nonlinear lighting conditions: Under nonlinear lighting conditions, collect any angle, any position and good product image with external interference, match with the template, and mark the qualified product as ‘OK’. It is expected that a good product image to be matched can match the template image (such as any template image given in Table 4, Table 5 and Table 6) normally under nonlinear lighting conditions.
- (3)
- Defect detection: Defect types include new defects, missing defects, external interference occlusion, etc., and the detected unqualified products are marked as “NG” (Not Good). It is expected that the defect can be correctly detected under both uniform illumination and nonlinear illumination.
Detection Type | Good Product Detection “OK” | Defect Detection “NG” | ||||||
---|---|---|---|---|---|---|---|---|
Experimental Condition | Uniform Illumination Any Angle, Any Position, There Are Distractions | Nonlinear Illumination Any Angle, Any Position, There Are Distractions | Any Illumination Parameters Above | |||||
Parameter | E = 50 lux | E = 100 lux | E = 150 lux | Left Illumination | Right Illumination | Front Illumination | Back Illumination | Any Template Image Above |
Template image | ——— | |||||||
Detection results (show part only) |
Detection Type | Good Product Detection “OK” | Defect Detection “NG” | ||||||
---|---|---|---|---|---|---|---|---|
Experimental Condition | Uniform Illumination Any Angle, Any Position, There Are Distractions | Nonlinear Illumination Any Angle, Any Position, There Are Distractions | Any Illumination Parameters Above | |||||
Parameter | E = 50 lux | E = 100 lux | E = 150 lux | Left Illumination | Right Illumination | Front Illumination | Back Illumination | Any Template Image Above |
Template image | ——— | |||||||
Detection results (show part only) |
Detection Type | Good Product Detection “OK” | Defect Detection “NG” | ||||||
---|---|---|---|---|---|---|---|---|
Experimental Condition | Uniform Illumination Any Angle, Any Position, There Are Distractions | Nonlinear Illumination Any Angle, Any Position, There Are Distractions | Any Illumination Parameters Above | |||||
Parameter | E = 50 lux | E = 100 lux | E = 150 lux | Left Illumination | Right Illumination | Front Illumination | Back Illumination | Any Template Image Above |
Template image | ——— | |||||||
Detection results (show part only) |
- (1)
- The average accuracy, precision and recall of the 3-hole metal stamping parts under various experimental conditions were 93.01%, 94.43% and 96.68%, respectively; The average accuracy, precision and recall of the mineral water bottle caps under each experimental condition were 94.44%, 95.07% and 95.62%, respectively. The average accuracy, precision and recall of the table hole cover under each experimental condition were 92.35%, 94.11% and 94.75%, respectively. The average accuracy, good product matching rate and bad product matching rate of product defect detection can be kept above 92%, which proves the effectiveness and practicability of the proposed method.
- (2)
- (2) Under uniform lighting conditions, when the light intensity E = 100 lux, the detection accuracy of the three products is higher than that under dark lighting (E = 50 lux) and bright lighting (E = 150 lux) conditions. The average values of detection accuracy under uniform illumination are 93.60%, 95.74% and 92.68%, respectively, as shown in Table 9. The overall detection accuracy remains basically unchanged, the lowest can be maintained above 92.68%, and a high defect detection rate can be maintained. This is because the contour matching algorithm based on shape features is adopted. The gradient features used are normalized gradient vector features, and the change in light intensity will not directly affect the gradient vector.
- (3)
- Under nonlinear illumination conditions, the image is subjected to nonlinear illumination in different directions, and the gradient direction vector of part of the contour will change compared with the gradient direction of the template. The detection accuracy of the three products fluctuated under nonlinear illumination conditions, and the average detection accuracy was 92.57%, 93.46% and 92.11%, respectively, as shown in Table 9. The overall detection accuracy is basically unchanged, and the defect detection rate is high, which indicates that the nonlinear illumination has little influence on the accuracy of the algorithm.
- (4)
- A statistical analysis was carried out on the comprehensive results of the matching accuracy of three kinds of products matched 100 times. It was found that the standard deviation of detection accuracy, good product matching rate and bad product matching rate were 0.0115, 0.0054 and 0.0106, and the confidence intervals were [92.57, 94.20], [93.56, 94.08] and [94.90, 96.74], respectively. This means that the true average of the experimental results with a standard deviation of 0.0115, 0.0054 and 0.0106 has a 95% probability of falling within the range of [92.57%, 94.20%], [93.56%, 94.58%] and [94.90%, 96.74%], which is consistent with the calculated detection mean value. It shows that the matching results of the proposed method are real and reliable.
- (5)
- In the presence of external interference sources, the outline of the item will be undetected, as shown in Figure 11. Especially when the external interference source is too close to the detected item, there is edge occlusion and there will be mismatching, so try to choose the appropriate template frame size. This paper mainly detects two types of defects: new defects and missing defects. When detecting new defects, if the new defects are not obvious or similar to the gray value of the background, the edge contour and other matching information of the defects cannot be correctly extracted, the algorithm will classify it as the background, and it cannot be correctly detected, as shown in Figure 11. Due to the obvious difference between the contour of the image to be matched and the contour of the template, the algorithm can accurately detect the missing flaw detection. Therefore, the algorithm is more suitable for the detection of missing defects, while the detection rate of new defects is low, and the overall defect detection rate remains at about 96%.
Product | Experimental Condition | Matching Accuracy of 100 Times | ||
---|---|---|---|---|
Accuracy | Precision | Recall | ||
3-hole stamping parts | Uniform illumination | 93.60% | 94.81% | 97.17% |
Nonlinear illumination | 92.57% | 94.15% | 96.32% | |
All conditions | 93.01% | 94.43% | 96.69% | |
Mineral water bottle cap | Uniform illumination | 95.74% | 95.46% | 97.17% |
Nonlinear illumination | 93.46% | 94.78% | 96.05% | |
All conditions | 94.43% | 95.07% | 96.53% | |
Table hole caps | Uniform illumination | 92.68% | 93.70% | 95.70% |
Nonlinear illumination | 92.11% | 94.42% | 94.05% | |
All conditions | 92.35% | 94.11% | 94.76% | |
Statistical analysis results of three kinds of products | Mean value | 93.33% | 94.55% | 96.05% |
Standard deviation | 0.0115 | 0.0054 | 0.0106 | |
Confidence interval | [92.57, 94.20] | [93.56, 94.58] | [94.90, 96.74] |
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- Jha, S.B.; Babiceanu, R.F. Deep CNN-based visual defect detection: Survey of current literature. Comput. Ind. 2023, 148, 103911. [Google Scholar] [CrossRef]
- Qu, J.J.; Luo, Y.S.; Chen, W.Y.; Wang, H.X. Research on the identification method of key parts of ship target based on contour matching. Aip. Adv. 2023, 13, 115011. [Google Scholar] [CrossRef]
- Dan, Y. Target Area Extraction Algorithm of Infrared Thermal Image Combining Target Detection with Matching Correction. Trait. Signal 2023, 40, 227–234. [Google Scholar]
- Xu, G.L.; Wu, Q.; Cheng, Y.H.; Yan, F.J.; Li, Z.H.; Yu, Q.D. A robust deformed image matching method for multi-source image matching. Infrared. Phys. Technol. 2021, 115, 103691. [Google Scholar] [CrossRef]
- Lewis, J.P. Fast Normalized Cross-Correlation. Circuits Syst. Signal Process. 1995, 120–123. [Google Scholar]
- Ma, S.K.; Li, X.H.; Liu, K.W.; Qiu, T.C.; Liu, Y.L. Research on Image Stitching Algorithm Based on Point-Line Consistency and Local Edge Feature Constraints. Entropy 2024, 26, 61. [Google Scholar] [CrossRef] [PubMed]
- Yang, W.; Mingquan, Z.; Pengfei, Z.; Guohua, G. Matching Method of Cultural Relic Fragments Constrained by Thickness and Contour Feature. IEEE Access 2020, 8, 25892–25904. [Google Scholar] [CrossRef]
- Hu, M.K. Visual Pattern Recognition by Moment Invariants. IRE Trans. Inf. Theory 1962, 8, 179–187. [Google Scholar]
- Du, Y.; Tong, M. Contour recognition of roadheader cutting head based on shape matching. Pattern Anal. Appl. PAA 2019, 22, 1643–1653. [Google Scholar] [CrossRef]
- Kim, S.W.; Kong, J.H.; Lee, S.W.; Lee, S. Recent Advances of Artificial Intelligence in Manufacturing Industrial Sectors: A Review. Int. J. Precis. Eng. Manuf. 2022, 23, 111–129. [Google Scholar] [CrossRef]
- Liu, R.Q.; Huang, M.; Gao, Z.M.; Cao, Z.Y.; Cao, P. MSC-DNet: An efficient detector with multi-scale context for defect detection on strip steel surface. Measurement 2023, 209, 112467. [Google Scholar] [CrossRef]
- Guo, Z.X.; Wang, C.S.; Yang, G.; Huang, Z.Y.; Li, G. MSFT-YOLO: Improved YOLOv5 Based on Transformer for Detecting Defects of Steel Surface. Sensors 2022, 22, 3467. [Google Scholar] [CrossRef] [PubMed]
- Feng, X.; Gao, X.; Luo, L. X-SDD: A New Benchmark for Hot Rolled Steel Strip Surface Defects Detection. Symmetry 2021, 13, 706. [Google Scholar] [CrossRef]
- Yang, R.; Singh, S.K.; Tavakkoli, M.; Karami, M.A.; Rai, R. Deep Learning Architecture for Computer Vision-based Structural Defect Detection. Appl. Intell. 2023, 53, 22850–22862. [Google Scholar] [CrossRef]
- Zhang, K.; Ma, X.; Li, X.G. Shape Matching Based on Multi-Scale Invariant Features. IEEE Access 2019, 7, 115637–115649. [Google Scholar]
- Arulananth, T.S.; Chinnasamy, P.; Babu, J.C.; Kiran, A.; Hemalatha, J.; Abbas, M. Edge detection using fast pixel based matching and contours mapping algorithms. PLoS ONE 2023, 18, e0289823. [Google Scholar] [CrossRef] [PubMed]
- Si, H.P.; Wang, Y.P.; Liu, Q.; Li, W.X.; Wan, L.; Song, J.Z.; Zhao, W.R.; Sun, C.X. A registration algorithm for the infrared and visible images of apple based on active contour model. Vis. Comput. 2024, 40, 2833–2855. [Google Scholar] [CrossRef]
- Zhang, X. Similarity invariant partial shape matching using coarse-to-fine strategy. J. Electron. Imaging 2014, 23, 53019.1–53019.17. [Google Scholar] [CrossRef]
- Wu, T.; Toet, A. Speed-up template matching through integral image based weak classifiers. J. Pattern Recognit. Res. 2014, 1, 1–12. [Google Scholar]
- Steger, C. Similarity Measures for Occlusion, Clutter, and Illumination Invariant Object Recognition, Pattern recognition. In Proceedings of the 23rd DAGM Symposium, Munich, Germany, 12–14 September 2001; pp. 148–154. [Google Scholar]
- Tan, P.; Li, X.-F.; Xu, J.-M.; Ma, J.-E.; Wang, F.-J.; Ding, J.; Fang, Y.-T.; Ning, Y. Catenary insulator defect detection based on contour features and gray similarity matching. J. Zhejiang Univ.-SCIENCE A 2020, 21, 64–73. [Google Scholar] [CrossRef]
- Wang, Y.F.; Huang, Q.; Hu, J. Adaptive enhancement for nonuniform illumination images via nonlinear mapping. J. Electron. Imaging 2017, 26, 053012. [Google Scholar] [CrossRef]
Test Stage | Scenarios | Variables | Parameter | Expectation |
---|---|---|---|---|
Template Establishment | 1 | Uniform light intensity | E = 50 lux | The same template can be matched normally under different light intensities. |
E = 100 lux | ||||
E = 150 lux | ||||
2 | Nonlinear illumination | Set a point light source to the left of the product | The same template can be matched normally under different lighting conditions. | |
Set a point light source to the right of the product | ||||
Set a point light source to the front of the product | ||||
Set a point light source to the back of the product | ||||
Contour Matching | 3 | Product defect | Added types of defects | Compared with the template, the product to be matched has obvious defects and can be identified normally. |
Missing type of defect | ||||
4 | External interference (Complex background) | Multiple interference sources | Under complex background or external interference conditions, good and defective products can be identified normally. |
Product | Dark Lighting Conditions | Normal Lighting Conditions | Bright Lighting Conditions |
---|---|---|---|
3-hole stamping parts | |||
Mineral water bottle cap | |||
Table hole cover |
Product | Left Illumination | Right Illumination | Front Illumination | Back Illumination |
---|---|---|---|---|
3-hole stamping parts | ||||
Mineral water bottle cap | ||||
Table hole cover |
Experimental Condition | Uniform Illumination | Nonlinear Illumination | Mean Time | ||||||
---|---|---|---|---|---|---|---|---|---|
Product | 50 lux | 100 lux | 150 lux | Left | Right | Front | Back | ||
3-hole stamping parts | 298 | 290 | 303 | 308 | 306 | 301 | 305 | 301.57 | |
Mineral water bottle cap | 213 | 206 | 210 | 209 | 216 | 221 | 209 | 212.00 | |
Table hole cover | 282 | 262 | 288 | 279 | 291 | 285 | 281 | 284.14 |
Product | Experimental Condition | Illumination Parameter | Matching Accuracy of 100 Times | ||
---|---|---|---|---|---|
Accuracy | Precision | Recall | |||
3-hole stamping parts | Uniform illumination | E = 50 lux | 93.21% | 94.62% | 97.21% |
E = 100 lux | 95.59% | 96.03% | 96.99% | ||
E = 150 lux | 92.01% | 93.77% | 97.32% | ||
Nonlinear illumination | Left illumination | 91.89% | 93.42% | 95.88% | |
Right illumination | 92.36% | 94.96% | 96.35% | ||
Front illumination | 92.56% | 93.85% | 95.42% | ||
Back illumination | 93.45% | 94.37% | 97.63% | ||
Mineral water bottle caps | Uniform illumination | E = 50 lux | 95.88% | 96.04% | 97.21% |
E = 100 lux | 97.32% | 97.63% | 96.99% | ||
E = 150 lux | 94.01% | 92.72% | 97.32% | ||
Nonlinear illumination | Left illumination | 93.95% | 94.78% | 97.35% | |
Right illumination | 92.35% | 95.96% | 96.91% | ||
Front illumination | 92.87% | 94.75% | 94.18% | ||
Back illumination | 94.66% | 93.61% | 95.74% | ||
Table hole caps | Uniform illumination | E = 50 lux | 93.45% | 94.11% | 95.41% |
E = 100 lux | 94.37% | 95.12% | 97.85% | ||
E = 150 lux | 90.21% | 91.87% | 93.84% | ||
Nonlinear illumination | Left illumination | 92.33% | 94.55% | 94.63% | |
Right illumination | 90.68% | 93.65% | 94.18% | ||
Front illumination | 91.85% | 94.69% | 93.55% | ||
Back illumination | 93.57% | 94.78% | 93.84% | ||
Average accuracy of 3-hole stamping parts/Mineral water bottle caps/Table hole caps | 93.01%/94.44%/92.35% | 94.43%/95.07%/ 94.11% | 96.68%/95.62%/94.75% |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Wu, H.; Luo, Z.; Sun, F.; Li, X.; Zhao, Y. An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms. Sensors 2024, 24, 3932. https://doi.org/10.3390/s24123932
Wu H, Luo Z, Sun F, Li X, Zhao Y. An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms. Sensors. 2024; 24(12):3932. https://doi.org/10.3390/s24123932
Chicago/Turabian StyleWu, Haorong, Ziqi Luo, Fuchun Sun, Xiaoxiao Li, and Yongxin Zhao. 2024. "An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms" Sensors 24, no. 12: 3932. https://doi.org/10.3390/s24123932
APA StyleWu, H., Luo, Z., Sun, F., Li, X., & Zhao, Y. (2024). An Improvement Method for Improving the Surface Defect Detection of Industrial Products Based on Contour Matching Algorithms. Sensors, 24(12), 3932. https://doi.org/10.3390/s24123932