Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

A generalized well neural network for surface defect segmentation in Optical Communication Devices via Template-Testing comparison

Published: 01 October 2023 Publication History

Abstract

Surface defect detection is an important task in the field of manufacturing, and dealing with imbalanced data is a challenge that has been addressed using methods such as anomaly detection and data augmentation. However, optical devices pose a particular challenge due to their characteristics of small batches and varying types, resulting in insufficient positive sample data and difficulty in predicting the data distribution of new batches. To address this issue, we propose a neural network that learns to compare the differences between templates and testing samples, rather than directly learning the representations of the samples. By collecting templates, the model can generalize to new batches. The challenge of extracting defect features by comparison is to remove background noise, such as displacements, deformations, and texture changes. We propose a Dual-Attention Mechanism (DAM) in the stage of feature extraction, which extracts the noise-free defect features using the non-position information of self-attention. In the stage of feature fusion, we introduce a Recurrent Residual Attention Mechanism (RRAM) to generate spatial masks that shield noise and enable multi-scale feature fusion. We evaluate our method on three datasets of Optical Communication Devices (OCDs), Printed Circuit Boards (PCBs) and Motor Commutator Surface Defects (MCSD), and demonstrate that it outperforms existing state-of-the-art methods. Our work provides a promising direction for addressing the challenge of surface defect detection in OCDs and can be generalized to other flexible manufacturing system (FMS).

Highlights

A Generalized Well Neural Network via Template-Testing comparison is proposed.
Dual-Attention and Recurrent Residual mechanisms proposed for noise elimination.
Our approach generalizes to new batches via template collection in FMS.

References

[1]
Andrychowicz M., Denil M., Gómez S., Hoffman M.W., Pfau D., Schaul T., Shillingford B., de Freitas N., Learning to learn by gradient descent by gradient descent, in: Lee D., Sugiyama M., Luxburg U., Guyon I., Garnett R. (Eds.), Advances in Neural Information Processing Systems, Vol. 29, Curran Associates, Inc., 2016, URL https://proceedings.neurips.cc/paper/2016/file/fb87582825f9d28a8d42c5e5e5e8b23d-Paper.pdf.
[2]
Badrinarayanan V., Kendall A., Cipolla R., Segnet: A deep convolutional encoder-decoder architecture for image segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 39 (12) (2017) 2481–2495.
[3]
Bao Y., Song K., Liu J., Wang Y., Yan Y., Yu H., Li X., Triplet-graph reasoning network for few-shot metal generic surface defect segmentation, IEEE Trans. Instrum. Meas. 70 (2021) 1–11.
[4]
Bergmann P., Batzner K., Fauser M., Sattlegger D., Steger C., The MVTec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection, Int. J. Comput. Vis. 129 (4) (2021) 1038–1059.
[5]
Božič J., Tabernik D., Skočaj D., Mixed supervision for surface-defect detection: From weakly to fully supervised learning, Comput. Ind. 129 (2021).
[6]
Cao H., Wang Y., Chen J., Jiang D., Zhang X., Tian Q., Wang M., Swin-unet: Unet-like pure transformer for medical image segmentation, in: Computer Vision–ECCV 2022 Workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part III, Springer, 2023, pp. 205–218.
[7]
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H., 2018. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision. ECCV, pp. 801–818.
[8]
Dong H., Song K., He Y., Xu J., Yan Y., Meng Q., PGA-net: Pyramid feature fusion and global context attention network for automated surface defect detection, IEEE Trans. Ind. Inform. 16 (12) (2019) 7448–7458.
[9]
Dong H., Song K., Wang Q., Yan Y., Jiang P., Deep metric learning-based for multi-target few-shot pavement distress classification, IEEE Trans. Ind. Inform. 18 (3) (2021) 1801–1810.
[10]
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al., An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In: International Conference on Learning Representations.
[11]
Douze, M., Szlam, A., Hariharan, B., Jégou, H., 2018. Low-shot learning with large-scale diffusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3349–3358.
[12]
Finn C., Abbeel P., Levine S., Model-agnostic meta-learning for fast adaptation of deep networks, in: Precup D., Teh Y.W. (Eds.), Proceedings of the 34th International Conference on Machine Learning, in: Proceedings of Machine Learning Research, vol. 70, PMLR, 2017, pp. 1126–1135. URL https://proceedings.mlr.press/v70/finn17a.html.
[13]
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., Lu, H., 2019. Dual attention network for scene segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3146–3154.
[14]
Gao H., Shou Z., Zareian A., Zhang H., Chang S.-F., Low-shot learning via covariance-preserving adversarial augmentation networks, Adv. Neural Inf. Process. Syst. 31 (2018).
[15]
Gao L., Zhang J., Yang C., Zhou Y., Cas-VSwin transformer: A variant swin transformer for surface-defect detection, Comput. Ind. 140 (2022).
[16]
Gong, D., Liu, L., Le, V., Saha, B., Mansour, M.R., Venkatesh, S., Hengel, A.v.d., 2019. Memorizing Normality to Detect Anomaly: Memory-Augmented Deep Autoencoder for Unsupervised Anomaly Detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. ICCV.
[17]
Guo M.-H., Liu Z.-N., Mu T.-J., Hu S.-M., Beyond self-attention: External attention using two linear layers for visual tasks, IEEE Trans. Pattern Anal. Mach. Intell. (2022).
[18]
Guo M.-H., Xu T.-X., Liu J.-J., Liu Z.-N., Jiang P.-T., Mu T.-J., Zhang S.-H., Martin R.R., Cheng M.-M., Hu S.-M., Attention mechanisms in computer vision: A survey, Comput. Vis. Media 8 (3) (2022) 331–368.
[19]
He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 770–778.
[20]
Huang Z., Wang X., Wei Y., Huang L., Shi H., Liu W., Huang T., CCNet: Criss-cross attention for semantic segmentation, IEEE Trans. Pattern Anal. Mach. Intell. (2020).
[21]
Jin Q., Meng Z., Pham T.D., Chen Q., Wei L., Su R., DUNet: A deformable network for retinal vessel segmentation, Knowl.-Based Syst. 178 (2019) 149–162.
[22]
Koch G., Zemel R., Salakhutdinov R., et al., Siamese neural networks for one-shot image recognition, in: ICML Deep Learning Workshop, Vol. 2, No. 1, Lille, 2015.
[23]
Kwon D., Ahn J., Kim J., Choi I., Jeong S., Lee Y.-S., Park J., Lee M., Siamese U-Net with healthy template for accurate segmentation of intracranial hemorrhage, in: Medical Image Computing and Computer Assisted Intervention–MICCAI 2019: 22nd International Conference, Shenzhen, China, October 13–17, 2019, Proceedings, Part III 22, Springer, 2019, pp. 848–855.
[24]
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P., 2017. Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision. pp. 2980–2988.
[25]
Ling Z., Zhang A., Ma D., Shi Y., Wen H., Deep siamese semantic segmentation network for PCB welding defect detection, IEEE Trans. Instrum. Meas. 71 (2022) 1–11.
[26]
Liu, B., Wang, X., Dixit, M., Kwitt, R., Vasconcelos, N., 2018. Feature space transfer for data augmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 9090–9098.
[27]
Long, J., Shelhamer, E., Darrell, T., 2015. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3431–3440.
[28]
Lu X., Wang W., Shen J., Crandall D., Luo J., Zero-shot video object segmentation with co-attention siamese networks, IEEE Trans. Pattern Anal. Mach. Intell. 44 (4) (2020) 2228–2242.
[29]
Luo X., Li S., Wang Y., Zhan T., Shi X., Liu B., MaMiNet: Memory-attended multi-inference network for surface-defect detection, Comput. Ind. 145 (2023).
[30]
Ma S., Song K., Niu M., Tian H., Wang Y., Yan Y., Shape consistent one-shot unsupervised domain adaptation for rail surface defect segmentation, IEEE Trans. Ind. Inform. (2023).
[31]
Mnih V., Heess N., Graves A., et al., Recurrent models of visual attention, Adv. Neural Inf. Process. Syst. 27 (2014).
[32]
Niu T., Li B., Li W., Qiu Y., Niu S., Positive-sample-based surface defect detection using memory-augmented adversarial autoencoders, IEEE/ASME Trans. Mechatronics 27 (1) (2022) 46–57,.
[33]
Niu S., Li B., Wang X., Peng Y., Region-and strength-controllable GAN for defect generation and segmentation in industrial images, IEEE Trans. Ind. Inform. 18 (7) (2021) 4531–4541.
[34]
Ren X., Lin W., Yang X., Yu X., Gao H., Data augmentation in defect detection of sanitary ceramics in small and Non-i.i.d datasets, IEEE Trans. Neural Netw. Learn. Syst. (2022) 1–10,.
[35]
Ronneberger O., Fischer P., Brox T., U-net: Convolutional networks for biomedical image segmentation, in: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, Springer, 2015, pp. 234–241.
[36]
Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., Gehler, P., 2022b. Towards Total Recall in Industrial Anomaly Detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR, pp. 14318–14328.
[37]
Tao X., Gong X., Zhang X., Yan S., Adak C., Deep learning for unsupervised anomaly localization in industrial images: A survey, IEEE Trans. Instrum. Meas. (2022).
[38]
Tian Z., Zhao H., Shu M., Yang Z., Li R., Jia J., Prior guided feature enrichment network for few-shot segmentation, IEEE Trans. Pattern Anal. Mach. Intell. 44 (2) (2020) 1050–1065.
[39]
Vinyals O., Blundell C., Lillicrap T., kavukcuoglu k., Wierstra D., Matching networks for one shot learning, in: Lee D., Sugiyama M., Luxburg U., Guyon I., Garnett R. (Eds.), Advances in Neural Information Processing Systems, Vol. 29, Curran Associates, Inc., 2016, URL https://proceedings.neurips.cc/paper/2016/file/90e1357833654983612fb05e3ec9148c-Paper.pdf.
[40]
Wang, X., Girshick, R., Gupta, A., He, K., 2018. Non-local neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 7794–7803.
[41]
Wang, F., Jiang, M., Qian, C., Yang, S., Li, C., Zhang, H., Wang, X., Tang, X., 2017. Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 3156–3164.
[42]
Wang Y., Yao Q., Kwok J.T., Ni L.M., Generalizing from a few examples: A survey on few-shot learning, ACM Comput. Surv. (Csur) 53 (3) (2020) 1–34.
[43]
Yun J.P., Shin W.C., Koo G., Kim M.S., Lee C., Lee S.J., Automated defect inspection system for metal surfaces based on deep learning and data augmentation, J. Manuf. Syst. 55 (2020) 317–324,. URL https://www.sciencedirect.com/science/article/pii/S027861252030042X.
[44]
Zhan Z., Zhou J., Xu B., Fabric defect classification using prototypical network of few-shot learning algorithm, Comput. Ind. 138 (2022).
[45]
Zhang, Z., Peng, H., 2019. Deeper and wider siamese networks for real-time visual tracking. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 4591–4600.
[46]
Zhang Y., Yang Q., A survey on multi-task learning, IEEE Trans. Knowl. Data Eng. 34 (12) (2022) 5586–5609,.
[47]
Zhuxi M., Li Y., Huang M., Huang Q., Cheng J., Tang S., A lightweight detector based on attention mechanism for aluminum strip surface defect detection, Comput. Ind. 136 (2022).

Cited By

View all

Index Terms

  1. A generalized well neural network for surface defect segmentation in Optical Communication Devices via Template-Testing comparison
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Please enable JavaScript to view thecomments powered by Disqus.

        Information & Contributors

        Information

        Published In

        cover image Computers in Industry
        Computers in Industry  Volume 151, Issue C
        Oct 2023
        437 pages

        Publisher

        Elsevier Science Publishers B. V.

        Netherlands

        Publication History

        Published: 01 October 2023

        Author Tags

        1. Flexible manufacturing system
        2. Surface defect segmentation
        3. Template-Testing comparison
        4. Attention mechanism
        5. Siamese networks

        Qualifiers

        • Research-article

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)0
        • Downloads (Last 6 weeks)0
        Reflects downloads up to 21 Dec 2024

        Other Metrics

        Citations

        Cited By

        View all

        View Options

        View options

        Media

        Figures

        Other

        Tables

        Share

        Share

        Share this Publication link

        Share on social media