Abstract
Surface defect classification plays a very important role in industrial production and mechanical manufacturing. However, there are currently some challenges hindering its use. The first is the similarity of different defect samples makes classification a difficult task. Second, the lack of defect samples leads to poor accuracies when using deep learning methods. In this paper, we first design a novel backbone network, ResMSNet, which draws on the idea of multi-scale feature extraction for small discriminative regions in defect samples. Then, we introduce few-shot learning for defect classification and propose a Relation-Prototypical network (RPNet), which combines the characteristics of ProtoNet and RelationNet and provides classification by linking the prototypes distances and the nonlinear relation scores. Next, we consider a more realistic scenario where the base dataset for training the model and target defect dataset for applying the model are usually obtained from domains with large differences, called cross-domain few-shot learning. Hence, we further improve RPNet to KD-RPNet inspired by knowledge distillation methods. Through extensive comparative experiments and ablation experiments, we demonstrate that either our ResMSNet or RPNet proves its effectiveness and KD-RPNet outperforms other state-of-the-art approaches for few-shot defect classification.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The dataset generated during the current study is not public because it contains proprietary information obtained by the authors through a license. Information on how to obtain it is available from the corresponding author upon reasonable request.
References
Afrasiyabi, A., Lalonde, J.-F., & Gagné, C. (2020). Associative alignment for few-shot image classification. In European conference on computer vision (pp. 18–35). Springer.
Allen, K., Shelhamer, E., Shin, H., & Tenenbaum, J. (2019). Infinite mixture prototypes for few-shot learning. In International conference on machine learning (pp. 232–241). PMLR.
Bao, Y., Song, K., Liu, J., Wang, Y., Yan, Y., Yu, H., & Li, X. (2021). Triplet-graph reasoning network for few-shot metal generic surface defect segmentation. IEEE Transactions on Instrumentation and Measurement, 70, 1–11.
Bertinetto, L., Henriques, J. F., Torr, P. H., & Vedaldi, A. (2018). Meta-learning with differentiable closed-form solvers. arXiv preprint arXiv:1805.08136
Çelik, H., Dülger, L., & Topalbekiroğlu, M. (2014). Development of a machine vision system: Real-time fabric defect detection and classification with neural networks. The Journal of The Textile Institute, 105(6), 575–585.
Chen, W.-Y., Liu, Y.-C., Kira, Z., Wang, Y.-C. F., & Huang, J.-B. (2019). A closer look at few-shot classification. arXiv preprint arXiv:1904.04232
Fei-Fei, L., Fergus, R., & Perona, P. (2006). One-shot learning of object categories. IEEE transactions on pattern analysis and machine intelligence, 28(4), 594–611.
Finn, C., Abbeel, P., & Levine, S. (2017). Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning (pp. 1126–1135). PMLR.
Fu, Y., Fu, Y., & Jiang, Y.-G. (2021). Meta-fdmixup: Cross-domain few-shot learning guided by labeled target data. arXiv preprint arXiv:2107.11978
Fu, Y., Xie, Y., Fu, Y., Chen, J., & Jiang, Y.-G. (2022). Me-d2n: Multi-expert domain decompositional network for cross-domain few-shot learning. In Proceedings of the 30th ACM international conference on multimedia (pp. 6609–6617).
Gao, S.-H., Cheng, M.-M., Zhao, K., Zhang, X.-Y., Yang, M.-H., & Torr, P. (2019). Res2net: A new multi-scale backbone architecture. IEEE transactions on pattern analysis and machine intelligence, 43(2), 652–662.
Gao, Z., Wu, Y., Jia, Y., & Harandi, M. (2021). Curvature generation in curved spaces for few-shot learning. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 8691–8700).
Ge, W., & Yu, Y. (2017). Borrowing treasures from the wealthy: Deep transfer learning through selective joint fine-tuning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1086–1095).
Ghiasi, G., Lin, T.-Y., & Le, Q. V. (2018). Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems, 31.
Guo, Y., Codella, N. C., Karlinsky, L., Codella, J. V., Smith, J. R., Saenko, K., Rosing, T., & Feris, R. (2020). A broader study of cross-domain few-shot learning. In European conference on computer vision (pp. 124–141). Springer.
Guo, Y., Shi, H., Kumar, A., Grauman, K., Rosing, T., & Feris, R. (2019). Spottune: Transfer learning through adaptive fine-tuning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 4805–4814).
Hu, S. X., Moreno, P. G., Xiao, Y., Shen, X., Obozinski, G., Lawrence, N. D., & Damianou, A. (2020). Empirical bayes transductive meta-learning with synthetic gradients. arXiv preprint arXiv:2004.12696
Hu, Y., & Ma, A. J. (2022). Adversarial feature augmentation for cross-domain few-shot classification. arXiv preprint arXiv:2208.11021
Islam, A., Chen, C.-F.R., Panda, R., Karlinsky, L., Feris, R., & Radke, R. J. (2021). Dynamic distillation network for cross-domain few-shot recognition with unlabeled data. Advances in Neural Information Processing Systems, 34, 3584–3595.
Kaftandjian, V., Zhu, Y. M., Dupuis, O., & Babot, D. (2005). The combined use of the evidence theory and fuzzy logic for improving multimodal nondestructive testing systems. IEEE Transactions on Instrumentation and Measurement, 54(5), 1968–1977.
Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980
Krizhevsky, A., & Hinton, G. (2009). Learning multiple layers of features from tiny images
Laenen, S., & Bertinetto, L. (2021). On episodes, prototypical networks, and few-shot learning. Advances in Neural Information Processing Systems, 34, 24581–24592.
Lake, B., Salakhutdinov, R., Gross, J., & Tenenbaum, J. (2011). One shot learning of simple visual concepts. In Proceedings of the annual meeting of the cognitive science society (Vol. 33).
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015). Human-level concept learning through probabilistic program induction. Science, 350(6266), 1332–1338.
Lee, K., Maji, S., Ravichandran, A., & Soatto, S. (2019) Meta-learning with differentiable convex optimization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 10657–10665).
Li, X., Sun, Q., Liu, Y., Zhou, Q., Zheng, S., Chua, T.-S., & Schiele, B. (2019). Learning to self-train for semi-supervised few-shot classification. Advances in Neural Information Processing Systems, 32.
Liu, Y., Lee, J., Park, M., Kim, S., Yang, E., Hwang, S. J., & Yang, Y. (2018). Learning to propagate labels: Transductive propagation network for few-shot learning. arXiv preprint arXiv:1805.10002
Misra, D. (2019). Mish: A self regularized non-monotonic activation function. arXiv preprint arXiv:1908.08681
Munkhdalai, T., Yuan, X., Mehri, S., & Trischler, A. (2018). Rapid adaptation with conditionally shifted neurons. In International conference on machine learning (pp. 3664–3673). PMLR.
Neogi, N., Mohanta, D. K., & Dutta, P. K. (2014). Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014(1), 1–19.
Oh, J., Yoo, H., Kim, C., & Yun, S.-Y. (2020). Boil: Towards representation change for few-shot learning. In International conference on learning representations.
Qiao, L., Shi, Y., Li, J., Wang, Y., Huang, T., & Tian, Y. (2019). Transductive episodic-wise adaptive metric for few-shot learning. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3603–3612).
Qiao, S., Liu, C., Shen, W., & Yuille, A. L. (2018). Few-shot image recognition by predicting parameters from activations. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7229–7238).
Ravichandran, A., Bhotika, R., & Soatto, S. (2019). Few-shot learning with embedded class models and shot-free meta training. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 331–339).
Ren, M., Triantafillou, E., Ravi, S., Snell, J., Swersky, K., Tenenbaum, J. B., Larochelle, H., & Zemel, R. S. (2018). Meta-learning for semi-supervised few-shot classification. arXiv preprint arXiv:1803.00676
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., & Bernstein, M. (2015). Imagenet large scale visual recognition challenge. International journal of computer vision, 115(3), 211–252.
Sa, L., Yu, C., Chen, Z., Zhao, X., & Yang, Y. (2021). Attention and adaptive bilinear matching network for cross-domain few-shot defect classification of industrial parts. In 2021 International joint conference on neural networks (IJCNN) (pp. 1–8). IEEE.
Saito, K., Kim, D., Sclaroff, S., Darrell, T., & Saenko, K. (2019). Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 8050–8058).
Snell, J., Swersky, K., & Zemel, R. (2017). Prototypical networks for few-shot learning. Advances in Neural Information Processing Systems, 30.
Snell, J., & Zemel, R. (2020). Bayesian few-shot classification with one-vs-each pólya-gamma augmented gaussian processes. In International conference on learning representations.
Song, K., & Yan, Y. (2013). A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science, 285, 858–864.
Sun, J., Lapuschkin, S., Samek, W., Zhao, Y., Cheung, N.-M., & Binder, A. (2021) Explanation-guided training for cross-domain few-shot classification. In 2020 25th international conference on pattern recognition (ICPR) (pp. 7609–7616). IEEE.
Sun, Q., Liu, Y., Chua, T.-S., & Schiele, B. (2019). Meta-transfer learning for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 403–412).
Sung, F., Yang, Y., Zhang, L., Xiang, T., Torr, P. H., & Hospedales, T. M. (2018). Learning to compare: Relation network for few-shot learning. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1199–1208).
Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2020). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3), 759–776.
Tseng, H.-Y., Lee, H.-Y., Huang, J.-B., & Yang, M.-H. (2020). Cross-domain few-shot classification via learned feature-wise transformation. In International conference on learning representations.
Tseng, H.-Y., Lee, H.-Y., Huang, J.-B., & Yang, M.-H. (2020). Cross-domain few-shot classification via learned feature-wise transformation. In International conference on learning representations.
Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-sne. Journal of Machine Learning Research, 9(11).
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra,D. (2016). Matching networks for one shot learning. Advances in Neural Information Processing Systems, 29.
Wang, H., & Deng, Z.-H. (2021). Cross-domain few-shot classification via adversarial task augmentation. arXiv preprint arXiv:2104.14385
Wang, Y., Xu, C., Liu, C., Zhang, L., & Fu, Y. (2020). Instance credibility inference for few-shot learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 12836–12845).
Wang, Z., Miao, Z., Zhen, X., & Qiu, Q. (2021). Learning to learn dense gaussian processes for few-shot learning. Advances in Neural Information Processing Systems, 34, 13230–13241.
Wei, B., Hao, K., Gao, L., & Tang, X.-S. (2020). Bioinspired visual-integrated model for multilabel classification of textile defect images. IEEE Transactions on Cognitive and Developmental Systems, 13(3), 503–513.
Wertheimer, D., Tang, L., & Hariharan, B. (2021). Few-shot classification with feature map reconstruction networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 8012–8021).
Xiao, W., Song, K., Liu, J., & Yan, Y. (2022). Graph embedding and optimal transport for few-shot classification of metal surface defect. IEEE Transactions on Instrumentation and Measurement, 71, 1–10.
Xu, W., Wang, H., & Tu, Z. (2020). Attentional constellation nets for few-shot learning. In International conference on learning representations.
Yi, L., Li, G., & Jiang, M. (2017). An end-to-end steel strip surface defects recognition system based on convolutional neural networks. Steel Research International, 88(2), 1600068.
Yoon, S. W., Seo, J., & Moon, J. (2019). Tapnet: Neural network augmented with task-adaptive projection for few-shot learning. In International conference on machine learning (pp. 7115–7123). PMLR.
Zhang, C., Cai, Y., Lin, G., & Shen, C. (2022). Deepemd: Differentiable earth mover’s distance for few-shot learning. IEEE Transactions on Pattern Analysis and Machine Intelligence.
Zhang, J., Su, H., Zou, W., Gong, X., Zhang, Z., & Shen, F. (2021). Cadn: A weakly supervised learning-based category-aware object detection network for surface defect detection. Pattern Recognition, 109, 107571.
Zhang, R., Che, T., Ghahramani, Z., Bengio, Y., & Song, Y. (2018). Metagan: An adversarial approach to few-shot learning. Advances in Neural Information Processing Systems, 31.
Funding
Funding was provided by National Natural Science Foundation of China (Grant Nos. 61973066 and 61471110), Major Science and Technology Projects of Liaoning (Grant No. 2021JH1/10400049), Fundation of Key Laboratory of Aerospace System Simulation (Grant No. 6142002200301), Fundation of Key Laboratory of Equipment Reliability (Grant No. WD2C20205500306), Open Research Projects of Zhejiang Lab (Grant No. 2019KD0AD01/006), and Major Science and Technology Innovation Engineering Projects of Shandong Province (Grant No. 2019JZZY010128).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we have no financial and personal relationships that can directly or indirectly influence the work submitted.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhao, J., Qian, X., Zhang, Y. et al. A knowledge distillation-based multi-scale relation-prototypical network for cross-domain few-shot defect classification. J Intell Manuf 35, 841–857 (2024). https://doi.org/10.1007/s10845-023-02080-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-023-02080-w