Abstract
It is important to detect surface defects for controlling product quality and prolonging equipment life. However, detecting surface defects quickly and accurately is still a great challenge due to the complexity of the environment and surface defects. Aiming at the issue, this paper proposes a two-stage attention-based feature fusion network (TAFFNet) to make full use of each level feature for surface defect segmentation. Specifically, the network uses Resnet50 as the backbone network to obtain features, and then extracts multi-scale feature by the atrous convolution feature extraction module. In order to make the features at all levels contain more defect information, the attention-based adjacent feature fusion module is applied to fuse features with adjacent layers; then use the attention-based high-level feature fusion module to merge features with all upper layers, so all level features not only contain multi-scale context but also obtain more defect details. Finally, all features are cascaded together to achieve accurate segmentation of surface defects. In addition, TAFFNet uses a hybrid loss function to overcome blurry boundaries. The experimental results on three surface defects datasets (SD900, MT, and CFD) show that the proposed network outperforms other 13 methods in terms of PR curve, F1, MAE, and mIoU.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Song, G., Song, K., & Yan, Y. (2020). EDRNet: Encoder-decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement, 69(12), 9709–9719.
Zhou, X., Wang, Y., Zhu, Q., Mao, J., Xiao, C., Lu, X., & Zhang, H. (2020). A surface defect detection framework for glass bottle bottom using visual attention model and wavelet transform. IEEE Transactions on Industrial Informatics, 16(4), 2189–2201.
Su, B., Chen, H., Chen, P., Bian, G., Liu, K., & Liu, W. (2021). Deep learning-based solar-cell manufacturing defect detection with complementary attention network. IEEE Transactions on Industrial Informatics, 17(6), 4084–4095.
Tang, Y., Zhang, A. A., Luo, L., Wang, G., & Yang, E. (2021). Pixel-level pavement crack segmentation with encoder-decoder network. Measurement, 184, 109914.
Nieniewski, M. (2020). Morphological detection and extraction of rail surface defects. IEEE Transactions on Instrumentation and Measurement, 69(9), 6870–6879.
Krummenacher, G., Ong, C. S., Koller, S., Kobayashi, S., & Buhmann, J. M. (2018). Wheel defect detection with machine learning. IEEE Transactions on Intelligent Transportation Systems, 19(4), 1176–1187.
Yapi, D., Allili, M. S., & Baaziz, N. (2018). Automatic fabric defect detection using learning-based local textural distributions in the contourlet domain. IEEE Transactions on Automation Science and Engineering, 15(3), 1014–1026.
Li, W., & Tsai, D. (2011). Defect inspection in low-contrast LCD images using hough transform-based nonstationary line detection. IEEE Transactions on Industrial Informatics, 7(1), 136–147.
Eren, L., Ince, T., & Kiranyaz, S. (2019). A generic intelligent bearing fault diagnosis system using compact adaptive 1D CNN classifier. Journal of Signal Processing Systems, 91(2), 179–189.
Rathore, M. S., & Harsha, S. P. (2022). Degradation pattern of high speed roller bearings using a data-driven deep learning approach. Journal of Signal Processing Systems. https://doi.org/10.1007/s11265-022-01761-8
Yuan, Y., Virupakshappa, K., & Oruklu, E. (2022). FPGA implementation of an ultrasonic flaw detection algorithm based on convolutional neural networks. Journal of Signal Processing Systems. https://doi.org/10.1007/s11265-022-01756-5
Borji, A., Frintrop, S., Sihite, D. N., & Itti, L. (2012). Adaptive object tracking by learning background context. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 23–30.
Abdulmunem, A., Lai, Y.-K., & Sun, X. (2016). Saliency guided local and global descriptors for effective action recognition. Computational Visual Media, 2(1), 97–106.
Zhu, L., Chen, J., Hu, X., Fu, C., Xu, X., Qin, J., & Heng, P. (2020). Aggregating attentional dilated features for salient object detection. IEEE Transactions on Circuits and Systems for Video Technology, 30(10), 3358–3371.
Pang, Y., Zhao, X., Zhang, L., & Lu, H. (2020). Multi-scale interactive network for salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 9410–9419.
Wang, B., Chen, Q., Zhou, M., Zhang, Z., & Gai, K. (2020). Progressive feature polishing network for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 12128–12135.
Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254–1259.
Harel, J., Koch C., & Perona P. (2006). Graph-based visual saliency. In Proceedings Neural Information Processing Systems (NIPS), pp 545–552.
Hou, X., & Zhang, L. (2007). Saliency Detection: A Spectral Residual Approach. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8.
Guo, C., Ma, Q., & Zhang, L.(2008). Spatio-temporal saliency detection using phase spectrum of quaternion fourier transform. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1–8.
Achanta, R., Hemami, S., Estrada, F., & Susstrunk, S. (2009). Frequency-tuned salient region detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1597–1604.
Cheng, M., Zhang, G., Mitra, N. J., Huang, X., & Hu, S. (2011). Global contrast based salient region detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 409–416.
Goferman, S., Zelnik-Manor, L., & Tal, A. (2012). Context-Aware Saliency Detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10), 1915–1926.
Xie, Y., Lu, H., & Yang, M. (2013). Bayesian saliency via low and mid level cues. IEEE Transactions on Image Processing, 22(5), 1689–1698.
Du, S., & Chen, S. (2014). Salient Object Detection via Random Forest. IEEE Signal Processing Letters, 21(1), 51–54.
Peng, H., Li, B., Ling, H., Hu, W., Xiong, W., & Maybank, S. J. (2017). Salient object detection via structured matrix decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(4), 818–832.
Tang, C., Wang, P., Zhang, C., & Li, W. (2017). Salient object detection via weighted low rank Matrix recovery. IEEE Signal Processing Letters, 24(4), 490–494.
Long, J., Shelhamer, E., & Darrell, T. (2015). Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440.
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. In Proceedings Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp 234–241.
Hou, Q., Cheng, M., Hu, X., Borji, A., Tu, Z., & Torr, P. (2017). Deeply supervised salient object detection with short connections. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 5300–5309.
Zhang P., Wang, D., Lu, H., Wang, H., & Ruan, X. (2017). Amulet: Aggregating multi-level convolutional features for salient object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 202–211.
Zhang, X., Wang, T., Qi, J., Lu, H., & Wang, G. (2018) Progressive attention guided recurrent network for salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 714–722.
Wu, Z., Su, L., & Huang, Q. (2019). Cascaded partial decoder for fast and accurate salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3902–3911.
Chen, S., Tan, X., Wang, B., & Hu, X. (2018). Reverse attention for salient object detection. In Proceedings of the European Conference on Computer Vision (ECCV), pp 236–252.
Liu, J., Hou, Q., Cheng, M., Feng, J., & Jiang, J. (2019). A simple pooling-based design for real-time salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 3912–3921.
Wei, J., Wang, S., & Huang, Q. (2020). F3Net: fusion, feedback and focus for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 12321–12328.
Chen, Z., Xu, Q., Cong, R., & Huang, Q. (2020). Global context-aware progressive aggregation network for salient object detection. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pp 10599–10606.
Qin, X., Zhang, Z., Huang, C., Gao, C., Dehghan, M., & Jagersand, M. (2019). BASNet: Boundary-aware salient object detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 7471–7481.
Deng, Z., Hu, X., Zhu, L., Xu, X., Qin, J., Han, G., & Heng, P. (2018). R3net: Recurrent residual refinement network for saliency detection. In Proceedings International Joint Conference on Artificial Intelligence (IJCAI), pp 684–690.
Zhao, J., Liu, J., Fan, D., Cao, Y., Yang, J., & Cheng, M. (2019). EGNet: Edge guidance network for salient object detection. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp 8778–8787.
Woo, S., Park J., Lee, J., & Kweon, I. S. (2018). CBAM: Convolutional block attention module. In Proceedings of the IEEE International Conference on Computer Vision (ECCV), pp 3–19.
Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017). Large kernel matters improve semantic segmentation by global convolutional network. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), pp 1743–1751.
Song, G., Song, K., & Yan, Y. (2020). Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering, 128, 106000.
Huang, Y., Qiu, C., & Yuan, K. (2020). Surface defect saliency of magnetic tile. The Visual Computer, 36(1), 85–96.
Shi, Y., Cui, L., Qi, Z., Meng, F., & Chen, Z. (2016). Automatic road crack detection using random structured forests. IEEE Transactions on Intelligent Transportation Systems, 17(12), 3434–3445.
Qin, X., Zhang, Z., Huang, C., Dehghan, M., & Jagersand, M. (2020). U2-net: Going deeper with nested u-structure for salient object detection. Pattern Recognition, 106, 107404.
Funding
This work is supported by the Fundamental Research Funds for the Central Universities of China (2021MS092).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflicts of Interest
We declare that we have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Cao, J., Yang, G. & Yang, X. TAFFNet: Two-Stage Attention-Based Feature Fusion Network for Surface Defect Detection. J Sign Process Syst 94, 1531–1544 (2022). https://doi.org/10.1007/s11265-022-01801-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-022-01801-3