Abstract
Machine vision-based inspection technologies are gaining considerable importance for automated monitoring and quality control of manufactured products in recent years due to the advent of Industry 4.0. The involvement of advanced deep learning methods is a significant factor contributing to the advent of robust vision-based solutions for improving inspection accuracy at a significantly lower cost in manufacturing industries. The requirement of computational resources and large training datasets hinders the deployment of these solutions to manufacturing shop floors. The present research work develops an image-based framework considering pre-trained Convolutional Neural Network (CNN), ResNet-101 to detect surface defects with the minimum training datasets and computational requirements. The outcomes of the proposed framework are substantiated through a case study of detecting commonly observed surface defects during the centerless grinding of tapered rollers. The image datasets consisting of standard tapered rollers and three common defect classes are captured and enriched further with the help of the data augmentation technique. The present work employs ResNet-101 for feature extraction combined with and multi-class Support Vector Machine (SVM) as a classifier to detect defective images. The effects of the feature extraction layer (fc1000) and pooling layer (pool5) activation are explored to achieve the desired prediction abilities. The testing trials demonstrate that the proposed framework effectively performs image classification, achieving 100% precision for the ‘Good’ class components. The study showed that the proposed approach could overcome the requirements of large training datasets and higher computational power for deep learning models. The proposed system can be of significant importance for Micro, Small, and Medium Enterprises (MSMEs) and Small and Medium-sized Enterprises (SMEs) as an alternative to conventional labor-intensive manual inspection techniques.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badmos, O., Kopp, A., Bernthaler, T., & Schneider, G. (2020). Image-based defect detection in lithium-ion battery electrode using convolutional neural networks. Journal of Intelligent Manufacturing, 31(4), 885–897. https://doi.org/10.1007/s10845-019-01484-x
Bengio, Y. (2012). Practical recommendations for gradient-based training of deep architectures. In Neural networks: Tricks of the trade (pp. 437–478). Springer. https://doi.org/10.1007/978-3-642-35289-8_26
Bhatt, P., Malhan, R. K., Rajendran, P., Shah, B., Thakar, S., Yoon, Y. J., & Gupta, S. K. (2021). Image-based surface defect detection using deep learning: A review. Journal of Computing and Information Science in Engineering. https://doi.org/10.1115/1.4049535
Bianco, S., Cadene, R., Celona, L., & Napoletano, P. (2018). Benchmark analysis of representative deep neural network architectures. IEEE Access, 6, 64270–64277. https://doi.org/10.1109/ACCESS.2018.2877890
Czimmermann, T., Ciuti, G., Milazzo, M., Chiurazzi, M., Roccella, S., Oddo, C. M., & Dario, P. (2020). Visual-based defect detection and classification approaches for industrial applications—A survey. Sensors, 20(5), 1459. https://doi.org/10.3390/s20051459
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009). Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248–255). https://doi.org/10.1109/CVPR.2009.5206848
Fang, X., Luo, Q., Zhou, B., Li, C., & Tian, L. (2020). Research progress of automated visual surface defect detection for industrial metal planar materials. Sensors, 20(18), 5136. https://doi.org/10.3390/s20185136
Fu, G., Sun, P., Zhu, W., Yang, J., Cao, Y., Yang, M. Y., & Cao, Y. (2019). A deep-learning-based approach for fast and robust steel surface defects classification. Optics and Lasers in Engineering, 121, 397–405. https://doi.org/10.1016/j.optlaseng.2019.05.005
Gao, Y., Li, X., Wang, X. V., Wang, L., & Gao, L. (2021). A review on recent advances in vision-based defect recognition towards industrial intelligence. Journal of Manufacturing Systems. https://doi.org/10.1016/j.jmsy.2021.05.008
Gunasekaran, A., Subramanian, N., & Ngai, W. T. E. (2019). Quality management in the 21st century enterprises: Research pathway towards industry 4.0. International Journal of Production Economics, 207, 125–129. https://doi.org/10.1016/j.ijpe.2018.09.005
Hao, R., Lu, B., Cheng, Y., Li, X., & Huang, B. (2020). A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-020-01670-2
Hasan, M. S. (2017). An application of pre-trained CNN for image classification. In 2017 20th international conference of computer and information technology (ICCIT) (pp. 1–6). IEEE. https://doi.org/10.1109/ICCITECHN.2017.8281779
Hashimoto, F., Gallego, I., Oliveira, J. F. G., Barrenetxea, D., Takahashi, M., Sakakibara, K., Stålfelt, H. O., Staadt, G., & Ogawa, K. (2012). Advances in centerless grinding technology. CIRP Annals—Manufacturing Technology, 61(2), 747–770. https://doi.org/10.1016/j.cirp.2012.05.003
He, D., Xu, K., & Zhou, P. (2019). Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers and Industrial Engineering, 128, 290–297. https://doi.org/10.1016/j.cie.2018.12.043
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778). https://doi.org/10.1109/CVPR.2016.90
Hernandez-Garcia, A. (2020). Data augmentation and image understanding. arXiv preprint arXiv:2012.14185
Industrial Machine Vision—Global Market Outlook (2020–2028). (2021). Stratistics MRC. https://www.strategymrc.com/report/industrial-machine-vision-market
Jiang, Q., Tan, D., Li, Y., Ji, S., Cai, C., & Zheng, Q. (2020). Object detection and classification of metal polishing shaft surface defects based on convolutional neural network deep learning. Applied Sciences, 10(1), 87. https://doi.org/10.3390/app10010087
Khan, M. A., Javed, M. Y., Sharif, M., Saba, T., & Rehman, A. (2019). Multi-model deep neural network based features extraction and optimal selection approach for skin lesion classification. In 2019 international conference on computer and information sciences (ICCIS) (pp. 1–7). IEEE. https://doi.org/10.1109/ICCISci.2019.8716400
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84–90. https://doi.org/10.1145/3065386
Kuo, C. F. J., Tsai, C. H., Wang, W. R., & Wu, H. C. (2019). Automatic marking point positioning of printed circuit boards based on template matching technique. Journal of Intelligent Manufacturing, 30(2), 671–685. https://doi.org/10.1007/s10845-016-1274-2
Le, X., Mei, J., Zhang, H., Zhou, B., & Xi, J. (2020). A learning-based approach for surface defect detection using small image datasets. Neurocomputing, 408, 112–120. https://doi.org/10.1016/j.neucom.2019.09.107
Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. In Nature (Vol. 521, Issue 7553, pp. 436–444). Nature Publishing Group. https://doi.org/10.1038/nature14539
Li, B. (2018). Application of machine vision technology in geometric dimension measurement of small parts. EURASIP Journal on Image and Video Processing, 2018(1), 1–8. https://doi.org/10.1186/s13640-018-0364-9
Li, X., Jia, X., Wang, Y., Yang, S., Zhao, H., & Lee, J. (2020). Industrial remaining useful life prediction by partial observation using deep learning with supervised attention. IEEE/ASME Transactions on Mechatronics, 25(5), 2241–2251. https://doi.org/10.1109/TMECH.2020.2992331
Lin, H., Li, B., Wang, X., Shu, Y., & Niu, S. (2019). Automated defect inspection of LED chip using deep convolutional neural network. Journal of Intelligent Manufacturing, 30(6), 2525–2534. https://doi.org/10.1007/s10845-018-1415-x
Liu, J., Guo, F., Gao, H., Li, M., Zhang, Y., & Zhou, H. (2021). Defect detection of injection molding products on small datasets using transfer learning. Journal of Manufacturing Processes, 70, 400–413. https://doi.org/10.1016/j.jmapro.2021.08.034
Ma, G., Yu, L., Yuan, H., Xiao, W., & He, Y. (2021). A vision-based method for lap weld defects monitoring of galvanized steel sheets using convolutional neural network. Journal of Manufacturing Processes, 64, 130–139. https://doi.org/10.1016/j.jmapro.2020.12.067
Matt, D. T., Modrák, V., & Zsifkovits, H. (Eds.). (2020). Industry 4.0 for SMEs: Challenges, opportunities and requirements. Springer Nature. https://doi.org/10.1007/978-3-030-25425-4
Meister, S., Wermes, M. A., Stüve, J., & Groves, R. M. (2021a). Review of image segmentation techniques for layup defect detection in the automated fiber placement process. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01774-3
Meister, S., Wermes, M., Stüve, J., & Groves, R. M. (2021b). Cross-evaluation of a parallel operating SVM–CNN classifier for reliable internal decision-making processes in composite inspection. Journal of Manufacturing Systems, 60, 620–639. https://doi.org/10.1016/j.jmsy.2021.07.022
Moru, D. K., & Borro, D. (2020). A machine vision algorithm for quality control inspection of gears. International Journal of Advanced Manufacturing Technology, 106(1–2), 105–123. https://doi.org/10.1007/s00170-019-04426-2
Neogi, N., Mohanta, D. K., & Dutta, P. K. (2014). Review of vision-based steel surface inspection systems. EURASIP Journal on Image and Video Processing, 2014(1), 1–19. https://doi.org/10.1186/1687-5281-2014-50
Ngan, H. Y., Pang, G. K., & Yung, N. H. (2011). Automated fabric defect detection—A review. Image and Vision Computing, 29(7), 442–458. https://doi.org/10.1016/j.imavis.2011.02.002
Oztemel, E., & Gursev, S. (2020). Literature review of industry 4.0 and related technologies. Journal of Intelligent Manufacturing, 31(1), 127–182. https://doi.org/10.1007/s10845-018-1433-8
Penumuru, D. P., Muthuswamy, S., & Karumbu, P. (2020). Identification and classification of materials using machine vision and machine learning in the context of industry 4.0. Journal of Intelligent Manufacturing, 31(5), 1229–1241. https://doi.org/10.1007/s10845-019-01508-6
Pham, D. T., & Alcock, R. J. (2002). Smart inspection systems: Techniques and applications of intelligent vision. Elsevier. https://doi.org/10.1016/B978-0-12-554157-2.X5000-7
Qi, S., Yang, J., & Zhong, Z. (2020). A review on industrial surface defect detection based on deep learning technology. In 2020 the 3rd international conference on machine learning and machine intelligence (pp. 24–30). https://doi.org/10.1145/3426826.3426832
Rawat, W., & Wang, Z. (2017). Deep convolutional neural networks for image classification: A comprehensive review. Neural Computation, 29(9), 2352–2449. https://doi.org/10.1162/neco_a_00990
Ren, Z., Fang, F., Yan, N., & Wu, Y. (2021). State of the art in defect detection based on machine vision. International Journal of Precision Engineering and Manufacturing-Green Technology. https://doi.org/10.1007/s40684-021-00343-6
Shen, H., Li, S., Gu, D., & Chang, H. (2012). Bearing defect inspection based on machine vision. Measurement, 45(4), 719–733. https://doi.org/10.1016/j.measurement.2011.12.018
Sikder, J., Das, U. K., & Chakma, R. J. (2021). Supervised learning-based cancer detection. International Journal of Advanced Computer Science and Applications (IJACSA). https://doi.org/10.14569/IJACSA.2021.01205101
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Suo, X., Liu, J., Dong, L., Shengfeng, C., Enhui, L., & Ning, C. (2021). A machine vision-based defect detection system for nuclear-fuel rod groove. Journal of Intelligent Manufacturing. https://doi.org/10.1007/s10845-021-01746-7
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9). https://doi.org/10.1109/CVPR.2015.7298594
Tabernik, D., Šela, S., Skvarč, J., & Skočaj, D. (2020). Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing, 31(3), 759–776. https://doi.org/10.1007/s10845-019-01476-x
Varshni, D., Thakral, K., Agarwal, L., Nijhawan, R., & Mittal, A. (2019). Pneumonia detection using CNN based feature extraction. In 2019 IEEE international conference on electrical, computer and communication technologies (ICECCT) (pp. 1–7). IEEE. https://doi.org/10.1109/ICECCT.2019.8869364
Wang, J., Fu, P., & Gao, R. X. (2019). Machine vision intelligence for product defect inspection based on deep learning and Hough transform. Journal of Manufacturing Systems, 51, 52–60. https://doi.org/10.1016/j.jmsy.2019.03.002
Xie, X. (2008). A review of recent advances in surface defect detection using texture analysis techniques. ELCVIA: Electronic Letters on Computer Vision and Image Analysis, 7, 1–22.
Yun, J. P., Shin, W. C., Koo, G., Kim, M. S., Lee, C., & Lee, S. J. (2020). Automated defect inspection system for metal surfaces based on deep learning and data augmentation. Journal of Manufacturing Systems, 55, 317–324. https://doi.org/10.1016/j.jmsy.2020.03.009
Acknowledgements
The authors wish to acknowledge the support of M/s. Kansara Modler Limited, Jodhpur, for providing tapered rollers for the generation of the labeled image dataset. The authors would also like to thank the Ministry of Education (MoE), India, for providing financial support to carry out this research work through the Prime Ministers Research Fellowship (PMRF) Scheme (Ref: PMRF/192002/488).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Singh, S.A., Desai, K.A. Automated surface defect detection framework using machine vision and convolutional neural networks. J Intell Manuf 34, 1995–2011 (2023). https://doi.org/10.1007/s10845-021-01878-w
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-021-01878-w