Abstract
In recent years, the industrial sector has evolved towards its fourth revolution. The quality control domain is particularly interested in advanced machine learning for computer vision anomaly detection. Nevertheless, several challenges have to be faced, including imbalanced datasets, the image complexity, and the zero-false-negative (ZFN) constraint to guarantee the high-quality requirement. This paper illustrates a use case for an industrial partner, where Printed Circuit Board Assembly (PCBA) images are first reconstructed with a Vector Quantized Generative Adversarial Network (VQGAN) trained on normal products. Then, several multi-level metrics are extracted on a few normal and abnormal images, highlighting anomalies through reconstruction differences. Finally, a classifier is trained to build a composite anomaly score thanks to the metrics extracted. This three-step approach is performed on the public MVTec-AD datasets and on the partner PCBA dataset, where it achieves a regular accuracy of 94.65% and 87.93% under the ZFN constraint.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Availability of data and materials
All public works and datasets have been cited in reference.
Code availability
Not applicable.
Notes
Some parts of the images have been blurred to guarantee the intellectual property of our industrial partner. The arguments described also apply to the hidden parts, where information can be extrapolated.
References
Abd Al Rahman, M., & Mousavi, A. (2020). A review and analysis of automatic optical inspection and quality monitoring methods in electronics industry. IEEE Access, 8, 183192–183271.
Abdallah, A., Maarof, M. A., & Zainal, A. (2016). Fraud detection system: A survey. Journal of Network and Computer Applications, 68, 90–113.
Akcay, S., Atapour-Abarghouei, A., & Breckon, T. P. (2019). Ganomaly: Semi-supervised anomaly detection via adversarial training. In: Computer Vision—ACCV 2018: 14th Asian conference on computer vision, Perth, Australia, December 2–6, 2018, revised selected papers, Part III 14 (pp. 622–637). Springer: Berlin
Akçay, S., Atapour-Abarghouei, A., & Breckon, T.P. (2019). Skip-ganomaly: Skip connected and adversarially trained encoder-decoder anomaly detection. In: 2019 International Joint Conference on Neural Networks (IJCNN) (pp. 1–8). IEEE.
Babic, M., Farahani, M. A., & Wuest, T. (2021). Image based quality inspection in smart manufacturing systems: A literature review. Procedia CIRP, 103, 262–267.
Bergmann, P., Fauser, M., Sattlegger, D., & Steger, C. (2019). Mvtec ad—A comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9592–9600).
Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(2), 281–305.
Bougaham, A., Bibal, A., Linden, I., & Frenay, B. (2021). Ganodip-gan anomaly detection through intermediate patches: A PCBA manufacturing case. In Third international workshop on learning with imbalanced domains: Theory and applications (pp. 104–117). PMLR.
Chawla, N. V., Bowyer, K. W., Hall, L. O., & Kegelmeyer, W. P. (2002). Smote: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16, 321–357.
Crispin, A., & Rankov, V. (2007). Automated inspection of PCB components using a genetic algorithm template-matching approach. The International Journal of Advanced Manufacturing Technology, 35, 293–300.
Down, M., Czubak, F., Gruska, G., Stahley, S., & Benham, D. (2010). Measurement system analysis. In AIAG Reference Manual: Chrysler Group LLC, Ford Motor Company, and General Motors Corporation, Southfield (pp. 103–123).
Eckstein, M. P. (2011). Visual search: A retrospective. Journal of Vision, 11(5), 14–14.
Erfani, S. M., Rajasegarar, S., Karunasekera, S., & Leckie, C. (2016). High-dimensional and large-scale anomaly detection using a linear one-class svm with deep learning. Pattern Recognition, 58, 121–134.
Esser, P., Rombach, R., & Ommer, B. (2021). Taming transformers for high-resolution image synthesis. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873–12883.
Filz, M.-A., Herrmann, C., & Thiede, S. (2020). Simulation-based assessment of quality inspection strategies on manufacturing systems. Procedia CIRP, 93, 777–782.
Han, C., Hayashi, H., Rundo, L., Araki, R., Shimoda, W., Muramatsu, S., Furukawa, Y., Mauri, G., & Nakayama, H. (2018). Gan-based synthetic brain MR image generation. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) (pp. 734–738). IEEE.
Hasoon, J. N., Fadel, A. H., Hameed, R. S., Mostafa, S. A., Khalaf, B. A., Mohammed, M. A., & Nedoma, J. (2021). Covid-19 anomaly detection and classification method based on supervised machine learning of chest x-ray images. Results in Physics, 31, 105045.
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local nash equilibrium. In: Advances in neural information processing systems (Vol. 30).
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A.A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125–1134).
Jia, H., Shi, J., & Chang, T.-S. (2004). An intelligent real-time vision system for surface defect detection. In Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004. (vol. 3, pp. 239–242). IEEE.
Karami, E., Prasad, S., & Shehata, M. (2017). Image matching using sift, surf, brief and orb: Performance comparison for distorted images. arXiv e-prints, 1710.
Kiran, B. R., Thomas, D. M., & Parakkal, R. (2018). An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos. Journal of Imaging, 4(2), 36.
Krawczyk, B., Woźniak, M., & Schaefer, G. (2014). Cost-sensitive decision tree ensembles for effective imbalanced classification. Applied Soft Computing, 14, 554–562.
Li, H., & Li, Y. (2022). Anomaly detection methods based on GAN: A survey. Applied Intelligence. https://doi.org/10.1007/s10489-022-03905-6
Liu, J., Song, K., Feng, M., Yan, Y., Tu, Z., & Zhu, L. (2021). Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection. Optics and Lasers in Engineering, 136, 106324.
Matteoli, S., Diani, M., & Corsini, G. (2010). A tutorial overview of anomaly detection in hyperspectral images. IEEE Aerospace and Electronic Systems Magazine, 25(7), 5–28.
Razavi, A., Van den Oord, A., & Vinyals, O. (2019). Generating diverse high-fidelity images with VQ-VAE-2. In Advances in neural information processing systems (Vol. 32)
Ren, Z., Fang, F., Yan, N., & Wu, Y. (2022). State of the art in defect detection based on machine vision. International Journal of Precision Engineering and Manufacturing-Green Technology, 9(2), 661–691.
Roth, K., Pemula, L., Zepeda, J., Schölkopf, B., Brox, T., & Gehler, P. (2021). Towards total recall in industrial anomaly detection. arXiv preprint arXiv:2106.08265
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training GANs. In: Advances in neural information processing systems (Vol. 29).
Sangalli, S., Erdil, E., Hötker, A., Donati, O., & Konukoglu, E. (2021). Constrained optimization to train neural networks on critical and under-represented classes. In Advances in neural information processing systems (Vol. 34).
Schlegl, T., Seeböck, P., Waldstein, S. M., Langs, G., & Schmidt-Erfurth, U. (2019). f-AnoGAN: Fast unsupervised anomaly detection with generative adversarial networks. Medical Image Analysis, 54, 30–44.
Schlegl, T., Seeböck, P., Waldstein, S.M., Schmidt-Erfurth, U., & Langs, G. (2017). Unsupervised anomaly detection with generative adversarial networks to guide marker discovery. In: International conference on information processing in medical imaging (pp. 146–157). Springer: Berlin
Sridhar, P., Arivan, S., Akshay, R., & Farhathullah, R. (2022). Anomaly detection using CNN with SVM. In 2022 8th international conference on smart structures and systems (ICSSS) (pp. 1–4). IEEE.
Van Den Oord, A., Vinyals, O., & Kavukcuoglu, K. (2017). Neural discrete representation learning. In Advances in neural information processing systems (Vol. 30).
Vergara-Villegas, O. O., Cruz-Sánchez, V. G., de Jesús Ochoa-Domínguez, H., de Jesús Nandayapa-Alfaro, M., & Flores-Abad, Á. (2014). Automatic product quality inspection using computer vision systems. In Lean manufacturing in the developing world: Methodology, case studies and trends from Latin America (pp. 135–156).
Wang, W.-C., Chen, S.-L., Chen, L.-B., & Chang, W.-J. (2016). A machine vision based automatic optical inspection system for measuring drilling quality of printed circuit boards. IEEE Access, 5, 10817–10833.
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018). Deep learning for smart manufacturing: Methods and applications. Journal of Manufacturing Systems, 48, 144–156.
Xia, X., Pan, X., Li, N., He, X., Ma, L., Zhang, X., & Ding, N. (2022). Gan-based anomaly detection: A review. Neurocomputing, 493, 497–535.
Acknowledgements
The authors thank Jérôme Fink and Géraldin Nanfack for their insightful comments and discussions on this paper.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
AB conceptualized the ideas, designed the algorithm, carried out the experiments and wrote the manuscript. MEA, IL and BF supervised the work, provided critical improvements, helped writing, reviewed and approved the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no competing or conflict of interests.
Ethics approval
The authors declare that this work is original, is not under consideration for publication elsewhere and has not been published previously. The authors approve the manuscript enclosed.
Consent to participate
Not applicable
Consent for publication
The authors consent to the publication of this work.
Additional information
Editors: Nuno Moniz, Paula Branco, Luís Torgo, Nathalie Japkowicz, Michal Wozniak and Shuo Wang.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Bougaham, A., El Adoui, M., Linden, I. et al. Composite score for anomaly detection in imbalanced real-world industrial dataset. Mach Learn 113, 4381–4406 (2024). https://doi.org/10.1007/s10994-023-06415-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10994-023-06415-9