Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

FRAnomaly: flow-based rapid anomaly detection from images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Detecting anomalies, such as defects in newly manufactured products or damage in long-used material structures, is a tedious task for humans. Given the current advances in the areas of artificial intelligence (AI) and computer vision, automation of visual quality control is possible and can be a reliable solution. Current methods that achieve state-of-the-art anomaly detection results often use features of different scales extracted from pre-trained convolutional neural networks (CNN). We propose employing multiple feature sets of the same scale alongside a normalizing flow model designed specifically for such sets. Such input features allow the creation of significantly smaller flow models with faster inference. We managed to achieve a decrease of up to 77.5% in the number of flow model parameters and 63.6% in inference time while still accomplishing results better than all other flow models and all but one non-flow method. Experimental evaluation on publicly available MVTec AD and MTD datasets showed a state-of-the-art level of performance of our models, thus proving that it is not necessary to use different scales to detect anomalies of different sizes. This research paves the way for real-time, efficient AI-based automation of visual inspection.

Graphical abstract

Anomaly detection pipeline of FRAnomaly

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

The datasets used for the development and evaluation of methods in this paper are publicly available:

1. MVTec AD [1]: https://www.mvtec.com/company/research/datasets/mvtec-ad

2. MTD [11]: https://github.com/abin24/Magnetic-tile-defect-datasets

References

  1. Bergmann P, Batzner K, Fauser M et al (2021) The mvtec anomaly detection dataset: a comprehensive real-world dataset for unsupervised anomaly detection. Int J Comput Vis 129(4):1038–1059. https://doi.org/10.1007/s11263-020-01400-4

    Article  Google Scholar 

  2. Chen X, You S, Tezcan KC et al (2020) Unsupervised lesion detection via image restoration with a normative prior. Med Image Anal 64(101):713. https://doi.org/10.1016/j.media.2020.101713

    Article  Google Scholar 

  3. Cordoni F, Bacchiega G, Bondani G et al (2022) A multi-modal unsupervised fault detection system based on power signals and thermal imaging via deep autoencoder neural network. Eng Appl Artif Intell 110(104):729. https://doi.org/10.1016/j.engappai.2022.104729

    Article  Google Scholar 

  4. Defard T, Setkov A, Loesch A et al (2021) Padim: a patch distribution modeling framework for anomaly detection and localization. In: Del Bimbo A, Cucchiara R, Sclaroff S et al (eds) Pattern recognition. ICPR international workshops and challenges. springer international publishing. Cham, pp 475–489, https://doi.org/10.1007/978-3-030-68799-1_35

  5. Deng J, Dong W, Socher R et al (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248–255, https://doi.org/10.1109/CVPR.2009.5206848

  6. Dinh L, Sohl-Dickstein J, Bengio S (2017) Density estimation using real NVP. In: 5th International conference on learning representations. ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net

  7. Gonzalez C, Horrocks T, Wedge D et al (2023) Anomaly detection in fourier transform infrared spectroscopy of geological specimens using variational autoencoders. Ore Geol Rev 158(105):478. https://doi.org/10.1016/j.oregeorev.2023.105478

    Article  Google Scholar 

  8. Goodfellow I, Pouget-Abadie J, Mirza M et al (2020) Generative adversarial networks. Commun ACM 63(11):139–144. https://doi.org/10.1145/3422622

    Article  MathSciNet  Google Scholar 

  9. Gudovskiy D, Ishizaka S, Kozuka K (2022) Cflow-ad: real-time unsupervised anomaly detection with localization via conditional normalizing flows. In: 2022 IEEE/CVF Winter conference on applications of computer vision (WACV), pp 1819–1828, https://doi.org/10.1109/WACV51458.2022.00188

  10. He L, Niu X, Chen T et al (2022) Spatio-temporal trajectory anomaly detection based on common sub-sequence. Appl Intell 52:1–23. https://doi.org/10.1007/s10489-021-02754-z

    Article  Google Scholar 

  11. Huang Y, Qiu C, Yuan K (2020) Surface defect saliency of magnetic tile. Vis Comput 36(1):85–96. https://doi.org/10.1007/s00371-018-1588-5

    Article  MathSciNet  Google Scholar 

  12. Jiang J, Zhu J, Bilal M et al (2023) Masked swin transformer unet for industrial anomaly detection. IEEE Trans Industr Inform 19(2):2200–2209. https://doi.org/10.1109/TII.2022.3199228

    Article  Google Scholar 

  13. Kingma DP, Welling M (2014) Auto-encoding variational bayes. In: Bengio Y, LeCun Y (eds) 2nd International conference on learning representations. ICLR 2014, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings

  14. Kingma DP, Salimans T, Jozefowicz R, et al (2016) Improved variational inference with inverse autoregressive flow. In: Proceedings of the 30th international conference on neural information processing systems. Curran Associates Inc., Red Hook, NY, USA, NIPS’16, p 4743-4751

  15. Kirichenko P, Izmailov P, Wilson AG (2020) Why normalizing flows fail to detect out-of-distribution data. In: Proceedings of the 34th International conference on neural information processing systems. Curran Associates Inc., Red Hook, NY, USA, NIPS’20

  16. Koetsier C, Fiosina J, Gremmel JN et al (2022) Detection of anomalous vehicle trajectories using federated learning. ISPRS Open Journal of Photogrammetry and Remote Sensing 4(100):013. https://doi.org/10.1016/j.ophoto.2022.100013

    Article  Google Scholar 

  17. Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  18. Kwon MS, Moon YG, Lee B et al (2023) Autoencoders with exponential deviation loss for weakly supervised anomaly detection. Pattern Recogn Lett 171:131–137. https://doi.org/10.1016/j.patrec.2023.05.016

    Article  Google Scholar 

  19. Liang Y, Zhang J, Zhao S et al (2023) Omni-frequency channel-selection representations for unsupervised anomaly detection. IEEE Trans Image Process 32:4327–4340. https://doi.org/10.1109/TIP.2023.3293772

    Article  Google Scholar 

  20. Luo G, Xie W, Gao R et al (2023) Unsupervised anomaly detection in brain mri: learning abstract distribution from massive healthy brains. Comput Biol Med 154(106):610. https://doi.org/10.1016/j.compbiomed.2023.106610

    Article  Google Scholar 

  21. Milković F, Filipović B, Subašić M et al (2021) Ultrasound anomaly detection based on variational autoencoders. In: 2021 12th International symposium on image and signal processing and analysis (ISPA), pp 225–229, https://doi.org/10.1109/ISPA52656.2021.9552041

  22. Mohamed AA, Alqahtani F, Shalaby A et al (2022) Texture classification-based feature processing for violence-based anomaly detection in crowded environments. Image and Vis Comput 124(104):488. https://doi.org/10.1016/j.imavis.2022.104488

    Article  Google Scholar 

  23. Murase H, Fukumizu K (2022) Algan: anomaly detection by generating pseudo anomalous data via latent variables. IEEE Access 10:44,259–44,270. https://doi.org/10.1109/ACCESS.2022.3169594

  24. Papamakarios G, Pavlakou T, Murray I (2017) Masked autoregressive flow for density estimation. In: Guyon I, Luxburg UV, Bengio S et al (eds) Advances in neural information processing systems, vol 30. Curran Associates Inc

  25. Pinaya WH, Tudosiu PD, Gray R et al (2022) Unsupervised brain imaging 3d anomaly detection and segmentation with transformers. Med Image Anal 79(102):475. https://doi.org/10.1016/j.media.2022.102475

    Article  Google Scholar 

  26. Posilović L, Medak D, Milković F et al (2022) Deep learning-based anomaly detection from ultrasonic images. Ultrasonics 124(106):737. https://doi.org/10.1016/j.ultras.2022.106737

    Article  Google Scholar 

  27. Rezende D, Mohamed S (2015) Variational inference with normalizing flows. In: Bach F, Blei D (eds) Proceedings of the 32nd international conference on machine learning, proceedings of machine learning research, vol 37. PMLR, Lille, France, pp 1530–1538

  28. Rippel O, Mertens P, König E et al (2021) Gaussian anomaly detection by modeling the distribution of normal data in pretrained deep features. IEEE Trans Instrum Meas 70:1–13. https://doi.org/10.1109/TIM.2021.3098381

    Article  Google Scholar 

  29. Roth K, Pemula L, Zepeda J et al (2022) Towards total recall in industrial anomaly detection. In: 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 14,298–14,308, https://doi.org/10.1109/CVPR52688.2022.01392

  30. Rudolph M, Wandt B, Rosenhahn B (2021) Same same but differnet: semi-supervised defect detection with normalizing flows. In: 2021 IEEE winter conference on applications of computer vision (WACV), pp 1906–1915, https://doi.org/10.1109/WACV48630.2021.00195

  31. Rudolph M, Wehrbein T, Rosenhahn B et al (2022) Fully convolutional cross-scale-flows for image-based defect detection. In: 2022 IEEE/CVF Winter conference on applications of computer vision (WACV), pp 1829–1838, https://doi.org/10.1109/WACV51458.2022.00189

  32. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning internal representations by error propagation. In: Rumelhart DE, Mcclelland JL (eds) Parallel distributed processing: explorations in the microstructure of cognition, vol 1. Foundations. MIT Press, Cambridge, MA, pp 318–362

  33. Sarafijanovic-Djukic N, Davis J (2019) Fast distance-based anomaly detection in images using an inception-like autoencoder. In: Kralj Novak P, Šmuc T, Džeroski S (eds) Discovery science. springer international publishing. Cham, pp 493–508, https://doi.org/10.1007/978-3-030-33778-0_37

  34. Sato J, Suzuki Y, Wataya T et al (2023) Anatomy-aware self-supervised learning for anomaly detection in chest radiographs. iScience 26(7):107,086. https://doi.org/10.1016/j.isci.2023.107086

  35. Shi Y, Yang J, Qi Z (2021) Unsupervised anomaly segmentation via deep feature reconstruction. Neurocomputing 424:9–22. https://doi.org/10.1016/j.neucom.2020.11.018

    Article  Google Scholar 

  36. Tan M, Le Q (2019) EfficientNet: Rethinking model scaling for convolutional neural networks. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of the 36th international conference on machine learning, proceedings of machine learning research, vol 97. PMLR, pp 6105–6114

  37. Tao X, Zhang D, Ma W et al (2022) Unsupervised anomaly detection for surface defects with dual-siamese network. IEEE Trans Ind Inf 18(11):7707–7717. https://doi.org/10.1109/TII.2022.3142326

    Article  Google Scholar 

  38. Wang Y, Yu Z, Zhu L (2022) Intrusion detection for high-speed railways based on unsupervised anomaly detection models. Applied Intelligence pp 1–14. https://doi.org/10.1007/s10489-022-03911-8

  39. Wu P, Harris CA, Salavasidis G et al (2021) Unsupervised anomaly detection for underwater gliders using generative adversarial networks. Eng Appl Artif Intell 104(104):379. https://doi.org/10.1016/j.engappai.2021.104379

    Article  Google Scholar 

  40. Yang J, Lyu M, Qi Z et al (2023) Deep feature inpainting for unsupervised visual anomaly detection. Procedia Comput Sci 221:901–911. https://doi.org/10.1016/j.procs.2023.08.067

    Article  Google Scholar 

  41. Zagoruyko S, Komodakis N (2016) Wide residual networks. In: Richard C. Wilson ERH, Smith WAP (eds) Proceedings of the British machine vision conference (BMVC). BMVA Press, pp 87.1–87.12, https://doi.org/10.5244/C.30.87

  42. Zavrtanik V, Kristan M, Skočaj D (2021) Reconstruction by inpainting for visual anomaly detection. Pattern Recognit 112(107):706. https://doi.org/10.1016/j.patcog.2020.107706

    Article  Google Scholar 

  43. Zhang X, Zheng Y, Zhao Z et al (2021) Deep learning detection of anomalous patterns from bus trajectories for traffic insight analysis. Knowl-Based Syst 217(106):833. https://doi.org/10.1016/j.knosys.2021.106833

    Article  Google Scholar 

  44. Zhang Z, Deng X (2021) Anomaly detection using improved deep svdd model with data structure preservation. Pattern Recognit Lett 148:1–6. https://doi.org/10.1016/j.patrec.2021.04.020

    Article  Google Scholar 

  45. Zhou Y, Liang X, Zhang W et al (2021) Vae-based deep svdd for anomaly detection. Neurocomputing 453:131–140. https://doi.org/10.1016/j.neucom.2021.04.089

Download references

Funding

This research was co-funded by the European Union through the European Regional Development Fund, under the grant KK.01.2.1.01.0151 (Smart UTX).

Author information

Authors and Affiliations

  1. Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, Zagreb, 10000, Croatia

    Fran Milković, Luka Posilović, Duje Medak, Marko Subašić & Sven Lončarić

  2. INETEC, Dolenica 28, Lučko, 10250, Croatia

    Marko Budimir

Authors
  1. Fran Milković
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luka Posilović
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Duje Medak
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marko Subašić
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Sven Lončarić
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Marko Budimir
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fran Milković.

Ethics declarations

Competing of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Milković, F., Posilović, L., Medak, D. et al. FRAnomaly: flow-based rapid anomaly detection from images. Appl Intell 54, 3502–3515 (2024). https://doi.org/10.1007/s10489-024-05332-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05332-1

Keywords

Search

Navigation