article

Novelty detection: a review—part 1: statistical approaches

Authors:

Sameer SinghAuthors Info & Claims

Signal Processing, Volume 83, Issue 12

Pages 2481 - 2497

https://doi.org/10.1016/j.sigpro.2003.07.018

Published: 01 December 2003 Publication History

Abstract

Novelty detection is the identification of new or unknown data or signal that a machine learning system is not aware of during training. Novelty detection is one of the fundamental requirements of a good classification or identification system since sometimes the test data contains information about objects that were not known at the time of training the model. In this paper we provide state-of-the-art review in the area of novelty detection based on statistical approaches. The second part paper details novelty detection using neural networks. As discussed, there are a multitude of applications where novelty detection is extremely important including signal processing, computer vision, pattern recognition, data mining, and robotics.

References

[1]

{1} L.D. Baker, T. Hofmann, A.K. McCallum, Y. Yang, A hierarchical probabilistic model for novelty detection in text, Technical Report, 1999.]]

[2]

{2} V. Barnett, T. Lewis, Outliers in Statistical Data, Wiley, NY, USA, 1994.]]

[3]

{3} J.C. Bezdek, R. Ehrlich, W. Full, FCM: the fuzzy c-means clustering algorithm, Computers and Geosciences, Vol. 10, 1984, pp. 191-203.]]

[4]

{4} C. Bishop, Novelty detection and neural network validation, Proceedings of the IEE Conference on Vision and Image Signal Processing, 1994, pp. 217-222.]]

[5]

{5} T. Brotherton, T. Johnson, G. Chadderdon, Classification and novelty detection using linear models and a class dependent-- elliptical basis function neural network, Proceedings of the IJCNN Conference, Anchorage, May 1998.]]

[6]

{6} C. Campbell, K.P. Bennett, A linear programming approach to novelty detection, in: Advances in NIPS, Vol. 14, MIT Press, Cambridge, MA, USA, 2001.]]

[7]

{7} G.A. Carpenter, M.A. Rubin, W.W. Streilein, ARTMAP-FD: familiarity discrimination applied to radar target recognition, Proceedings of the International Conference on Neural Networks, Vol. III, Houston, TX, 1997, pp. 1459-1464.]]

[8]

{8} C.K. Chow, On optimum recognition error and reject tradeoff, IEEE Trans. Inform. Theory IT-16 (1) (January 1970) 41-46.]]

Digital Library

[9]

{9} L.P. Cordella, C. De Stefano, F. Tortorella, M. Vento, A method for improving classification reliability of multilayer perceptrons, IEEE Trans. Neural Networks 6 (5) (1995) 1140-1147.]]

Digital Library

[10]

{10} T. Cover, P. Hart, Nearest neighbor pattern classification, IEEE Trans. Inform. Theory 13 (1967) 21-27.]]

[11]

{11} D. Dasgupta, S. Forrest, Novelty-detection in time series data using ideas from immunology, Proceedings of the International Conference on Intelligent Systems, Reno, Nevada, 1996.]]

[12]

{12} D. Dasgupta, F.A. Gonzalez, An immunogenetic approach to intrusion detection, Division of Computer Science, University of Memphis, Technical Report CS-01-001, 2001.]]

[13]

{13} D. Dasgupta, N.S. Majumdar, Anomaly detection in multidimensional data using negative selection algorithm, Proceedings of the IEEE Conference on Evolutionary Computation, Hawaii, May 2002, pp. 1039-1044.]]

Digital Library

[14]

{14} D. Dasgupta, F. Nino, A comparison of negative and positive selection algorithms in novel pattern detection, Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics, Vol. 1, Nashville, TN, 2000, pp. 125-130.]]

[15]

{15} M.J. Desforges, P.J. Jacob, J.E. Cooper, Applications of probability density estimation to the detection of abnormal conditions in engineering, Proceedings of the Institute of Mechanical Engineers, Vol. 212, 1998, pp. 687-703.]]

[16]

{16} R.O. Duda, P.E. Hart, D.G. Stork, Pattern Classification, Wiley, NY, USA, 2001.]]

Digital Library

[17]

{17} M. Elad, Y. Hel-Or, R. Keshet, Rejection based classifier for face detection, Pattern Recognition Lett. 23 (2002) 1459-1471.]]

[18]

{18} R.A. Fisher, L.H.C. Tippett, Limiting forms of the frequency distribution of the largest and smallest member of a sample, Proc. Camb. Philos. Soc. 24 (1928) 180-190.]]

[19]

{19} P. Foggia, C. Sansone, F. Tortorella, M. Vento, Multiclassification: reject criteria for the Bayesian combiner, Pattern Recognition 32 (1999) 1435-1447.]]

[20]

{20} S. Forrest, A.S. Perelson, L. Allen, R. Cherukuri, Self-nonself discrimination in a computer, Proceedings of the IEEE Symposium on Research in Security and Privacy, Oakland, CA, USA, May 1994, pp. 202-212.]]

Digital Library

[21]

{21} G. Fumera, F. Roli, G. Giacinto, Reject option with multiple thresholds, Pattern Recognition 33 (2000) 2099-2101.]]

[22]

{22} R.S. Guh, F. Zorriassatine, J.D.T. Tannock, On-line control chart pattern detection and discrimination--a neural network approach, Artificial Intell. Eng. 13 (1999) 413-425.]]

[23]

{23} S.E. Guttormsson, R.J. Marks II, M.A. El-Sharkawi, Elliptical novelty grouping for on-line short-turn detection of excited running rotors, IEEE Trans. Energy Conversion 14(1) (March 1999).]]

[24]

{24} L.K. Hanson, C. Liisberg, P. Salamon, The error-reject tradeoff, Open Systems Inform. Dynamics 4 (1997) 159-184.]]

Digital Library

[25]

{25} L.K. Hanson, S. Sigurdsson, T. Kolenda, F.A. Nielson, U. Kjems, J. Larsen, Modelling text with generalizable Gaussian mixtures, Proceedings of IEEE ICASSP '2000, Vol. 6, Istanbul, Turkey, 2000, pp. 3494-3497.]]

Digital Library

[26]

{26} M.E. Hellman, The nearest neighbour classification with a reject option, IEEE Trans. Systems Sci. Cybernet. 6 (3) (July 1970) 179-185.]]

[27]

{27} S.J. Hickinbotham, J. Austin, Neural networks for novelty detection in airframe strain data, Proceedings of IEEE IJCNN, Como, Italy, 2000.]]

Digital Library

[28]

{28} N. Japkowicz, C. Myers, M. Gluck, A novelty detection approach to classification, Proceedings of the 14th IJCAI, Conference, Montreal, 1995, pp. 518-523.]]

[29]

{29} M.F. Jiang, S.S. Tseng, C.M. Su, Two-phase clustering algorithm for outliers detection, Pattern Recognition Lett. 22 (2001) 691-700.]]

Digital Library

[30]

{30} S.P. King, D.M. King, P. Anuzis, K. Astley, L. Tarassenko, P. Hayton, S. Utete, The use of novelty detection techniques for monitoring high-integrity plant, Proceedings of the 2002 International Conference on Control Applications, Vol. 1, Cancun, Mexico, 2002, pp. 221-226.]]

[31]

{31} E.M. Knorr, R.T. Ng, V. Tucakov, Distance-based outliers: algorithms and applications, VLDB J. 8 (3-4) (2000) 237-253.]]

Digital Library

[32]

{32} M. Lauer, A mixture approach to novelty detection using training data with outliers, in: L. De Raedt, P. Flach (Eds.), Proceedings of the 12th European Conference on Machine Learning, Springer, Freiburg, Germany, 2001, pp. 300-311.]]

Digital Library

[33]

{33} J. Laurikkala, M. Juhola, E. Kentala, Informal identification of outliers in medical data, Intelligent Data Analysis in Medicine and Pharmacology (IDAMAP-2000), Berlin, August 2000.]]

[34]

{34} C. Manikopoulos, S. Papavassiliou, Network intrusion and fault detection: a statistical anomaly approach, IEEE Comm. Mag. 40 (October 2002).]]

Digital Library

[35]

{35} G. Manson, Identifying damage sensitive, environment insensitive features for damage detection, Proceedings of the IES Conference, Swansea, UK, 2002.]]

[36]

{36} G. Manson, G. Pierce, K. Worden, On the long-term stability of normal condition for damage detection in a composite panel, Proceedings of the 4th International Conference on Damage Assessment of Structures, Cardiff, UK, June 2001.]]

[37]

{37} G. Manson, G. Pierce, K. Worden, T. Monnier, P. Guy, K. Atherton, Long term stability of normal condition data for novelty detection, Proceedings of the 7th International Symposium on Smart Structures and Materials, California, USA, 2000.]]

[38]

{38} A. Nairac, T. Corbett-Clark, R. Ripley, N. Townsend, L. Tarassenko, Choosing an appropriate model for novelty detection, Proceedings of the 5th IEEE International Conference on Artificial Neural Networks, Cambridge, 1997, pp. 227-232.]]

[39]

{39} A. Nairac, N. Townsend, R. Carr, S. King, P. Cowley, L. Tarassenko, A system for the analysis of jet engine vibration data, Integrated Comput. Aided Eng. 6 (1999) 53-65.]]

Digital Library

[40]

{40} T. Odin, D. Addison, Novelty detection using neural network technology, Proceedings of the COMADEN Conference, Houston, TX, 2000.]]

[41]

{41} L. Parra, G. Deco, S. Miesbach, Statistical independence and novelty detection with information preserving non-linear maps, Neural Comput. 8 (2) (1995) 260-269.]]

Digital Library

[42]

{42} N.J. Pizzi, R.A. Vivanco, R.L. Somorjai, Evident: a functional magnetic resonance image analysis system, Artif. Intell. Med. 21 (2001) 263-269.]]

Digital Library

[43]

{43} S.J. Roberts, Novelty detection using extreme value statistics, IEE Proc. Vision, Image Signal Process. 146 (3) (1999) 124-129.]]

[44]

{44} S.J. Roberts, Extreme value statistics for novelty detection in biomedical signal processing, Proceedings of the 1st International Conference on Advances in Medical Signal and Information Processing, 2002, pp. 166-172.]]

[45]

{45} S. Roberts, L. Tarassenko, A probabilistic resource allocating network for novelty detection, Neural Comput. 6 (1994) 270-284.]]

Digital Library

[46]

{46} R. Ruotolo, C. Surace, A statistical approach to damage detection through vibration monitoring, Proceedings of the 5th Pan American Congress of Applied Mechanics, Puerto, Rico, 1997.]]

[47]

{47} R. Saunders, J.S. Gero, The importance of being emergent, Proceedings of the Artificial Intelligence in Design, 2000.]]

Digital Library

[48]

{48} G. Scarth, M. McIntyre, B. Wowk, R. Somorjai, Detection of novelty in functional images using fuzzy clustering, Proceedings of the 3rd Meeting ISMRM, Nice, France, 1995, p. 238.]]

[49]

{49} S. Singh, M. Markou, An approach to novelty detection applied to the classification of image regions, IEEE Trans. Knowledge Data Eng., 2003, in press.]]

Digital Library

[50]

{50} C. Spence, L. Parra, P. Sajda, Detection, synthesis and compression in mammographic image analysis with a hierarchical image probability model, IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, MMBIA 2001, Kauai, HI, 2001, pp. 3-10.]]

Digital Library

[51]

{51} C.D. Stefano, C. Sansone, M. Vento, To reject or not to reject: that is the question--an answer in case of neural classifiers, IEEE Trans. Systems, Man Cybernet. Part C 30 (1) (2000) 84-94.]]

Digital Library

[52]

{52} A.O. Tarakanov, V.A. Skormin, Pattern recognition by immunocomputing, Proceedings of the 2002 Congress on Evolutionary Computation, CEC '02., Vol. 1, Honolulu, HI, 2002, pp. 938-943.]]

[53]

{53} L. Tarassenko, Novelty detection for the identification of masses in mammograms, Proceedings of the 4th IEE International Conference on Artificial Neural Networks, Vol. 4, Cambridge, UK, 1995, pp. 442-447.]]

[54]

{54} L. Tarassenko, A. Nairac, N. Townsend, P. Cowley, Novelty detection in jet engines, IEE Colloquium on Condition Monitoring, Imagery, External Structures and Health, Birmingham, UK, 1999, pp. 41-45.]]

[55]

{55} D.M.J. Tax, R.P.W. Duin, Outlier detection using classifier instability, in: Advances in Pattern Recognition, the Joint IAPR International Workshops, Sydney, Australia, 1998, pp. 593-601.]]

Digital Library

[56]

{56} D.M.J. Tax, R.P.W. Duin, Data description in subspaces, International Conference on Pattern Recognition, Vol. 2, Barcelona, 2000.]]

[57]

{57} A. Webb, Statistical Pattern Recognition, Arnold, Paris, 1999.]]

[58]

{58} F. Wei, M. Miller, S.J. Stolfo, L. Wenke, P.K. Chan, Using artificial anomalies to detect unknown and known network intrusions, Proceedings of the IEEE International Conference on Data Mining, ICDM 2001, San Jose, CA, 2001, pp. 123-130.]]

Digital Library

[59]

{59} K. Yamanishi, J. Takeuchi, G. Williams, On-line unsupervised outlier detection using finite mixtures with discounting learning algorithms, Proceedings of the 6th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, USA, August 2000, pp. 320-324.]]

Digital Library

[60]

{60} Y. Yang, X. Liu, A re-examination of text categorization methods, Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, 1999, pp. 42-49.]]

Digital Library

[61]

{61} Y. Yang, T. Pierce, J. Carbonell, A study on retrospective and on-line event detection, Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, Melbourne, Australia, 1998, pp. 28-36.]]

Digital Library

[62]

{62} Y. Yang, J. Zhang, J. Carbonell, C. Jin, Topic-conditioned novelty detection International Conference on Knowledge Discovery and Data Mining, July 2002.]]

Digital Library

[63]

{63} D.Y. Yeung, C. Chow, Parzen window network intrusion detectors, Proceedings of the International Conference on Pattern Recognition, Quebec, Canada, 2002.]]

Digital Library

[64]

{64} D.Y. Yeung, Y. Ding, Host-based intrusion detection using dynamic and static behavioral models, Pattern Recognition 36 (2002) 229-243.]]

Cited By

Wang YSong MCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)UniL: Point Cloud Novelty Detection through Multimodal Pre-trainingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680988(8355-8364)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680988
Jamonnak SGuo JHe WGou LRen L(2024)OW-Adapter: Human-Assisted Open-World Object Detection with a Few ExamplesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332657730:1(694-704)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326577
Benedetti LBoniardi EChiani LGhirri JMastropietro MCappozzo ADenti F(2024)Variational inference for semiparametric Bayesian novelty detection in large datasetsAdvances in Data Analysis and Classification10.1007/s11634-023-00569-z18:3(681-703)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11634-023-00569-z
Show More Cited By

Index Terms

Novelty detection: a review—part 1: statistical approaches

Recommendations

Novelty detection: a review—part 2: neural network based approaches

Novelty detection is the identification of new or unknown data or signal that a machine learning system is not aware of during training. In this paper we focus on neural network-based approaches for novelty detection. Statistical approaches are covered ...
Maximum Likelihood Estimation from Uncertain Data in the Belief Function Framework

We consider the problem of parameter estimation in statistical models in the case where data are uncertain and represented as belief functions. The proposed method is based on the maximization of a generalized likelihood criterion, which can be ...
Type-2 fuzzy Gaussian mixture models

This paper presents a new extension of Gaussian mixture models (GMMs) based on type-2 fuzzy sets (T2 FSs) referred to as T2 FGMMs. The estimated parameters of the GMM may not accurately reflect the underlying distributions of the observations because of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Signal Processing

Signal Processing Volume 83, Issue 12

December 2003

201 pages

ISSN:0165-1684

Issue’s Table of Contents

Publisher

Elsevier North-Holland, Inc.

United States

Publication History

Published: 01 December 2003

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

239
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang YSong MCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)UniL: Point Cloud Novelty Detection through Multimodal Pre-trainingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680988(8355-8364)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680988
Jamonnak SGuo JHe WGou LRen L(2024)OW-Adapter: Human-Assisted Open-World Object Detection with a Few ExamplesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332657730:1(694-704)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326577
Benedetti LBoniardi EChiani LGhirri JMastropietro MCappozzo ADenti F(2024)Variational inference for semiparametric Bayesian novelty detection in large datasetsAdvances in Data Analysis and Classification10.1007/s11634-023-00569-z18:3(681-703)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1007/s11634-023-00569-z
Hendrickx KPerini LVan der Plas DMeert WDavis J(2024)Machine learning with a reject option: a surveyMachine Language10.1007/s10994-024-06534-x113:5(3073-3110)Online publication date: 1-May-2024
https://dl.acm.org/doi/10.1007/s10994-024-06534-x
Agrahari SSrivastava SSingh A(2024)Review on novelty detection in the non-stationary environmentKnowledge and Information Systems10.1007/s10115-023-02018-x66:3(1549-1574)Online publication date: 1-Mar-2024
https://dl.acm.org/doi/10.1007/s10115-023-02018-x
Jin QKerstens KVan de Woestyne I(2024)Convex and nonconvex nonparametric frontier-based classification methods for anomaly detectionOR Spectrum10.1007/s00291-024-00751-546:4(1213-1239)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00291-024-00751-5
Li CFu YZhang RLiang HWang CLi J(2023)An Anomaly Detection Approach Based on Integrated LSTM for IoT Big DataSecurity and Communication Networks10.1155/2023/89039802023Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1155/2023/8903980
Wang HYin hRen QZhang JTang K(2023)A Dynamic Printing Equipment Production Status Anomaly Detection ModelProceedings of the 2023 International Conference on Electronics, Computers and Communication Technology10.1145/3637494.3638732(208-214)Online publication date: 17-Nov-2023
https://dl.acm.org/doi/10.1145/3637494.3638732
Zhou DHe J(2023)Rare Category Analysis for Complex Data: A ReviewACM Computing Surveys10.1145/362652056:5(1-35)Online publication date: 27-Nov-2023
https://dl.acm.org/doi/10.1145/3626520
Li ZZhu YVan Leeuwen M(2023)A Survey on Explainable Anomaly DetectionACM Transactions on Knowledge Discovery from Data10.1145/360933318:1(1-54)Online publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1145/3609333
Show More Cited By

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents