tutorial

On detection of novel categories and subcategories of images using incongruence

Authors:

Teofilo de Campos,

Rita CucchiaraAuthors Info & Claims

ICMR '14: Proceedings of International Conference on Multimedia Retrieval

Pages 337 - 344

https://doi.org/10.1145/2578726.2578769

Published: 01 April 2014 Publication History

Abstract

Novelty detection is a crucial task in the development of autonomous vision systems. It aims at detecting if samples do not conform with the learnt models. In this paper, we consider the problem of detecting novelty in object recognition problems in which the set of object classes are grouped to form a semantic hierarchy. We follow the idea that, within a semantic hierarchy, novel samples can be defined as samples whose categorization at a specific level contrasts with the categorization at a more general level. This measure indicates if a sample is novel and, in that case, if it is likely to belong to a novel broad category or to a novel sub-category. We present an evaluation of this approach on two hierarchical subsets of the Caltech256 objects dataset and on the SUN scenes dataset, with different classification schemes. We obtain an improvement over Weinshall et al. and show that it is possible to bypass their normalisation heuristic. We demonstrate that this approach achieves good novelty detection rates as far as the conceptual taxonomy is congruent with the visual hierarchy, but tends to fail if this assumption is not satisfied.

References

[1]

{Almajai et al., 2012} Almajai, I., Yan, F., deCampos, T., Khan, A., Christmas, W., Windridge, D., and Kittler, J. (2012). Anomaly detection and knowledge transfer in automatic sports video annotation. In Studies in Computational Intelligence. Springer.

[2]

{Biederman, 1987} Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Rev., 97:115--147.

[3]

{Chandola et al., 2009} Chandola, V., Banerjee, A., and Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys, 41(3).

Digital Library

[4]

{Chatfield et al., 2011} Chatfield, K., Lempitsky, V., Vedaldi, A., and Zisserman, A. (2011). The devil is in the details: an evaluation of recent feature encoding methods. In British Machine Vision Conference.

[5]

{Deng et al., 2012} Deng, J., Krause, J., Berg, A. C., and Fei-Fei, L. (2012). Hedging your bets: Optimizing accuracy-specificity trade-offs in large scale visual recognition. In IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[6]

{Douze et al., 2011} Douze, M., Ramisa, A., and Schmid, C. (2011). Combining attributes and fisher vectors for efficient image retrieval. In IEEE Conference on Computer Vision and Pattern Recognition, pages 745--752.

Digital Library

[7]

{Duchenne et al., 2011} Duchenne, O., Joulin, A., and Ponce, J. (2011). A graph-matching kernel for object categorization. In IEEE International Conference on Computer Vision.

Digital Library

[8]

{Everingham et al., 2010} Everingham, M., Gool, L. J. V., Williams, C. K. I., Winn, J. M., and Zisserman, A. (2010). The PASCAL visual object classes (VOC) challenge. International Journal of Computer Vision, 88(2).

Digital Library

[9]

{Eysenck and Keane, 2005} Eysenck, M. W. and Keane, M. T. (2005). Cognitive psychology: a student's handbook. Psychology Press.

[10]

{Griffin et al., 2007} Griffin, G., Holub, A. D., and Perona, P. (2007). The Caltech 256 object category data-set. Technical Report CNS-TR-2007-001, California Institute of Technology.

[11]

{Kittler et al., 2014} Kittler, J., Christmas, W., deCampos, T., Windridge, D., Yan, F., Illingworth, J., and Osman, M. (2014). Domain anomaly detection in machine perception: A system architecture and taxonomy. IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]

{Lampert et al., 2009} Lampert, C. H., Nickisch, H., and Harmeling, S. (2009). Learning to detect unseen object classes by betweenclass attribute transfer. In IEEE Conference on Computer Vision and Pattern Recognition.

[13]

{Lazebnik et al., 2006} Lazebnik, S., Schmid, C., and Ponce, J. (2006). Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In IEEE Conference on Computer Vision and Pattern Recognition, pages 2169--2178. IEEE Computer Society.

Digital Library

[14]

{Markou and Singh, 2003a} Markou, M. and Singh, S. (2003a). Novelty detection: A review - part 1: Statistical approaches. Journal of Signal Processing, 83.

Digital Library

[15]

{Markou and Singh, 2003b} Markou, M. and Singh, S. (2003b). Novelty detection: A review - part 2: Neural network based approaches. Journal of Signal Processing, 83.

Digital Library

[16]

{Pauwels and Ambekar, 2011} Pauwels, E. J. and Ambekar, O. (2011). One class classification for anomaly detection: Support vector data description revisited. In Advances in Data Mining. Applications and Theoretical Aspects, volume 6870 of Lecture Notes in Computer Science. Springer.

Digital Library

[17]

{Perronnin and Dance, 2007} Perronnin, F. and Dance, C. (2007). Fisher kernels on visual vocabularies for image categorization. In IEEE Conference on Computer Vision and Pattern Recognition.

[18]

{Perronnin et al., 2010} Perronnin, F., Sanchez, J., and Mensink, T. (2010). Improving the Fisher kernel for large-scale image classification. In IEEE European Conference on Computer Vision.

Digital Library

[19]

{Rodner et al., 2011} Rodner, E., Wacker, E. S., Kemmler, M., and Denzler, J. (2011). One-class classification for anomaly detection in wire ropes with gaussian processes in a few lines of code. In IAPR Conference on Machine Vision Applications.

[20]

{Rohrbach et al., 2011} Rohrbach, M., Stark, M., and Schiele, B. (2011). Evaluating knowledge transfer and zero-shot learning in a large-scale setting. In IEEE Conference on Computer Vision and Pattern Recognition.

Digital Library

[21]

{Schölkopf et al., 2001} Schölkopf, B., Platt, J., Shawe-Taylor, J., Smola, A., and Williamson, R. (2001). Estimating the support of a high-dimensional distribution. Neural Computation, 13(7):1443--1471.

Digital Library

[22]

{Vedaldi and Fulkerson, 2008} Vedaldi, A. and Fulkerson, B. (2008). VLFeat: An open and portable library of computer vision algorithms.

[23]

{Vedaldi and Zisserman, 2012} Vedaldi, A. and Zisserman, A. (2012). Efficient additive kernels via explicit feature maps. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(3).

Digital Library

[24]

{Wang et al., 2012} Wang, G., Hoiem, D., and Forsyth, D. (2012). Learning image similarity from Flickr groups using fast kernel machines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2177--2188.

Digital Library

[25]

{Weinshall et al., 2012} Weinshall, D., Zweig, A., Hermansky, H., Kombrink, S., Ohl, F. W., Anemuller, J., Bach, J. H., Gool, L. V., Nater, F., Pajdla, T., Havlena, M., and Pavel, M. (2012). Beyond novelty detection: Incongruent events, when general and specific classifiers disagree. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(10).

Digital Library

[26]

{Xiao et al., 2010} Xiao, J., Hays, J., Ehinger, K., Oliva, A., and Torralba, A. (2010). Sun database: Large-scale scene recognition from abbey to zoo. In IEEE Conference on Computer Vision and Pattern Recognition, pages 3485--3492.

Cited By

Kittler JZor C(2019)Delta Divergence: A Novel Decision Cognizant Measure of Classifier IncongruenceIEEE Transactions on Cybernetics10.1109/TCYB.2018.282535349:6(2331-2343)Online publication date: Jun-2019
https://doi.org/10.1109/TCYB.2018.2825353
Chen TLu SFan J(2018)S-CNNIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2017.275693640:10(2522-2528)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1109/TPAMI.2017.2756936
Kittler JKaloskampis IZor CXu YHicks YWang W(2018)Intelligent Signal Processing Mechanisms for Nuanced Anomaly Detection in Action Audio-Visual Data Streams2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2018.8461595(6563-6567)Online publication date: 15-Apr-2018
https://dl.acm.org/doi/10.1109/ICASSP.2018.8461595
Show More Cited By

Index Terms

On detection of novel categories and subcategories of images using incongruence
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Machine learning
2. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

Hierarchical Novelty Detection
Advances in Intelligent Data Analysis XVI
Abstract
Hierarchical classification is commonly defined as multi-class classification where the classes are hierarchically nested. Many practical hierarchical classification problems also share features with multi-label classification (i.e., each data ...
Hierarchical Few-Shot Object Detection: Problem, Benchmark and Method
MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Few-shot object detection (FSOD) is to detect objects with a few examples. However, existing FSOD methods do not consider hierarchical fine-grained category structures of objects that exist widely in real life. For example, animals are taxonomically ...
Large margin distribution multi-class supervised novelty detection
Abstract
As one of state-of-the-art supervised novelty detection models, support vector machine-supervised novelty detection (SVM-SND) can recognize whether a test instance is a novelty or which class the test instance comes from if it is a ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICMR '14: Proceedings of International Conference on Multimedia Retrieval

April 2014

564 pages

ISBN:9781450327824

DOI:10.1145/2578726

Conference Chairs:
Mohan Kankanhalli
National University of Singapore
,
Stefan Rueger
The Open University, UK
,
R. Manmatha
A9.com, USA
,
General Chairs:
Joemon Jose
University of Glasgow, UK
,
Keith van Rijsbergen
University of Glasgow, UK

Copyright © 2014 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

In-Cooperation

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2014

Check for updates

Author Tags

Qualifiers

Tutorial
Research
Refereed limited

Conference

ICMR '14

ICMR '14: International Conference on Multimedia Retrieval

April 1 - 4, 2014

Glasgow, United Kingdom

Acceptance Rates

ICMR '14 Paper Acceptance Rate 21 of 111 submissions, 19%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
125
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kittler JZor C(2019)Delta Divergence: A Novel Decision Cognizant Measure of Classifier IncongruenceIEEE Transactions on Cybernetics10.1109/TCYB.2018.282535349:6(2331-2343)Online publication date: Jun-2019
https://doi.org/10.1109/TCYB.2018.2825353
Chen TLu SFan J(2018)S-CNNIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2017.275693640:10(2522-2528)Online publication date: 1-Oct-2018
https://dl.acm.org/doi/10.1109/TPAMI.2017.2756936
Kittler JKaloskampis IZor CXu YHicks YWang W(2018)Intelligent Signal Processing Mechanisms for Nuanced Anomaly Detection in Action Audio-Visual Data Streams2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2018.8461595(6563-6567)Online publication date: 15-Apr-2018
https://dl.acm.org/doi/10.1109/ICASSP.2018.8461595
Ponti MKittler JRiva MCampos TZor C(2017)A decision cognizant KullbackLeibler divergencePattern Recognition10.1016/j.patcog.2016.08.01861:C(470-478)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1016/j.patcog.2016.08.018

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents