Article

Categorization of natural scenes: local vs. global information

Authors:

Adrian Schwaninger,

Christian Wallraven,

Heinrich H. BülthoffAuthors Info & Claims

APGV '06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization

Pages 33 - 40

https://doi.org/10.1145/1140491.1140498

Published: 28 July 2006 Publication History

Abstract

Understanding the robustness and rapidness of human scene categorization has been a focus of investigation in the cognitive sciences over the last decades. At the same time, progress in the area of image understanding has prompted computer vision researchers to design computational systems that are capable of automatic scene categorization. Despite these efforts, a framework describing the processes underlying human scene categorization that would enable efficient computer vision systems is still missing. In this study, we present both psychophysical and computational experiments that aim to make a further step in this direction by investigating the processing of local and global information in scene categorization. In a set of human experiments, categorization performance is tested when only local or only global image information is present. Our results suggest that humans rely on local, region-based information as much as on global, configural information. In addition, humans seem to integrate both types of information for intact scene categorization. In a set of computational experiments, human performance is compared to two state-of-the-art computer vision approaches that model either local or global information.

References

[1]

Biederman, I. 1972. Perceiving real-world scenes. Science 177, 43, 77--80.

[2]

Chang, C.-C., and Lin, C.-J. 2001. LIBSVM: a library for support vector machines. Software available at: http://www.csie.ntu.edu.tw.

[3]

Fei-Fei, L., and Perona, P. 2005. A bayesian hierarchical model for learning natural scene categories. In IEEE Conf. on Computer Vision and Pattern Recogntion CVPR'05.

Digital Library

[4]

Fei-Fei, L., Van Rullen, R., Koch, C., and Perona, P. 2005. Why does natural scene categorization require little attention? exploring attentional requirements for natural and synthetic stimuli. Visual Cognition 12, 6, 893--924.

[5]

Hayward, W. 2003. After the viewpoint debate: where next in object recognition? Trends in Cognitive Sciences 7, 10, 425--427.

[6]

Henderson, J. 2005. Introduction to real-world scene perception. Visual Cognition: Special Issue on Real-World Scene Perception 12, 849--851.

[7]

Henderson, J., Ed. 2005. Visual Cognition: Special Issue on Real-World Scene Perception, vol. 12.

[8]

Jain, R., Kasturi, R., and Schunck, B. 1995. Machine Vision. McGraw-Hill, Inc.

Digital Library

[9]

McCotter, M., Gosselin, F., Sowden, P., and Schyns, P. 2005. The use of visual information in natural scenes. Visual Cognition 12, 6, 938--953.

[10]

Mojsilovic, A., Gomes, J., and Rogowitz, B. 2004. Semantic-friendly indexing and querying of images based on the extraction of the objective semantic cues. International Journal of Computer Vision 56, 1/2 (January), 79--107.

Digital Library

[11]

Oliva, A., and Torralba, A. 2001. Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision 42, 3 (March), 145--175.

Digital Library

[12]

Oliva, A. 2005. Gist of a scene. In Neurobiology of Attention., L. Itti, G. Rees, and J. Tsotsos, Eds. Academic Press, Elsevier, 251--256.

[13]

Rogowitz, B., Frese, T., Smith, J., Bouman, C., and Kalin, E. 1997. Perceptual image similarity experiments. In SPIE Conference on Human Vision and Electronic Imaging, 576--590.

[14]

Rosch, E., Simpson, C., and Miller, R. 1976. Structural bases of typicality effects. Journal of Experimental Psychology: Human Perception and Performance 2, 491--502.

[15]

Schwaninger, A., Vogel, J., Hofer, F., and Schiele, B. A psychophysically plausible model for typicality ranking of natural scenes. Transactions of Applied Perception. Under revision.

Digital Library

[16]

Schwaninger, A., Lobmaier, J. S., and Collishaw, S. M. 2002. Role of featural and configural information in familiar and unfamiliar face recognition. In 2nd Conference on Biologically Motivated Computer Vision BMCV, Springer, Lecture Notes in Computer Science, 2525, Tübingen, Germany.

Digital Library

[17]

Schwaninger, A., Carbon, C., and Leder, H. 2003. Expert face processing: Specialization and constraints. In Development of face processing, G. Schwarzer and H. Leder, Eds. 81--97.

[18]

Schyns, P., and Oliva, A. 1994. From blobs to boundary edges: evidence for time- and spatial-scale dependent scene recognition. Psychological Science 5, 195--200.

[19]

Szummer, M., and Picard, R. 1998. Indoor-outdoor image classification. In Workshop on Content-based Access of Image and Video Databases.

Digital Library

[20]

Thorpe, S., Fize, D., and Marlot, C. 1996. Speed of processing in the human visual system. Nature 381, 520--522.

[21]

Torralba, A., Murphy, K. P., and Freeman, W. T. 2004. Contextual models for object detection using boosted random fields. Tech. Rep. AIM-2004-008, MIT, AI Lab, April.

[22]

Tversky, B., and Hemenway, K. 1983. Categories of environmental scenes. Cognitive Psychology 15, 121--149.

[23]

Vailaya, A., Figueiredo, M., Jain, A., and Zhang, H. 2001. Image classification for content-based indexing. IEEE Transactions on Image Processing 10, 1 (January), 117--130.

Digital Library

[24]

Vogel, J., and Schiele, B. Semantic modeling of natural scenes for content-based image retrieval. International Journal of Computer Vision. In press.

Digital Library

[25]

Walker Renninger, L., and Malik, J. 2004. When is scene identification just texture recognition? Vision Research 44, 4 (April), 2301--2311.

[26]

Wallraven, C., Schwaninger, A., and Bülthoff, H. Learning from humans: computational modeling of face recognition. Network: Computation in Neural Systems. In press.

[27]

Wichmann, F., Sharpe, L., and Gegenfurtner, K. 2002. The contribution of color to recognition memory for natural scenes. Journal of Experimental Psychology: Learning, Memory and Cognition 28(3), 509--520.

Cited By

Yu YWang XChen LWang YXu G(2023)Cooperative linear regression model for image set classificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120558230:COnline publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120558
Leipzig JBakis YWang XElhamod MDiamond KDahdul WKarpatne AMaga MMabee PBart HGreenberg J(2021)Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish SpeciesMetadata and Semantic Research10.1007/978-3-030-71903-6_1(3-12)Online publication date: 18-Mar-2021
https://doi.org/10.1007/978-3-030-71903-6_1
Borji AItti L(2014)Human vs. Computer in Scene and Object RecognitionProceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition10.1109/CVPR.2014.22(113-120)Online publication date: 23-Jun-2014
https://dl.acm.org/doi/10.1109/CVPR.2014.22
Show More Cited By

Recommendations

Categorization of natural scenes: Local versus global information and the role of color

Categorization of scenes is a fundamental process of human vision that allows us to efficiently and rapidly analyze our surroundings. Several studies have explored the processes underlying human scene categorization, but they have focused on processing ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

APGV '06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization

July 2006

181 pages

ISBN:1595934294

DOI:10.1145/1140491

Conference Chairs:
Roland W. Fleming
MPI for Biological Cybernetics
,
Sunghee Kim
Gettysburg College

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 July 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

APGV06

Sponsor:

SIGGRAPH

APGV06: ACM Symposium on Applied Perception in Graphics and Visualization

July 28 - 29, 2006

Massachusetts, Boston, USA

Acceptance Rates

Overall Acceptance Rate 19 of 33 submissions, 58%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
630
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu YWang XChen LWang YXu G(2023)Cooperative linear regression model for image set classificationExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.120558230:COnline publication date: 15-Nov-2023
https://dl.acm.org/doi/10.1016/j.eswa.2023.120558
Leipzig JBakis YWang XElhamod MDiamond KDahdul WKarpatne AMaga MMabee PBart HGreenberg J(2021)Biodiversity Image Quality Metadata Augments Convolutional Neural Network Classification of Fish SpeciesMetadata and Semantic Research10.1007/978-3-030-71903-6_1(3-12)Online publication date: 18-Mar-2021
https://doi.org/10.1007/978-3-030-71903-6_1
Borji AItti L(2014)Human vs. Computer in Scene and Object RecognitionProceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition10.1109/CVPR.2014.22(113-120)Online publication date: 23-Jun-2014
https://dl.acm.org/doi/10.1109/CVPR.2014.22
Werghi NBnouni NGazzah S(2013)3D face matching using structured ordered facial patterns10th International Multi-Conferences on Systems, Signals & Devices 2013 (SSD13)10.1109/SSD.2013.6564025(1-6)Online publication date: Mar-2013
https://doi.org/10.1109/SSD.2013.6564025
Werghi N(2013)Face-tree: A compact discrete 3D face shape representation2013 1st International Conference on Communications, Signal Processing, and their Applications (ICCSPA)10.1109/ICCSPA.2013.6487271(1-6)Online publication date: Feb-2013
https://doi.org/10.1109/ICCSPA.2013.6487271
Serrano-Talamantes JAviléS-Cruz CVillegas-Cortez JSossa-Azuela J(2013)Self organizing natural scene image retrievalExpert Systems with Applications: An International Journal10.1016/j.eswa.2012.10.06440:7(2398-2409)Online publication date: 1-Jun-2013
https://dl.acm.org/doi/10.1016/j.eswa.2012.10.064
Meng XWang ZWu L(2012)Building global image features for scene recognitionPattern Recognition10.1016/j.patcog.2011.06.01245:1(373-380)Online publication date: 1-Jan-2012
https://dl.acm.org/doi/10.1016/j.patcog.2011.06.012
Huang KTao DYuan YLi XTan T(2011)Biologically Inspired Features for Scene Classification in Video SurveillanceIEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics10.1109/TSMCB.2009.203792341:1(307-313)Online publication date: 1-Feb-2011
https://dl.acm.org/doi/10.1109/TSMCB.2009.2037923
Shi DWang L(2011)An algorithm of scenes description and analysis based on MRF2011 International Conference on Mechatronic Science, Electric Engineering and Computer (MEC)10.1109/MEC.2011.6025760(1513-1516)Online publication date: Aug-2011
https://doi.org/10.1109/MEC.2011.6025760
Parikh D(2011)Recognizing jumbled imagesProceedings of the 2011 International Conference on Computer Vision10.1109/ICCV.2011.6126283(519-526)Online publication date: 6-Nov-2011
https://dl.acm.org/doi/10.1109/ICCV.2011.6126283
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten