Abstract
Most previous studies on visual saliency have only focused on static or dynamic 2D scenes. Since the human visual system has evolved predominantly in natural three dimensional environments, it is important to study whether and how depth information influences visual saliency. In this work, we first collect a large human eye fixation database compiled from a pool of 600 2D-vs-3D image pairs viewed by 80 subjects, where the depth information is directly provided by the Kinect camera and the eye tracking data are captured in both 2D and 3D free-viewing experiments. We then analyze the major discrepancies between 2D and 3D human fixation data of the same scenes, which are further abstracted and modeled as novel depth priors. Finally, we evaluate the performances of state-of-the-art saliency detection models over 3D images, and propose solutions to enhance their performances by integrating the depth priors.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. TPAMI (1998)
Toet, A.: Computational versus psychophysical bottom-up image saliency: A comparative evaluation study. TPAMI (2011)
Kinect: http://www.xbox.com/kinect
http://nicolas.burrus.name/index.php/Research/Kinect/Calibration
Ramanathan, S., Katti, H., Sebe, N., Kankanhalli, M., Chua, T.-S.: An Eye Fixation Database for Saliency Detection in Images. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 30–43. Springer, Heidelberg (2010)
Avraham, T., Lindenbaum, M.: Esaliency (extended saliency): Meaningful attention using stochastic image modeling. TPAMI (2010)
Bruce, N., Tsotsos, J.: Saliency based on information maximization. In: NIPS (2006)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS (2006)
Tsotsos, J., Culhane, S., Wai, W., Lai, Y., Davis, N.: Modelling visual attention via selective tuning. Artificial Intelligence (1995)
Wolfe, J., Horowitz, T.: What attributes guide the deployment of visual attention and how do they do it? Neuroscience (2004)
Jansen, L., Onat, S., Konig, P.: Influence of disparity on fixation and saccades in free viewing of natural scenes. JOV (2009)
Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: ICCV (2009)
Cerf, M., Frady, E., Koch, C.: Faces and text attract gaze independent of the task: Experimental data and computer model. JOV (2010)
Quan, H., Schiaatti, L.: Examination of 3d visual attention in stereoscopic video content. SPIE-IST Electronic Imaging (2011)
Nakayama, K., Silverman, G.: Serial and parallel processing of visual feature conjunctions. Nature (1986)
Jang, Y.-M., Ban, S.-W., Lee, M.: Stereo Saliency Map Considering Affective Factors in a Dynamic Environment. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds.) ICONIP 2007, Part II. LNCS, vol. 4985, pp. 1055–1064. Springer, Heidelberg (2008)
Aziz, M., Mertsching, B.: Pre-attentive detection of depth saliency using stereo vision. In: AIPR (2010)
Frintrop, S., Rome, E., Núchter, A., Surmann, H.: A bimodal laser-based attention system. CVIU (2005)
Ouerhani, N., Hugli, H.: Computing visual attention from scene depth. In: ICPR (2000)
Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: SLIC Superpixels. EPFL Technical Report (2010)
Velichkovsky, B., Pomplun, M., Rieser, J., Ritter, H.: Eye-movement-based research paradigms. In: Visual Attention and Cognition (2009)
Ouerhani, N., Wartburg, R., Hugli, H.: Empirical validation of the saliency-based model of visual attention. Electronic Letters on CVIA (2004)
Meur, O., Chevet, J.: Relevance of a feed-forward model of visual attention for goal-oriented and free-viewing tasks. TIP (2010)
Achanta, R., Hemami, S., Estrada, F., Susstrunk, S.: Frequency-tuned salient region detection. In: CVPR (2009)
Hou, X., Zhang, L.: Dynamic visual attention: searching for coding length increments. In: NIPS (2008)
Seo, H., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. JOV (2009)
Lang, C., Liu, G., Yu, J., Yan, S.: Saliency detection by multi-task sparsity pursuit. TIP (2011)
Itti, L., Koch, C.: Feature combination strategies for saliency-based visual attention systems. J. Electronic Imaging (2001)
Ni, B., Wang, G., Moulin, P.: HuDaAct: A color-depth video database for human daily activity recognition. In: ICCV Workshop (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lang, C., Nguyen, T.V., Katti, H., Yadati, K., Kankanhalli, M., Yan, S. (2012). Depth Matters: Influence of Depth Cues on Visual Saliency. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7573. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33709-3_8
Download citation
DOI: https://doi.org/10.1007/978-3-642-33709-3_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33708-6
Online ISBN: 978-3-642-33709-3
eBook Packages: Computer ScienceComputer Science (R0)