Nothing Special   »   [go: up one dir, main page]

skip to main content
research-article

Markov chain based computational visual attention model that learns from eye tracking data

Published: 01 November 2014 Publication History

Abstract

We use Markov chain to model the visual attention.Our visual attention model is based on low level and high level image features.We use the real eye tracking data to train our visual attention model.We measure performances of attention models by comparing them with human fixations.Our model is more consistency with the attentional deployment of humans. Computational visual attention models are a topic of increasing importance in computer understanding of images. Most existing attention models are based on bottom-up computation that often does not match actual human attention. To address this problem, we propose a novel visual attention model that is learned from actual eye tracking data. We use a Markov chain to model the relationship between the image feature and the saliency, then train a support vector regression (SVR) from true eye tracking data to predict the transition probabilities of the Markov chain. Finally, a saliency map predicting user's attention is obtained from the stationary distribution of this chain. Our experimental evaluations on several benchmark datasets demonstrate that the results of the proposed approach are comparable with or outperform the state-of-art models on prediction of human eye fixations and interest region detection.

References

[1]
G. Boccignone, M. Ferraro, Modelling gaze shift as a constrained random walk, Phys. A - Stat. Mech. Appl., 331 (2004) 207-218.
[2]
C. Brezinski, M. Redivo-Zaglia, The PageRank vector: properties, computation, approximation, and acceleration, SIAM J. Matrix Anal. Appl., 28 (2006) 551-575.
[3]
M. Cerf, E.P. Frady, C. Koch, Faces and text attract gaze independent of the task: experimental data and computer model, J. Vision, 9 (2009) 10.
[4]
C. Chang, C.-J. Lin, Libsvm: a library for support vector machines. Linux J., 2001.
[5]
B.A. Draper, A. Lionelle, Evaluation of selective attention under similarity transformations, Comput. Vis. Image Understanding, 100 (2005) 152-171.
[6]
W. Einhäuser, M. Spain, P. Perona, Objects predict fixations better than early saliency, J. Vision, 8 (2008).
[7]
E. Erdem, A. Erdem, Visual saliency estimation by nonlinearly integrating features using region covariances, J. Vision, 13 (2013) 11.
[8]
M. Everingham, L. Van Gool, C.K.I. Williams, J. Winn, A. Zisserman, n.d., The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results.
[9]
P.F. Felzenszwalb, D.A. Mcallester, D. Ramanan, A discriminatively trained, multiscale, deformable part model, in: Computer Vision and Pattern Recognition, 2008, pp. 1-8, http://dx.doi.org/10.1109/CVPR.2008.4587597.
[10]
S. Frintrop, E. Rome, H.I. Christensen, Computational visual attention systems and their cognitive foundations: a survey, ACM Trans. Appl. Percept., 7 (2010) 1-39.
[11]
H. Fu, Z. Chi, D. Feng, Attention-driven image interpretation with application to image retrieval, Pattern Recognit., 39 (2006) 1604-1621.
[12]
D. Gao, V. Mahadevan, N. Vasconcelos, On the plausibility of the discriminant center-surround hypothesis for visual saliency, J. Vision, 8 (2008).
[13]
H. Greenspan, S. Belongie, R. Goodman, P. Perona, S. Rakshit, C.H. Anderson, Overcomplete steerable pyramid filters and rotation invariance, in: Computer Vision and Pattern Recognition, 1994, pp. 222-228, http://dx.doi.org/10.1109/CVPR.1994.323833.
[14]
J. Harel, C. Koch, P. Perona, Graph-based visual saliency, in: Neural Information Processing Systems, 2006, pp. 545-552.
[15]
L. Itti, C. Koch, E. Niebur, A model of saliency-based visual attention for rapid scene analysis, IEEE Trans. Pattern Anal. Mach. Intell., 20 (1998) 1254-1259.
[16]
T. Judd, K.A. Ehinger, F. Durand, A. Torralba, Learning to predict where humans look, in: International Conference on Computer Vision, 2009, pp. 2106-2113, http://dx.doi.org/10.1109/ICCV.2009.5459462.
[17]
W. Kienzle, F.A. Wichmann, B. Schölkopf, M.O. Franz, A nonparametric approach to bottom-up visual saliency, in: Neural Information Processing Systems, 2006, pp. 689-696.
[18]
Z. Liang, H. Fu, Z. Chi, D.D. Feng, Refining a region based attention model using eye tracking data, in: Image Processing, IEEE International Conference, 2010, pp. 1105-1108, http://dx.doi.org/10.1109/ICIP.2010.5651804.
[19]
H. Liu, I. Heynderickx, Studying the added value of visual attention in objective image quality metrics based on eye movement data, in: Image Processing, IEEE International Conference, 2009, pp. 3097-3100, http://dx.doi.org/10.1109/ICIP.2009.5414466.
[20]
O.L. Meur, P.L. Callet, D. Barba, D. Thoreau, A coherent computational approach to model bottom-up visual attention, IEEE Trans. Pattern Anal. Mach. Intell., 28 (2006) 802-817.
[21]
N. Otsu, A threshold selection method from gray-level histograms, IEEE Trans. Syst. Man Cybernet., 9 (1979) 62-66.
[22]
C.M. Privitera, L.W. Stark, Algorithms for defining visual regions-of-interest: comparison with eye fixations, IEEE Trans. Pattern Anal. Mach. Intell., 22 (2000) 970-982.
[23]
M. Rubinstein, A. Shamir, S. Avidan, Improved seam carving for video retargeting, ACM Trans. Graphics, 27 (2008).
[24]
U. Rutishauser, D. Walther, C. Koch, P. Perona, Is bottom-up attention useful for object recognition? in: Computer Vision and Pattern Recognition, 2004, pp. 37-44, http://dx.doi.org/10.1109/CVPR.2004.1315142.
[25]
J. Sivic, B.C. Russell, A.A. Efros, A. Zisserman, W.T. Freeman, Discovering objects and their localization in images, in: International Conference on Computer Vision, 2005, pp. 370-377, http://dx.doi.org/10.1109/ICCV.2005.77.
[26]
A. Torralba, A. Oliva, M.S. Castelhano, J.M. Henderson, Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search, Psychol. Rev., 113 (2006) 766-786.
[27]
P.A. Viola, M.J. Jones, Robust real-time face detection, Int. J. Comput. Vision, 57 (2004) 137-154.
[28]
D. Walther, C. Koch, Modeling attention to salient proto-objects, Neural Networks, 19 (2006) 1395-1407.
[29]
Z. Wang, L. Lu, A.C. Bovik, Foveation scalable video coding with automatic fixation selection, IEEE Trans. Image Process., 12 (2003) 243-254.
[30]
L. Zhang, M.H. Tong, T.K. Marks, H. Shan, G.W. Cottrell, SUN: a Bayesian framework for saliency using natural statistics, J. Vision, 8 (2008) 32.
[31]
Q. Zhao, C. Koch, Learning saliency-based visual attention: a review, Signal Process., 93 (2013) 1401-1407.

Cited By

View all
  • (2021)Markov chain to analyze web usability of a university website using eye tracking dataStatistical Analysis and Data Mining10.1002/sam.1151214:4(331-341)Online publication date: 10-May-2021
  1. Markov chain based computational visual attention model that learns from eye tracking data

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image Pattern Recognition Letters
      Pattern Recognition Letters  Volume 49, Issue C
      November 2014
      264 pages

      Publisher

      Elsevier Science Inc.

      United States

      Publication History

      Published: 01 November 2014

      Author Tags

      1. Attention model
      2. Eye tracking
      3. Human fixations
      4. Markov chain
      5. Saliency

      Qualifiers

      • Research-article

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)0
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 24 Nov 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2021)Markov chain to analyze web usability of a university website using eye tracking dataStatistical Analysis and Data Mining10.1002/sam.1151214:4(331-341)Online publication date: 10-May-2021

      View Options

      View options

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media