Abstract
Several salient object detection approaches have been published which have been assessed using different evaluation scores and datasets resulting in discrepancy in model comparison. This calls for a methodological framework to compare existing models and evaluate their pros and cons. We analyze benchmark datasets and scoring techniques and, for the first time, provide a quantitative comparison of 35 state-of-the-art saliency detection models. We find that some models perform consistently better than the others. Saliency models that intend to predict eye fixations perform lower on segmentation datasets compared to salient object detection algorithms. Further, we propose combined models which show that integration of the few best models outperforms all models over other datasets. By analyzing the consistency among the best models and among humans for each scene, we identify the scenes where models or humans fail to detect the most salient object. We highlight the current issues and propose future research directions.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Judd, T., Ehinger, K., Durand, F.: Learning to predict where humans look. In: ICCV (2009)
Itti, L., Koch, C., Niebur, E.: A model of saliency-based visual attention for rapid scene analysis. IEEE Trans. PAMI 20(11), 1254–1259 (1998)
Harel, J., Koch, C., Perona, P.: Graph-based visual saliency. In: NIPS (2006)
Bruce, N.D.B., Tsotsos, J.K.: Saliency based on information maximization. In: NIPS (2005)
Hou, X., Zhang, L.: Saliency detection: A spectral residual approach. In: CVPR (2007)
Hou, X., Zhang, L.: Dynamic attention: Searching for coding length increments. In: NIPS (2008)
Garcia-Diaz, A., Fdez-Vidal, X.R., Pardo, X.M., Dosil, R.: Decorrelation and Distinctiveness Provide with Human-Like Saliency. In: Blanc-Talon, J., Philips, W., Popescu, D., Scheunders, P. (eds.) ACIVS 2009. LNCS, vol. 5807, pp. 343–354. Springer, Heidelberg (2009)
Seo, H.J., Milanfar, P.: Static and space-time visual saliency detection by self-resemblance. Journal of Vision 9, 1–27 (2009)
Walther, D., Koch, C.: Modeling attention to salient proto-objects. Neural Net. (2006)
Zhang, L., Tong, M.H., Marks, T.K., Shan, H., Cottrell, G.W.: SUN: A Bayesian framework for saliency using natural statistics. JOV (2008)
Tatler, B.W.: The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor bases and image feature distributions. J. Vision 14(7) (2007)
Zhai, Y., Shah, M.: Visual attention detection in video sequences using spatiotemporal cues. ACM Multimedia (2006)
Cheng, M.M., Zhang, G.X., Mitra, N.J., Huang, X., Hu, S.M.: Global contrast based salient region detection. In: CVPR (2011)
Achanta, R., Estrada, F.J., Wils, P., Süsstrunk, S.: Salient Region Detection and Segmentation. In: Gasteratos, A., Vincze, M., Tsotsos, J.K. (eds.) ICVS 2008. LNCS, vol. 5008, pp. 66–75. Springer, Heidelberg (2008)
Goferman, S., Zelnik-Manor, L., Tal, A.: Context-aware saliency detection. In: CVPR (2010)
Achanta, R., Hemami, S., Estrada, F., Süsstrunk, S.: Frequency-tuned salient region detection. In: CVPR (2009)
Wang, J., Sun, J., Quan, L., Tang, X., Shum, H.Y.: Picture collage. In: CVPR (2006)
Wang, M., Konrad, J., Ishwar, P., Jing, Y., Rowley, H.: Image saliency: from intrinsic to extrinsic context. In: CVPR (2011)
Rosin, P.L.: A simple method for detecting salient regions. Pattern Rec. (2009)
Goferman, S., Tal, A., Zelnik-Manor, L.: Puzzle-like collage. In: EuroGraphics (2010)
Zhang, W., Wu, Q.M.J., Wang, G., Yin, H.: An adaptive computational model for salient object detection. IEEE Trans. on Multimedia 12(4) (2010)
Feng, J., Wei, Y., Tao, L., Zhang, C., Sun, J.: Salient object detection by composition. In: ICCV (2011)
Mehrani, P., Veksler, O.: Saliency segmentation based on learning and graph cut. In: BMVC (2010)
Lu, Y., Zhang, W., Lu, H., Xue, X.: Salient object detection using concavity context. In: ICCV (2011)
Klein, D.A., Frintrop, S.: Center-surround divergence of feature statistics for salient object detection. In: ICCV (2011)
Wang, L., Xue, J., Zheng, N., Hua, G.: Automatic Salient object extraction with contextual cue. In: ICCV (2011)
Chang, K.Y., Liu, T.L., Chen, H.T., Lai, S.H.: Fusing generic objectness and visual saliency for salient object detection. In: ICCV (2011)
Khuwuthyakorn, P., Robles-Kelly, A., Zhou, J.: Object of Interest Detection by Saliency Learning. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 636–649. Springer, Heidelberg (2010)
Rahtu, E., Kannala, J., Salo, M., Heikkilä, J.: Segmenting Salient Objects from Images and Videos. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 366–379. Springer, Heidelberg (2010)
Kanan, C., Cottrell, G.: Robust classification of objects, faces, and flowers using natural image. In: CVPR (2010)
Jiang, H., Wang, J., Yuan, Z., Liu, T., Zheng, N., Li, S.: Automatic salient object segmentation based on context and shape prior. In: BMVC (2011)
Li, J., Tian, Y., Huang, T., Gao, W.: A dataset and evaluation methodology for visual saliency in video. In: Int. Conf. on Multimedia and Expo., pp. 442–445 (2009)
Liu, T., Sun, J., Zheng, N., Tang, X., Shum, H.: Learning to detect a salient object. In: CVPR (2007)
Gopalakrishnan, V., Hu, Y., Rajan, D.: Random walks on graphs to model saliency in images. In: CVPR (2009)
Marchesotti, L., Cifarelli, C., Csurka, G.: A framework for visual saliency detection with applications to image thumbnailing. In: ICCV (2009)
Li, J., Levine, M.D., An, X., He, H.: Saliency detection based on frequency and spatial domain analysis. In: BMVC (2011)
Holtzman-Gazit, M., Zelnik-Manor, L., Yavneh, I.: Salient edges: A multi scale approach. In: ECCV, Workshop on Vision for Cognitive Tasks (2010)
Luo, Y., Yuan, J., Xue, P., Tian, Q.: Saliency Density Maximization for Object Detection and Localization. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 396–408. Springer, Heidelberg (2011)
Deng, Q., Luo, Y.: Edge-based method for detecting salient objects. Opt. Eng. 50 (2011)
Valenti, R., Sebe, N., Gevers, T.: Image saliency by isocentric curvedness and color. In: ICCV (2009)
Zhang, Q., Liu, H., Shen, J., Gu, G., Xiao, H.: An improved computational approach for salient region detection. Journal of Computers (2010)
Li, H., Ngan, K.N.: A co-saliency model of image pairs. IEEE Trans. Image Process (2011)
Ge, F., Wang, S.: New benchmark for image segmentation evaluation. Journal of Electronic Imaging 16(3) (2007)
Estrada, F.J., Jepson, A.D.: Benchmarking image segmentation algorithms. IJCV (2009)
Ancuti, C.O., Ancuti, C., Bekaert, P.: CVPR (2011)
Movahedi, V., Elder, J.H.: Design and perceptual validation of performance measures for salient object segmentation. In: POCV (2010)
Alpert, S., Galun, M., Basri, R., Brandt, A.: Image segmentation by probabilistic bottom-up aggregation and cue integration. In: CVPR (2007)
Ge, F., Wang, S., Liu, T.: Image-segmentation evaluation from the perspective of salient object extraction. In: CVPR (2006)
Mishra, A.K., Aloimonos, Y., Fah, C.L., Kassim, A.: Active visual segmentation. IEEE Trans. PAMI (2011)
Siagian, C., Koch, C.: Salient segmentation using contours and region growing (submitted)
Ma, Y.F., Zhang, H.J.: Contrast-based image attention analysis by using fuzzy growing. ACM Multimedia, 374–381 (2003)
Liu, Z., Xue, Y., Yan, H., Zhang, Z.: Efficient saliency detection based on Gaussian models. IET Image Processing 5(2), 122–131 (2011)
Liu, Z., Xue, Y., Shen, L., Zhang, Z.: Nonparametric saliency detection using kernel density estimation. In: ICIP, pp. 253–256 (2010)
Li, J., Tian, Y., Huang, T., Gao, W.: Probabilistic multi-task learning for visual saliency estimation in video. IJCV 90(2), 150–165 (2010)
Achanta, R., Susstrunk, S.: Saliency detection for content-aware image resizing. In: ICIP (2009)
Guo, C., Zhang, L.: A novel multiresolution spatiotemporal saliency detection model and Its applications in image and video compression. IEEE Trans. on Image Processing (2010)
Huang, T.H., Cheng, K.Y., Chuang, Y.Y.: A collaborative benchmark for region of interest detection algorithms. In: CVPR (2009)
Masciocchi, C.M., Mihalas, S., Parkhurst, D., Niebur, E.: Everyone knows what is interesting: salient locations which should be fixated. Journal of Vision (2009)
Itti, L.: Automatic Foveation for Video Compression using a neurobiological model of visual attention. IEEE Trans. Image Process (2004)
Ma, Y., Hua, X., Lu, L., Zhang, H.: A generic framework of user aattention model and its application in video summarization. IEEE Trans. Multimedia (2005)
Borji, A., Itti, L.: State-of-the-art in visual attention modeling. IEEE Trans. Pattern Anal. Mach. Intell (2012)
Gao, D., Vasconcelos, N.: Discriminant saliency for visual recognition from cluttered scenes. In: NIPS (2004)
Kienzle, W., Franz, M.O., Schölkopf, B., Wichmann, F.A.: Center-surround patterns emerge as optimal predictors for human saccade targets. J. Vision (2009)
Judd, T.: Understanding and predicting where people look. Phd Thesis, MIT (2011)
Itti, L., Dhavale, N., Pighin, F.: SPIE (2003)
Koehler, K., Guo, F., Zhang, S., Eckstein, M.: Vision Science Society (2011)
Borji, A., Sihite, D.N., Itti, L.: Quantitative analysis of human-model agreement in visual saliency modeling: a comparative study. IEEE Trans. Image Processing (2012)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Borji, A., Sihite, D.N., Itti, L. (2012). Salient Object Detection: A Benchmark. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds) Computer Vision – ECCV 2012. ECCV 2012. Lecture Notes in Computer Science, vol 7573. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33709-3_30
Download citation
DOI: https://doi.org/10.1007/978-3-642-33709-3_30
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33708-6
Online ISBN: 978-3-642-33709-3
eBook Packages: Computer ScienceComputer Science (R0)