Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Active learning with sampling by joint global-local uncertainty for salient object detection

  • S.I.: Human-in-the-loop Machine Learning and its Applications
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The training of the SOD model relies on abundant annotated data, which needs laborious and expensive manual labeling. The generated pseudo-labels for reducing the annotation of the salient object will inevitably introduce noise, which will degrade the performance of the model and cannot fully represent the ground truth of manual labeling. To address this issue, we propose a novel active sampling strategy for salient object detection. The method is made up of two parts: a prediction module and an active learning module. The prediction module predicts the saliency of the image and provides the saliency prediction map for the active learning module. Then, the active learning module measures the global uncertainty and local uncertainty of the prediction map, aiming to select the most informative samples for the model. The selected samples are manually annotated and added to the training set to retrain the prediction model. Experimental results on DUTS dataset indicate that the amount of data can be reduced by 48.3% with competitive performance compared with the state-of-the-art SOD model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Xuebin Q, Shida H, Xiucheng Y, Masood D, Qiming Q, Jagersand M (2018) Accurate outline extraction of individual building from very high-resolution optical images. IEEE Geosci Remote Sens Lett 15(11):1775–1779

    Article  Google Scholar 

  2. Timor K, Michael B (2001) Saliency, scale and image description. Int J Comput Vis 45(2):83–105

    Article  MATH  Google Scholar 

  3. Xin X, Jinshan T, Xiaoming L, Xiaolong Z (2010) Human behavior understanding for video surveillance: recent advance. In: 2010 IEEE international conference on systems, man and cybernetics. IEEE, pp 3867–3873

  4. Xuebin Q, Shida H, Zichen Z, Masood D, Martin J (2018) Bylabel: a boundary based semi-automatic image annotation tool. In: IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1804–1813

  5. Martin J (1995) Saliency maps and attention selection in scale and spatial coordinates: An information theoretic approach. In: Proceedings of IEEE international conference on computer vision. IEEE, pp 195–202

  6. Roey M, Eli S, Lihi Z-M (2019) Saliency driven image manipulation. Mach Vis Appl 30(2):189–202

    Article  Google Scholar 

  7. Hyemin L, Daijin K (2018) Salient region-based online object tracking. In: IEEE winter conference on applications of computer vision (WACV). IEEE, pp 1170–1177

  8. Xuebin Q, Shida H, Camilo Perez Q, Abhineet S, Masood D, Martin J (2017) Real-time salient closed boundary tracking via line segments perceptual grouping. In: IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 4284–4289

  9. Prakhar G, Shubh G, Ajaykrishnan J, Sourav P, Ritwik S (2018) Saliency prediction for mobile user interfaces. In: IEEE Winter conference on applications of computer vision (WACV). IEEE, pp 1529–1538

  10. Lijun W, Huchuan L, Yifan W, Mengyang F, Dong W, Baocai Y, Xiang R (2017) Learning to detect salient objects with image-level supervision. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 136–145

  11. Tie L, Zejian Y, Jian S, Jingdong W, Nanning Z, Xiaoou T, Heung-Yeung S (2010) Learning to detect a salient object. IEEE Trans Pattern Anal Mach Intell 33(2):353–367

    Google Scholar 

  12. Qibin H, Ming-Ming C, Xiaowei H, Ali B, Zhuowen T, Philip HST (2017) Deeply supervised salient object detection with short connections. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3203–3212

  13. Xin X, Nan M, Xiaolong Z, Bo L (2016) Covariance descriptor based convolution neural network for saliency computation in low contrast images. In: 2016 international joint conference on neural networks (IJCNN). IEEE, pp 616–623

  14. Linzhao W, Lijun W, Huchuan L, Pingping Z, Xiang R (2016) Saliency detection with recurrent fully convolutional networks. In: European conference on computer vision. Springer, pp 825–841

  15. Nan M, Xin X, Xiaolong Z, Hong Z (2018) Salient object detection using a covariance-based cnn model in low-contrast images. Neural Comput Appl 29(8):181–192

    Article  Google Scholar 

  16. Kuang-Jui H, Yen-Yu L, Yung-Yu C (2019) Weakly supervised salient object detection by learning a classifier-driven map generator. IEEE Trans Image Process 28(11):5435–5449

    Article  MathSciNet  MATH  Google Scholar 

  17. Yu Z, Yunzhi Z, Huchuan L, Lihe Z, Mingyang Q, Yizhou Y (2019) Multi-source weak supervision for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 6074–6083

  18. Shuwei H, Yuan Z, Sun-Yuan K (2017) Semi-supervised saliency classifier based on a linear feedback control system model. In: International joint conference on neural networks (IJCNN). IEEE, pp 3130–3137

  19. Duc Tam N, Maximilian D, Chaithanya Kumar M, Thi Phuong Nhung N, Thi Hoai Phuong N, Zhongyu L, Thomas B (2019) Deepusps: deep robust unsupervised saliency prediction with self-supervision. arXiv preprint arXiv:1909.13055

  20. Lihe Z, Jiayu S, Tiantian W, Yifan M, Huchuan L (2019) Visual saliency detection via kernelized subspace ranking with active learning. IEEE Trans Image Process 29:2258–2270

    MATH  Google Scholar 

  21. Wenguan W, Qiuxia L, Huazhu F, Jianbing S, Haibin L, Ruigang Y (2019) Salient object detection in the deep learning era: an in-depth survey. arXiv preprint arXiv:1904.09146

  22. Xin X, Lei L, Xiaolong Z, Weili G, Ruimin H (2021) Rethinking data collection for person re-identification: active redundancy reduction. Pattern Recognition, p 107827

  23. Shuai S, Zeming L, Tianyuan Z, Chao P, Gang Y, Xiangyu Z, Jing L, Jian S (2019) Objects365: a large-scale, high-quality dataset for object detection. In: Proceedings of the IEEE/CVF international conference on computer vision. pp 8430–8439

  24. Zhuolin J, Larry SD (2013) Submodular salient region detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2043–2050

  25. Chuan Y, Lihe Z, Huchuan L, Xiang R, Ming-Hsuan Y (2013) Saliency detection via graph-based manifold ranking. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3166–3173

  26. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  27. Karen S, Andrew Z (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  28. Xin X, Shiqin W, Zheng W, Xiaolong Z, Ruimin H (2021) Exploring image enhancement for salient object detection in low light images. ACM Trans Multimedia Comput Commun Appl 17(1):1–19

    Article  Google Scholar 

  29. Guanbin L, Yizhou Y (2015) Visual saliency based on multiscale deep features. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 5455–5463

  30. Nian L, Junwei H, Dingwen Z, Shifeng W, Tianming L (2015) Predicting eye fixations using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 362–370

  31. Rui Z, Wanli O, Hongsheng L, Xiaogang W (2015) Saliency detection by multi-context deep learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1265–1274

  32. Hisham C, Jubin J, Deepu R (2016) Backtracking scspm image classifier for weakly supervised top-down saliency. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5278–5287

  33. Qiong Y, Li X, Jianping S, Jiaya J (2013) Hierarchical saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1155–1162

  34. Shuhan C, Xiuli T, Ben W, Xuelong H (2018) Reverse attention for salient object detection. In: Proceedings of the European conference on computer vision (ECCV). pp 234–250

  35. Zijun D, Xiaowei H, Lei Z, Xuemiao X, Jing Q, Guoqiang H, Pheng-Ann H (2018) R3net: Recurrent residual refinement network for saliency detection. In: Proceedings of the 27th international joint conference on artificial intelligence. AAAI Press, pp 684–690

  36. Nian L, Junwei H, Ming-Hsuan Y (2018) Picanet: Learning pixel-wise contextual attention for saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3089–3098

  37. Olaf R, Philipp F, Thomas B (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 234–241

  38. Runmin W, Mengyang F, Wenlong G, Dong W, Huchuan L, Errui D (2019) A mutual learning method for salient object detection with intertwined multi-supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 8150–8159

  39. Linzhao W, Lijun W, Huchuan L, Pingping Z, Xiang R (2018) Salient object detection with recurrent fully convolutional networks. IEEE Trans Pattern Anal Mach Intell 41(7):1734–1746

    Google Scholar 

  40. Ting Z, Xiangqian W (2019) Pyramid feature attention network for saliency detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 3085–3094

  41. Xuebin Q, Zichen Z, Chenyang H, Chao G, Masood D, Martin J (2019) Basnet: Boundary-aware salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 7479–7489

  42. Jun W, Shuhui W, Zhe W, Chi S, Qingming H, Qi T (2020) Label decoupling framework for salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp 13025–13034

  43. Chunshui C, Yongzhen H, Zilei W, Liang W, Ninglong X, Tieniu T (2018) Lateral inhibition-inspired convolutional neural network for visual attention and saliency detection. In: Proceedings of the AAAI conference on artificial intelligence. vol 32

  44. Guanbin L, Yuan X, Liang L (2018) Weakly supervised salient object detection using image labels. In: Proceedings of the AAAI conference on artificial intelligence. vol 32

  45. Dingwen Z, Junwei H, Yu Z (2017) Supervision by fusion: towards unsupervised learning of deep salient object detector. In: Proceedings of the IEEE international conference on computer vision. pp 4048–4056

  46. Jing Z, Tong Z, Yuchao D, Mehrtash H, Richard H (2018) Deep unsupervised saliency detection: a multiple noisy labeling perspective. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 9029–9038

  47. Xin L, Fan Y, Hong C, Wei L, Dinggang S (2018) Contour knowledge transfer for salient object detection. In: Proceedings of the European conference on computer vision (ECCV). pp 355–370

  48. Jianming Z, Stan S, Zhe L, Xiaohui S, Brian P, Radomir M (2015) Minimum barrier salient object detection at 80 fps. In: Proceedings of the IEEE international conference on computer vision. pp 1404–1412

  49. Jianming Z, Stan S (2015) Exploiting surroundedness for saliency detection: a boolean map approach. IEEE Trans Pattern Anal Mach Intell 38(5):889–902

    Google Scholar 

  50. Wangjiang Z, Shuang L, Yichen W, Jian S (2014) Saliency optimization from robust background detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 2814–2821

  51. Bowen J, Lihe Z, Huchuan L, Chuan Y, Ming-Hsuan Y (2013) Saliency detection via absorbing markov chain. In: Proceedings of the IEEE international conference on computer vision. pp 1665–1672

  52. Xiaohui L, Huchuan L, Lihe Z, Xiang R, Ming-Hsuan Y (2013) Saliency detection via dense and sparse reconstruction. In: Proceedings of the IEEE international conference on computer vision. pp 2976–2983

  53. Liang L, Keze W, Deyu M, Wangmeng Z, Lei Z (2017) Active self-paced learning for cost-effective and progressive face identification. IEEE Trans Pattern Anal Mach Intell 40(1):7–19

    Google Scholar 

  54. Keze W, Xiaopeng Y, Dongyu Z, Lei Z, Liang L (2018) Towards human-machine cooperation: Self-supervised sample mining for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1605–1613

  55. Xin L, Yuhong G (2014) Multi-level adaptive active learning for scene classification. In: European conference on computer vision. Springer, pp 234–249

  56. Keze W, Dongyu Z, Ya L, Ruimao Z, Liang L (2016) Cost-effective active learning for deep image classification. IEEE Trans Circuits Syst Video Technol 27(12):2591–2600

    Google Scholar 

  57. Yarin G, Riashat I, Zoubin G (2017) Deep bayesian active learning with image data. In: International conference on machine learning. PMLR, pp 1183–1192

  58. William HB, Tim G, Andreas N, Jan MK (2018) The power of ensembles for active learning in image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 9368–9377

  59. David DL, Jason C (1994) Heterogeneous uncertainty sampling for supervised learning. In: Machine learning proceedings. Elsevier, pp 148–156

  60. Sudheendra V, Ashish K (2010) Visual recognition and detection under bounded computational resources. In: IEEE computer society conference on computer vision and pattern recognition. IEEE, pp 1006–1013

  61. Hoi SCH, Jin R, Zhu J, Lyu MR (2009) Semisupervised svm batch mode active learning with applications to image retrieval. ACM Trans Inf Syst 27(3):1–29

    Article  Google Scholar 

  62. Alexander V, Vittorio F, Joachim MB (2012) Weakly supervised structured output learning for semantic segmentation. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 845–852

  63. Dana A (1988) Queries and concept learning. Mach Learn 2(4):319–342

    Article  MathSciNet  MATH  Google Scholar 

  64. Dana A (2004) Queries revisited. Theor Comput Sci 313(2):175–194

    Article  MathSciNet  MATH  Google Scholar 

  65. King RD, Whelan KE, Jones FM, Reiser PGK, Bryant CH, Muggleton SH, Kell DB, Oliver SG (2004) Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 427(6971):247–252

    Article  Google Scholar 

  66. Eric BB, Kenneth L (1992) Query learning can work poorly when a human oracle is used. In: International joint conference on neural networks. vol 8, p 8

  67. Les EA, David AC, Richard EL (1990) Training connectionist networks with queries and selective sampling. In: Advances in neural information processing systems. Citeseer, pp 566–573

  68. David C, Les A, Richard L (1994) Improving generalization with active learning. Mach Learn 15(2):201–221

    Article  Google Scholar 

  69. Ido D, Sean PE (1995) Committee-based sampling for training probabilistic classifiers. In: Machine learning proceedings. Elsevier, pp 150–157

  70. Mitchell TM (1982) Generalization as search. Artif Intell 18(2):203–226

    Article  MathSciNet  Google Scholar 

  71. Saining X, Zhuowen T (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision. pp 1395–1403

  72. Kaiming H, Xiangyu Z, Shaoqing R, Jian S (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 770–778

  73. Fisher Y, Vladlen K (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint. arXiv:1511.07122

  74. Sergey I, Christian S (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning. PMLR, pp 448–456

  75. Hahnloser RHR, Sebastian Seung H, Slotine J-J (2003) Permitted and forbidden sets in symmetric threshold-linear networks. Neural Comput 15(3):621–638

    Article  MATH  Google Scholar 

  76. Gellért M, Wenjie L, Raquel U (2017) Deeproadmapper: extracting road topology from aerial images. In: Proceedings of the IEEE international conference on computer vision. pp 3438–3446

  77. De Boer P-T, Kroese DP, Mannor S, Rubinstein RY (2005) A tutorial on the cross-entropy method. Ann Oper Res 134(1):19–67

    Article  MathSciNet  MATH  Google Scholar 

  78. Zhou W, Eero PS, Alan CB (2003) Multiscale structural similarity for image quality assessment. In: The Thrity-Seventh asilomar conference on signals, systems & computers. IEEE, vol 2, pp 1398–1402

  79. Paul J (1912) The distribution of the flora in the alpine zone. New Phytol 11(2):37–50

    Article  Google Scholar 

  80. Yin L, Xiaodi H, Christof K, James MR, Alan LY (2014) The secrets of salient object segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 280–287

  81. Cheng M-M, Mitra NJ, Huang X, Shi-Min H (2014) Salientshape: group saliency in image collections. Vis Comput 30(4):443–453

    Article  Google Scholar 

  82. Federico P, Philipp K, Yael P, Alexander H (2012) Saliency filters: contrast based filtering for salient region detection. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 733–740

  83. Deng-Ping F, Cheng G, Yang C, Bo R, Ming-Ming C, Ali B (2018) Enhanced-alignment measure for binary foreground map evaluation. arXiv preprint arXiv:1805.10421

  84. Deng-Ping F, Ming-Ming C, Yun L, Tao L, Ali B (2017) Structure-measure: a new way to evaluate foreground maps. In: Proceedings of the IEEE international conference on computer vision. pp 4548–4557

  85. Radhakrishna A, Sheila H, Francisco E, Sabine S (2009) Frequency-tuned salient region detection. In: IEEE conference on computer vision and pattern recognition. IEEE, pp 1597–1604

  86. Diederik PK, Jimmy B (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  87. Xavier G, Yoshua B (2010) Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR workshop and conference proceedings, pp 249–256

  88. Lu Z, Ju D, Huchuan L, You H, Gang W (2018) A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1741–1750

  89. Xiaoning Z, Tiantian W, Jinqing Q, Huchuan L, Gang W (2018) Progressive attention guided recurrent network for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 714–722

  90. Tiantian W, Lihe Z, Shuo W, Huchuan L, Gang Y, Xiang R, Ali B (2018) Detect globally, refine locally: a novel approach to saliency detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3127–3135

  91. Tiantian W, Ali B, Lihe Z, Pingping Z, Huchuan L (2017) A stagewise refinement model for detecting salient objects in images. In: Proceedings of the IEEE international conference on computer vision. pp 4019–4028

  92. Pingping Z, Dong W, Huchuan L, Hongyu W, Xiang R (2017) Amulet: aggregating multi-level convolutional features for salient object detection. In: Proceedings of the IEEE international conference on computer vision. pp 202–211

  93. Pingping Z, Dong W, Huchuan L, Hongyu W, Baocai Y (2017) Learning uncertain convolutional features for accurate saliency detection. In: Proceedings of the IEEE international conference on computer vision. pp 212–221

Download references

Acknowledgements

This work was supported by the Natural Science Foundation of China (U1803262, 62006165).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, 430065, China

    Longfei Li, Haidong Fu & Xin Xu

  2. Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, 430065, China

    Haidong Fu & Xin Xu

Authors
  1. Longfei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haidong Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Haidong Fu or Xin Xu.

Ethics declarations

Conflict of interest

No conflict of interest exits in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Fu, H. & Xu, X. Active learning with sampling by joint global-local uncertainty for salient object detection. Neural Comput & Applic 35, 23387–23399 (2023). https://doi.org/10.1007/s00521-021-06395-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06395-8

Keywords

Search

Navigation