Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

DFANet: A Dual-Stream Deep Feature Aware Network for Multi-focus Image Fusion

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2024)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 15045))

Included in the following conference series:

  • 25 Accesses

Abstract

Multi-focus image fusion is a crucial technique for improving image clarity by combining images taken at different focal depths. However, existing deep learning-based methods tend to overlook crucial details in the feature extraction and fusion phases, substantially affecting the quality of the fused images. To address this, in the feature extraction phase, we crafted an effective feature extraction block that boosts feature recognition by modeling channel interdependencies and captures global information through global average pooling. This block excels in gathering positional details from feature maps, significantly boosting clarity and detail retention. In the feature fusion phase, we introduce a novel feature fusion module (SFLE) that integrates the Local Energy (LE) operator to uncover intensity and detail distribution in source images, with Spatial Frequency (SF) measuring pixel value changes. This fusion technique ensures vital details are preserved with greater fidelity, facilitating a seamless integration of features from the input images. Accordingly, we present a novel dual-stream deep feature aware network (DFANet). Extensive experiments show that our model outperforms existing methods both qualitatively and quantitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 74.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Amin-Naji, M., Aghagolzadeh, A., Ezoji, M.: Ensemble of CNN for multi-focus image fusion. Inf. Fusion 51, 201–214 (2019)

    Article  Google Scholar 

  2. Aymaz, S., Köse, C., Aymaz, Ş: Multi-focus image fusion for different datasets with super-resolution using gradient-based new fusion rule. Multimed. Tools Appl. 79, 13311–13350 (2020)

    Article  Google Scholar 

  3. Bai, X., Zhang, Y., Zhou, F., Xue, B.: Quadtree-based multi-focus image fusion using a weighted focus-measure. Inf. Fusion 22, 105–118 (2015)

    Article  Google Scholar 

  4. Bai, X., Zhou, F., Xue, B.: Fusion of infrared and visual images through region extraction by using multi-scale center-surround top-hat transform. Opt. Express 19(9), 8444–8457 (2011)

    Article  Google Scholar 

  5. Bao, P., Xia, W., Yang, K., Chen, W., Chen, M., Xi, Y., Niu, S., Zhou, J., Zhang, H., Sun, H., Wang, Z., Zhang, Y.: Convolutional sparse coding for compressed sensing CT reconstruction. IEEE Trans. Med. Imaging 38(11), 2607–2619 (2019)

    Article  Google Scholar 

  6. Duan, Z., Luo, X., Zhang, T.: Multi-focus image fusion via gradient guidance progressive network. In: ICME, pp. 2159–2164. IEEE (2023)

    Google Scholar 

  7. Farid, M.S., Mahmood, A., Al-Maadeed, S.A.: Multi-focus image fusion using content adaptive blurring. Inf. Fusion 45, 96–112 (2019)

    Article  Google Scholar 

  8. Guan, Y., Xu, R., Yao, M., Wang, L., Xiong, Z.: Mutual-guided dynamic network for image fusion. In: ACM MM, pp. 1779–1788 (2023)

    Google Scholar 

  9. Han, Y., Cai, Y., Cao, Y., Xu, X.: A new image fusion performance metric based on visual information fidelity. Inf. Fusion 14(2), 127–135 (2013)

    Article  Google Scholar 

  10. He, K., Sun, J., Tang, X.: Guided image filtering. TPAMI 35(6), 1397–1409 (2012)

    Article  Google Scholar 

  11. Kinga, D., Adam, J.B., et al.: A method for stochastic optimization. In: ICLR, vol. 5, p. 6. San Diego, California (2015)

    Google Scholar 

  12. Li, J., Guo, X., Lu, G., Zhang, B., Xu, Y., Wu, F., Zhang, D.: DRPL: deep regression pair learning for multi-focus image fusion. IEEE Trans. Image Process. 29, 4816–4831 (2020)

    Article  Google Scholar 

  13. Li, S., Kang, X., Hu, J.: Image fusion with guided filtering. IEEE Trans. Image Process. 22(7), 2864–2875 (2013)

    Article  Google Scholar 

  14. Li, S., Kwok, J.T., Wang, Y.: Combination of images with diverse focuses using the spatial frequency. Inf. Fusion 2(3), 169–176 (2001)

    Article  Google Scholar 

  15. Liu, Y., Chen, X., Peng, H., Wang, Z.: Multi-focus image fusion with a deep convolutional neural network. Inf. Fusion 36, 191–207 (2017)

    Article  Google Scholar 

  16. Liu, Y., Liu, S., Wang, Z.: Multi-focus image fusion with dense sift. Inf. Fusion 23, 139–155 (2015)

    Article  Google Scholar 

  17. Liu, Y., Wang, L., Cheng, J., Chen, X.: Multiscale feature interactive network for multifocus image fusion. IEEE Trans. Instrum. Meas. 70, 1–16 (2021)

    Article  Google Scholar 

  18. Lu, J., Tan, K., Li, Z., Chen, J., Ran, Q., Wang, H.: Multi-focus Image Fusion Using Residual Removal and Fractional Order Differentiation Focus Measure, Signal, Image and Video Processing, pp. 1–16 (2024)

    Google Scholar 

  19. Ma, B., Yin, X., Wu, D., Shen, H., Ban, X., Wang, Y.: End-to-end learning for simultaneously generating decision map and multi-focus image fusion result. Neurocomputing 470, 204–216 (2022)

    Article  Google Scholar 

  20. Ma, B., Zhu, Y., Yin, X., Ban, X., Huang, H., Mukeshimana, M.: SESF-fuse: an unsupervised deep model for multi-focus image fusion. Neural Comput. Appl. 33, 5793–5804 (2021)

    Article  Google Scholar 

  21. Ma, X., Wang, Z., Hu, S.: Multi-focus image fusion based on multi-scale sparse representation. J. Visual Commun. Image Represent. 81, 103328 (2021)

    Article  Google Scholar 

  22. Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: Proceedings of International Conference on 3D Vision, pp. 565–571. IEEE (2016)

    Google Scholar 

  23. Nejati, M., Samavi, S., Shirani, S.: Multi-focus image fusion using dictionary-based sparse representation. Inf. Fusion 25, 72–84 (2015)

    Article  Google Scholar 

  24. Ni, B., Pei, Y., Moulin, P., Yan, S.: Multilevel depth and image fusion for human activity detection. IEEE Trans. Cybern. 43(5), 1383–1394 (2013)

    Article  Google Scholar 

  25. Qin, X., Zhang, Z., Huang, C., Dehghan, M., Zaiane, O.R., Jagersand, M.: U2-net: going deeper with nested u-structure for salient object detection. Pattern Recognit. 106, 107404 (2020)

    Article  Google Scholar 

  26. Qu, G., Zhang, D., Yan, P.: Information measure for performance of image fusion. Electron. Lett. 38(7), 1–2 (2002)

    Article  Google Scholar 

  27. Rajalingam, B., Priya, R., Bhavani, R.: Hybrid multimodal medical image fusion using combination of transform techniques for disease analysis. Procedia Comput. Sci. 152, 150–157 (2019)

    Article  Google Scholar 

  28. Wang, Q., Shen, Y., Jin, J.: Performance evaluation of image fusion techniques. Image Fusion Algorithms Appl. 19, 469–492 (2008)

    Article  Google Scholar 

  29. Wang, Y., Xiao, Y., Lu, J., Tan, B., Cao, Z., Zhang, Z., Zhou, J.: Discriminative multi-view dynamic image fusion for cross-view 3-d action recognition. IEEE Trans. Neural Netw. Learn, Syst. 33(10), 5332–5345 (2022)

    Google Scholar 

  30. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  31. Xu, H., Ma, J., Jiang, J., Guo, X., Ling, H.: U2fusion: a unified unsupervised image fusion network. IEEE Trans. Pattern Anal. Mach. Intell. 44(1), 502–518 (2020)

    Article  Google Scholar 

  32. Xydeas, C.S., Petrovic, V., et al.: Objective image fusion performance measure. Electron. Lett. 36(4), 308–309 (2000)

    Article  Google Scholar 

  33. Yang, Y., Huang, S., Gao, J., Qian, Z.: Multi-focus image fusion using an effective discrete wavelet transform based algorithm. Meas. Sci. Rev. 14(2), 102–108 (2014)

    Article  Google Scholar 

  34. Zhang, X.: Deep learning-based multi-focus image fusion: a survey and a comparative study. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 4819–4838 (2021)

    Google Scholar 

  35. Zhang, Y., Bai, X., Wang, T.: Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure. Inf. Fusion 35, 81–101 (2017)

    Article  Google Scholar 

  36. Zhang, Y., Liu, Y., Sun, P., Yan, H., Zhao, X., Zhang, L.: IFCNN: a general image fusion framework based on convolutional neural network. Inf. Fusion 54, 99–118 (2020)

    Article  Google Scholar 

  37. Zheng, K., Cheng, J., Liu, Y.: CCSR-Net: unfolding coupled convolutional sparse representation for multi-focus image fusion. In: PRCV, pp. 285–297. Springer (2023)

    Google Scholar 

  38. Zhou, Z., Li, S., Wang, B.: Multi-scale weighted gradient-based fusion for multi-focus images. Inf. Fusion 20, 60–72 (2014)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun, 130012, China

    Yuye Dong, Libo Zhao, Xiongfei Li & Xiaoli Zhang

  2. College of Computer Science and Technology, Jilin University, Changchun, 130012, China

    Yuye Dong, Libo Zhao, Xiongfei Li & Xiaoli Zhang

Authors
  1. Yuye Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Libo Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiongfei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoli Zhang .

Editor information

Editors and Affiliations

  1. Peking University, Beijing, China

    Zhouchen Lin

  2. Nankai University, Tianjin, China

    Ming-Ming Cheng

  3. Chinese Academy of Sciences, Beijing, China

    Ran He

  4. Xinjiang University, Urumqi, Xinjiang, China

    Kurban Ubul

  5. Xinjiang University, Urumqi, China

    Wushouer Silamu

  6. Peking University, Beijing, China

    Hongbin Zha

  7. Tsinghua University, Beijing, China

    Jie Zhou

  8. Chinese Academy of Sciences, Beijing, China

    Cheng-Lin Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2025 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Dong, Y., Zhao, L., Li, X., Zhang, X. (2025). DFANet: A Dual-Stream Deep Feature Aware Network for Multi-focus Image Fusion. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2024. Lecture Notes in Computer Science, vol 15045. Springer, Singapore. https://doi.org/10.1007/978-981-97-8499-8_22

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-8499-8_22

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-8498-1

  • Online ISBN: 978-981-97-8499-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation