Abstract
As a member of low-level visual tasks, image super-resolution (SR) is now mostly implemented by deep learning. Although the deeper convolution neural network can bring larger receptive field, it will increase the amount of calculation, make the training difficult and reduce efficiency. In addition, the feature information obtained by each channel plays a different and important role in the detail recovery during the SR process. To settle the above problems and improve the performance, we develop a multi-branch attention SR model. The main network contains multiple residual bodies, which are composed of several residual units. Additionally, we construct a multi-branch attention mechanism, which divides all channels into equal parts. Then, the network learns the relationship between channels and focuses more on the high-frequency feature channels. Experimental results show that the proposed algorithm is superior to the state-of-the-art algorithms in terms of subjective visual quality and objective evaluation criteria.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Greenspan, H.: Super-resolution in medical imaging. Comput J 52(1), 43–63 (2009)
Lillesand, T., Kiefer, R.W., Chipman, J.: Remote Sensing and Image Interpretation. Wliley, New York (2014)
Lobanov, A.P.: Resolution limits in astronomical images. arXiv:astro-ph/0503225 (2005)
Harris, J.L.: Diffraction and resolving power (1917–1983). J. Opt. Soc. Am. 54(7), 931–933 (1964)
Goodman, J.W.: Introduction to Fourier Optics. McGraw-Hill, London (1968)
Tsai, R.Y., Huang, T.S.: Multi-frame image restoration and registration. Adv. Comput. Vis. Image Process. 1(2), 317–339 (1984)
Irani, M.: Improving resolution by image registration. GVGIP Graph. Models Image Process. 53, 231–239 (1991)
Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. Publ. IEEE Sig. Process. Soc. 5(6), 996–1011 (1996)
Gao, X., Zhang, K., Tao, D., et al.: Image super-resolution with sparse neighbor embedding. IEEE Trans. Image Process. 21(7), 3194–3205 (2012)
Dong, C., Loy, C., He, K., Tang, X.: Learning a deep convolutional network for image super-resolution. In: ECCV (2014)
Kim, J., Lee, J.K., Lee, K.M.: Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016)
Lim B, Son S, Kim H, et al. Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE conference on computer vision and pattern recognition workshops (CVPRW). IEEE Computer Society (2017)
Zhang, Y., et al.: Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European Conference on Computer Vision (ECCV) (2018)
Kim, J., Kwon Lee, J., Mu Lee, K.: Deeply-recursive convolutional network for image super-resolution. In: CVPR (2016)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In: CVPR (2018)
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: Esrgan: enhanced super-resolution generative adversarial networks. arXiv:1809.00219 (2018)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)
Ma, C., Rao, Y., Cheng, Y., et al.: Structure-preserving super resolution with gradient guidance. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7769–7778 (2020)
Ren, H., El-Khamy, M., Lee, J.: Image super resolution based on fusing multiple convolution neural networks. In: CVPRW (2017)
Hu, Y., Gao, X., Li, J., Huang, Y., Wang, H.: Single image super-resolution via cascaded multi-scale cross network. arXiv:1802.08808 (2018)
Hui, Z., Wang, X., Gao, X.: Fast and accurate single image super-resolution via information distillation network. In: CVPR (2018)
Hui, Z., et al.: Lightweight Image Super-Resolution with Information Multi-distillation Network. arXiv:1909.11856v1 (2019)
Choi, J., Kim, M.: A deep convolutional neural network with selection units for super-resolution. In: CVPRW (2017)
Kim, J.-H., Choi, J.-H., Cheon, M., Lee, J.-S.: Ram: residual attention module for single image super-resolution. arXiv:1811.12043 (2018)
Dai, T., Cai, J., Zhang, Y., et al.: Second-order attention network for single image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 11065–11074 (2019)
Li, X., Wang, W., Hu, X., et al.: Selective kernel networks In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2020)
Timofte, R., Agustsson, E., Van Gool, L., Yang, M.-H., Zhang, L., Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M. et al.: Ntire 2017 challenge on single image super-resolution: methods and results. In: CVPRW (2017)
Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the 23rd British Machine Vision Conference (BMVC). BMVA Press, pp. 135.1–135.10 (2012)
Zeyde, R., Elad, M., Protter, M.: On single image scale-up using sparse-representations. In: International Conference on Curves and Surfaces (2010)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: ICCV (2001)
Huang, J.-B., Singh, A., Ahuja, N.: Single image super-resolution from transformed self-exemplars. In: CVPR (2015)
Fujimoto, A., Ogawa, T., Yamamoto, K., Matsui, Y., Yamasaki, T., Aizawa, K.: Manga109 dataset and creation of metadata. In: International Workshop on coMics ANalysis, Processing and Understanding (2016)
Dong, C., Loy, C.C., Tang, X.: Accelerating the super-resolution convolutional neural network. In: ECCV (2016)
Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Deep Laplacian pyramid networks for fast and accurate super-resolution. In: CVPR (2017)
Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In: CVPR (2017)
Wang, C., Li, Z., Shi, J.: Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358 (2019)
Haris, M., Shakhnarovich, G., Ukita, N.: Deep back-projection networks for super-resolution. In: CVPR (2018)
Li, Z., et al.: Feedback network for image super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2019)
Acknowledgments
This research was supported by the National Natural Science Foundation of China (61573182) and by the Fundamental Research Funds for the Central Universities (NS2020025).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Yang, X., Guo, Y., Li, Z. et al. Image super-resolution network based on a multi-branch attention mechanism. SIViP 15, 1397–1405 (2021). https://doi.org/10.1007/s11760-021-01870-0
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-021-01870-0