research-article

Image super resolution by dilated dense progressive network

Authors:

Pourya Shamsolmoali,

Masoumeh zareapoor,

Jie YangAuthors Info & Claims

Volume 88, Issue C

Pages 9 - 18

https://doi.org/10.1016/j.imavis.2019.03.006

Published: 01 August 2019 Publication History

Abstract

Image super-resolution (SR) is an interesting topic in computer vision. However, it remains challenging to achieve high-resolution image from the corresponding low-resolution version due to inherent variability, high dimensionality, and small ground targets images. In this paper, a new model based on dilated convolutional neural network is proposed to improve the image resolution. Recently, deep learning methods have led to significant improvements and completely outpace other models. However, these methods have not fully exploited all the features of the original low-resolution image, because of complex imaging conditions and the degradation process. To address this issue, we proposed an effective model based on dilated dense network operations to accelerate deep networks for image SR, which support the exponential growth of the receptive field parallel by increasing the filter size. In particular, residual network and skip connections are used for deep recovery. The experimental evaluations on several datasets prove the efficiency and stability of the proposed model. The proposed model not only achieves state-of-the-art performance but also has more efficient computation.

References

[1]

J. Jiang, X. Ma, C. Chen, T. Lu, Z. Wang, J. Ma, Single image superresolution via locally regularized anchored neighborhood regression and nonlocal means, IEEE Trans. Multimedia 19 (1) (2017) 15–26.

[2]

C. Dong, C.C. Loy, K. He, X. Tang, Image super-resolution using deep convolutional networks, IEEE Trans. Pattern Anal. Mach. Intell. 38 (2) (2016) 295–307.

Digital Library

[3]

K. Jiang, Z. Wang, P. Yi, J. Jiang, A progressively enhanced network for video satellite imagery Superresolution, IEEE Signal Processing Lett. 25 (11) (2018) 1630–1634.

[4]

W. Shi, J. Caballero, F. Huszar, J. Totz, A.P. Aitken, R. Bishop, D. Rueckert, Z. Wang, Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, in: CVPR, 2016.

[5]

R. Timofte, E. Agustsson, L. Van Gool, M.-H. Yang, L. Zhang, B. Lim, S. Son, H. Kim, S. Nah, K.M. Lee, Ntire 2017 challenge on single image super-resolution: methods and results, in: CVPRW, 2017.

[6]

R. Timofte, R. Rothe, L. Van Gool, Seven ways to improve example-based single image super resolution, in: CVPR, 2016.

[7]

H. Zhang, V.M. Patel, Density-aware single image deraining using a multi-stream dense network, in: CVPR, 2018.

[8]

C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, Photo-realistic single image super-resolution using a generative adversarial network, in: CVPR, 2017.

[9]

N. Akhtar, F. Shafait, A. Mian, Bayesian sparse representation for hyperspectral image super resolution, in: CVPR, 2015, pp. 3631–3640.

[10]

C. Dong, C.C. Loy, X. Tang, Accelerating the superresolution convolutional neural network, in: ECCV, 2016.

[11]

Y. Zhang, Y. Tian, Y. Kong, B. Zhong, Y. Fu, Residual dense network for image super-resolution, in: CVPR, 2018.

[12]

Y. Zhang, K. Li, K. Li, L. Wang, B. Zhong, Y. Fu, Image super-resolution using very deep residual channel attention networks, in: ECCV, 2018.

[13]

T. Tong, G. Li, X. Liu, Q. Gao, Image super-resolution using dense skip connections, in: ICCV, 2017.

[14]

W. Shi, J. Caballero, C. Ledig, X. Zhuang, W. Bai, K. Bhatia, A.M.S.M. de Marvao, T. Dawes, D. ORegan, D. Rueckert, Cardiac image super-resolution with global correspondence using multi-atlas patchmatch, in: MICCAI, 2013.

[15]

C. Dong, C.C. Loy, K. He, X. Tang, Learning a deep convolutional network for image super-resolution, in: ECCV, 2014.

[16]

B. Lim, S. Son, H. Kim, S. Nah, K.M. Lee, Enhanced deep residual networks for single image super-resolution, in: CVPRW, 2017.

[17]

Z. Huang, L. Wang, G. Meng, C. Pan, Image super-resolution via deep dilated convolutional networks, in: ICIP, 2017.

[18]

Fisher Yu, Vladlen Koltun, Multi-scale context aggregation by dilated convolutions, in: abs/1511.07122, ICLR, 2016.

[19]

M. Zareapoor, M.E. Celebi, J. Yang, Diverse adversarial network for image super-resolution, Signal Processing: Image Communication 74 (2019) 191–200.

[20]

J. Kim, J. Kwon Lee, K. Mu Lee, Accurate image superresolution using very deep convolutional networks, in: CVPR, 2016.

[21]

J. Kim, J. Kwon Lee, K. Mu Lee, Deeply-recursive convolutional network for image super-resolution, in: CVPR, 2016.

[22]

H. Chen, Y. Zhang, M.K. Kalra, F. Lin, Y. Chen, P. Liao, J. Zhou, G. Wang, Low-dose CT with a residual encoder-decoder convolutional neural network (RED-CNN), IEEE Trans. Med. Imaging 36 (12) (2017) 2524–2535.

[23]

Y. Tai, J. Yang, X. Liu, Image super-resolution via deep recursive residual network, in: CVPR, 2017.

[24]

H. Zhang, V.M. Patel, Densely connected pyramid dehazing network, in: CVPR, 2018.

[25]

Y. Romano, J. Isidoro, P. Milanfar, RAISR: rapid and accurate image super resolution, IEEE Trans. Comput. Imaging 3 (1) (2017) 110–125.

[26]

S. Mei, X. Yuan, J. Ji, Y. Zhang, S. Wan, Q. Du, Hyperspectral image spatial super-resolution via 3D full convolutional neural network, Remote Sens. 9 (11) (2017) 1139.

[27]

S. Galliani, C. Lanaras, D. Marmanis, E. Baltsavias, K. Schindler, Learned Spectral Super-resolution arXiv PreprintarXiv:1703.09470, 2017.

[28]

H. Zhu, X. Tang, J. Xie, W. Song, F. Mo, X. Gao, Spatio-temporal super-resolution reconstruction of remote-sensing images based on adaptive multi-scale detail enhancement, Sensors 18 (2) (2018) 498.

[29]

N. Akhtar, A. Mian, Hyperspectral recovery from RGB images using Gaussian processes, IEEE Trans. Pattern Anal. Mach. Intell. 14 (8) (2018).

[30]

P. Cui, S. Jin, L. Yu, F. Wang, W. Zhu, S. Yang, Cascading outbreak prediction in networks: A data-driven approach, in: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2013, pp. 901–909.

Digital Library

[31]

Z. Wang, A.C. Bovik, H.R. Sheikh, E.P. Simoncelli, Image quality assessment: from error visibility to structural similarity, IEEE Trans. Image Process. 13 (4) (2004) 600–612.

Digital Library

[32]

C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z. Wang, W. Shi, Photo-realistic single image super-resolution using a generative adversarial network, in: CVPR, 2017, pp. 4681–4690.

[33]

W.-S. Lai, J.-B. Huang, N. Ahuja, M.-H. Yang, Deep laplacian pyramid networks for fast and accurate super resolution, in: CVPR, 2017.

[34]

W.-S. Lai, J.-B. Huang, N. Ahuja, M.-H. Yang, Fast and accurate image super-resolution with deep Laplacian pyramid networks, in: CVPR, 2018.

[35]

O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: International 11 Conference on Medical Image Computing and Computer-assisted Intervention, Springer, 2015, pp. 234–241.

[36]

P.F. Christ, M.E.A. Elshaer, F. Ettlinger, S. Tatavarty, M. Bickel, P. Bilic, M. Rempfler, M. Armbruster, F. Hofmann, M. DAnastasi, Automatic liver and lesion segmentation in CT using cascaded fully convolutional neural networks and 3D conditional random fields, in: International Conference on Medical Image Computing and Computer-Assisted Intervention, 2016, pp. 415–423.

Digital Library

[37]

X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: AISTATS, 2011.

[38]

V. Nair, G.E. Hinton, Rectified Linear Units Improve Restricted Boltzman Machines, in: ICML, 2010.

[39]

P. Wang, P. Chen, Y. Yuan, D. Liu, Z. Huang, X. Hou, G. Cottrell, Understanding Convolution for Semantic Segmentation, (arXiv preprint arXiv:1702.08502) 2017.

[40]

E. Romera, J.M. Alvarez, L.M. Bergasa, R. Arroyo, Efficient convnet for real-time semantic segmentation, in: Intelligent Vehicles Symposium (IV), 2017, pp. 1789–1794.

[41]

D. Martin, C. Fowlkes, D. Tal, J. Malik, A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in: ICCV, 2001.

[42]

M. Bevilacqua, A. Roumy, C. Guillemot, M.L. AlberiMorel, Low-complexity single-image super-resolution based on nonnegative neighbor embedding, in: BMVC, 2012.

[43]

R. Zeyde, M. Elad, M. Protter, On single image scale-up using sparse-representations, in: Proc. 7th Int. Conf. Curves Surf, 2010.

[44]

J.-B. Huang, A. Singh, N. Ahuja, Single image superresolution from transformed self-exemplars, in: CVPR, 2015.

[45]

H. Chen, X. He, L. Qing, Q. Teng, C. Ren, SGCRSR: sequential gradient constrained regression for single image super-resolution, Signal Process. Image Commun. 66 (2018) 1–18.

[46]

C. Zhang, W. Liu, J. Liu, C. Liu, C. Shi, Sparse representation and adaptive mixed samples regression for single image super-resolution, Signal Process. Image Commun. 67 (2018) 79–89.

[47]

J. Xiao, E. Liu, L. Zhao, Y.F. Wang, W. Jiang, Detail enhancement of image super-resolution based on detail synthesis, Signal Process. Image Commun. 50 (2017) 21–33.

Cited By

Dan JQu ZWang XLi FGu JMa B(2021)Scale adaptive and lightweight super-resolution with a selective hierarchical residual networkProceedings of the 2021 5th International Conference on Innovation in Artificial Intelligence10.1145/3461353.3461376(8-14)Online publication date: 5-Mar-2021
https://dl.acm.org/doi/10.1145/3461353.3461376

Index Terms

Image super resolution by dilated dense progressive network

Index terms have been assigned to the content through auto-classification.

Recommendations

Single image resolution enhancement by efficient dilated densely connected residual network
Abstract
Convolution Neural Networks have been widely applied in single image super-resolution (SR). Recent works have shown the superior performance of deep networks for SR tasks. With just an increase in the model’s depth, more features and ...
Highlights
- We introduce a progressive dilated convolution network for image super-resolution.
Dilated projection correction network based on autoencoder for hyperspectral image super-resolution
Abstract
This paper focuses on improving the spatial resolution of the hyperspectral image (HSI) by taking the prior information into consideration. In recent years, single HSI super-resolution methods based on deep learning have achieved good ...
Highlights
- We transform the single HSI SR problem into the abundance domain.
- We deploy a ...
Image super-resolution: use of self-learning and gabor prior
ACCV'12: Proceedings of the 11th Asian conference on Computer Vision - Volume Part III

Recent approaches on single image super-resolution (SR) have attempted to exploit self-similarity to avoid the use of multiple images. In this paper, we propose an SR method based on self-learning and Gabor prior. Given a low resolution (LR) test image ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Image and Vision Computing

Image and Vision Computing Volume 88, Issue C

Aug 2019

132 pages

ISSN:0262-8856

Issue’s Table of Contents

Elsevier B.V.

Publisher

Butterworth-Heinemann

United States

Publication History

Published: 01 August 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dan JQu ZWang XLi FGu JMa B(2021)Scale adaptive and lightweight super-resolution with a selective hierarchical residual networkProceedings of the 2021 5th International Conference on Innovation in Artificial Intelligence10.1145/3461353.3461376(8-14)Online publication date: 5-Mar-2021
https://dl.acm.org/doi/10.1145/3461353.3461376

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents