research-article

Distortion detection and removal integrated method for image restoration

Authors:

Shujuan HouAuthors Info & Claims

Volume 127, Issue C

https://doi.org/10.1016/j.dsp.2022.103528

Published: 01 July 2022 Publication History

Abstract

Image restoration has been the focus of research in image processing, and current methods mainly target specific single distortion or hybrid distortion with known distortion types. However, real-world images generally affected hybrid distortions of unknown quantity and type, there is no restoration method applicable to such complex hybrid distortion. Therefore, we propose a distortion detection and removal integrated method. Firstly, the distortion detection module is designed based on the idea of multi-label classification, which can detect the type and level of distortion in the distorted image. The type and level of distortion can be used as a piece of prior information to control the subsequent image restoration process. Then, the image restoration module is designed based on controllable residual connection, which enables the network to restore images with different types and levels of distortion, and the attention parallel convolution block is designed by using a parallel convolution layer and coordinated attention mechanism to enhance the feature extraction ability of the network and improve the quality of the restored image. The experimental results show that the proposed method achieves superior performance.

References

[1]

N. Kwak, J. Yoo, S. Lee, Image restoration by estimating frequency distribution of local patches, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 6684–6692.

[2]

K. Zhang, W. Zuo, Y. Chen, et al., Beyond a Gaussian denoiser: residual learning of deep CNN for image denoising, IEEE Trans. Image Process. 26 (7) (2017) 3142–3155.

Digital Library

[3]

K. Zhang, W. Zuo, L. Zhang, FFDNet toward a fast and flexible solution for CNN-based image denoising, IEEE Trans. Image Process. 27 (9) (2018) 4608–4622.

[4]

S. Guo, Z. Yan, K. Zhang, et al., Toward convolutional blind denoising of real photographs, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019, pp. 1712–1722.

[5]

C. Tian, Y. Xu, W. Zuo, Image denoising using deep CNN with batch renormalization, Neural Netw. 121 (2020) 461–473.

Digital Library

[6]

S. Nah, T.H. Kim, K.M. Lee, Deep multi-scale convolutional neural network for dynamic scene deblurring, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2017, pp. 257–265.

[7]

O. Kupyn, V. Budzan, M. Mykhailych, et al., DeblurGAN: blind motion deblurring using conditional adversarial networks, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 8183–8192.

[8]

X. Tao, H. Gao, X. Shen, et al., Scale-recurrent network for deep image deblurring, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 8174–8182.

[9]

Y. Zhang, W. Li, Z. Li, et al., Dual attention per-pixel filter network for spatially varying image deblurring, Digit. Signal Process. 113 (2021).

[10]

W. Ren, L. Si, Z. Hua, et al., Single image dehazing via multi-scale convolutional neural networks, in: European Conference on Computer Vision, Springer, Cham, 2016.

[11]

B. Li, X. Peng, Z. Wang, et al., AOD-Net: all-in-one dehazing network, in: 2017 IEEE International Conference on Computer Vision (ICCV), IEEE, 2017, pp. 4780–4788.

[12]

D. Chen, M. He, Q. Fan, et al., Gated context aggregation network for image dehazing and deraining, in: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), IEEE Computer Society, 2019, pp. 1375–1383.

[13]

D. Ren, W. Zuo, Q. Hu, et al., Progressive image deraining networks: a better and simpler baseline, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2019, pp. 3932–3941.

[14]

X. Fu, J. Huang, X. Ding, et al., Clearing the skies: a deep network architecture for single-image rain removal, IEEE Trans. Image Process. 26 (6) (2017) 2944–2956.

Digital Library

[15]

X. Li, J. Wu, Z. Lin, et al., Recurrent squeeze-and-excitation context aggregation net for single image deraining, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 254–269.

[16]

C. Dong, Y. Deng, C.C. Loy, et al., Compression artifacts reduction by a deep convolutional network, in: 2015 IEEE International Conference on Computer Vision (ICCV), IEEE, 2015, pp. 576–584.

[17]

X. Zhang, W. Yang, Y. Hu, et al., Dmcnn: dual-domain multi-scale convolutional neural network for compression artifacts removal, in: 2018 25th IEEE International Conference on Image Processing (ICIP), IEEE, 2018, pp. 390–394.

[18]

J. Li, Y. Wang, H. Xie, et al., Learning a single model with a wide range of quality factors for jpeg image artifacts removal, IEEE Trans. Image Process. 29 (2020) 8842–8854.

[19]

K. Yu, C. Dong, L. Lin, et al., Crafting a toolchain for image restoration by deep reinforcement learning, in: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 2443–2452.

[20]

M. Suganuma, X. Liu, T. Okatani, Attention-based adaptive selection of operations for image restoration in the presence of unknown combined distortions, in: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2020.

[21]

Xin Li, Xin Jin, Jianxin Lin, Tao Yu, Sen Liu, Yaojun Wu, Wei Zhou, Zhibo Chen, Learning disentangled feature representation for hybrid-distorted image restoration, in: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020.

[22]

Simonyan, K.; Zisserman, A. (2014): Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556.

[23]

K. He, X. Zhang, S. Ren, et al., Deep residual learning for image recognition, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778.

[24]

J. He, C. Dong, Y. Qiao, Interactive multi-dimension modulation with dynamic controllable residual learning for image restoration, in: European Conference on Computer Vision (ECCV), 2020.

[25]

X. Ding, Y. Guo, G. Ding, et al., ACNet: strengthening the kernel skeletons for powerful CNN via asymmetric convolution blocks, in: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE, 2019.

[26]

S. Nie, C. Ma, D. Chen, et al., A dual residual network with channel attention for image restoration, in: European Conference on Computer Vision, Springer, Cham, 2020, pp. 352–363.

[27]

Y. Zhang, K. Li, K. Li, et al., Image super-resolution using very deep residual channel attention networks, in: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 286–301.

Digital Library

[28]

X. Zhang, T. Wang, J. Wang, et al., Pyramid channel-based feature attention network for image dehazing, Comput. Vis. Image Underst. 197 (2020).

[29]

H. Wang, Y. Zhu, B. Green, et al., Axial-deeplab: stand-alone axial-attention for panoptic segmentation, in: European Conference on Computer Vision, Springer, Cham, 2020, pp. 108–126.

[30]

Q. Hou, D. Zhou, J. Feng, Coordinate attention for efficient mobile network design, in: 2021 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2021.

[31]

E. Agustsson, R. Timofte, NTIRE 2017 challenge on single image super-resolution: dataset and study, in: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), IEEE, 2017.

[32]

Kingma, D.P.; Ba, J. (2014): Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980.

Index Terms

Distortion detection and removal integrated method for image restoration
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Unified Restoration Method for Different Degraded Images
ICOIP '10: Proceedings of the 2010 International Conference on Optoelectronics and Image Processing - Volume 02

Blurred images are caused by many factors such as defocus, motion, and atmospheric turbulence. Due to the unknown various factors that cannot be distinguished in the blurred image, it is necessary to propose a unified method for image restoration. In ...
A Blind Image Restoration Method Based on PSF Estimation
WCSE '09: Proceedings of the 2009 WRI World Congress on Software Engineering - Volume 02

In order to improve the quality of the restored image, a blind image restoration method is proposed, by estimating the blur function of the imaging model. Firstly, the parameters of the Gaussian point spread function (PSF) of the observed image are ...
Blurred image restoration: A fast method of finding the motion length and angle

Motion blur in photographic images is a result of camera movement or shake. Methods such as Blind Deconvolution are used when information about the direction and size of blur is not known. Restoration methods, such as Lucy and Richardson or Wiener ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Digital Signal Processing

Digital Signal Processing Volume 127, Issue C

Jul 2022

671 pages

ISSN:1051-2004

Issue’s Table of Contents

Copyright © 2022.

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 July 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents