research-article

A parallel and serial denoising network

Authors:

Chia-Wen LinAuthors Info & Claims

Volume 231, Issue C

https://doi.org/10.1016/j.eswa.2023.120628

Published: 30 November 2023 Publication History

Abstract

Convolutional neural networks (CNNs) have performed well in image denoising. Although some CNNs enlarge convolutional kernels and increase stacked convolutional layers to overcome the locality defect of convolutional operations, they may increase computational costs. In this paper, we propose a parallel and serial denoising network (PSDNet) for image denoising to preserve image texture. Specifically, the proposed PSDNet contains a parallel block (PB), a serial block (SB), and a reconstruction block (RB). A PB uses two heterogeneous sub-networks with a deformable convolution in a parallel way to extract comparative information for better-recovering image texture. A SB utilizes an enhanced residual dense architecture via combinations of a batch normalization, ReLU, and convolutional layer in a serial way to refine obtained features for obtaining more accurate noise information. A RB is responsible for reconstructing images. Experimental results reveal that our PSDNet is very effective in image denoising, according to quantitative analysis and visual analysis. Codes can be obtained at https://github.com/hellloxiaotian/PSDNet.

Highlights

•

Heterogeneous architecture with deformable convolution can better filter noise.

•

An enhanced residual architecture is used to remove redundant features.

•

Combining a parallel and serial way can improve effects of images denoising.

•

Proposed network is effective for synthesized and real noisy image denoising.

References

[1]

Bao, L., Yang, Z., Wang, S., Bai, D., & Lee, J. (2020). Real image denoising based on multi-scale residual dense block and cascaded U-Net with block-connection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops (pp. 448–449).

[2]

Buades A., Coll B., Morel J.-M., A review of image denoising algorithms, with a new one, Multiscale Modeling and Simulation 4 (2) (2005) 490–530.

[3]

Burger H.C., Schuler C.J., Harmeling S., Image denoising: Can plain neural networks compete with BM3D?, in: 2012 IEEE conference on computer vision and pattern recognition, IEEE, 2012, pp. 2392–2399.

[4]

Chang Y., Yan L., Fang H., Zhong S., Liao W., HSI-DeNet: Hyperspectral image restoration via convolutional neural network, IEEE Transactions on Geoscience and Remote Sensing 57 (2) (2018) 667–682.

[5]

Chen Y., Pock T., Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (6) (2016) 1256–1272.

[6]

Chen R., Pu D., Tong Y., Wu M., Image-denoising algorithm based on improved K-singular value decomposition and atom optimization, CAAI Transactions on Intelligence Technology 7 (1) (2022) 117–127.

[7]

Chen Z., Zhou Z., Adnan S., Joint low-rank prior and difference of Gaussian filter for magnetic resonance image denoising, Medical & Biological Engineering & Computing 59 (3) (2021) 607–620.

[8]

Cho S.I., Kang S.-J., Gradient prior-aided CNN denoiser with separable convolution-based optimization of feature dimension, IEEE Transactions on Multimedia 21 (2) (2018) 484–493.

[9]

Dabov K., Foi A., Katkovnik V., Egiazarian K., Image denoising by sparse 3-D transform-domain collaborative filtering, IEEE Transactions on Image Processing 16 (8) (2007) 2080–2095.

[10]

Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., et al. (2017). Deformable convolutional networks. In Proceedings of the IEEE international conference on computer vision (pp. 764–773).

[11]

Dey S., Bhattacharya R., Schwenker F., Sarkar R., Median filter aided CNN based image denoising: An ensemble aprroach, Algorithms 14 (4) (2021) 109.

[12]

Ephraim Y., Malah D., Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator, IEEE Transactions on Acoustics, Speech and Signal Processing 32 (6) (1984) 1109–1121.

[13]

Fan C.-M., Liu T.-J., Liu K.-H., Chiu C.-H., Selective residual M-net for real image denoising, in: 2022 30th European signal processing conference, IEEE, 2022, pp. 469–473.

[14]

Fang F., Li J., Yuan Y., Zeng T., Zhang G., Multilevel edge features guided network for image denoising, IEEE Transactions on Neural Networks and Learning Systems 32 (9) (2020) 3956–3970.

[15]

Franzen R., Kodak24, 1999, URL http://www.r0k.us/graphics/kodak/.

[16]

Gai S., Bao Z., New image denoising algorithm via improved deep convolutional neural network with perceptive loss, Expert Systems with Applications 138 (2019).

[17]

Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014). Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2862–2869).

[18]

Gurrola-Ramos J., Dalmau O., Alarcón T.E., A residual dense U-net neural network for image denoising, IEEE Access 9 (2021) 31742–31754.

[19]

Harris C.R., Millman K.J., Van Der Walt S.J., Gommers R., Virtanen P., Cournapeau D., et al., Array programming with numpy, Nature 585 (7825) (2020) 357–362.

[20]

He K., Zhang X., Ren S., Sun J., Deep residual learning for image recognition, in: 2016 IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778,.

[21]

Healey G.E., Kondepudy R., Radiometric CCD camera calibration and noise estimation, IEEE Transactions on Pattern Analysis and Machine Intelligence 16 (3) (1994) 267–276.

[22]

Herbreteau S., Kervrann C., DCT2net: an interpretable shallow CNN for image denoising, IEEE Transactions on Image Processing 31 (2022) 4292–4305.

[23]

Hore A., Ziou D., Image quality metrics: PSNR vs. SSIM, in: 2010 20th international conference on pattern recognition, IEEE, 2010, pp. 2366–2369.

[24]

Ioffe S., Szegedy C., Batch normalization: Accelerating deep network training by reducing internal covariate shift, in: International conference on machine learning, PMLR, 2015, pp. 448–456.

[25]

Jafarbigloo S.K., Danyali H., Nuclear atypia grading in breast cancer histopathological images based on CNN feature extraction and LSTM classification, CAAI Transactions on Intelligence Technology 6 (4) (2021) 426–439.

[26]

Jain V., Seung S., Natural image denoising with convolutional networks, Advances in Neural Information Processing Systems 21 (2008).

[27]

Jia, X., Liu, S., Feng, X., & Zhang, L. (2019). Focnet: A fractional optimal control network for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 6054–6063).

[28]

Jin Y., Yu L., Li G., Fei S., A 6-DOFs event-based camera relocalization system by CNN-LSTM and image denoising, Expert Systems with Applications 170 (2021).

[29]

Jonides J., Lewis R.L., Nee D.E., Lustig C.A., Berman M.G., Moore K.S., The mind and brain of short-term memory, Annual Review of Psychology 59 (2008) 193–224.

[30]

Kim, D.-W., Ryun Chung, J., & Jung, S.-W. (2019). Grdn: Grouped residual dense network for real image denoising and gan-based real-world noise modeling. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops.

[31]

Kingma D.P., Ba J., Adam: A method for stochastic optimization, 2014, arXiv preprint arXiv:1412.6980.

[32]

Lee B., Ku B., Kim W., Ko H., Two-stream learning-based compressive sensing network with high-frequency compensation for effective image denoising, IEEE Access 9 (2021) 91974–91982.

[33]

Li Z., Wu J., Learning deep cnn denoiser priors for depth image inpainting, Applied Sciences 9 (6) (2019) 1103.

[34]

Liu, P., Zhang, H., Zhang, K., Lin, L., & Zuo, W. (2018). Multi-level wavelet-CNN for image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops (pp. 773–782).

[35]

Lucas A., Iliadis M., Molina R., Katsaggelos A.K., Using deep neural networks for inverse problems in imaging: beyond analytical methods, IEEE Signal Processing Magazine 35 (1) (2018) 20–36.

[36]

Luo E., Chan S.H., Nguyen T.Q., Adaptive image denoising by targeted databases, IEEE Transactions on Image Processing 24 (7) (2015) 2167–2181.

[37]

Makitalo M., Foi A., Optimal inversion of the generalized anscombe transformation for Poisson-Gaussian noise, IEEE Transactions on Image Processing 22 (1) (2012) 91–103.

[38]

Malfait M., Roose D., Wavelet-based image denoising using a Markov random field a priori model, IEEE Transactions on Image Processing 6 (4) (1997) 549–565.

[39]

Mao X., Shen C., Yang Y.-B., Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections, Advances in Neural Information Processing Systems 29 (2016).

[40]

Martin D., Fowlkes C., Tal D., Malik J., A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics, in: Proceedings eighth IEEE international conference on computer vision. ICCV 2001, Vol. 2, IEEE, 2001, pp. 416–423.

[41]

Nam, S., Hwang, Y., Matsushita, Y., & Kim, S. J. (2016). A holistic approach to cross-channel image noise modeling and its application to image denoising. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1683–1691).

[42]

Ofir N., Keller Y., Multi-scale processing of noisy images using edge preservation losses, in: 2020 25th international conference on pattern recognition, IEEE, 2021, pp. 1–8.

[43]

Panda A., Naskar R., Pal S., Exponential linear unit dilated residual network for digital image denoising, Journal of Electronic Imaging 27 (5) (2018).

[44]

Park, B., Yu, S., & Jeong, J. (2019). Densely connected hierarchical network for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops.

[45]

Pytorch A.D.I., Pytorch, 2018.

[46]

Rajni R., Anutam A., Image denoising techniques-an overview, International Journal of Computer Applications 86 (16) (2014) 13–17.

[47]

Ronneberger O., Fischer P., Brox T., U-net: Convolutional networks for biomedical image segmentation, in: International conference on medical image computing and computer-assisted intervention, Springer, 2015, pp. 234–241.

[48]

Roth S., Black M.J., Fields of experts: A framework for learning image priors, in: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol. 2, IEEE, 2005, pp. 860–867.

[49]

Scetbon M., Elad M., Milanfar P., Deep k-svd denoising, IEEE Transactions on Image Processing 30 (2021) 5944–5955.

[50]

Schmidt, U., & Roth, S. (2014). Shrinkage fields for effective image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2774–2781).

[51]

Shim, G., Park, J., & Kweon, I. S. (2020). Robust reference-based super-resolution with similarity-aware deformable convolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8425–8434).

[52]

Sun X., Kottayil N.K., Mukherjee S., Cheng I., Adversarial training for dual-stage image denoising enhanced with feature matching, in: International conference on smart multimedia, Springer, 2018, pp. 357–366.

[53]

Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., et al. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1–9).

[54]

Tai, Y., Yang, J., Liu, X., & Xu, C. (2017). Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE international conference on computer vision (pp. 4539–4547).

[55]

Tang Y., Sun J., Jiang A., Chen Y., Zhou L., Adaptive graph filtering with intra-patch pixel smoothing for image denoising, Circuits, Systems, and Signal Processing 40 (11) (2021) 5381–5400.

[56]

Thomas, H., Qi, C. R., Deschaud, J.-E., Marcotegui, B., Goulette, F., & Guibas, L. J. (2019). Kpconv: Flexible and deformable convolution for point clouds. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 6411–6420).

[57]

Tian C., Xu Y., Fei L., Wang J., Wen J., Luo N., Enhanced CNN for image denoising, CAAI Transactions on Intelligence Technology 4 (1) (2019) 17–23.

[58]

Tian C., Xu Y., Li Z., Zuo W., Fei L., Liu H., Attention-guided CNN for image denoising, Neural Networks 124 (2020) 117–129.

[59]

Tian C., Xu Y., Zuo W., Image denoising using deep CNN with batch renormalization, Neural Networks 121 (2020) 461–473.

[60]

Tian C., Xu Y., Zuo W., Du B., Lin C.-W., Zhang D., Designing and training of a dual CNN for image denoising, Knowledge-Based Systems 226 (2021).

[61]

Tian C., Xu Y., Zuo W., Zhang B., Fei L., Lin C.-W., Coarse-to-fine CNN for image super-resolution, IEEE Transactions on Multimedia 23 (2020) 1489–1502.

[62]

Tian C., Zheng M., Zuo W., Zhang B., Zhang Y., Zhang D., Multi-stage image denoising with the wavelet transform, Pattern Recognition (2022).

[63]

Wang Y., Chang D., Zhao Y., A new blind image denoising method based on asymmetric generative adversarial network, IET Image Processing 15 (6) (2021) 1260–1272.

[64]

Wang T., Qin Z., Zhu M., An ELU network with total variation for image denoising, in: International conference on neural information processing, Springer, 2017, pp. 227–237.

[65]

Wang X.-Y., Yang H.-Y., Fu Z.-K., A new wavelet-based image denoising using undecimated discrete wavelet transform and least squares support vector machine, Expert Systems with Applications 37 (10) (2010) 7040–7049.

[66]

Wu C., Chen X., Ji D., Zhan S., Image denoising via residual network based on perceptual 1oss, Journal of Image and Graphics 23 (10) (2018) 1483–1491.

[67]

Wu X., Liu M., Cao Y., Ren D., Zuo W., Unpaired learning of deep image denoising, in: European conference on computer vision, Springer, 2020, pp. 352–368.

[68]

Xu J., Li H., Liang Z., Zhang D., Zhang L., Real-world noisy image denoising: A new benchmark, 2018, arXiv preprint arXiv:1804.02603.

[69]

Xu M., Xie X., NFCNN: toward a noise fusion convolutional neural network for image denoising, Signal, Image and Video Processing 16 (1) (2022) 175–183.

[70]

Yang X., Xu Y., Quan Y., Ji H., Image denoising via sequential ensemble learning, IEEE Transactions on Image Processing 29 (2020) 5038–5049.

[71]

Yu X., Fu Z., Ge C., A multi-scale generative adversarial network for real-world image denoising, Signal, Image and Video Processing 16 (1) (2022) 257–264.

[72]

Yu, S., Park, B., & Jeong, J. (2019). Deep iterative down-up cnn for image denoising. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops.

[73]

Yuan Q., Zhang Q., Li J., Shen H., Zhang L., Hyperspectral image denoising employing a spatial–spectral deep residual convolutional neural network, IEEE Transactions on Geoscience and Remote Sensing 57 (2) (2018) 1205–1218.

[74]

Yue Z., Yong H., Zhao Q., Meng D., Zhang L., Variational denoising network: Toward blind noise modeling and removal, Advances in Neural Information Processing Systems 32 (2019).

[75]

Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., et al. (2021). Multi-stage progressive image restoration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 14821–14831).

[76]

Zhang, C., & Kim, J. (2019). Object detection with location-aware deformable convolution and backward attention filtering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 9452–9461).

[77]

Zhang Y., Li K., Li K., Zhong B., Fu Y., Residual non-local attention networks for image restoration, 2019, arXiv preprint arXiv:1903.10082.

[78]

Zhang, Y., Tian, Y., Kong, Y., Zhong, B., & Fu, Y. (2018). Residual dense network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2472–2481).

[79]

Zhang K., Zuo W., Chen Y., Meng D., Zhang L., Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising, IEEE Transactions on Image Processing 26 (7) (2017) 3142–3155.

Digital Library

[80]

Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017). Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3929–3938).

[81]

Zhang K., Zuo W., Zhang L., Ffdnet: Toward a fast and flexible solution for CNN-based image denoising, IEEE Transactions on Image Processing 27 (9) (2018) 4608–4622.

[82]

Zhao W., Lv Y., Liu Q., Qin B., Detail-preserving image denoising via adaptive clustering and progressive PCA thresholding, IEEE Access 6 (2017) 6303–6315.

[83]

Zhao, H., Shao, W., Bao, B., & Li, H. (2019). A simple and robust deep convolutional approach to blind image denoising. In Proceedings of the IEEE/CVF international conference on computer vision workshops.

[84]

Zheng M., Zhi K., Zeng J., Tian C., You L., A hybrid CNN for image denoising, Journal of Artificial Intelligence and Technology (2022).

[85]

ZhiPing Q., YuanQi Z., Yi S., XiangBo L., A new generative adversarial network for texture preserving image denoising, in: 2018 eighth international conference on image processing theory, tools and applications, IEEE, 2018, pp. 1–5.

[86]

Zhu J., Fang L., Ghamisi P., Deformable convolutional neural networks for hyperspectral image classification, IEEE Geoscience and Remote Sensing Letters 15 (8) (2018) 1254–1258.

[87]

Zoran D., Weiss Y., From learning models of natural image patches to whole image restoration, in: 2011 international conference on computer vision, IEEE, 2011, pp. 479–486.

Recommendations

Multi-stage image denoising with the wavelet transform
Highlights
- A dynamic convolution is used into a CNN to address limitations in depth and width of lightweight CNNs for pursuing good denoising performance.
- The combination of a signal processing technique and discriminative learning technique is ...
Abstract
Deep convolutional neural networks (CNNs) are used for image denoising via automatically mining accurate structure information. However, most of existing CNNs depend on enlarging depth of designed networks to obtain better denoising performance, ...
Image Denoising Based on the Dyadic Wavelet Transform
ICCIMA '03: Proceedings of the 5th International Conference on Computational Intelligence and Multimedia Applications

Since subsampling does not take place in image dyadic wavelet transform at each level, image representation in dyadic wavelet domain compared with wavelet series reconstruction is very redundant and part of disturbance of image dyadic wavelet ...
Edge-preserving image denoising using a deep convolutional neural network
Highlights
- This paper makes use of a deep CNN for image denoising.
- The network is trained ...
Abstract
This paper introduces a novel denoising approach making use of a deep convolutional neural network to preserve image edges. The network is trained by using the edge map obtained from the well-known Canny algorithm and aims at ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Expert Systems with Applications: An International Journal

Expert Systems with Applications: An International Journal Volume 231, Issue C

Nov 2023

1599 pages

ISSN:0957-4174

Issue’s Table of Contents

Copyright © 2023.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 30 November 2023

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents