research-article

Dynamic Neural Network for Lossy-to-Lossless Image Coding

Authors:

Tassnim Dardouri,

Mounir Kaaniche,

Amel Benazza-Benyahia,

Jean-Christophe PesquetAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 31

Pages 569 - 584

https://doi.org/10.1109/TIP.2021.3132825

Published: 01 January 2022 Publication History

Abstract

Lifting-based wavelet transform has been extensively used for efficient compression of various types of visual data. Generally, the performance of such coding schemes strongly depends on the lifting operators used, namely the prediction and update filters. Unlike conventional schemes based on linear filters, we propose, in this paper, to learn these operators by exploiting neural networks. More precisely, a classical Fully Connected Neural Network (FCNN) architecture is firstly employed to perform the prediction and update. Then, we propose to improve this FCNN-based Lifting Scheme (LS) in order to better take into account the input image to be encoded. Thus, a novel dynamical FCNN model is developed, making the learning process adaptive to the input image contents for which two adaptive learning techniques are proposed. While the first one resorts to an iterative algorithm where the computation of two kinds of variables is performed in an alternating manner, the second learning method aims to learn the model parameters directly through a reformulation of the loss function. Experimental results carried out on various test images show the benefits of the proposed approaches in the context of lossy and lossless image compression.

References

[1]

T. Guo, H. S. Mousavi, T. H. Vu, and V. Monga, “Deep wavelet prediction for image super-resolution,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Honolulu, HI, USA, Jul. 2017, pp. 104–113.

[2]

M. Antonini, M. Barlaud, P. Mathieu, and I. Daubechies, “Image coding using wavelet transform,” IEEE Trans. Image Process., vol. 1, no. 2, pp. 205–220, Apr. 1992.

Digital Library

[3]

D. Taubman, “High performance scalable image compression with EBCOT,” IEEE Trans. Image Process., vol. 9, no. 7, pp. 1158–1170, Jul. 2000.

Digital Library

[4]

B. Pesquet-Popescu and V. Bottreau, “Three-dimensional lifting schemes for motion compensated video compression,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., vol. 3, Salt Lake City, UT, USA, May 2001, pp. 1793–1796.

[5]

M. Kaaniche, A. Benazza-Benyahia, B. Pesquet-Popescu, and J. C. Pesquet, “Vector lifting schemes for stereo image coding,” IEEE Trans. Image Process., vol. 18, no. 11, pp. 2463–2475, Nov. 2009.

Digital Library

[6]

Y. Xing, M. Kaaniche, B. Pesquet-Popescu, and F. Dufaux, “Adaptive nonseparable vector lifting scheme for digital holographic data compression,” Appl. Opt., vol. 54, no. 1, p. A98, Jan. 2015.

[7]

A. R. Calderbank, I. Daubechies, W. Sweldens, and B.-L. Yeo, “Wavelet transforms that map integers to integers,” Appl. Comput. Harmon. Anal., vol. 5, no. 3, pp. 332–369, Jul. 1998.

[8]

J.-H. Jacobsen, A. W. M. Smeulders, and E. Oyallon, “I-RevNet: Deep invertible networks,” in Proc. Int. Conf. Learn. Represent., Vancouver, BC, Canada, May 2018, pp. 1–11.

[9]

W. Sweldens, “The lifting scheme: A custom-design construction of biorthogonal wavelets,” Appl. Comput. Harmon. Anal., vol. 3, no. 2, pp. 186–200, Apr. 1996.

[10]

I. Daubechies and W. Sweldens, “Factoring wavelet transforms into lifting steps,” J. Fourier Anal. Appl., vol. 4, no. 3, pp. 247–269, 1998.

[11]

F. J. Hampson and J.-C. Pesquet, “M-band nonlinear subband decompositions with perfect reconstruction,” IEEE Trans. Image Process., vol. 7, no. 11, pp. 1547–1560, Nov. 1998.

Digital Library

[12]

J. Sole and P. Salembier, “Generalized lifting prediction optimization applied to lossless image compression,” IEEE Signal Process. Lett., vol. 14, no. 10, pp. 695–698, Oct. 2007.

[13]

Y. Liu and K. Ngi Ngan, “Weighted adaptive lifting-based wavelet transform for image coding,” IEEE Trans. Image Process., vol. 17, no. 4, pp. 500–511, Apr. 2008.

Digital Library

[14]

M. Kaaniche, J.-C. Pesquet, A. Benazza-Benyahia, and B. Pesquet-Popescu, “Two-dimensional non separable adaptive lifting scheme for still and stereo image coding,” in Proc. IEEE Int. Conf. Acoust., Speech Signal Process., Dallas, TX, USA, Mar. 2010, pp. 1298–1301.

[15]

A. Gouze, M. Antonini, M. Barlaud, and B. Macq, “Design of signal-adapted multidimensional lifting scheme for lossy coding,” IEEE Trans. Image Process., vol. 13, no. 12, pp. 1589–1603, Dec. 2004.

Digital Library

[16]

M. Kaaniche, B. Pesquet-Popescu, A. Benazza-Benyahia, and J.-C. Pesquet, “Adaptive lifting scheme with sparse criteria for image coding,” EURASIP J. Adv. Signal Process., vol. 2012, no. 1, pp. 1–22, Jan. 2012.

[17]

A. Benazza-Benyahia, J.-C. Pesquet, J. Hattay, and H. Masmoudi, “Block-based adaptive vector lifting schemes for multichannel image coding,” EURASIP Int. J. Image Video Process., vol. 2007, no. 1, p. 10, Jan. 2007.

[18]

B. Pesquet-Popescu, “Two-stage adaptive filter bank,” European Patent 1 119 911, Jul. 27, 1999.

[19]

M. Kaaniche, A. Benazza-Benyahia, B. Pesquet-Popescu, and J.-C. Pesquet, “Non-separable lifting scheme with adaptive update step for still and stereo image coding,” Signal Process., vol. 91, no. 12, pp. 2767–2782, Dec. 2011.

[20]

W. Ding, F. Wu, X. Wu, S. Li, and H. Li, “Adaptive directional lifting-based wavelet transform for image coding,” IEEE Trans. Image Process., vol. 16, no. 2, pp. 416–427, Feb. 2007.

Digital Library

[21]

E. Martinez-Enriquez, J. Cid-Sueiro, F. Diaz-de-Maria, and A. Ortega, “Directional transforms for video coding based on lifting on graphs,” IEEE Trans. Circuits Syst. Video Technol., vol. 28, no. 4, pp. 933–946, Apr. 2018.

[22]

G. Todericiet al., “Full resolution image compression with recurrent neural networks,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Las Vegas, NV, USA, Jul. 2016, pp. 5306–5314.

[23]

J. Ballé, V. Laparra, and E. P. Simoncelli, “End-to-end optimized image compression,” in Proc. Int. Conf. Learn. Represent. (ICLR), Toulon, France, Apr. 2017, pp. 1–27.

[24]

O. Rippel and L. Bourdev, “Real-time adaptive image compression,” in Proc. Int. Conf. Mach. Learn., Sydney, NSW, Australia, Aug. 2017, pp. 1–9.

[25]

M. Li, W. Zuo, S. Gu, D. Zhao, and D. Zhang, “Learning convolutional networks for content-weighted image compression,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake City, UT, USA, Jun. 2018, pp. 3214–3223.

[26]

J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in Proc. Int. Conf. Learn. Represent., Vancouver, BC, Canada, May 2018, pp. 1–47.

[27]

D. Minnen, J. Ballé, and G. Toderici, “Joint autoregressive and hierarchical priors for learned image compression,” in Proc. Int. Conf. Neural Inf. Process. Syst., Montréal, QC, Canada, Dec. 2018, pp. 10794–10803.

[28]

E. Agustsson, M. Tschannen, F. Mentzer, R. Timofte, and L. Van Gool, “Generative adversarial networks for extreme learned image compression,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), New Orleans, LO, USA, Oct. 2019, pp. 1–31.

[29]

M. A. Yilmaz and A. Murat Tekalp, “Effect of architectures and training methods on the performance of learned video frame prediction,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Taipei, Taiwan, Sep. 2019, pp. 1–5.

[30]

I. Schiopu and A. Munteanu, “Macro-pixel prediction based on convolutional neural networks for lossless compression of light field images,” in Proc. 25th IEEE Int. Conf. Image Process. (ICIP), Athens, Greece, Oct. 2018, pp. 445–449.

[31]

J. Li, B. Li, J. Xu, R. Xiong, and W. Gao, “Fully connected network-based intra prediction for image coding,” IEEE Trans. Image Process., vol. 27, no. 7, pp. 3236–3247, Jul. 2018.

[32]

T. Dumas, A. Roumy, and C. Guillemot, “Context-adaptive neural network-based prediction for image compression,” IEEE Trans. Image Process., vol. 29, pp. 679–693, 2020.

Digital Library

[33]

D. Liu, H. Ma, Z. Xiong, and F. Wu, “CNN-based DCT-like transform for image compression,” in Proc. Int. Conf. MultiMedia Modeling (MMM), Bangkok, Thailand, Jan. 2018, pp. 61–72.

[34]

E. Ahanonu, M. Marcellin, and A. Bilgin, “Lossless image compression using reversible integer wavelet transforms and convolutional neural networks,” in Proc. Data Compress. Conf., Snowbird, UT, USA, Mar. 2018, p. 395.

[35]

P. Akyazi and T. Ebrahimi, “Learning-based image compression using convolutional autoencoder and wavelet decomposition,” in Proc. Conf. Comput. Vis. Pattern Recognit. Workshops, Long Beach, CA, USA, Jun. 2019, pp. 1–5.

[36]

H. Ma, D. Liu, R. Xiong, and F. Wu, “IWave: CNN-based wavelet-like transform for image compression,” IEEE Trans. Multimedia, vol. 22, no. 7, pp. 1667–1679, Jul. 2020.

[37]

I. Schiopu and A. Munteanu, “Deep-learning-based lossless image coding,” IEEE Trans. Circuits Syst. Video Technol., vol. 30, no. 7, pp. 1829–1842, Jul. 2020.

Digital Library

[38]

I. Schiopu and A. Munteanu, “A study of prediction methods based on machine learning techniques for lossless image coding,” in Proc. IEEE Int. Conf. Image Process. (ICIP), Abu Dhabi, United Arab Emirates, Oct. 2020, pp. 1–5.

[39]

F. Mentzer, E. Agustsson, M. Tschannen, R. Timofte, and L. Van Gool, “Practical full resolution learned lossless image compression,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Long Beach, CA, USA, Jun. 2019, pp. 1–14.

[40]

F. Mentzer, L. Van Gool, and M. Tschannen, “Learning better lossless compression using lossy compression,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, Jun. 2020, pp. 6637–6646.

[41]

T. Dardouri, M. Kaaniche, A. Benazza-Benyahia, and J.-C. Pesquet, “Optimized lifting scheme based on a dynamical fully connected network for image coding,” in Proc. Int. Conf. Image Process., Abu Dhabi, United Arab Emirates, Oct. 2020, pp. 1–5.

[42]

Y.-K. Sun, “A two-dimensional lifting scheme of integer wavelet transform for lossless image compression,” in Proc. Int. Conf. Image Process., vol. 1, Singapore, Oct. 2004, pp. 497–500.

[43]

Y. Wang, X. Fan, C. Jia, D. Zhao, and W. Gao, “Neural network based inter prediction for HEVC,” in Proc. IEEE Int. Conf. Multimedia Expo (ICME), San Diego, CA, USA, Jul. 2018, pp. 1–6.

[44]

D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” in Proc. Int. Conf. Learn. Represent., San Siego, CA, USA, May 2015, pp. 1–15.

[45]

N. Asuni and A. Giachetti, “Test images: A large-scale archive for testing visual devices and basic image processing algorithms,” in Proc. Eurograph. Italian Conf., Cagliari, Italy, Sep. 2014, pp. 1–3.

[46]

N. Asuni and A. Giachetti, “TESTIMAGES: A large data archive for display and algorithm testing,” J. Graph. Tools, vol. 17, no. 4, pp. 113–125, Oct. 2013.

[47]

E. Prashnani, H. Cai, Y. Mostofi, and P. Sen, “PieAPP: Perceptual image-error assessment through pairwise preference,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit., Salt Lake City, UT, USA, Jun. 2018, pp. 1–10.

[48]

G. Bjøntegaard, “Calculation of average PSNR differences between RD curves,” ITU, Austin, TX, USA, Tech. Rep. SG16 VCEG-M33, 2001.

[49]

G. J. Sullivan, J.-R. Ohm, W.-J. Han, and T. Wiegand, “Overview of the high efficiency video coding (HEVC) standard,” IEEE Trans. Circuits Syst. Video Technol., vol. 22, no. 12, pp. 1649–1668, Dec. 2012.

Digital Library

[50]

F. Bossen, Common HM Test Conditions and Software Reference Configurations, document JCTVC-G1100, San Jose, CA, USA, 2012.

[51]

G. Valenzise, A. Purica, V. Hulusic, and M. Cagnazzo, “Quality assessment of deep-learning-based image compression,” in Proc. Int. Workshop Multimedia Signal Process., Vancouver, BC, Canada, Aug. 2018, pp. 1–6.

[52]

O. Dhifallah, M. Kaaniche, and A. Benazza-Benyahia, “Efficient joint multiscale decomposition for color stereo image coding,” in Proc. Eur. Signal Image Process. Conf., Lisbon, Portugal, Sep. 2014, pp. 1–5.

Cited By

He JZhang TZhang ZYu TZhang Y(2024)Efficient Dynamic Correspondence NetworkIEEE Transactions on Image Processing10.1109/TIP.2023.333459433(228-240)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3334594
Dardouri TKaaniche MBenazza-Benyahia ADauphin GPesquet J(2024)Joint Learning of Fully Connected Network Models in Lifting Based Image CodersIEEE Transactions on Image Processing10.1109/TIP.2023.333327933(134-148)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3333279
Khan ZDardouri TKaaniche MDauphin G(2023)NNCD-IQA: A new neural networks based compressed database for image quality assessmentMultimedia Tools and Applications10.1007/s11042-022-13842-882:9(13951-13971)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13842-8

Recommendations

Lossless-by-Lossy Coding for Scalable Lossless Image Compression

This paper presents a method of scalable lossless image compression by means of lossy coding. A progressive decoding capability and a full decoding for the lossless rendition are equipped with the losslessly encoded bit stream. Embedded coding is ...
Subband/transform image coding for lossy and lossless compression
Deep Lossy Plus Residual Coding for Lossless and Near-Lossless Image Compression
Lossless and near-lossless image compression is of paramount importance to professional users in many technical fields, such as medicine, remote sensing, precision engineering and scientific research. But despite rapidly growing research interests in ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 31, Issue

2022

3518 pages

ISSN:1057-7149

Issue’s Table of Contents

1941-0042 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See https://www.ieee.org/publications/rights/index.html for more information.

Publisher

IEEE Press

Publication History

Published: 01 January 2022

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

He JZhang TZhang ZYu TZhang Y(2024)Efficient Dynamic Correspondence NetworkIEEE Transactions on Image Processing10.1109/TIP.2023.333459433(228-240)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3334594
Dardouri TKaaniche MBenazza-Benyahia ADauphin GPesquet J(2024)Joint Learning of Fully Connected Network Models in Lifting Based Image CodersIEEE Transactions on Image Processing10.1109/TIP.2023.333327933(134-148)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3333279
Khan ZDardouri TKaaniche MDauphin G(2023)NNCD-IQA: A new neural networks based compressed database for image quality assessmentMultimedia Tools and Applications10.1007/s11042-022-13842-882:9(13951-13971)Online publication date: 1-Apr-2023
https://dl.acm.org/doi/10.1007/s11042-022-13842-8

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents