Abstract
In recent years, several loss functions have been proposed for the image reconstruction task of convolutional autoencoders (CAEs). In this paper, a performance analysis of a CAE with respect to different loss functions is presented. Quality of reconstruction is analyzed using the mean Square error (MSE), binary cross-entropy (BCE), Sobel, Laplacian, and Focal binary loss functions. To evaluate the performance of different loss functions, a vanilla autoencoder is trained on eight datasets having diversity in terms of application domains, image dimension, color space, and the number of images in the dataset. MSE, peak signal to noise ratio (PSNR), and structural similarity index (SSIM) metrics have been used as the performance measures on all eight datasets. The assessment shows that the MSE loss function outperforms two datasets with a small image dimension and a large number of images. At the same time, BCE excels on six datasets with high image dimensions and a small number of training samples in datasets compared with the Sobel and Laplacian loss functions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Odaibo, S.: Tutorial: deriving the standard variational autoencoder (VAE) loss function. arXiv preprint arXiv:1907.08956 (2019)
Kingma, D.P., Welling, M.: An introduction to variational autoencoders. arXiv preprint arXiv:1906.02691 (2019)
Gondara, L.: Medical image denoising using convolutional denoising autoencoders. In: 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW), pp. 241–246. IEEE (2016)
Fazlali, H., Shirani, S., McDonald, M., Brown, D., Kirubarajan, T.: Aerial image dehazing using a deep convolutional autoencoder. Multimedia Tools Appl. 79(39), 29493–29511 (2020)
Hong, J.-P., Cho, S.-J., Lee, J., Ji, S.-W., Ko, S.-J.: Single image deblurring based on auxiliary Sobel loss function. In: 2020 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp. 1–3. IEEE (2020)
Chen, M., Shi, X., Zhang, Y., Wu, D., Guizani, M.: Deep features learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans. Big Data 7, 750–758 (2017)
Chow, J.K., Su, Z., Wu, J., Tan, P.S., Mao, X., Wang, Y.-H.: Anomaly detection of defects on concrete structures with the convolutional autoencoder. Adv. Eng. Inform. 45, 101105 (2020)
Bai, J., Dai, X., Wu, Q., Xie, L.: Limited-view CT reconstruction based on autoencoder-like generative adversarial networks with joint loss. In: 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 5570–5574. IEEE (2018)
Liu, X., Gherbi, A., Wei, Z., Li, W., Cheriet, M.: Multispectral image reconstruction from color images using enhanced variational autoencoder and generative adversarial network. IEEE Access 9, 1666–1679 (2020)
Pandey, R.K., Saha, N., Karmakar, S., Ramakrishnan, A.G.: MSCE: an edge-preserving robust loss function for improving super-resolution algorithms. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11306, pp. 566–575. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04224-0_49
Zhao, H., Gallo, O., Frosio, I., Kautz, J.: Loss functions for image restoration with neural networks. IEEE Trans. Comput. imaging 3(1), 47–57 (2016)
Li, S., Xu, X., Nie, L., Chua, T.-S.: Laplacian-steered neural style transfer. In: Proceedings of the 25th ACM International Conference on Multimedia, pp. 1716–1724 (2017)
Zhu, Q., Wang, H., Zhang, R.: Wavelet loss function for auto-encoder. IEEE Access 9, 27101–27108 (2021)
Ephraim, Y., Malah, D.: Speech enhancement using a minimum mean-square error log-spectral amplitude estimator. IEEE Trans. Acoust. Speech Signal Process. 33(2), 443–445 (1985)
Creswell, A., Arulkumaran, K., Bharath, A.A.: On denoising autoencoders trained to minimise binary cross-entropy. arXiv preprint arXiv:1708.08487 (2017)
Lu, Z., Chen, Y.: Single image super resolution based on a modified U-net with mixed gradient loss. arXiv preprint arXiv:1911.09428 (2019)
Kanopoulos, N., Vasanthavada, N., Baker, R.L.: Design of an image edge detection filter using the Sobel operator. IEEE J. Solid State Circuits 23(2), 358–367 (1988)
Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollar, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), October 2017
Wang, C., Deng, C., Wang, S.: Imbalance-XGBoost: leveraging weighted and focal losses for binary label-imbalanced classification with XGBoost. Pattern Recognit. Lett. 136, 190–197 (2020)
Ma, X., Huang, H., Wang, Y., Romano, S., Erfani, S., Bailey, J.: Normalized loss functions for deep learning with noisy labels. In: International Conference on Machine Learning, pp. 6543–6553. PMLR (2020)
Pogorelov, K., et al.: KVASIR: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 164–169 (2017)
Chakrabarty, N.: Brain MRI images for brain tumor detection. https://www.kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection (2019)
Rahman, T., et al.: Exploring the effect of image enhancement techniques on COVID-19 detection using chest X-ray images. Comput. Biol. Med. 132, 104319 (2021)
Krizhevsky, A., Nair, V., Hinton, G.: CIFAR-10 (Canadian institute for advanced research), vol. 5, vol. 4 (2010). http://www.cs.toronto.edu/kriz/cifar.html
Vicente, T.F.Y., Hou, L., Yu, C.-P., Hoai, M., Samaras, D.: Large-scale training of shadow detectors with noisily-annotated shadow examples. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9910, pp. 816–832. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46466-4_49
Nene, S.A., Nayar, S.K., Murase, H., et al.: Columbia object image library (COIL-100) (1996)
LeCun, Y., Cortes, C.: MNIST handwritten digit database (2010). http://yann.lecun.com/exdb/mnist/
Eitz, M., Hays, J., Alexa, M.: How do humans sketch objects? ACM Trans. Graph. (Proc. SIGGRAPH) 31(4), 44:1–44:10 (2012)
Silva, E.A., Panetta, K., Agaian, S.S.: Quantifying image similarity using measure of enhancement by entropy. In: Mobile Multimedia/Image Processing for Military and Security Applications 2007, vol. 6579, p. 65790U. International Society for Optics and Photonics (2007)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Khare, N., Thakur, P.S., Khanna, P., Ojha, A. (2022). Analysis of Loss Functions for Image Reconstruction Using Convolutional Autoencoder. In: Raman, B., Murala, S., Chowdhury, A., Dhall, A., Goyal, P. (eds) Computer Vision and Image Processing. CVIP 2021. Communications in Computer and Information Science, vol 1568. Springer, Cham. https://doi.org/10.1007/978-3-031-11349-9_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-11349-9_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11348-2
Online ISBN: 978-3-031-11349-9
eBook Packages: Computer ScienceComputer Science (R0)