Companion Classification Losses for Regression Problems
Pages 219 - 230
Abstract
By their very nature, regression problems can be transformed into classification problems by discretizing their target variable. Within this perspective, in this work we investigate the possibility of improving the performance of deep machine learning models in regression scenarios through a training strategy that combines different classification and regression objectives. In particular, we train deep neural networks using the mean squared error along with categorical cross-entropy and the novel Fisher loss as companion losses. Finally, we will compare experimentally the results of these companion loss methods with the ones obtained using the standard mean squared loss.
References
[1]
Abadi, M., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/, software available from tensorflow.org
[2]
Bengio Y, Courville A, and Vincent P Representation learning: a review and new perspectives IEEE Trans. Pattern Anal. Mach. Intell. 2013 35 8 1798-1828
[3]
Caruana R Multitask learning Mach. Learn. 1997 28 41-75
[4]
Chang CC and Lin CJ LIBSVM: a library for support vector machines ACM Trans. Intell. Syst. Technol. (TIST) 2011 2 3 1-27
[5]
Chen, J., Cheng, L., Yang, X., Liang, J., Quan, B., Li, S.: Joint learning with both classification and regression models for age prediction. In: Journal of Physics: Conference Series, vol. 1168, p. 032016. IOP Publishing (2019)
[6]
Chollet, F., et al.: Keras (2015). https://github.com/fchollet/keras
[7]
Christensen RHB ordinal-regression models for ordinal data R Packag. Version 2015 28 2015
[8]
Diaz-Vico D and Dorronsoro JR Deep least squares fisher discriminant analysis IEEE Trans. Neural Netw. Learn. Syst. 2019 31 8 2752-2763
[9]
Díaz-Vico D, Fernández A, and Dorronsoro JR Sanjurjo González H, Pastor López I, García Bringas P, Quintián H, and Corchado E Companion losses for deep neural networks Hybrid Artificial Intelligent Systems 2021 Cham Springer 538-549
[10]
Díaz-Vico, D., Fernández, A., Dorronsoro, J.R.: Companion losses for ordinal regression. In: Garcia Bringas, P., et al. (eds.) Hybrid Artificial Intelligent Systems. HAIS 2022. LNCS, vol. 13469, pp. 211–222. Springer, Cham (2022).
[11]
Glocker B, Pauly O, Konukoglu E, and Criminisi A Fitzgibbon A, Lazebnik S, Perona P, Sato Y, and Schmid C Joint classification-regression forests for spatially structured multi-object segmentation Computer Vision – ECCV 2012 2012 Heidelberg Springer 870-881
[12]
Liu M, Zhang J, Adeli E, and Shen D Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins DL, and Duchesne S Deep multi-task multi-channel learning for joint classification and regression of brain status Medical Image Computing and Computer Assisted Intervention – MICCAI 2017 2017 Cham Springer 3-11
[13]
Liu M, Zhang J, Adeli E, and Shen D Joint classification and regression via deep multi-task multi-channel learning for Alzheimer’s disease diagnosis IEEE Trans. Biomed. Eng. 2018 66 5 1195-1206
[14]
Paszke, A., et al.: Automatic differentiation in pytorch. In: Advances in Neural Information Processing Systems, vol. 32, pp. 8024–8035. Curran Associates, Inc. (2019)
[15]
Schulter, S., Leistner, C., Wohlhart, P., Roth, P.M., Bischof, H.: Alternating regression forests for object detection and pose estimation. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), December 2013
[16]
Schulter, S., Leistner, C., Wohlhart, P., Roth, P.M., Bischof, H.: Accurate object detection with joint classification-regression random forests. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2014
[17]
Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1(6), 80–83 (1945). https://www.jstor.org/stable/3001968
[18]
Zhang Z, Dai G, Xu C, and Jordan MI Regularized discriminant analysis, ridge regression and beyond J. Mach. Learn. Res. 2010 11 2199-2228
Recommendations
Robust Losses in Deep Regression
Hybrid Artificial Intelligent SystemsAbstractWhat is the noise distribution of a given regression problem is not known in advance and, given that the assumption on which noise is present is reflected on the loss to be used, a consequence is that neither the loss choice should be fixed ...
Multi-style learning with denoising autoencoders for acoustic modeling in the internet of things (IoT)
Multi-style Learning (Multi-style training+Deep Learning) relies on multiple layers of nonlinear data transformations to synthesize additional examples of representative variations.Rather than artificially creating data samples, a deep denoising ...
An overview on the shrinkage properties of partial least squares regression
The aim of this paper is twofold. In the first part, we recapitulate the main results regarding the shrinkage properties of partial least squares (PLS) regression. In particular, we give an alternative proof of the shape of the PLS shrinkage factors. It ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Sep 2023
788 pages
ISBN:978-3-031-40724-6
DOI:10.1007/978-3-031-40725-3
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 05 September 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025