Extensive semi-quantitative regression
Abstract
In this paper, we propose and solve a new machine learning problem called the extensive semi-quantitative regression, where the information about some target values is incomplete; we only know their lower bounds and/or upper bounds instead of their exact values. To employ the information efficiently in extensive semi-quantitative regression, we introduce a local graph to capture the geometric structure for the samples with the exact target values and the target bounds, and construct a graph-based support vector regressor, called ESQ-SVR. The efficiency of our ESQ-SVR is supported by the results of preliminary experiments conducted on both the artificial and real world datasets. HighlightsWe propose a new problem called extensive semi-quantitative regression.A graph-based support vector regressor is used to solve the above problem.Our approach could capture the geometric structure for the samples.It also bounded the target values and the value ranges.The efficiency of the approach is supported by the results of experiments.
References
[1]
N.R. Draper, H. Smith, E. Pownell, Wiley, New York, 1966.
[2]
D.G. Kleinbaum, Applied regression analysis and multivariable methods, CengageBrain. Com. (2007).
[3]
D.C. Montgomery, E.A. Peck, G.G. Vining, Wiley, 2012.
[4]
C.-J. Lu, T.-S. Lee, C.-C. Chiu, Financial time series forecasting using independent component analysis and support vector regression, Decis. Support Syst., 47 (2009) 115-125.
[5]
S. Chatterjee, A.S. Hadi, Regression Analysis by Example, John Wiley & Sons, 2013.
[6]
T.T. Wu, Y.F. Chen, T. Hastie, E. Sobel, K. Lange, Genome-wide association analysis by lasso penalized logistic regression, Bioinformatics, 25 (2009) 714-721.
[7]
X. Shao, C. Tan, C. Voss, S. Li, N. Deng, G. Bader, A regression framework incorporating quantitative and negative interaction data improves quantitative prediction of pdz domain-peptide interaction from primary sequence, Bioinformatics, 27 (2011) 383-390.
[8]
P.J. Rousseeuw, Least median of squares regression, J. Am. Stat. Assoc., 79 (1984) 871-880.
[9]
X. Liu, L.-F. Lee, Two-stage least squares estimation of spatial autoregressive models with endogenous regressors and many instruments, Econom. Rev., 32 (2013) 734-753.
[10]
G. Mateos, J.A. Bazerque, G.B. Giannakis, Distributed sparse linear regression, IEEE Trans. Signal Process., 58 (2010) 5262-5276.
[11]
D.F. Specht, A general regression neural network, IEEE Trans. Neural Netw., 2 (1991) 568-576.
[12]
S.S. Haykin, S.S. Haykin, S.S. Haykin, S.S. Haykin, Prentice Hall, New York, 2009.
[13]
G. Mateos, G.B. Giannakis, Robust nonparametric regression via sparsity control with application to load curve data cleansing, IEEE Trans. Signal Process., 60 (2012) 1571-1584.
[14]
V. Kekatos, G.B. Giannakis, Sparse volterra and polynomial regression models: recoverability and estimation, IEEE Trans. Signal Process., 59 (2011) 5907-5920.
[15]
N.Y. Deng, Y.J. Tian, C.H. Zhang, CRC Press, 2013.
[16]
A.J. Smola, B. Schlkopf, A tutorial on support vector regression, Stat. Comput., 14 (2004) 199-222.
[17]
M. Soltani, T.B. Moghaddam, M.R. Karim, S. Shamshirband, C. Sudheer, Stiffness performance of polyethylene terephthalate modified asphalt mixtures estimation using support vector machine-firefly algorithm, Measurement, 63 (2015) 232-239.
[18]
S. Shamshirband, D. Petkovic, H. Javidnia, A. Gani, Sensor data fusion by support vector regression methodology: a comparative study, IEEE Sens. J., 15 (2015) 850-854.
[19]
S. Shamshirband, A. Tavakkoli, C.B. Roy, S. Motamedi, K.-I. Song, R. Hashim, S.M. Islam, Hybrid intelligent model for approximating unconfined compressive strength of cement-based bricks with odd-valued array of peat content (029%), Powder Technol., 284 (2015) 560-570.
[20]
A. Sanchez, Advanced support vector machines and kernel methods, Neurocomputing, 55 (2003) 5-20.
[21]
J.D. Lafferty, L.A. Wasserman, Statistical analysis of semi-supervised regression, Adv. Neural Inf. Process. Syst., 21 (2007) 801-808.
[22]
O.L. Mangasarian, J.W. Shavlik, E.W. Wild, Knowledge-based kernel approximation, J. Mach. Learn. Res., 5 (2004) 1127-1141.
[23]
D. Zheng, J. Wang, Y. Zhao, Non-flat function estimation with a multi-scale support vector regression, Neurocomputing, 70 (2006) 420-429.
[24]
Z.H. Zhou, M.Li, Semi-supervised regression with co-training, in: Ijcai-05, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, UK, July 30-August 2005, pp. 908916.
[25]
F. Lauer, G. Bloch, Incorporating prior knowledge in support vector regression, Mach. Learn., 70 (2008) 89-118.
[26]
J. Zhou, B. Duan, J. Huang, N. Li, Incorporating prior knowledge and multi-kernel into linear programming support vector regression, Soft Comput., 19 (2014) 2047-2061.
[27]
J.R. Chen, B.H. Chang, J.E. Allen, M.A. Stiffler, G. MacBeath, Predicting pdz domain-peptide interactions from primary sequences, Nat. Biotechnol., 26 (2008) 1041-1045.
[28]
S. Satchell, J. Knight, Forecasting Volatility in the Financial Markets, Butterworth-Heinemann, 2011.
[29]
M. Koetter, T. Poghosyan, Real estate prices and bank stability, J. Bank. Financ., 34 (2010) 1129-1138.
[30]
D. Basak, S. Pal, D.C. Patranabis, Support vector regression, Neural Inf. Process.-Lett. Rev., 11 (2007) 203-224.
[31]
T. Hastie, R. Tibshirani, J. Friedman, Linear Methods for Regression, Springer, 2009.
[32]
O. Chapelle, B. Schlkopf, A. Zien, MIT Press, Cambridge, 2006.
[33]
X. Zhu, A.B. Goldberg, Introduction to semi-supervised learning, Synth. Lect. Artif. Intell. Mach. Learn., 3 (2009) 1-130.
[34]
M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization:a geometric framework for learning from labeled and unlabeled example, J. Mach. Learn. Res., 7 (2006) 2399-2434.
[35]
V.N. Vapnik, The Nature of Statistical Learning Theory, Springer New York Inc., New York, NY, USA, 1995.
[36]
B. Scholkpf, A. Smola, MA: MIT Press, Cambridge, 2002.
[37]
M. Belkin, P. Niyogi, V. Sindhwani, Manifold regularization:a geometric framework for learning from labeled and unlabeled examples, J. Mach. Learn. Res., 7 (2006) 2399-2434.
[38]
S. Melacci, M. Belkin, Laplacian support vector machines trained in the primal, J. Mach. Learn. Res., 12 (2011) 1149-1184.
[39]
O.L. Mangasarian, Nonlinear Programming, SIAM, 1994
[40]
S. Sonnenburg, G. Rtsch, C. Schfer, B. Schlkopf, Large scale multiple kernel learning, J. Mach. Learn. Res., 7 (2006) 1531-1565.
[41]
L. Chen, I.W. Tsang, D. Xu, Laplacian embedded regression for scalable manifold regularization, IEEE Trans. Neural Netw. Learn. Syst., 23 (2012) 902-915.
[42]
O.L. Mangasarian, E.W. Wild, Nonlinear knowledge in kernel approximation, IEEE Trans. Neural Netw., 18 (2007) 300-306.
[43]
C.E. Rasmussen, Gaussian Processes for Machine Learning, MIT Press, 2006.
[44]
G.-B. Huang, H. Zhou, X. Ding, R. Zhang, Extreme learning machine for regression and multiclass classification, IEEE Trans. Syst. Man Cybern. Part B: Cybern., 42 (2012) 513-529.
[45]
G.-B. Huang, Q.-Y. Zhu, C.-K. Siew, Extreme learning machine:theory and application, Neurocomputing, 70 (2006) 489-501.
[46]
G.-B. Huang, L. Chen, C.-K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Netw., 17 (2006) 879-892.
[47]
C.-C. Chang, C.-J. Lin, Libsvm:a library for support vector machines, ACM Trans. Intell. Syst. Technol., 2 (2011) 27.
[48]
MATLAB, The MathWorks, Inc, 2007. URL http://www.mathworks.com
[49]
Y. Shao, C. Zhang, Z. Yang, L. Jing, N. Deng, -twin support vector machine for regression, Neural Comput. Appl., 23 (2013) 175-185.
[50]
S. Weisberg, Wiley, 2005.
[51]
R.G. Staudte, S.J. Sheather, Wiley-Interscience, 2011.
[52]
M.A. Stiffler, J.R. Chen, V.P. Grantcharova, Y. Lei, D. Fuchs, J.E. Allen, L.A. Zaslavskaia, G. MacBeath, Pdz domain binding selectivity is optimized across the mouse proteome, Sci. Signal., 317 (2007) 364.
[53]
R.A. Johnson, D.W. Wichern, Prentice Hall Upper Saddle River, NJ, 2002.
[54]
S. Mak, L. Choy, W. Ho, Region-specific estimates of the determinants of real estate investment in china, Urban Stud., 49 (2012) 741-755.
Recommendations
TSVR: An efficient Twin Support Vector Machine for regression
The learning speed of classical Support Vector Regression (SVR) is low, since it is constructed based on the minimization of a convex quadratic function subject to the pair groups of linear inequality constraints for all training samples. In this paper ...
A heuristic weight-setting strategy and iteratively updating algorithm for weighted least-squares support vector regression
Weighted least-squares support vector machine (WLS-SVM) is an improved version of least-squares support vector machine (LS-SVM). It adds weights on error variables to correct the biased estimation of LS-SVM. Traditional weight-setting algorithm for WLS-...
Efficient SVM Regression Training with SMO
The sequential minimal optimization algorithm (SMO) has been shown to be an effective method for training support vector machines (SVMs) on classification tasks defined on sparse data sets. SMO differs from most SVM algorithms in that it does not ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © Elsevier B.V.
Publisher
Elsevier Science Publishers B. V.
Netherlands
Publication History
Published: 19 December 2016
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Feb 2025