Article

Nonlinear independent component analysis with minimal nonlinear distortion

Authors:

Laiwan ChanAuthors Info & Claims

ICML '07: Proceedings of the 24th international conference on Machine learning

Pages 1127 - 1134

https://doi.org/10.1145/1273496.1273638

Published: 20 June 2007 Publication History

Abstract

Nonlinear ICA may not result in nonlinear blind source separation, since solutions to nonlinear ICA are highly non-unique. In practice, the nonlinearity in the data generation procedure is usually not strong. Thus it is reasonable to select the solution with the mixing procedure close to linear. In this paper we propose to solve nonlinear ICA with the "minimal nonlinear distortion" principle. This is achieved by incorporating a regularization term to minimize the mean square error between the mixing mapping and the best-fitting linear one. As an application, the proposed method helps to identify linear, non-Gaussian, and acyclic causal models when mild nonlinearity exists in the data generation procedure. Using this method to separate daily returns of a set of stocks, we successfully identify their linear causal relations. The resulting causal relations give some interesting insights into the stock market.

References

[1]

Almeida, L. B. (2003). MISEP --- linear and nonlinear ICA based on mutual information. Journal of Machine Learning Research, 4, 1297--1318.

Digital Library

[2]

Almeida, L. B. (2005). Separating a real-life nonlinear image mixture. Journal of Machine Learning Research, 6, 1199--1229.

Digital Library

[3]

Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7, 1129--1159.

Digital Library

[4]

Bishop, C. (1993). Curvature-driven smoothing: a learning algorithm for feedforward networks. IEEE Trans. on Neural Networks, 4, 882--884.

Digital Library

[5]

Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. J. Amer. Statist. Assoc., 96, 1348--1360.

[6]

Harmeling, S., Ziehe, A., Kawanabe, M., & Müüller, K. (2003). Kernel-based nonlinear blind source separation. Neural Computation, 15, 1089--1124.

Digital Library

[7]

Hyvärinen, A. (1999). Fast and robust fixed-point algorithms for independent component analysis. IEEE Trans. on Neural Networks, 10(3), 626--634.

Digital Library

[8]

Hyvärinen, A., & Karthikesh, R. (2000). Sparse priors on the mixing matrix in independent component analysis. Proc. 2nd Int. Workshop on ICA and BSS (ICA2000) (pp. 477--452). Helsinki, Finland.

[9]

Hyvärinen, A., & Pajunen, P. (1999). Nonlinear independent component analysis: Existence and uniqueness results. Neural Networks, 12, 429--439.

Digital Library

[10]

Jutten, C., & Karhunen, J. (2003). Advances in nonlinear blind source separation. Proc. 4th Int. Symp. on ICA and BSS (ICA2003) (pp. 245--256). Invited paper in the special session on nonlinear ICA and BSS.

[11]

Jutten, C., & Taleb, A. (2000). Source separation: From dusk till dawn. 2nd Int. Workshop on ICA and BSS (ICA 2000) (pp. 15--26). Helsinki, Finland.

[12]

Poggio, T., Torre, V., & Koch, C. (1985). Computational vision and regularization theory. Nature, 317, 314--319.

Digital Library

[13]

Shimizu, S., Hoyer, P., Hyvärinen, A., & Kerminen, A. (2006). A linear non-Gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7, 2003--2030.

Digital Library

[14]

Taleb, A., & Jutten, C. (1999). Source separation in post-nonlinear mixtures. IEEE Trans. on Signal Processing, 47, 2807--2820.

Digital Library

[15]

Tan, Y., Wang, J., & Zurada, J. M. (2001). Nonlinear blind source separation using a radial basis function network. IEEE Trans. on Neural Networks, 12, 124--134.

Digital Library

[16]

Tikhonov, A. N., & Arsenin, V. A. (1977). Solutions of ill-posed problems. Washington: Winston & Sons.

[17]

Valpola, H. (2000). Nonlinear independent component analysis using ensemble learning: Theory. Proc. 2nd Int. Workshop on ICA and BSS (ICA2000) (pp. 251--256). Helsinki, Finland.

Cited By

Kao LChiu CLu CYang J(2013)Integration of nonlinear independent component analysis and support vector regression for stock price forecastingNeurocomputing10.1016/j.neucom.2012.06.03799(534-542)Online publication date: 1-Jan-2013
https://dl.acm.org/doi/10.1016/j.neucom.2012.06.037
Almeida MAlmeida L(2012)Nonlinear separation of show-through image mixtures using a physical model trained with ICASignal Processing10.1016/j.sigpro.2011.09.02392:4(872-884)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1016/j.sigpro.2011.09.023
Dai WWu JLu C(2012)Combining nonlinear independent component analysis and neural network for the prediction of Asian stock market indexesExpert Systems with Applications: An International Journal10.1016/j.eswa.2011.09.14539:4(4444-4452)Online publication date: 1-Mar-2012
https://dl.acm.org/doi/10.1016/j.eswa.2011.09.145
Show More Cited By

Nonlinear independent component analysis with minimal nonlinear distortion
1. Computing methodologies

Recommendations

A post nonlinear geometric algorithm for independent component analysis

Simple linear independent component analysis (ICA) algorithms work efficiently only in linear mixing environments. Whereas, a nonlinear ICA model, which is more complicated, would be more practical for general applications as it can work with both ...
Enhanced nonlinear damping for a class of singularly perturbed interconnected nonlinear systems

In this paper, we propose a method of enhanced nonlinear damping control for a class of singularly perturbed interconnected nonlinear systems (SPINSs). Instead of simply canceling out the interconnection between slow and fast subsystems, the proposed ...
Nonlinear observers: a circle criterion design and robustness analysis

Globally convergent observers are designed for a class of systems with monotonic nonlinearities. The approach is to represent the observer error system as the feedback interconnection of a linear system and a time-varying multivariable sector ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '07: Proceedings of the 24th international conference on Machine learning

June 2007

1233 pages

ISBN:9781595937933

DOI:10.1145/1273496

Editor:
Zoubin Ghahramani
University of Cambridge, United Kingdom

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Machine Learning Journal

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 June 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ICML '07 & ILP '07

Sponsor:

ICML '07 & ILP '07: The 24th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 20 - 24, 2007

Oregon, Corvalis, USA

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)1

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kao LChiu CLu CYang J(2013)Integration of nonlinear independent component analysis and support vector regression for stock price forecastingNeurocomputing10.1016/j.neucom.2012.06.03799(534-542)Online publication date: 1-Jan-2013
https://dl.acm.org/doi/10.1016/j.neucom.2012.06.037
Almeida MAlmeida L(2012)Nonlinear separation of show-through image mixtures using a physical model trained with ICASignal Processing10.1016/j.sigpro.2011.09.02392:4(872-884)Online publication date: 1-Apr-2012
https://dl.acm.org/doi/10.1016/j.sigpro.2011.09.023
Dai WWu JLu C(2012)Combining nonlinear independent component analysis and neural network for the prediction of Asian stock market indexesExpert Systems with Applications: An International Journal10.1016/j.eswa.2011.09.14539:4(4444-4452)Online publication date: 1-Mar-2012
https://dl.acm.org/doi/10.1016/j.eswa.2011.09.145
Lu C(2012)Hybridizing nonlinear independent component analysis and support vector regression with particle swarm optimization for stock index forecastingNeural Computing and Applications10.1007/s00521-012-1198-523:7-8(2417-2427)Online publication date: 17-Oct-2012
https://doi.org/10.1007/s00521-012-1198-5
Lu CWu JFan CChiu C(2009)Forecasting stock price using Nonlinear independent component analysis and support vector regression2009 IEEE International Conference on Industrial Engineering and Engineering Management10.1109/IEEM.2009.5372995(2370-2374)Online publication date: Dec-2009
https://doi.org/10.1109/IEEM.2009.5372995
Lu CChiu CYang J(2009)Integrating Nonlinear Independent Component Analysis and Neural Network in Stock Price PredictionProceedings of the 22nd International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems: Next-Generation Applied Intelligence10.1007/978-3-642-02568-6_62(614-623)Online publication date: 26-Jun-2009
https://dl.acm.org/doi/10.1007/978-3-642-02568-6_62
Zhang KChan L(2007)Kernel-based nonlinear independent component analysisProceedings of the 7th international conference on Independent component analysis and signal separation10.5555/1776684.1776724(301-308)Online publication date: 9-Sep-2007
https://dl.acm.org/doi/10.5555/1776684.1776724
Zhang KChan L(2007)Kernel-Based Nonlinear Independent Component AnalysisIndependent Component Analysis and Signal Separation10.1007/978-3-540-74494-8_38(301-308)Online publication date: 2007
https://doi.org/10.1007/978-3-540-74494-8_38

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents