Abstract
Reservoir computing such as Echo State Network (ESN) and Liquid State Machine (LSM) has been successfully applied in dynamical system modeling. However, there is an antagonistic trade-off between the non-linear mapping capacity and the short-term memory capacity in single-reservoir networks, especially when the input signals contain high non-linearity and short-term dependencies. To address this problem, we propose a novel reservoir computing model called Dual-Reservoir Network (DRN), which connects two reservoirs with an unsupervised encoder such as PCA. Specifically, we allow these two adversarial capacities to be decoupled and enhanced in the dual reservoirs respectively. In our experiments, we first verify DRN’s feasibility on an extended polynomial system, which allows us to control the nonlinearity and short-term dependencies of data. In addition, we demonstrate the effectiveness of DRN on the synthesis and real-world time series predictions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bingham, E., Mannila, H.: Random projection in dimensionality reduction: applications to image and text data. In: Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 245–250. ACM (2001)
Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Reservoir computing and extreme learning machines for non-linear time-series data analysis. Neural Netw. 38, 76–89 (2013)
Butcher, J., Verstraeten, D., Schrauwen, B., Day, C., Haycock, P.: Extending reservoir computing with random static projections: a hybrid between extreme learning and RC. In: 18th European Symposium on Artificial Neural Networks (ESANN 2010), pp. 303–308. D-Side (2010)
Gallicchio, C., Micheli, A.: Architectural and markovian factors of echo state networks. Neural Netw. 24(5), 440–456 (2011)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(13), 489–501 (2006). Neural Networks Selected Papers from the 7th Brazilian Symposium on Neural Networks (SBRN 2004)
Jaeger, H.: The echo state approach to analysing and training recurrent neural networks-with an erratum note. German National Research Center for Information Technology GMD Technical report, Bonn, Germany, vol. 148(34), p. 13 (2001)
Jaeger, H., Haas, H.: Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science 304(5667), 78–80 (2004)
Jaeger, H., Lukoeviius, M., Popovici, D., Siewert, U.: Optimization and applications of echo state networks with leaky-integrator neurons. Neural Netw. 20(3), 335–352 (2007). Echo State Networks and Liquid State Machines
LukošEvičIus, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Comput. Sci. Rev. 3(3), 127–149 (2009)
Tang, J., Deng, C., Huang, G.B.: Extreme learning machine for multilayer perceptron. IEEE Trans. Neural Netw. Learn. Syst. 27(4), 809–821 (2016)
Time Series Data Library: daily minimum temperatures in Melbourne, Australia, 1981–1990. https://datamarket.com/data/set/2324/daily-minimum-temperatures-in-melbourne-australia-1981-1990
Verstraeten, D., Dambre, J., Dutoit, X., Schrauwen, B.: Memory versus non-linearity in reservoirs. In: The 2010 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2010)
Wold, S., Esbensen, K., Geladi, P.: Principal component analysis. Chemometr. Intell. Lab. Syst. 2(1), 37–52 (1987). Proceedings of the Multivariate Statistical Workshop for Geologists and Geochemists
Acknowledgment
This work is supported by the National Natural Science Foundation of China (Grant Nos. 61502174, 61402181), the Natural Science Foundation of Guangdong Province (Grant Nos. S2012010009961, 2015A030313215), the Science and Technology Planning Project of Guangdong Province (Grant No. 2016A040403046), the Guangzhou Science and Technology Planning Project (Grant Nos. 201704030051, 2014J4100006), the Opening Project of Guangdong Province Key Laboratory of Big Data Analysis and Processing (Grant No. 2017014), and the Fundamental Research Funds for the Central Universities (Grant No. D2153950).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Ma, Q., Shen, L., Zhuang, W., Chen, J. (2017). Decouple Adversarial Capacities with Dual-Reservoir Network. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10638. Springer, Cham. https://doi.org/10.1007/978-3-319-70139-4_48
Download citation
DOI: https://doi.org/10.1007/978-3-319-70139-4_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70138-7
Online ISBN: 978-3-319-70139-4
eBook Packages: Computer ScienceComputer Science (R0)