research-article

Computational power of neural networks: a characterization in terms of Kolmogorov complexity

Authors:

J. L. Balcazar,

R. Gavalda,

H. T. SiegelmannAuthors Info & Claims

IEEE Transactions on Information Theory, Volume 43, Issue 4

Pages 1175 - 1183

https://doi.org/10.1109/18.605580

Published: 01 September 2006 Publication History

Abstract

The computational power of recurrent neural networks is shown to depend ultimately on the complexity of the real constants (weights) of the network. The complexity, or information contents, of the weights is measured by a variant of resource-bounded Kolmogorov (1965) complexity, taking into account the time required for constructing the numbers. In particular, we reveal a full and proper hierarchy of nonuniform complexity classes associated with networks having weights of increasing Kolmogorov complexity

Cited By

View all

Barmpalias GLewis-Pye A(2019)Compression of Data Streams Down to Their Information ContentIEEE Transactions on Information Theory10.1109/TIT.2019.289663865:7(4471-4485)Online publication date: 14-Jun-2019
https://dl.acm.org/doi/10.1109/TIT.2019.2896638
Šíma JPlátek M(2019)One Analog Neuron Cannot Recognize Deterministic Context-Free LanguagesNeural Information Processing10.1007/978-3-030-36718-3_7(77-89)Online publication date: 12-Dec-2019
https://dl.acm.org/doi/10.1007/978-3-030-36718-3_7
Šíma J(2019)Counting with Analog NeuronsArtificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation10.1007/978-3-030-30487-4_31(389-400)Online publication date: 17-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-30487-4_31
Show More Cited By

Computational power of neural networks: a characterization in terms of Kolmogorov complexity
1. Computing methodologies
  1. Machine learning
    1. Machine learning approaches
2. Theory of computation
  1. Computational complexity and cryptography

Recommendations

On the Computational Power of Max-Min Propagation Neural Networks

We investigate the computational power of max-min propagation (MMP) neural networks, composed of neurons with maximum (Max) or minimum (Min) activation functions, applied over the weighted sums of inputs. The main results presented are that a single-...
Computational complexity of neural networks: a survey

We survey some of the central results in the complexity theory of discrete neural networks, with pointers to the literature. Our main emphasis is on the computational power of various acyclic and cyclic network models, but we also discuss briefly the ...
The computational power of interactive recurrent neural networks

In classical computation, rational-and real-weighted recurrent neural networks were shown to be respectively equivalent to and strictly more powerful than the standard Turing machine model. Here, we study the computational power of recurrent neural ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

IEEE Transactions on Information Theory Volume 43, Issue 4

July 1997

296 pages

ISSN:0018-9448

Issue’s Table of Contents

Publisher

IEEE Press

Publication History

Published: 01 September 2006

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Barmpalias GLewis-Pye A(2019)Compression of Data Streams Down to Their Information ContentIEEE Transactions on Information Theory10.1109/TIT.2019.289663865:7(4471-4485)Online publication date: 14-Jun-2019
https://dl.acm.org/doi/10.1109/TIT.2019.2896638
Šíma JPlátek M(2019)One Analog Neuron Cannot Recognize Deterministic Context-Free LanguagesNeural Information Processing10.1007/978-3-030-36718-3_7(77-89)Online publication date: 12-Dec-2019
https://dl.acm.org/doi/10.1007/978-3-030-36718-3_7
Šíma J(2019)Counting with Analog NeuronsArtificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation10.1007/978-3-030-30487-4_31(389-400)Online publication date: 17-Sep-2019
https://dl.acm.org/doi/10.1007/978-3-030-30487-4_31
(2018)Quasi-periodic -expansions and cut languagesTheoretical Computer Science10.1016/j.tcs.2018.02.028720:C(1-23)Online publication date: 11-Apr-2018
https://dl.acm.org/doi/10.1016/j.tcs.2018.02.028
Wu YFeng J(2018)Development and Application of Artificial Neural NetworkWireless Personal Communications: An International Journal10.1007/s11277-017-5224-x102:2(1645-1656)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s11277-017-5224-x
Šíma J(2018)Three Analog Neurons Are Turing UniversalTheory and Practice of Natural Computing10.1007/978-3-030-04070-3_36(460-472)Online publication date: 12-Dec-2018
https://dl.acm.org/doi/10.1007/978-3-030-04070-3_36
Cabessa JVilla A(2012)The expressive power of analog recurrent neural networks on infinite input streamsTheoretical Computer Science10.1016/j.tcs.2012.01.042436(23-34)Online publication date: 1-Jun-2012
https://dl.acm.org/doi/10.1016/j.tcs.2012.01.042
Beggs ECosta JLoff BTucker J(2008)Oracles and Advice as MeasurementsProceedings of the 7th international conference on Unconventional Computing10.1007/978-3-540-85194-3_6(33-50)Online publication date: 25-Aug-2008
https://dl.acm.org/doi/10.1007/978-3-540-85194-3_6
Šorel MŠíma J(2004)Robust RBF finite automataNeurocomputing10.1016/j.neucom.2003.12.00562:C(93-110)Online publication date: 1-Dec-2004
https://dl.acm.org/doi/10.1016/j.neucom.2003.12.005
Šíma JOrponen P(2003)General-purpose computation with neural networksNeural Computation10.1162/08997660332251873115:12(2727-2778)Online publication date: 1-Dec-2003
https://dl.acm.org/doi/10.1162/089976603322518731
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Abstract

Cited By

Recommendations

On the Computational Power of Max-Min Propagation Neural Networks

Computational complexity of neural networks: a survey

The computational power of interactive recurrent neural networks

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Get Access

Login options

Full Access

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations