Nothing Special   »   [go: up one dir, main page]

Skip to main content

Generality’s Price

Inescapable Deficiencies in Machine-Learned Programs

  • Conference paper
Learning Theory and Kernel Machines

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 2777))

  • 5551 Accesses

Abstract

This paper investigates some delicate tradeoffs between the generality of an algorithmic learning device and the quality of the programs it learns successfully. There are results to the effect that, thanks to small increases in generality of a learning device, the computational complexity of some successfully learned programs is provably unalterably suboptimal. There are also results in which the complexity of successfully learned programs is asymptotically optimal and the learning device is general, but, still thanks to the generality, some of those optimal, learned programs are provably unalterably information deficient — in some cases, deficient as to safe, algorithmic extractability/provability of the fact that they are even approximately optimal. The paper is on the borderline between learning theory, complexity theory and logic.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Barzdin, J.A.: Two theorems on the limiting synthesis of functions. In: Theory of Algorithms and Programs, Latvian State University, Riga, U.S.S.R, vol. 210, pp. 82–88 (1974)

    Google Scholar 

  2. Blum, L., Blum, M.: Toward a mathematical theory of inductive inference. Information and Control 28, 125–155 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  3. Blum, M.: A machine independent theory of the complexity of recursive functions. Journal of the ACM 14, 322–336 (1967)

    Article  MATH  Google Scholar 

  4. Case, J., Chen, K., Jain, S.: Costs of general purpose learning. Theoretical Computer Science 259, 455–473 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  5. Chen, K.: Tradeoffs in machine inductive inference, Ph.D. thesis, SUNY at Buffalo (1981)

    Google Scholar 

  6. Cormen, T., Leiserson, C., Rivest, R., Stein, C.: Introduction to algorithms, 2nd edn. MIT Press, Cambridge (2001)

    MATH  Google Scholar 

  7. Chew, P., Machtey, M.: A note on structure and looking back applied to the relative complexity of computable functions. Journal of Computer and System Sciences 22, 53–59 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  8. Case, J., Smith, C.: Comparison of identification criteria for machine inductive inference. Theoretical Computer Science 25, 193–220 (1983)

    Article  MATH  MathSciNet  Google Scholar 

  9. Gold, E.M.: Language identification in the limit. Information and Control 10, 447–474 (1967)

    Article  MATH  Google Scholar 

  10. Hartmanis, J., Stearns, R.: On the computational complexity of algorithms. Transactions of the American Mathematical Society 117, 285–306 (1965)

    Article  MATH  MathSciNet  Google Scholar 

  11. Hopcroft, J., Ullman, J.: Introduction to automata theory languages and computation. Addison-Wesley Publishing Company, Reading (1979)

    MATH  Google Scholar 

  12. Jones, N.: Computability and complexity from a programming perspective. MIT Press, Cambridge (1997)

    MATH  Google Scholar 

  13. Jain, S., Osherson, D., Royer, J., Sharma, A.: Systems that learn: An introduction to learning theory, 2nd edn. MIT Press, Cambridge (1999)

    Google Scholar 

  14. Ladner, R.: On the structure of polynomial time reducibility. Journal of the ACM 22, 155–171 (1975)

    Article  MATH  MathSciNet  Google Scholar 

  15. Landweber, L., Lipton, R., Robertson, E.: On the structure of sets in NP and other complexity classes. Theoretical Computer Science 15, 181–200 (1981)

    Article  MATH  MathSciNet  Google Scholar 

  16. Li, M., Vitányi, P.: An introduction to Kolmogorov complexity and its applications, 2nd edn. Springer, Heidelberg (1997)

    MATH  Google Scholar 

  17. Mendelson, E.: Introduction to mathematical logic, 4th edn. Chapman & Hall, London (1997)

    MATH  Google Scholar 

  18. Royer, J., Case, J.: Subrecursive programming systems: Complexity & succinctness. Birkhäuser, Basel (1994)

    MATH  Google Scholar 

  19. Regan, K.: The topology of provability in complexity theory. Journal of Computer and System Sciences 36, 384–432 (1988)

    Article  MathSciNet  Google Scholar 

  20. Rogers, H.: Theory of recursive functions and effective computability. Mc- Graw Hill, New York (1967); Reprinted MIT Press, Cambridge (1987)

    Google Scholar 

  21. Schoning, U.: A uniform approach to obtain diagonal sets in complexity classes. Theoretical Computer Science 18, 95–103 (1982)

    Article  MathSciNet  Google Scholar 

  22. Schmidt, D.: The recursion-theoretic structure of complexity classes. Theoretical Computer Science 38, 143–156 (1985)

    Article  MATH  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2003 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Case, J., Chen, KJ., Jain, S., Merkle, W., Royer, J.S. (2003). Generality’s Price. In: Schölkopf, B., Warmuth, M.K. (eds) Learning Theory and Kernel Machines. Lecture Notes in Computer Science(), vol 2777. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_50

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-45167-9_50

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-40720-1

  • Online ISBN: 978-3-540-45167-9

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics