Abstract
THE receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented1–4 and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms5,6. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding7–12. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties13–18, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal8,12 that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.
This is a preview of subscription content, access via your institution
Access options
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Buy this article
- Purchase on SpringerLink
- Instant access to full article PDF
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
References
Hubel, D. H. & Wiesel, T. N. J. Physiol., Lond. 195, 215–244 (1968).
De Valois, R. L., Albrecht, D. G. & Thorell, L. G. Vision Res. 22, 545–559 (1982).
Jones, J. P. & Palmer, L. A. J. Neurophysiol. 58, 1233–1258 (1987).
Parker, A. J. & Hawken, M. J. J. opt. Soc. Am. A 5, 598–605 (1988).
Daugman, J. G. Computational Neuroscience (ed. Schwartz, E.) 403–423 (MIT Press, Cambridge, MA, 1990).
Field, D. J. in Wavelets, Fractals, and Fourier Transforms (eds Farge, M., Hunt, J. & Vascillicos, C.) 151–193 (Oxford Univ. Press, 1993).
Srinivasan, M. V., Laughlin, S. B. & Dubs, A. Proc. R. Soc. Lond. B216, 427–459 (1982).
Field, D. J. J. opt. Soc. Am. A4, 2379–2394 (1987).
Atick, J. J. Network 3, 213–251 (1992).
van Hateren, J. H. Nature 360, 68–70 (1992).
Ruderman, D. L. Network 5, 517–548 (1994).
Field, D. J. Neur. Comput. 6, 559–601 (1994).
Barrow, H. G. in IEEE First Int. Conf. on Neural Networks Vol. 4, (eds Caudill, M. & Butler, C.) 115–121 (Institute of Electrical and Electronics Engineers, 1994).
Sanger, T. D. in Advances in Neural Information Processing Systems Vol. I (ed. Touretzky, D.) 11–19 (Morgan-Kaufmann, 1989).
Hancock, P. J. B., Baddeley, R. J. & Smith, L. S. Network 3, 61–72 (1992).
Law, C. C. & Cooper, L. N. Proc. natn. Acad. Sci. U.S.A. 91, 7797–7801 (1994).
Fyfe, C. & Baddeley, R. Network 6, 333–344 (1995).
Schmidhuber, J., Eldracher, M. & Foltin, B. Neur. Comput. 8 773–786 (1996).
Barlow, H. B. Neur. Comput. 1, 295–311 (1989).
Linsker, R. Computer 105–117 (March, 1988).
Olshausen, B. A. & Field, D. J. Network 7, 333–339 (1996).
Daugman, J. G. IEEE Trans. biomed. Engng. 36, 107–114 (1989).
Harpur, G. F. & Prager, R. W. Network 7, 277–284 (1996).
Foldiak, P. Biol. Cybernet. 64, 165–170 (1990).
Zemel, R. S. thesis, Univ. Toronto (1993).
Intrator, N. Neur. Comput. 4, 98–107 (1992).
Bell, A. J. & Sejnowski, T. J. Neur. Comput. 7, 1129–1159 (1995).
Saund, E. Neur. Comput. 7, 51–71 (1995).
Hinton, G. E., Dayan, P., Frey, B. J. & Neal, R. M. Science 268, 1158–1161 (1995).
Lu, Z. L., Chubb, C. & Sperling, G. Technical Report MBS 96-15 (Institute for Mathematical Behavioral Sciences, University of California at Irvine, 1996).
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). https://doi.org/10.1038/381607a0
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1038/381607a0
This article is cited by
-
Distinguishing examples while building concepts in hippocampal and artificial networks
Nature Communications (2024)
-
Rejoinder of “Identifiability of latent-variable and structural-equation models: from linear to nonlinear"
Annals of the Institute of Statistical Mathematics (2024)
-
Cervical cancer classification using sparse stacked autoencoder and fuzzy ARTMAP
Neural Computing and Applications (2024)
-
Interpretable Task-inspired Adaptive Filter Pruning for Neural Networks Under Multiple Constraints
International Journal of Computer Vision (2024)
-
Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding
Science China Physics, Mechanics & Astronomy (2024)