Papeer logo
ExploreHubPricingPrivacy
https://www.nature.com/articles/381607a0

Emergence of simple-cell receptive field properties by learning a sparse code for natural images

  • Letter
  • Published: 13 June 1996

Nature volume 381, pages 607–609 (1996) Cite this article

Abstract

THE receptive fields of simple cells in mammalian primary visual cortex can be characterized as being spatially localized, oriented1–4 and bandpass (selective to structure at different spatial scales), comparable to the basis functions of wavelet transforms5,6. One approach to understanding such response properties of visual neurons has been to consider their relationship to the statistical structure of natural images in terms of efficient coding7–12. Along these lines, a number of studies have attempted to train unsupervised learning algorithms on natural images in the hope of developing receptive fields with similar properties13–18, but none has succeeded in producing a full set that spans the image space and contains all three of the above properties. Here we investigate the proposal8,12 that a coding strategy that maximizes sparseness is sufficient to account for these properties. We show that a learning algorithm that attempts to find sparse linear codes for natural scenes will develop a complete family of localized, oriented, bandpass receptive fields, similar to those found in the primary visual cortex. The resulting sparse image code provides a more efficient representation for later stages of processing because it possesses a higher degree of statistical independence among its outputs.

This is a preview of subscription content, access via your institution

Access options

Subscribe to this journal

Receive 51 print issues and online access

199,00 € per year

only 3,90 € per issue

Buy this article

  • Purchase on SpringerLink
  • Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Additional access options:

Similar content being viewed by others

References

  1. Hubel, D. H. & Wiesel, T. N. J. Physiol., Lond. 195, 215–244 (1968).

    Article CAS Google Scholar

  2. De Valois, R. L., Albrecht, D. G. & Thorell, L. G. Vision Res. 22, 545–559 (1982).

    Article CAS Google Scholar

  3. Jones, J. P. & Palmer, L. A. J. Neurophysiol. 58, 1233–1258 (1987).

    Article CAS Google Scholar

  4. Parker, A. J. & Hawken, M. J. J. opt. Soc. Am. A 5, 598–605 (1988).

    Article ADS CAS Google Scholar

  5. Daugman, J. G. Computational Neuroscience (ed. Schwartz, E.) 403–423 (MIT Press, Cambridge, MA, 1990).

    Google Scholar

  6. Field, D. J. in Wavelets, Fractals, and Fourier Transforms (eds Farge, M., Hunt, J. & Vascillicos, C.) 151–193 (Oxford Univ. Press, 1993).

    Google Scholar

  7. Srinivasan, M. V., Laughlin, S. B. & Dubs, A. Proc. R. Soc. Lond. B 216, 427–459 (1982).

    ADS CAS PubMed Google Scholar

  8. Field, D. J. J. opt. Soc. Am. A 4, 2379–2394 (1987).

    Article ADS CAS Google Scholar

  9. Atick, J. J. Network 3, 213–251 (1992).

    Article Google Scholar

  10. van Hateren, J. H. Nature 360, 68–70 (1992).

    Article ADS CAS Google Scholar

  11. Ruderman, D. L. Network 5, 517–548 (1994).

    Article Google Scholar

  12. Field, D. J. Neur. Comput. 6, 559–601 (1994).

    Article Google Scholar

  13. Barrow, H. G. in IEEE First Int. Conf. on Neural Networks Vol. 4, (eds Caudill, M. & Butler, C.) 115–121 (Institute of Electrical and Electronics Engineers, 1994).

    Google Scholar

  14. Sanger, T. D. in Advances in Neural Information Processing Systems Vol. I (ed. Touretzky, D.) 11–19 (Morgan-Kaufmann, 1989).

    Google Scholar

  15. Hancock, P. J. B., Baddeley, R. J. & Smith, L. S. Network 3, 61–72 (1992).

    Article Google Scholar

  16. Law, C. C. & Cooper, L. N. Proc. natn. Acad. Sci. U.S.A. 91, 7797–7801 (1994).

    Article ADS CAS Google Scholar

  17. Fyfe, C. & Baddeley, R. Network 6, 333–344 (1995).

    Article Google Scholar

  18. Schmidhuber, J., Eldracher, M. & Foltin, B. Neur. Comput. 8 773–786 (1996).

    Article Google Scholar

  19. Barlow, H. B. Neur. Comput. 1, 295–311 (1989).

    Article Google Scholar

  20. Linsker, R. Computer 105–117 (March, 1988).

  21. Olshausen, B. A. & Field, D. J. Network 7, 333–339 (1996).

    Article CAS Google Scholar

  22. Daugman, J. G. IEEE Trans. biomed. Engng. 36, 107–114 (1989).

    Article CAS Google Scholar

  23. Harpur, G. F. & Prager, R. W. Network 7, 277–284 (1996).

    Article CAS Google Scholar

  24. Foldiak, P. Biol. Cybernet. 64, 165–170 (1990).

    Article CAS Google Scholar

  25. Zemel, R. S. thesis, Univ. Toronto (1993).

  26. Intrator, N. Neur. Comput. 4, 98–107 (1992).

    Article Google Scholar

  27. Bell, A. J. & Sejnowski, T. J. Neur. Comput. 7, 1129–1159 (1995).

    Article CAS Google Scholar

  28. Saund, E. Neur. Comput. 7, 51–71 (1995).

    Article Google Scholar

  29. Hinton, G. E., Dayan, P., Frey, B. J. & Neal, R. M. Science 268, 1158–1161 (1995).

    Article ADS CAS Google Scholar

  30. Lu, Z. L., Chubb, C. & Sperling, G. Technical Report MBS 96-15 (Institute for Mathematical Behavioral Sciences, University of California at Irvine, 1996).

Download references

Author information

Author notes

  1. Bruno A. Olshausen

Present address: Center for Neuroscience, UC Davis, Davis, California, 95616, USA

Authors and Affiliations

  1. Department of Psychology, Uris Hall, Cornell University, Ithaca, New York, 14853, USA

Bruno A. Olshausen & David J. Field

Authors

  1. Bruno A. Olshausen

You can also search for this author inPubMed Google Scholar

  1. David J. Field

You can also search for this author inPubMed Google Scholar

About this article

Cite this article

Olshausen, B., Field, D. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609 (1996). https://doi.org/10.1038/381607a0

Download citation