List of Contributors
L.F. Abbott, Volen Center for Complex Systems and Department of Biology, Brandeis University, Waltham, MA 02454-9110, USA E. Ahissar, Department of Neurobiology, The Weizmann Institute of Science, Rehovot 76100, Israel L.A. Baccal~, Av. Prof. Luciano Gualberto, Tray. 3, #158,CEP 05508-900, Sao Paulo, SR Brazil J.G. Bjaalie, Department of Anatomy, Institute of Basic Medical Sciences, University of Oslo, RO. Box 1105 Blindern, N-0317 Oslo, Norway E. Covey, Department of Psychology, University of Washington, Box 351525, Seattle, WA 98195, USA E. De Schutter, Born-Bunge Foundation, University of Antwerp, Universiteitsplein 1, B2610 Antwerp, Belgium S.A. Deadwyler, Department of Physiology and Pharmacology, Wake University School of Medicine, Winston-Salem, NC 27157-1084, USA H.R. Dinse, Institute for Neuroinformatics, Theoretical Biology, Ruhr University Bochum, Bochum, Germany J.R Donoghue, Department of Neuroscience and Division of Applied Mathematics, Brown University, Providence, RI 02912, USA R.R Erickson, Departments of Psychology, Experimental and Neurobiology, Duke University, Durham, NC 27708, USA E.E. Fetz, Department of Physiology and Biophysics and the Regional Primate Research Center, University of Washington, Seattle, WA 98195, USA W.A. Freiwald, Institute for Brain Research, University of Bremen, FB2, RO. Box 330440, D-28334 Bremen, Germany B.H. Gaese, Institut ftir Biologie II, RWTH Aachen, Kopernikusstr. 16, D-52074 Aachen, Germany Y. Garbourg, Department of Physiology and Neuroscience, New York University School of Medicine, 550 First Avenue, New York, NY 10016, USA R.E. Hampson, Department of Physiology and Pharmacology, Wake Forest University School of Medicine, Winston-Salem, NC 27157-1083, USA M.T. Harrison, Department of Neuroscience and Division of Applied Mathematics, Brown University, Providence, RI 02912, USA N.G. Hatsopoulos, Department of Neuroscience and Division of Applied Mathematics, Brown University, Providence, RI 02912, USA N. Jain, Department of Psychology, Vanderbilt University, 301 Wilson Hall, 111 21st Avenue South, Nashville, TN 37240, USA D. Jancke, Institute for Neuroinformatics, Theoretical Biology, Ruhr University Bochum, Bochum, Germany. Present address: The Weizmann Institute of Science, Rehovot, Israel
vi J.H. Kaas, Department of Psychology, Vanderbilt University, 301 Wilson Hall, 111 21st Ave. South, Nashville, TN 37240, USA J.S. Kauer, Department of Neuroscience, Tufts University School of Medicine, 136 Harrison Avenue, Boston, MA 02111, USA A.K. Kreiter, Institute for Brain Research, University of Bremen, FB2, P.O. Box 330440, D-28334 Bremen, Germany D. Margoliash, Department of Organismal Biology and Anatomy, The University of Chicago, 1027 E. 57th Street, Chicago, IL 60637, USA J.T. McIlwain, Department of Neuroscience, Brown University, Providence, RI 02912, USA S.L. Moody, Department of Computer Science, Wellesley College, 106 Central St, Wellesley, MA 02481, USA M.A.L. Nicolelis, Department of Neurobiology, Box 3209, Duke University, Bryan Research Building, Room 333, 101 Research Drive, Durham, NC 27710, USA S.I. Perlmutter, Department of Physiology and Biophysics and the Regional Primate Research Center, University of Washington, Seattle, WA 98195, USA S.M. Potter, Division of Biology 156-29, California Institute of Technology, Pasadena, CA 91125, USA Y. Prut, Department of Physiology and Biophysics and the Regional Regional Primate Research Center, University of Washington, Seattle, WA 98195, USA H.-X. Qi, Department of Psychology, Vanderbilt University, 301 Wilson Hall, 111 21st Avenue South, Nashville, TN 37240, USA R.C. Reid, Department of Neurobiology, Harvard Medical School, 220 Longwood Avenue, Boston, MA 02115, USA E. Salinas, Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute for Biological Studies, 10010 North Torrey Pines Road, La Jolla, CA 92037, USA K. Sameshima, Disc. Medical Informatics and Functional Neurosurgery Lab., School of Medicine, University of Sao Paulo, Sao Paulo, Brazil C. Schwarz, Eberhard-Karls-Universit~t Ttibingen, Department of Cognitive Neurology, Auf der Morgenstelle 15, D-72076 Tttbingen, Germany T.J. Sejnowski, Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, CA 92037, USA and Department of Biology, University of California at San Diego, La Jolla, CA 92093, USA M. Shuler, Biomedical Engineering, Duke University, Bryan Research Building, Room 333, 101 Research Drive, Durham, NC 27710, USA J.D. Simeral, Department of Physiology and Pharmacology, Wake Forest University School of Medicine, Winston-Salem, NC 27157-1083, USA W. Singer, Max-Planck Institute for Brain Research, Deutschordenstr. 46, D-60528 Frankfurt/Main, Germany J.P. Welsh, Department of Physiology and Neuroscience, New York University, School of Medicine, 550 First Avenue, New York, NY 10016, USA J. White, Department of Neuroscience, Tufts University School of Medicine, 136 Harrison Avenue, Boston, MA 02111, USA S.P. Wise, Laboratory of Systems Neuroscience, National Institute of Mental Health, National Institutes of Health, 49 Convent Drive, MSC 4401, Bldg. 49 Room B1EE17, Bethesda, MD 20892-4401, USA
vii M. Zacksenhouse, Faculty of Mechanical Engineering, Technion - - Israel Institute of Technology, Haifa, 32000, Israel K. Zhang, Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute for Biological Studies, La Jolla, CA 92037, USA
ix
Preface Although neuroscientists have long recognized the relevance of investigating the principles that underlie the interactions of large populations of neurons in behaving animals, they had to wait a long time for the introduction of experimental and analytical methods to begin exploring the physiological properties of large neural ensembles. Fortunately, during the last two decades a variety of such techniques have been introduced into the arsenal of tools currently employed in neuroscience. As a result, today one only needs to stroll through the scientific exhibits of major neuroscience conferences to realize that the field of neural ensemble physiology is rapidly becoming one of the premier areas of modern brain research. Three years ago, motivated by the growing interest in neural ensemble research, Dennis Glanzman, the director of the Theoretical and Computational Neuroscience Research Program at the National Institute of Mental Health, and I organized a workshop on "Advances in Neural Population Coding" in Bethesda, Maryland. The main goal of that meeting was to bring together a distinguished group of neuroscientists to debate current and future developments in the area of neural ensemble physiology. By all accounts, the meeting was a great success. Soon after we departed from Bethesda, it occurred to me that the new and exciting ideas discussed in that meeting deserved to be disseminated to a broader community. The book that you are now holding was born out of this motivation. Those who participated in the Bethesda meeting kindly agreed to participate in this adventure and provided articles describing some of the issues they discussed in their presentations. However, since the workshop did not cover all aspects of neural population coding, it was decided that several invited articles should be added to make the volume as comprehensive as possible. Thus, several colleagues, who were not able to participate in the original meeting, were kind enough to accept my invitation and contribute to this book with great enthusiasm. After two years of hard work, it is with great pleasure that I introduce the end product of this collective effort, which reflects the research of many scientists working in laboratories around the world. This book is divided into six sections. The first one contains a historical overview of the concept of neural population coding. The second section introduces a series of new experimental paradigms and analytical techniques for investigating potential neural coding schemes. Then, the next four sections focus on recent advances in population coding in a broad range of areas of brain research, which include: sensory (Section III) and motor (Section IV) systems, learning (Section V), and cognitive neuroscience (Section VI). I would like to take this opportunity to thank those who made this project come to fruition. First, I would like to thank Dr. Dennis Glanzman and the National Institute of Mental Health for supporting the organization of the Bethesda meeting on population coding. I would also like to thank all the authors who contributed to this book. Finally, I would like to acknowledge the continuous support and enthusiasm provided by Mrs. Jenny Henzen, the Publishing Editor of Neuroscience for Elsevier. Miguel A. L. Nicoletis
M.A.L. Nicolelis (Ed.)
Progress in BrainResearch, Vol. 130 © 2001 Elsevier Science B.V. All rights reserved
CHAPTER 1
Population coding: a historical sketch J a m e s T. M c I l w a i n * Department of Neuroscience, Brown University, Providence, RI 02912, USA
Although the idea of population (or ensemble or distributed) coding was introduced almost 200 years ago, it did not attract substantial attention until relatively recently. One reason for this, aptly stated by Hinton et al. (1986), is that distributed representations are "less familiar and harder to think about than local representations" (p. 77). Another reason is that neuroscience is only now emerging from an era in which the widespread use of microelectrodes focused experimental research on the behavior of single neurons and the possibility that their individual properties could account for much of what the brain does. As someone whose work began in the era of single-unit recording, I can testify personally to the seductiveness of this latter view. As you sit in a darkened laboratory with your attention riveted to the sounds of the audiomonitor and probe a neuron's receptive field with a tiny visual stimulus, it is easy to forget that the cell you are listening to is but one of many that are responding to the stimulus. And from there it is but a small step to the assumption that the cell is a 'labeled line' for that aspect of the stimulus which produces the most vigorous response. This short essay is not the place for an exhaustive survey of all studies past and present that speak to the relative importance of single neurons versus ensembles of neurons in the mediation of sensory, motor and perceptual events. My intention is rather to highlight certain threads in this story, admitting at the
*Corresponding author: J.T. McIlwain, Department of Neuroscience, Brown University, Providence, RI 02912, USA.
outset that my perspective is inevitably constrained by the literature that has informed my own work. The debate forms a fascinating chapter in the history of neuroscience and deserves the attention of a professional historian, who, I believe, will recognize distinct parallels With the arguments over localization of function that occupied scientists in the eighteenth century. If one substitutes 'representation' for 'function' and 'single neuron' for 'area' in the arguments of the older debate, the underlying commonalties become clear. As is well known, the localization approach reached absurd heights in phrenology, which triggered a reaction among neurologists and other scientists that ranged from extremely holistic views to more moderate positions that recognized certain degrees of specialization in cortical areas, but held that cooperative activity was essential to brain function. The important contributions to this debate by such figures as Jackson, Hebb, Luria and Lashley are noted by Hinton et al. (1986) and discussed in detail by Finger (1994) and by Clark and O~Malley (1968). As suggested earlier, though, the interesting issues today are those that concern the role of individual neurons. The earliest invocation of the notion of population coding of which I am aware is that of Thomas Young in his trichromatic theory of color vision. Young (1802) argued that " ... as it is almost impossible to conceive each sensitive point on the retina to contain an infinite number of particles, each capable of vibrating in perfect unison with every possible undulation, it becomes necessary to suppose the number limited, for instance, to the three principal colours, red, yellow, and blue ... and that each of the par-
ticles is capable of being put in motion less or more forcibly, by undulations differing less or more from a perfect unison" (pp. 20-21). It is now generally accepted that visible wavelengths are represented in the retinas of most humans by the ratio of their effects on three distinct cone types, each sensitive to a broad, but not identical, range of wavelengths, and that the activity of any one cone type is ambiguous with respect to the chromatic composition of the impinging light. It is worth noting that Young was led to his conclusion by the realization that it was physically impossible to have at each point of the retina a labeled line for each color of the spectrum that the eye can distinguish. In an interesting inversion of this reasoning, modern investigators often invoke the idea of population coding when the broad tuning of individual neurons cannot account for the exquisite discriminative capacities of the circuits of which they form a part. Thomas Young was a Fellow of Trinity College in Cambridge University, an institution that has provided other major chapters in the development of the idea of distributed representation. From the Cambridge Physiological Laboratory came the study by Adrian et al. (1931) in which they showed that cutaneous afferents in the frog had large receptive fields on the body surface. In discussing the limits such large fields might place on localization of punctate cutaneous stimuli, the authors concluded: "There is no reason to suppose that the widespread distribution of the sensory endings of a single fibre will necessarily interfere with the exact localization of a stimulus. Owing to the overlapping of the area of distribution of different fibres the stimulation of any point on the skin will cause impulse discharges in several fibres and the particular combination of fibres in action, together with the relative intensity of the discharge in each, would supply all the data needed for localization." (p. 384) Carl Pfaffman, working somewhat later in the Physiological Laboratory, was prompted by the relatively nonspecific responses of the cat's primary gustatory afferents to advance what is sometimes called the 'cross-fiber pattern' theory of gustatory coding. Pfaffman (1941) concluded that "In such a system, sensory quality does not depend simply on the 'all or nothing' activation of some particular fiber
group alone, but on the pattern of other fibers active" (p. 255). There is some irony in the fact that the most recent contribution to this debate from Cambridge is what is perhaps the strongest statement of the labeledline hypothesis. The argument of Horace Barlow (1972) is based on the assumption that the brain is organized to achieve specific representations with activity in the minimum number of neurons. Perception occurs when there is activity in a small number of high-level neurons "each of which corresponds to a pattern of external events on the order of complexity of events symbolized by a word" (p. 371). Two decades earlier, Barlow (t953) had introduced the term 'detector' to characterize a cell whose activity signals the presence of a specific stimulus object. Discussing the behavior of one type of ganglion cell in the retina of the frog, Barlow wrote: "The receptive field of an 'on-off' unit would be nicely filled by the image of a fly at 2 in. distance and it is difficult to avoid the conclusion that the 'on-off' units are matched to the stimulus and act as 'fly detectors."' (p. 86) Referring to another class of frog ganglion cells, the 'off' units, Barlow addressed the issue of spatial localization with an argument reminiscent of that of Adrian et al. (1931), but modified to emphasize the centrality of the single neuron in the coding process. "A population of ganglion cells with over-lapping receptive fields of the same size as the expected image is a neat method of judging the centre of a large object, for there will be a single, unique, ganglion cell whose field is completely filled by the image, and which will, therefore, be maximally excited." (p. 87) With the publication of their paper entitled 'What the frog's eye tells the frog's brain', Lettvin et al. (1959) gave the idea of the 'feature-detector neuron' a major boost. In this influential paper the authors applied the 'detector' terminology to four classes of retinal ganglion ceils, and in the following decades, the concept of single neurons as labeled lines was routinely invoked by others in discussing the behavior of neurons elsewhere in the visual system. This practice, however, did not go unchallenged. One cautionary voice was that of Robert Erickson, whose reviews reminded readers that the broad tuning of most
sensory neurons for various stimulus dimensions was not consistent with such a neat story (Erickson, 1968, 1974). Erickson recalled the distributed nature of the widely accepted trichromatic theory and also observed that the feature-detector idea is in effect an extension of MUller's law of specific nerve energy to the single unit level. Another critical observer, apparently exasperated by what he perceived to be an excessive application of the feature-detector idea to psychological processes, referred to the era as the 'psychobiological silly season' (Uttal, 1971). Even as the visual system was providing the greater part of the evidence advanced in support of the feature-detector/labeled-line idea, Stephen Kuftier (1952) expressed doubts in his pioneering study of the receptive fields of ganglion cells in the cat's retina: "Since receptive fields overlap, even the smallest light spot will excite numerous ganglion cells. For such reasons psychophysical conclusions based only on individual cell responses will have to remain limited." (p. 290) Here Kuffler also expressed a concern that " ... the usefuless of the 'receptive field' concept may largely be lost, if it should really include a very large area." Although he did not elaborate, Kuffier must have been uneasy about the assumption that the behavior of a single cell is critical for the spatial localization of a stimulus. Subsequent work by the author of this essay showed that some ganglion cells in the cat can indeed be activated from areas distant from the localized 'on' and 'off' zones described by Kuffler (Mcllwain, 1964) 1, and numerous studies have since revealed important contributions of areas beyond the 'classical' receptive field to the behavior of neurons throughout the visual system (reviewed in Allman et al., 1985). The 1970s witnessed efforts to quantify earlier observations that stimulation at a given point on the retina would potentially activate a significant number
tit was my great good fortune to have the opportunity to demonstrate these findings to the late Stephen Kuffler when he visited UCLA in 1963. For his gracious assistance in helping an unknown postdoc publish his then rather heretical observation, I am forever in his debt.
of cells. This direction was prompted in some measure by knowledge that the point-spread function of a linear optical system, i.e. the distribution of light in the image of a point source, completely defines the system and can be used to predict the distribution of light in the image of any object (Papoulis, 1968). In an analysis of the cat's retina, Burkhardt Fischer (1973) estimated that at least 60 ganglion cells had receptive-field centers that overlapped at a point, an aggregate he termed the 'Punktbild', which translates approximately as 'point image'. Fischer's results suggested that this number was invariant across the retina and from this he concluded that a compact collection of axons from any Punktbild would occupy the same cross sectional area of the optic nerve regardless of its retinal origin. An analogous concept was advanced by Cleland et al. (1975) in the form of the 'coverage factor', defined as "equivalent to the number of receptive field centres which would be transfixed if a pin was pushed into the visual map at a particular point" (p. 169). Subsequent work on the visual cortex (Hubel and Wiesel, 1974; Albus, 1975; Dow et al., 1981; Van Essen et al., 1984; Grinvald et al., 1994) and superior colliculus (McIlwain, 1975) indicated that cells with receptive fields containing a given visual point occupied significant areas of these central stations in the visual pathway. The distributed representation of a visual point in the superior colliculus was mirrored in the widespread distribution of activity preceding saccades, suggesting that the active populations were involved in converting the retinotopic location of the saccade target to a representation of the metrics of the impending saccade (Mcllwain, 1975; Sparks et al., 1976). Here, then, was a case in which insight into the structure of the distributed code was facilitated by knowledge of the requirements of the output system, an advantage currently denied to those working on parts of the visual system concerned with perceptual processes. As neurophysiologists were coming to grips with the need to consider the consequences of broadly distributed neural activity representing sensory or motor events, there emerged more or less independently a stream of theoretical work that envisaged distributed representations as the neural substrates of memory. Pioneering models based on this idea were advanced by Anderson (1970) and Kohonen (1972)
and interviews with some of the early workers in the field have been collected by Anderson and Rosenfeld (1998). The development of optical holography lent credence to the notion that information could be stored in non-local fashion and retrieved, and there was an early effort to apply the holographic model to brain function (for review see Willshaw, 1989). Over the past two decades neur0physiological and theoretical streams have merged significantly and have moved in directions that are beyond the scope of this essay. One example, though, will serve to illustrate this convergence. In 1984, G.E. Hinton published a demonstration of the efficiency of what he called 'coarse coding' for the representation of features (Hinton, 1984). He concluded that "The central result is a surprising one. If you want to encode features accurately using as few units as possible, it pays to use units that are very coarsely tuned, so that each feature activates many different units and each unit is activated by many different features" (p. 11). Thomas Young would be pleased. This brief sketch suggests that a reliable sign or 'signature' of a system that employs a distributed code is that its neurons taken one at the time are broadly tuned along a dimension that the system nonetheless appears to resolve with a high degree of precision. For visual cells, the dimension may be retinal location, wavelength, orientation, speed, direction or, at higher levels, form and position in head- or body-centered coordinates. Neurons in the auditory system respond to a wide range of sound frequencies and intensities, and the broad tuning of olfactory and gustatory neurons is well established. There is ample evidence that this characteristic applies as well to the primary motor cortex and superior colliculus. As noted above, in certain cases progress has been made in estimating the distribution of activity in response to stimulation at some locus along the dimension involved. These results a r e a useful starting point because they may be related to the idea o f the point-spread function of the system and the body of theory developed around that concept. However, because neural systems are notoriously non-linear, the convolution techniques suitable for linear systems cannot be applied directly, and a major challenge is to develop methods to map the real distributions of activity to stimuli more complex than points. Multiunit recording and optical imaging
methods offer great promise here, but there is also a need for a strong theoretical underpinning for such efforts. If the brain uses distributed codes, as seems certainly to be the case, does this mean that neurons cannot be 'labeled lines'? Clearly, to support a code of any complexity, active populations must be discriminable from one another, which means that differences among the individual cells are important. Neurons cannot respond equally well to everything and form useful representations of different things. Thus, the sharp dichotomy between distributed coding and labeled lines seems to be a false one and the critical question is 'labeled how and with what'. Barlow's metaphor of the neuron as 'word' assigns more semantic stability to a cell's discharge than the evidence supports, but perhaps the role of the cell is analogous to that of the letter whose meaning in any instance is inseparable from its role in the ensemble that forms the word. Indeed, the task facing neuroscience is not unlike that encountered by scholars who deciphered the writing systems of the ancient Near East from the scraps of evidence at their disposal. Did the individual symbols represent words, syllables or letters and could the language be translated into one they already knew? Shrewd hypotheses, rigorous analysis and pure luck all played major roles in the success of that enterprise (Chadwick, 1958; Pope, 1975), and the same will probably be true for an eventual understanding of distributed neural codes.
References Adrian, E.D., Cattell, M. and Hoagland, H. (1931) Sensory discharges in single cutaneous nerve fibers. J. Physiol. (Lond.), 72: 377-391. Albus, K. (1975) A quantitative study of the projection area of the central and paracentral visual field in area 17 of the cat. I. Precision of the topography.Exp. Brain Res., 24: 159-179. Allman, J., Miezin, F. and McGuinness, E. (1985) Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annu. Rev. Neurosci., 8: 407-430. Anderson, J.A. (1970) Two models of memory organisation using interactive traces. Math. Biosci., 8: 137-160. Anderson, J.A. and Rosenfeld, E. (1998) Talking Nets: An Oral History of Neural Networks. MIT Press, Cambridge,MA. Barlow, H.B. (1953) Summation and inhibition in the frog's retina. J. Physiol. (Lond.), 119: 69-88.
Barlow, H.B. (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception, 1:371-394. Chadwick, J. (1958) The Decipherment of Linear B. Cambridge University Press, New York. Clark, E. and O'Malley, C.D. (1968) The Human Brain and Spinal Cord. University of California Press, Berkeley, CA. Cleland, B.G., Levick, W.R. and W~ssle, H. (1975) Physiological identification of a morphological class of cat retinal ganglion cells. J. Physiol. (Lond.), 248: 151-171. Dow, B.M., Snyder, A.Z., Vautin, R.G. and Bauer, R. (1981) Magnification factor and receptive field size in foveal striate cortex of the monkey. Exp. Brain Res., 44: 213-228. Erickson, R.R (1968) Stimulus coding in topographic and nontopographic afferent modalities. Psych. Rev., 75: 447-465. Erickson, R.E (1974) Parallel 'population' neural coding in feature extraction. In: F.O. Schmitt and EG. Worden (Eds.), The Neurosciences. Third Study Program. MIT Press, Cambridge, pp. 155-169. Finger, S. (1994) Origins of Neuroscience. Oxford University Press, New York. Fischer, B. (1973) Overlap of receptive field centers and representation of the visual field in the cat's optic tract. Vision Res., 13: 2113-2120. Grinvald, A., Lieke, E.E., Frostig, R.D. and Hildesheim, R. (1994) Cortical point-spread function and long range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual cortex. J. Neurosci., 14: 2545-2569. Hinton, G.E. (1984) Distributed representations. Technical Report CMU-CS-84-157. Carnegie-Mellon, Pittsburgh, p. 11. Hinton, G.E., McClelland, J.L. and Rumelhart, D.E. (1986) Distributed representations. In: D.E. Rumelhart and J.L. McClelland (Eds.), Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Volume 1: Foundations. Bradford Books, Cambridge, MA, pp. 77-109. Hubel, D.H. and Wiesel, T.N. (1974) Uniformity of monkey striate cortex: a parallel relationship between field size, scatter and magnification factor. J. Comp. Neurol., 158: 295-306.
Kohonen, T. (1972) Correlation matrix memories. IEEE Trans. Comput, 21: 353-359. Kuffler, S.W. (1952) Neurons in the retina: organization, inhibition and excitation problems. Cold Spring Harb. Symp. Quant. Biol., 17: 281-292. Lettvin, J.Y., Maturana, H.R., McCulloch, W.S. and Pitts, W.H. (1959) What the frog's eye tells the frog's brain. Proc. Inst. Radio Eng., 47: 1940-1951. McIlwain, J.T. (1964) Receptive fields of optic tract axons and lateral geniculate ceils: peripheral extent and barbiturate sensitivity. J. Neurophysiol., 27: 1154-1173. McIlwain, J.T. (1975) Visual receptive fields and their images in the superior colliculus of the cat. J. Neurophysiol., 38: 219230. Papoulis, A. (1968) Systems and Transforms With Applications In Optics. McGraw-Hill, New York. Pfaffman, C. (1941) Gustatory afferent impulses. J. Cell. Comp. Physiol., 17: 243-258. Pope, M. (1975) The Story of Decipherment: from Egyptian Hieroglyphics to Maya Script. Thames and Hudson, London. Sparks, D.L., Holland, R. and Guthrie, B.L. (1976) Size and distribution of movement fields in the monkey superior colliculus. Brain Res., 113: 21-34. Uttal, W.R. (1971) The psychobiological silly season - - OK what happens when neurophysiological data become psychological theories? J. Gen. Psychol., 84: 151-166. Van Essen, D.C., Newsome, W.T. and Maunsell, J.H.R. (1984) The visual field representation in striate cortex of the macaque monkey: asymmetries, anisotropies and individual variability. Vision Res., 24: 429-448. Willshaw, D. (1989) Holography, associative memory, and inductive generalization. In: G.E. Hinton and J.A. Anderson (Eds.), Parallel Models of Associative Memory. Lawrence Erlbaum, Hitlsdale, NJ, pp. 103-127. Young, T. (1802) On the theory of light and colours. Phil. Trans. R. Soc. Lond., 92: 12-48.
M.A.L. Nicolelis (Ed.)
Progress in Brain Research,Vol. 130 © 2001 Elsevier Science B.V. All rights reserved
CHAPTER 2
The evolution and implications of population and modular neural coding ideas Robert R Erickson * Departments of Psychology, Experimental, and Neurobiology, Duke University, Durham, NC 27708, USA
Introduction An examination of the evolution of the maj or current neural coding ideas reveals interesting and surprising steps in each that illuminate and clarify the others. The major aspects of these models allow most to be placed roughly into two major categories: population coding is one, and modular coding the other. Since these are approximately orthogonal modes of thinking, each throws the other into high relief. Within each the role of the temporal course of neural activity can be important. To bring order to the various ideas, the meanings of some of the terms used will be closely examined. This seems justifiable on two grounds. The first is that different researchers use the same terms in different ways, different terms are used for very similar or identical ideas, and terms are often simply not defined. The second is that simplification, a standardization of useful definitions (see Appendix), helps provide a coherent account of the ideas. Herein an attempt is made to have the definitions used I as close to typical contemporary meaning as possible. * Corresponding author: Robert R Erickson, Department of Psychology, Duke University, Durham, NC 27708, USA. Tel.: +1-919-660-5718; Fax: +1-919-660-5726; E-mail: eric @psych.duke.edu l Research, both past and present, will be described using modem terms, reduced to a few standard definitions (see Appendix). This is done to cast varieties of research into the common mold sought herein. For example, Gall and
Both the modular and population ideas about neural coding had their first formal statements in the early 1800s. In 1811 and 1825 Gall expressed the view that specific functions are carried out by correspondingly specific structures of the brain - modules in m o d e m terms. In 1802 and 1807, Young made a surprisingly astute and concise statement about the nature and value of population coding - in miniature - - in his theory of color vision.
Modularity Gall's modular approach was dictated by the implicit assumption that our words are usefully competent to represent our behavioral functions 2, and that each
Young are described as using modular and population ideas, although they did not use these terms, and the 'fit' may not be exact. Contemporary researchers may be surprised to see such words attached to their research. If not explicitly stated otherwise, such use of these words is the present author's. 2 Many scientists are sensitive to the influence of our words on our understanding. For example S.J. Gould discussed the problems of the verbal categorizing imperative thus: "The human mind seems to work as a categorizing device ... This deeply (perhaps innately) ingrained habit of thought causes us particular trouble when we need to analyze the many continua that form so conspicuous a part of our surrounding world. Continua are rarely so smooth and gradual in their flux that we cannot specify certain points or episodes as decidedly more interesting,
10 word-function is mediated by a distinct part of the cortex dedicated to that function (for an accessible resource see Boring, 1929, 1942). One aspect of his basic idea was certainly well-reasoned; that the importance of a given function for a particular species was reflected in the size of its neural representation. Thus primates have large visual areas, birds large cerebelli, and bats large auditory areas - - a very acceptable and m o d e m point; Gall was an excellent comparative neuroanatomist. He ventured further with the reasonable assumption that, given variance between individuals within a species, here humans, there will be idiosyncratic differences in the size of various brain areas resulting in differences in brain shape. A musically gifted individual should have more neurons in a certain brain system (think Mozart), manifest in a slightly different brain shape. These enlargements would impress themselves onto the shape of the skull, which is easy to measure. This kind of thinking is only human. Given the properties of language, we simplify nature by packaging it into our categorical words - - what else is
or more tumultuous in their rates of change, than the vast majority of moments along the sequence. We therefore falsely choose these crucial episodes as boundaries for fixed (verbal) categories, and we veil nature's continuity in the wrappings of our mental habits ... We must also remember another insidious aspect of our tendency to divide continua into fixed categories. These divisions are not neutral; they are established for definite purposes by partisans of particular viewpoints." And Kimble wrote (1996) "Language is the agent of cognition, the currency of thinking, the tool-box of communication and the custodian of culture. To be useful, it must map onto the world with some precision. Unfortunately, however, the fact that it does so encourages the faith that the fit is perfect and that truth is in the dictionary: If there is a word for it, there must be a corresponding item of reality. If there are two words, there must be two realities and they must be different." William James (1890) (a foundational psychologist and brother of the author Henry James, who may have been a greater, but less systematic psychologist than William), recognized the problem: "Whenever we have made a word.., to denote a certain group of phenomena, we are prone to suppose a substantive entity existing beyond the phenomena. [And] the lack of a word quite often leads (to the idea) that no entity can be there ... "
possible? 3 In biology, identifiable body structures have always been given names, and then the functions of these structures were given names - - an approach validated over time by its usefulness. But a potentially powerful and core problem with this approach is that we cannot be certain that our words match the functions of the brain; we need imagination here, and some good luck! Fodor (1983) used the term 'modularity' to refer to the identification of brain structures with singular functions, both adequately identifiable with our words. In his definition, each structure/function is 'encapsulated' or insulated from other structure/functions, and thus 'impenetrable' by other functions 4. Bell, in 1811, and Magendie, in 1822 (see Boring, 1929, 1942 for accessable reviews) took Gall's modular line of thinking one step further, ascribing sensory and motor activity to the dorsal and ventral roots, and then another step was taken by Mueller (1838) by naming separate functions for the variously distinct sensory nerves. Further advances in ideas about neural function pretty well depended on advances in techniques. After Schwann showed that the brain is composed of cells, in 1839, Helmholtz (1862) continued the modular logic to the level of the individual neuron, with each auditory receptor and its
3 That the answer to this question is not easy should be cause for concern for all scientists. 4 Modules can be defined mathematically as 'crisp' sets. Crisp sets are groups in which the events are totally inpenetrable and encapsulated; each event is completely described by the set. If a person (event) is Swedish, this term totally describes his nationality, with no other terms needed to complete the description. Also, the 'Swedish' set cannot be used to define any other trait of the individual. On the other hand, 'fuzzy sets' allow other than complete membership of an event in each of several sets - - an event may have a 'grade of membership' of 0.6 in one set, 0.3 in another, and 0.5 in a third set, and each set includes various degrees of memberships from a variety of other events. One person (event) may be tall, musical and agreeable to various degrees (three fuzzy sets), and another person may have different ratings on these scales. Combinations of such differences allow definition of a great many more events than there are sets. This is analogous to population coding, as used herein.
11 neuron providing the code for a particular tone. With the next technological step, recordings of the electrical activity of individual neurons, Adrian (1928) and Bronk (Adrian and Bronk, 1928) found that the only neural signal available to each neuron was an 'all-ornothing' electrical spike. Since the size and form of the spike were unmodifiable in normal function, and since rate changes appeared to encode stimulus intensity, then the identity of the stimulus (pain, touch, vision etc.) could only be encoded by which neurons were active. This supported Helmholtz's idea in audition that these individual neural structures were labeled as to their function (one structure for each tone). A fitting epithet soon followed: 'labeled-line' coding (see Perkel and Bullock, 1968)5 wherein each neuron ('line') encoded its particular ('labeled') function. The work of Hubel and Wiesel established the presence of labeled 'straight line detectors' 6. This view of the role of individual neurons in neural coding " . . . holds that the output of one neuron can be interpreted without reference to the output of its neighbors" (Stevens and Zador, 1995). This is an exact description of modularity, but at the single neuron level. Other neural organizations specialized for situations of importance to a species, such as 'face cells' and species-specialized auditory neurons were discovered as our technical abilities evolved. On a larger scale, separate areas of the cortex were found for different aspects of visual behavior (e.g. Mishkin, 1983; Zeki, 1993) which presaged the exciting development of a great variety of complex modules, facilitated by the new techniques of neuroimaging. A sampling of these include executive control (Knight, 1994; Cohen et al., 1997; Godefroy et al., 1999); theory of mind (Peterson and Siegal,
1999); shyness and sociability (Schmidt, 1999); spatial and object memory (Mecklinger, 1998); semantic memory (Gabrieli et al., 1996); integration of relations (Waltz et al., 1999); abstract vs. specific object recognition Marsolek (1999); sense of self (Craik et al., 1999) etc. The point of the above is that the same style of modular thinking has been 'reinvented' at the singleneuron and larger levels many times since at least Gall. The historically continuous idea that certain easily identifiable structures have independent functions each of which could be identified by one of our words could hardly have been otherwise. For better or worse, to be verbal is to be Gallian. In this paper, modular coding refers to the situation in which a group of neurons totally fulfills one simply namable function, and participates in no other functions. This collection of neurons, and the event encoded, constitute a module. This definition closely follows Fodor 7, whose main functional criteria are that a module be 'encapsulated', and 'impenetrable' by other functions. Structurally, Thus modularity here includes any single-function system from single 'labeled-line' neurons or groups of neurons, to brain areas or neurotransmitter systems, or any other single-function system. In older, simpler terms, this is the idea that neural structure and function are two sides of the same coin.
5 Perkel and Bullock discuss many forms of neural codes, including 'modular', although that term had not yet come into existence. Their landmark definition of 'labeled-lines' gave the term much more flexibility than seen in current usage. They also provide an informative discussion of coding in the temporal nature of a neural response. 6 Hubel and Wiesel (1968) saw the implications of their work to be that, since there are very many neurons in the cortex, there could be modular labeled-lines for each conceivable visual object.
7The term 'modularity' was used by Fodor (1983) to incorporate a failed structuralist psychology (Titchener, 1898) into the science of neural organization. According to that view, for which Fodor uses Gall as his neural model, human behavior is divided up into a number of informationally encapsulated structures. As an example, he speculated without formal rationale that comprehension or production of language may not use the same computer as one used for the understanding of categories such as 'animals' or 'cows'; thus modularity.
Population coding At about the same time that Gall was fighting the main body of medical science, which contended that the brain was something of a bowl of soup filled with liquid 'humors', a brilliant linguist, who had the insight to decipher hieroglyphics using the Rosetta stone, came up with a very exciting idea
12 about the economics of neural coding. This was Thomas Young, whose conception was orthogonal to Gall's modularization s. His idea was propelled by a consideration of the economical representation of information, which also appears to have had a role in his analysis of the Rosetta stone 9. His hypothesis for the encoding of color is succinctly put in what are arguably the two most powerful sentences in the history of neuroscience.
His is a very illuminating and properly scientific note in that it begins with an abstract definition of the issue to be addressed - - neural economy; whatever was in the eye, there was not room enough at each point in the retina for separate receptive 'particles', each 'labeled' for each perceivable color, at each point in the retina. His postulate moved us from a situation where the encoding of many colors is economically impossible, to one which can encode " a variety of traits beyond all calculation." This is an unusual mode of thinking in that it is not based on our words and techniques; his idea was driven by a precisely defined logical problem, a salutary scientific process that cannot be claimed by (at least) Gall. His entire comment sets in its most elemental form the idea and rationale of parallel, distributed, population coding, to provide the means for the immense power of the brain to handle extremely large amounts of information at great speed. The simplicity of the model, set in the context of three neurons, makes the idea altogether clear, and thus generalizable to other more complex situations; this simplicity also makes it easy to ignore when considering these larger issues. The cryptic but core idea here is that the nervous system accepts as a final code information distributed across neurons. This is a nearly impossible idea to accept when it comes to these larger issues; in fact it is hard to believe that it can really be a code for even color unless this distributed activity is 'read out' somewhere onto 'orange' and 'blue-green' cells somewhere in the depths of the brain. But such 'readout' would completely destroy the economy of his core idea of distributed information; that it is not economically realistic that each event - - here, all the different hues, saturations and intensities - could possibly be separately represented centrally for each visual location by separately labeled individual neurons. .
"Now, as it is almost impossible to conceive each sensitive point of the retina to contain an infinite number of particles (receptors), each capable of vibrating in perfect unison (responding) with every possible undulation (wavelength), it becomes necessary to suppose the number limited, for instance (to three); and that each of the particles is capable of being put in motion less or more forcibly, by undulations differing less or more from a perfect unison; for instance, the undulations of green light will affect equally the particles in unison with yellow and blue, and produce the same effect as a light composed of those two species;... " (Young, 1802 - - see Teevan and Bimey, 1961, Ch. 1). " ... the different proportions, in which (the sensations) may be combined, afford a variety of traits beyond all calculation." (Young, 1807 - - see Teevan and Birney, 1961, Ch. 1) 10.
8 Although contemporaries, Gall and Young were unaware of each other's work, a provincialism symbolic of the main theme of this paper. 9 Young's genius was to realize that the hieroglyphic markings could have either phonetic (alphabetic) or ideographic meaning; up to his work, it was considered to be either one or the other, which proved to be a fatal obstacle. He found that some of the symbols had phonetic (alphabetic) roles, and thus in a combinational way, could have immense power in representing much information; this relates closely to his color vision theory and is a clear demonstration of population coding. Interestingly he also found that some symbols were the equivalents of modular representations; to various degrees they had only one meaning (or related meanings), that meaning was completely given by the symbol, and the symbol did not have other meanings. These are slight overstatements, but they give the flavor of the two kinds of codes. 1°I discuss this quote (see Erickson, 1978, 1984) with some change in the meaning given by Young and
.
.
Helmholtz. They present their arguments in terms of sensations - - 'red' for one neural channel, and 'green' and 'blue' for the other two. The manufacture of a single 'blue-green' sensation in response to activation of the 'blue' and 'green' channels was handed over to higher neural centers. This maneuver does not explain anything. Herein it does no violence to their economic logic to avoid the use of their phenomenal terms.
13
60-
50.
0.1 M NH4CI
40,
STtMULU$ ~ CONTINUUM
~ Q
$ R
Neuron I
$ S
P
Q
R
$
P
Stimulus P
I
2
3
30.
impulses
2
Q
R
Q
I
2
20,
3
11o01. 1o 11o $
10"
0
2
3
. . . . . . . . . A
B
C
D
E
F
G
~ ~",~ H
I
J
K
L
M
chorda tympani fibers [rat] P
Q
R
R
3 I NEURONS
•
5
S
I
2
3
Fig. 1. Illustration of Young's population coding idea for color vision, and the applicability of this idea to other systems. (A) The tuning curves of three sensory neurons are drawn along their stimulus dimension, and their responses to four stimuli are shown. This example applies to all sensory and other systems where the dimension may be direction of movement, memory, or other functions. (B) From A. Modular consideration of the individual neuron as the functional unit. Note that each responds to the stimuli in various degrees, leaving the definition of the stimulus ambiguous within each neuron, and confusing the identity of the stimulus with stimulus intensity. (C) From A. Consideration of the population of three neurons as the functional unit. Each stimulus gives an unambiguous pattern across the population; intensity is coded variations in the height of each pattern (not shown). (Erickson, 1968)
The essence of Young's idea is given in Fig. 1. In Panel A, the several curves represent the logically arranged responsiveness of neurons along their continuum. In Young's case, as illustrated by Helmholtz (1860), the continuum is visual wavelength. Since such bell-shaped curves are found in all sensory, motor and other systems, they have been given the general rubric 'Neural Response Functions' (NRFs - - Erickson et al., 1965). Fig. 1B shows the rood-
Fig. 2. An example of a population response. Thirteen taste neurons are arranged along the abscissa with their amounts of response to three stimuli. Each neuron responds to all three stimuli such that within each neuron the identities of the stimuli are confused, along with their intensities (see Fig. 1B). The pattern of response across the neurons gives an unequivocal identification of each stimulus and, in the height of the pattern, the intensity of the stimulus (see Fig. 1C). The similarities of the stimuli are given in the similarities of their population responses (NH4C1 is very similar in taste to KC1, but neither tastes like NaC1). (Erickson, 1963).
ular (labeled line) approach to neural function; here the meaning of a neuron's activity is sought exclusively within its own activity, without the context of the other neurons' activity (see Stevens and Zador, 1995). Clearly each neuron in Fig. 1B, by itself, is confused about the identity and intensity of any stimulus; neuron 1 gives the same response to stimuli P and Q, and in neuron 2 a slightly more intense stimulus R would give the same response as stimulus Q. Young's view is given in Fig. 1C; here the response evoked across the population of three neurons gives an unequivocal code for each of a very large number of stimuli (Erickson, 1968, 1974, 1982, 1984; Rolls et al., 1997a,b) as illustrated in Fig. 2 (from Erickson, 1963). The height of a pattern defines the intensity of a given stimulus. This is the population idea at the micro level. The orientation taken here is that if the nervous system finds this principle acceptable and powerful at a very small level, it need not give it up at more molar levels where the economic demands are much larger. In this paper, neural 'population coding' refers to
the essential but idiosyncratic participation of neu-
14 rons in the representation of a particular event," thus information is spread out over various neurons each of which has a differential role in the code. This broad definition includes color coding as the simplest and most easily examined example. As a standard but more complicated example in visual coding, the location, form, color and movement of an object appear to be represented in separate neural areas. In population coding, these neural codes for location and form etc. need not be brought back together into a common neural pool for binding, but are a completed code in this distributed population state. Young's distributed theory, by whatever name, makes coding of many complex issues possible with limited neural resources. Many investigators since Yong have seen the importance of population coding 21. It is notable that these workers were and are regularly unaware of the ideas of their heritage or present family, and thus cannot benefit from them or cite them. The reinventots all came to this position from a recognition that the great breadth of each neuron's sensitivity (NRF) requires that each event must be encoded by many neurons of diverse sensitivity, and that no one neuron (or class of neurons) could unequivocally encode any event. These workers were each totally original. If the rubric 'population' or 'across-fiber pattern' had been invented, communication and progress would have been facilitated. The many researchers who see breadth of tuning of individual neurons as a basic problem to be eliminated neurally (perhaps by lateral inhibition), or phenomenologically to clean up the neural message, will not be discussed other than to observe that they are uncritically carrying the Gallian idea to the level of the single neuron. The economic problem of neural coding solved by broad NRFs and the 'population' scheme has escaped notice except by Young, and perhaps Adrian and Sperry in somesthesis. A brief illustration of some reinventions of population coding in various areas follows - - the point being how frequently a good idea will be discovered. (In-
tl Far more are the number of researchers who disclosed the kind of broad NRFs without attempting hypotheses about their possible roles (see Erickson, 1968 for an early review).
clusions in parentheses within quoted materials are by the present writer.) Olfaction The earliest advocates of population coding were driven to this idea on the basis of the broad tuning of the afferents - - which for them ruled out the possibility of Mueller's idea of 'Specific Nerve Energies' (Mueller, 1838) at the single neuron level. Lord Adrian (1956), the first scientist to record the electrical activity of individual afferents in any system, in summary of his earlier work in olfaction stated that since the afferents are so broadly tuned across stimuli, stimuli must be encoded in the "relative amounts of activity across mitral cells" 12. Using homey, easy-to understand language he said; "One (mitral cell) will respond more readily to tea than to coffee, and another more readily to coffee than tea." His observation was that since each mitral cell responded broadly but idiosyncratically to diverse stimuli (see Fig. 1A), the code for any stimulus could exist only in the pattern of amounts of activity across the differentially responding neurons (Fig. 1C) - - unbeknown to Adrian an exact restatement of Young's principle. The many workers in olfaction following Adrian, using the most sophisticated neural recording and brain imaging techniques, have confirmed his conclusions about great NRF breadth, and have accepted the idea of 'across-fiber patterning' for olfaction (for a review see Erickson, 2000). Other terms are sometimes used for the same coding idea (see Appendix), such as 'combinatorial' coding (Ressler et al., 1994; Fredrick and Korsching, 1998; Malnic et al., 1999). The broadly tuned NRFs are sometimes called 'generalists' (see Schneider, 1955; Pilpel and Lancet, 1999). The use of only one term for the same event would be helpful. 12Lord Adrian was a truly prolific research scientist, especially considering the lack of relevant prior work and the extremely primitive nature of his recording instruments. He worked in all peripheral afferent systems, as well as their central projections, providing extremely perceptive observations along the way, comments that are well worth reading today. The only omitted area was taste; this he assigned to Pfaffmann.
15 Gustation As in olfaction, the remarkable breadth of tuning of the afferents played a primary role in Pfaffmann's espousal (1941) of population coding in taste. He stated that "In such a system, sensory quality does not depend simply on the 'all or nothing' activation of some particular fiber groups alone, but on the pattern of other fibers active." Zotterman (1958), one of Pfaffmann's colleagues, echoed his assertion thus: " ... even so-called primary taste sensations are built up from a composite input pattern of several taste fibre types." Erickson, one of Pfaffmann's students, in following this line of thinking concluded: " ... the neural message for gustatory quality is a pattern made up of the amount of neural activity across many neural elements" (1963). He used the term 'across-fiber pattern' for population codes, and continued to develop this systems idea as a general neural code across diverse neural functions. Di Lorenzo (1989) developed a vector model to analyze this pattern of activity across taste neurons similar to that invented by Georgopoulos for motor organization (see Georgopoulos, 1995 for a review). At the present, there is divided opinion over whether there are five 'basic', 'modular' tastes ('sweet', 'sour', 'salty', 'bitter' and 'umami') each with its own private 'labeled lines', or a population AFP code across the many broadly tuned NRFs resulting in many tastes. A clarification of what the term 'basic taste' means would help resolve this debate. At any rate taste, as olfaction, demonstrates 'penetrable non-encapsulation' with each neuron responding to many stimuli, and each tastant modulating the perceptual effects of other stimuli, There could be exceptions, for example if there were a separate system encapsulated for homeostasis of sodium. But this would not serve for the encoding of the vast array o f different taste stimuli among which animals must differentiate (see Erickson, 2000 for a review). Somesthesis The history of somesthesis is replete with reinventions of Young's theory. Adrian (1928) and Adrian et al. (1931) extended population coding from olfaction to somesthesis, this time mentioning not only
to the fact of broad tuning of each afferent, but the accuracy of a code based on broad NRFs. "If there is much overlapping in the areas supplied by the terminal branches of different fibres it might be possible to localize the stimulus more exactly by comparing the relative intensities of discharge in the different fibres." (Adrian, 1928, p. 92). In the same line, Nafe (1929) restated Mueller's 'doctrine of specific nerve energies': "The specific accompaniment of sensory excitation is correlated with the number of nervous pulses and their temporal and spatial relations." In recordings from individual ciliary (trigeminal) neurons from the cat cornea, Tower (1943) found very broad receptive fields (a quarter or more of cornea, plus adjacent sclera and conjunctiva) whose sensitivity functions resembled the NRFs portrayed in Fig. 1A. She observed that in their overlap, a few of these neurons could accurately portray the location of the stimulus. She generalized this idea to all systems which have broad NRFs; it turns out that this includes all systems for which adequate data are available (Erickson, 1968, 1974, 1982, 1984). In describing the development of broad and overlapping NRFs of somatosensory neurons, Sperry (1959) speculated that "Precise localization is further enhanced by the overlapping of the terminal connections formed by the fibers in the skin", and in the legend of the accompanying figure mentioned, "Overlap of sensory fibers permits a subject to localize a pinprick accurately." His drawing is a 2dimensional map comparable to an overhead view of the single dimension portrayed in Fig. 1A, including the form of the NRFs and their overlap. Loomis and Collins (1978) explicitly use Young's model to account for tactile perception. Johansson and Vllabo (1980) come to similar conclusions from quantitative considerations of tactile sensitivity. Many other workers in the skin senses have made similar references to AFP coding: a few of these include Mountcastle (1980) in the description of profiles of neural activity across distributed, ensemble activity; Johnson and Hsiao (1992) used population responses for tactual form and texture; Merzenich and deCharms (1996) used AFP logic for localization; Ray and Doetsch (1990) and Doetsch (2000) used AFPs for the skin senses in general; Surmeier et al. (1986) described an AFP code for heat; and
16 Nicolelis et al. (1995) described complex population responses which over time shift to other populations as a code for localization. Audition Given the breadth of auditory neuron NRFs, Helmholtz (1862) cannot be correct in assigning a separate modularly labeled neuron for each tone; this function is obviously carried by a population code. Azimuth is also encoded in AFPs (e.g. see Eisenmann, 1974; Eggermont, 1998). Using point stimuli, Knudsen (1982) showed broadly tuned and distributed auditory and visual responses in the rectum of the owl. Covey (2000) has reviewed AFP coding in the auditory system of bats. Motor systems It should be noted that vector analyses of efferent activity in motor systems assumes AFP coding - a population response summed in a special way; McIlwain (1975, 1991) used vector analysis of efferent activity from the superior colliculus, as did Georgopoulos and his colleagues for cortical motor neurons (see Lukashink and Georgopoulas, 1993; Georgopoulos, 1995); the response curves for these neurons closely resemble the curves in Fig. 1A. Eye movements were found to be driven by populations of neurons by Lee et al. (1988) and Law et al. (1998). Muscle sense Llinas and Welsh (1994) used AFP coding for movements, and Johansson et al. (1995) used "ensemble coding" to account for the accuracy of information about muscle stretch. Joint position Mountcastle et al. (1963) showed that afferents sensitive to joint position respond monotonically across most of the joint angle, with maximal responses at full flexion or extension. This instance raises the point that NRFs need not be bell-shaped as in Fig. 1A; they simply need only to be simple (smooth) and broad to fulfil their AFP coding role.
Vestibular sense Adrian (1943) was the first to describe the broadly tuned activity of vestibular afferents. Fernandez et al. (1972) found each vestibular afferent neuron to be sensitive throughout all head positions, with NRFs as in Fig. 1A. Vision That broad NRFs require distributed coding has been obvious for many workers in various aspects of vision. The following quotes describe the heart of AFP coding exactly. Hartline (1940) spoke of activity in the peripheral nerve as follows: "It is evident that illumination of a given element of area on the retina results in a specific pattern of activity in a specific group of optic nerve fibers. The particular fibers involved, and the distribution of activity among them, are characteristic of the location on the retina of the particular element of area illuminated. Corresponding to different points on the retina are different patterns of nerve activity; even two closely adjacent point do not produce quite the same distribution of activity, although they may excite many fibers in common." Hartline, with his students Miller and Ratlliff (see Miller et al., 1961), showed that the visual phenomenon of heightening of contrast at visual borders ('Mach bands') depended on an AFP of activity distributed across a population of neurons 13 Nelson (1975) showed broad NRFs for stereopsis, requiring AFP coding. Since a visual point produces widespread activity in the superior colliculus, McI1wain (1975, 1991) knew that each neuron had a very broad spatial NRF, and thus the spatial code could only reside in the AFP across many of these cells. He (1986) beautifully summarized the issue of population coding, using visual location in the superior colliculus as an example: " ... the traditional question of recepfive-field analysis is turned on its head: instead of asking 'Here is a cell: where are the points that it sees?' the question becomes,
L3Each demonstration of the 'Mach band' neural population effect was obtained from one neuron, the stimulus being moved so that this neuron would play the part of all neurons involved in the natural situation.
17 'Here is a point; where are the cells (constituting the AFP) that see it?' "Gochin et al. (1994) advocated AFP coding in inferotemporal cortex for visual form, as did Spinelli (1966) in the retina and Spinelli et al. (1970) in the cortex. Bishop (1970) predicted that "The discrimination of form may always be the property of groups of neurons, perhaps always of large assemblies of neurons and never in any sense the trigger feature for one neuron." For AFP coding for faces see Young and Yamane (1992); Rolls et al. (1997a,b); in inferotemporal cortex see Vogels (1999) for visual categories, and Booth and Rolls (1998) for complex objects. Memory Lashley (1951) set the stage for population coding for memory. Because of the broad and overlapping neural areas involved, he concluded that neurons supporting memory must be broadly tuned across many functions - - that is, no individual neuron could be given over to a particular memory. Many others followed this population format. John (1972) found neural responses in response to memory to be extensively distributed in large assemblies of nerve cells. Abbott et al. (1996) and Sakurai (1998) proposed broad neural tuning with population coding for memory - - all close to Lashley's position. General Hebb (1949) presented a 'big view' of neural coding and plasticity in responses of neural populations (large sets of cell-assemblies). Lashley had the courage to raise issues that are important rather than convenient (e.g. Lashley, 1929, 1942, 1951) although he resolved few. Within a spatial and temporal patterning model he sensibly addressed 'untouchable' issues such as sensory equivalence and motor equivalence, insight, generalization, reasoning, and temporal order in behavior (such as language). In his discussions of neural organization in general, he concluded (1929) that information must be encoded in patterns of activity across neurons thus; "The problem of reaction to ratio (both the stimuli and AFPs consist of ratios - - i.e. the characteristics of a figure, or the amounts of activity across neurons) thus seems to underlie all phases of behavior, to such
a degree that we might be justified in saying that the unit of neural organization i s . . . the mechanism... by which reaction to a ratio is produced." Barnes et al. (1997) discussed cognitive maps as residing in populations of hippocampal cells. According to Llinas and Pare (1996) cognition requires spatial and temporal 'patterns of activity' across neuronal assemblies. Sperry (1969) and Mountcastle (1980) presented the idea that consciousness could arise as an emergent property of spatial and temporal patterns of distributed neural activity, and Tononi and Edelman (1998) argue for a 'central core' hypothesis for consciousness; that consciousness is not represented in one place, or over the whole brain, but by activity distributed across a few specific brain areas (each of which is here considered a module) which work together as an ensemble. The basic tenet of Gestalt neuroscience is that information is spread out as a pattern across functionally heterogeneous brain tissue; their further comments about electrical field theory which have caused them grief are not an essential part of the theory. Whether or not acknowledged, Gestalt-type thinking pervades ideas about neural organization, especially for visual form. In a general statement about information representation, Goldman-Rakic (1988a) presents the opinion that cortical function be approached " ... in terms of information processing functions and systems rather than traditional but artificially segregated sensory, motor, or limbic components and individual neurons within only one of these components," Erickson has applied the AFP population model to neural functions in general (Erickson, 1968, 1974, 1978, 1982, 1984, 1986, 2000). It might be fairly said that in a very basic sense, almost all of these reinventions of population coding did not, at their core, advance us past Young's 1802 and 1807 statements. With better communication and systematization, workers would have been spared the labor of reinvention, and could instead have been building on the preceding data and thought. Modular and population coding Modularity usually implies population coding, even if this is not made explicit. On the one hand, a number of functionally diverse neurons usually act as a popu-
18 lation to precisely define a putative modular function, such as the form or color of an object. On the other hand, several modules may work together as a population code; three examples follow. Modules for color, location, form and movement could cooperate as a population to represent a complex stimulus such as a face (Edelman and Mountcastle, 1978; Young and Yamane, 1992; Zeki and Bartels, 1998), or to represent body-object relationships (Graziano and Gross, 1995), with each of these modules performing its function by way of the population response of neurons of diverse function. A second example may be seen in the work of Posner and Petersen (1990) and Posner (1992) who posit four modular functions for directed attention: orientation of gaze, word form, word meaning, and vigilance. Each of these functions is posited to be as modular as the optic nerve is for vision, but they cooperate as a population in providing directed attention. Again, the various 'modular' aspects of pain - - discriminative (where is it, how strong?) affective (negativity), and cognitive (aspects of pain influenced by attention etc) - - appear to be given to various brain areas, acting in distributed form to produce the unitary experience of pain (Casey, 1999). But to group several such modules together as a higher-order encapsulated module would seem to allow further dilution of the modularity concept to the point of meaninglessness. To provide coherent behavior, modules must function in parallel - - putting the concept of modular encapsulation and insularity in question. Thus modularity includes and requires the concept of population coding. While the brain necessarily operates on population principles, the need for modularity has not been shown.
Issues for the modular and population neural coding schemes When are modular or population coding appropriate ? An efficient neural science should be characterized by its ability to anticipate in which cases its proposed codes would be appropriate; after the fact arguments are doomed to succeed. Several rationales for the appropriateness of given coding mechanisms are examined next.
Rapidity of neural processing Rapidity of neural processing is certainly a major issue given the size of the brain and the complexity of the networks. For complex issues, the many synaptic delays needed for the formation of a module of themselves could slow the organism to a very slow pace. But Treisman and Kanwisher (1998) argue that given their compact nature, modules should provide very rapid processing. On the other hand, the implication of parallel processing is that when the relevant neural activity reaches its distributed form, the processing is complete. In other words, even highly complex processing could be accomplished in the very short time it takes neural activity to reach its distributed destinations: e.g. the various visual areas for form, location, color etc. In that distributed form, information is immediately available for the full appreciation of the visual scene without the further generation of labeled 'higher-order' neurons or modules. This extreme rapidity of population coding may be difficult to accept, especially in complex cases, but it is the exact implication of Young's theory. Importance The importance of an event (e.g. visual, auditory, motor etc) for a given species is a first principle of neural organization in neuroethology. And the importance of a function seems to be a primary rationale for nomination to modularity, although this was not a first principle in Fodor's presentation (Fodor, 1983). For example, Desimone (1991) suggested modularity for face perception in humans because of the importance of this function. But no detailed rationale is available that importance requires modularity. On the other hand, in the Neural Mass Differences (NMD) aspect of population coding, it is reasoned that functions require neural mass in proportion to their complexity and importance to the species in question (Erickson, 1986). Singularity Just on their face, some simple functions seem to be singular, which is another way of saying 'encapsulated' and 'impenetrable'. Certain organ systems,
19 such as the kidney and heart are mono-functional, or 'modular'. The eye and optic nerve are clearly modular in that they provide vision and nothing else, and no other neurons provide visual input; other firstorder sensory neuron systems are similarly modular (Mueller, 1838). Alpha-motoneurons are modular in that they cause contraction of specified muscles. The giant axon of the crayfish performs only one function, escape, encapsulated and inpenetrable even from guidance, this very simple act is truly modular. Pheromone systems, based on olfactory 'grandmother neurons', provide a reasonable example of modular coding by neurons dedicated to eliciting one behavior, such as finding a mate (unlike the crayfish example, the behavior is guided (penetrated) as necessary to find the target). The terms 'non-combinatorial' or 'focal' are used for such 'labeled-line' pheromonal coding (Fredrick and Korsching, 1998). 'Releaser mechanisms', simple behaviors triggered by circumscribed stimuli (e.g. Ingle and Crews, 1985), are at about the same level of singularity as pheromonal systems. Neural modularity seems appropriate for very simple, circumscribed behaviors. If modular singularity is sufficiently identified above, can it be used for more complex functions? There appears to be a single-function song production system in some birds, and perhaps a language module in humans and some great apes, certainly functions of great importance to these species. The dissociability of symptoms is one method used to define modular singularity; in the 'theory of mind' module (see Stone et al., 1998; Tager-Flusberg and Boshart, 1998), singularity is defined in its dissociability from other behaviors. That is, the capacity for a theory of mind is disturbed in autism, but not in Downs or William's syndrome. Alas, it is very difficult to prove an idea true, especially with one dissociation, because the next test may prove it false (Popper, 1962). An extensive theoretical rationale for modularity would make reasonable tests possible. In the AFP view, all activities are necessarily interwoven and interpenetrable as the organism functions coherently and simultaneously on many fronts. Many activities are carried on at the same time, and in sensible relation to each other. That is, vision is part of the complex fabric of coordinated and interdependent organismic activities - - vision is neither encapsulated nor impenetrable.
Economics and complexity Brains are finite. Modules are very expensive of neural resources. An adequate definition of modules should include a logical and quantitative discussion of which and how many a brain needs and can afford. As discussed for singularity above, modules seem appropriate only for the simplest of functions, yet they are nominated for the most complex, from directed attention to theory of mind. With across-fiber pattern models, very large amounts of information can be expressed with a few neurons. If each neuron has 10 discriminable levels of activity, 13 neurons could carry 1013 patterns, a large quantity with respect to the total number of ('grandmother') neurons or possible modules in a human brain (see Fig. 2). This model is thus very appropriate for high information loads. Detection of the enormous number (here 1013) of potential antigens could be detected with a few (here 13) broadly tuned chemoreceptors (see Lancet and Ben-Aire, 1993; Pilpel and Lancet, 1999). One million proteins could easily be generated by 34,000 genes in their various combinations. The Oxford English Dictionary is non cramped by the 26 letters on which it is based, and all of music is based on 12 notes (Bernstein, 1970). Modules or functions? There is a difference between a set of neurons modularly dedicated for a particular function and neurons usually involved in this activity. That is, organisms certainly direct their attention, and over repeated instances many of the same neurons must be involved. This does not require, and it does not seem economically reasonable, that this is the only function for these neurons. In the population view, neurons are usually multi-purpose so that those used during directed attention, for example, will probably also be involved in other activities; thus 'directed attention' and its components may be defined as functions while not necessarily requiring modularity. Readout It is a primary characteristic of population coding that not only are input and output information in AFP form, but for the same economic reasons all
20 functions internal to these must be in population form. The information is at all times expressed in distributed patterns with no readout agencies needed to interpret these patterns. The absence of a readout system provides for the great economy and rapidity of processing large amounts of information. As for color vision, motor responses or other 'internal' productions occur in the time for neural activity to reach its distributed form - - this being the 'final' code. An internal modular 'readout' concept implies multiple massive, cumbersome, and time-consuming changes of format between input and output.
(1979) nominated the last four for specific neural networks or special-purpose neural processes on the basis of the existence of such areas for language and faces. Could any of these be accepted or rejected as modules on a priori principles? Is modularity to be accepted for any function after the finding of its singularity? Is anything that seems to be 'encapsulated' a likely module? Why? Perhaps we share Gall's fatal flaw of not having a logical theory of exactly what to look for. If a logical and clearly predictive theory is not developed, the future may see a swing away from modularity as it has from Gall.
Predictability Fine coding, coarse coding, and hyperacuity When classifying neural functions with words, the difficulty arises about which words should have modular representations. For example, what a priori arguments indicate whether the following are modular functions, or not? Visual color, form and location, sex, self esteem, religiousness, music, passage of time/continuity, and causality. A sense of self, decency, honor, power, fear, humor, patriotism, humility, fatigue or love. Theory of mind, directed attention, executive control, reading, anxiety, language, response error detection, insight, or memory in its many varieties. Comprehension of speech, animals, objects, word meaning, tunes, lyrics, my grandmother. Attention, aggression, creativity, family love, erotic love, love of mankind. Fear, anger, bladder/bowel control or hunger. The first nine putative modules were included by Gall among his listing of over 50 human faculties 14. Geschwind
14Many of Gall's choices of modular human faculties seem entirely modern, without rationale. Some modules include self-defense, property, disposition to quarrel, courage, homicide/suicide/destruction, camivorism, cunning, covetousness, propensity to steal, pride/hauteur, vanity, ambition, cautiousness, memory (separately of things, people, facts), mathematics, constructiveness, understanding of locality/space (cognitive maps?), music, painting, language, understanding of relationships (comparisons), wit, metaphysical depth of thought, poetry, moral sense, mimicry, god/religion, perseverance, knowing the interior of man by his exterior (theory of mind?), mimicry/pantomime, propagation, self-defense, and satire. He gives reasonable discussions of many of
'Fine' and 'coarse' coding are terms that have been employed to describe the breadth of tuning of individual neurons (NRF extent). These terms roughly correspond to modular and population coding, respectively. 'Hyperacuity' refers to the fact that precision of neural function is much greater than either fine or coarse tuning would imply. 'Fine' tuning seems close to 'labeled-line' coding, a modular idea ('specialists', see Schneider, 1955). Fine tuning means that the breadth of the NRFs are small. However, such tuning is small only with respect to the total relevant dimension; this tuning is actually large with respect to the discriminative capacities of the system - - called 'hyperacuity'. For example, the spatial extent of the NRF of each retinal ganglion cell is indeed small considering the spatial extent of the total retina, but very large with respect to visual spatial acuity. This type of coding was termed 'topographic' by Erickson (1968) for events which are represented across neural space. For example, visual and somesthetic space are laid out as maps across neural tissue. Since in this situation the availability of neurons is great, the NRFs are relatively small in order to restrict the total mass of the neural input. This restriction evidently conserves neural mass to that sufficient for each sensory event (Erickson, 1986); looked at another way, broad or coarse somesthetic or retinal NRFs would produce
these, such as their natural history and specific (disassociation) effects of disease.
21 unnecessarily large CNS activity; imagine the input from a point stimulus if each somesthetic or retinal neuron were responsive to the total skin or retinal surface. Rolls et al. (1997a,b) provide a quantification of how the amount of information in population responses for faces etc. is sufficient with a small neural mass ('narrow' NRFs). Coarse tuning, on the other hand, refers to the representation of information by neurons with broad NRFs ('generalists'; see Appendix and Schneider, 1955) as in taste, olfaction, color, temperature and vestibular sensitivity. In the modular view, this breadth is seen as a detriment to the accurate portrayal of an encapsulated event since each neuron is evidently involved in more than one function. However, as described above, broad NRFs are essential for the accurate representation of large amounts of information, especially when the quantity of neurons available is small. For example, the extent of the NRF of a retinal or lateral geniculate cell across the color dimension is very large. Evidently, this is because there are so few neurons available for color coding at each visual location that they must be broadly tuned to both include the total wavelength dimension and to maximize the mass of the neural input. This kind of coding is termed 'non-topographic' (Erickson, 1968) since the relevant dimension is not laid out across neural space. In brief, 'fine' and 'coarse' tuning are equivalent in that the NRF is always much broader than the acuity would suggest from the labeled-line point of view ('hyperacuity', next). In each case the breadth of tuning is probably evolutionarily designed to provide sufficient neural mass differences (NMDs) between discriminable events over the population of responding neurons; fine and coarse tuning are by design, not by mistake. Hyperacuity. The fine precision seen in neural systems in general, but emphasized in sensory systems, is a puzzle given the breadth of tuning of the neurons involved; this is called 'hyperacuity'. For example the fine discriminations between wavelengths and spatial resolutions in vision are far finer than the NRF widths of the neurons involved. Hyperacuity is a large problem if the modular view is assumed since the individual neurons respond to many different stimuli (broad NRFs), but are unifunctional. In the population view, the acuity of the
system is not limited by the NRF width in either fine or coarse coding (Erickson, 1968; however see Zhang and Sejnowski, 1998). The baseline in Fig. 1A may be considered as a small space of skin or retina ('fine' coding), or as the full dimension for color, taste or olfaction, attitude of the head, temperature, arm position, or direction of limb movement ('coarse coding') in all of which hyperacuity is evident. As one example, visual 'simple cells' were first considered to be detectors for straight lines of particular orientation; that these neurons are about as broadly tuned as possible, often spanning nearly 180°, has been shown by Henry et al. (1974); Soodak et al. (1987), and Vogels and Orban (1991). These neurons still provide exquisite orientation sensibility since in population coding acuity does not depend on the breadth of the NRF but on the size of the resultant NMDs, the latter being independent of NRF width (Erickson, 1986). Summary The test for modularity will come when some researcher predicts not only that a certain function must be based on a neural module, but also clearly explains why. A statement of this nature would be vastly helpful whether it was upheld or falsified. It might be even more useful to make clear which functions could not be modules, and why. Population coding models have a strong rationale, the first being the economic one presented by Young. This APF model is clear and testable. It accommodates itself to many issues as described herein, perhaps most importantly to the economy of neural resources, rapidity of action, and congruence with known facts of neural organization such as the puzzlingly broad sensitivities of individual neurons. It is especially appropriate for large and complex issues of high information load such as the concepts nominated for modularity. Why are events taken apart? Neural mass and neural mass differences
The puzzle of distributed wherein several aspects of a to different brain areas, has "Why was the image taken
coding, for example visual image are given two parts. The first is apart?" The second is
22 "How does it get put back together?" These are addressed in turn. Neural mass One rationale for taking things apart (e.g. visual form, location, color, movement) is that this maneuver gives each aspect of an informationally dense event its own substantial piece of neural tissue. This is similar to a sculptor using a large piece of granite to produce a detailed and complex figure. Thus in taking things apart, each important aspect of an event is given its own large piece of granite for careful and detailed sculpting (see Churchland and Sejnowski, 1992). The analogous amount of neural tissue involved (areas under the three curves in Fig. 2) is termed 'neural mass' (NM) herein (Erickson, 1986). It has been argued that discriminability is dependent on total differences in activity produced by the perceived events summed over all participating neurons (NMDs, Erickson, 1986) (absolute differences between the curves in Fig. 2). The size of an NMD is a positive function of the NM, which is the amount of activity summed over neurons, and the time over which the summation occurs (NM = N × F x T, where N = number of neurons, F = amount of activity evoked in each neuron, and T = time over which summation occurs). If it is important for one aspect of an event to receive detailed analysis, then a large number of neurons especially responsive to that event would help make this possible; e.g. if the form of an object is important (e.g. a face), then assigning form to its own group of neurons especially sensitive to this function would provide a large NM, and a large potential for NMDs. Conversely, the size of the NM elicited by a given function should indicate the importance (at least the degrees of differentiation) of that function for that species, as assumed by Gall. The careful expenditure of neurons is not a central point in modular thinking, but is of first importance in AFP coding. Constancy of neural mass That the neural mass evoked for certain functions must be important is strongly suggested by the fact that functional populations are often of a standard size. This means that a definable number of neurons
is required for that function (NM = K). For example, Hubel and Wiesel (1974) indicated a constancy in the size of the cortical representation of visual stimuli of various eccentricity, and Capuano and McIlwain (1981) McIlwain (1986) and show that 'point images' in the superior colliculus are about the same size for stimuli of various eccentlicity. Merzenich et al. (1984) suggested that when the cortical neural area available to a digit increases (i.e. when there is a loss of input from adjacent digits), the size of the receptive fields for each neuron decreases proportionally; this results in a constancy of neural mass activated for each point stimulus. Perhaps such 'neural quanta' are a general rule for diverse neural functions (Erickson, 1982, 1986). Quantification of neural mass Quantification of how much neural mass is required for the representation of an event is eventually an important issue in distributed neural coding. Beginning attempts include those by Erickson (1986), Ray and Doetsch (1990), Nagai et al. (1995), Rolls et al. (1997a,b), and Zhang and Sejnowski (1998). In general, these reports show that up to a point (perhaps the 'neural quantum' mentioned above) increased neural mass provides for greater information representation. As examples, Nagai et al. (1995) showed with taste neurons that the amount of information decreases gracefully with reductions in their numbers; further, a greater loss of information accompanies loss of the more strongly responding neurons - - those which have more variance across different stimuli and thus more potential for NMDs. The capacity for strongly responding neurons to allow large differences between neural representations (NMDs) is shown in Fig. 2. Similarly, Rolls et al. (1997b) showed that with visual neurons sensitive to faces, the amount of information is a positive function of the number of neurons. Abbott et al. (1996) showed information degradation with lessening of number of neurons. Lashley (1931) alluded to the power in numbers in his concept of 'mass action'. Some functions of large importance to a species, as song in birds and language in humans, have developed large neural masses suggesting relatively great information capacity.
23
How do events get put back together? Binding and readout. Goldman-Rakic (1988a,b) posed that the bringing back together of separated aspects of an event, 'binding', is one of the major problems for modem neuroscience. This issue of 'binding' is certainly of primacy for neuroscientists; once different aspects of information about an event are spread across neural tissue, how do they get put back together again in consciousness or other coherent behavior such as movement? Farmer (1998) states the problem well: It is assumed that " ... at some (spatial) point information must be gathered together as a single percept." Damasio and Damasio (1992) suggest that the components of a concept (e.g. for a cup of coffee, the components would be the cup form and color, the coffee aroma and warmth etc.) would be brought back together into the same ensemble of neurons. 'Binding' is a problem only with assumption of the 'modular' point of view. In population terms, the distributed aspects of an event need not be brought back together to correspond with a human construct. To accept the fact that the brain functions in terms of distributed populations of neurons is to understand, as Young did, that this distributed format is
the final language of the brain; the brain does not bind events back together again. This is very clear in motor output (or for any coherent production of the brain) wherein the motor neuron activity underlying a coordinated movement is in the form of a population and is not further coalesced. Sitting requires the correct level of activity of all motor neurons, except perhaps for those of the middle ear. The output, whether in motor neurons, an emotional state, or a memory etc. is in the same kind of population, distributed form as the input (sensory, or from other brain sources). Why would there be a conceptual difference in neural organization (e.g. modularity) between intervening processing levels and input and output? In all these, the same economic problems are evident. The simplest and most conservative position is that neural information does not take categorically different forms in different parts of the nervous system - - until proven otherwise. From this point of view, the inclusion of a second (modular) code in neural organization is not necessary and may not be helpful.
In population coding, various aspects of an event may be brought together into the same ensemble to permit the generation of neural differences between discriminable events (NMDs) across a common pool of neurons. This would be in line with the convergence of relevant neural information onto common ensembles (e.g. see Damasio and Damasio, 1992; Zeki, 1993). However the information in the ensemble would never leave population form (Erickson, 1978, 1986). In summary, as for the representation of color in Young's theory, great economic problems would be caused if the various aspects of an event were brought back together as expected in binding; the degree of economic embarrassment would be a rapidly increasing function of the complexity of the event to be 'reassembled'.
The role of temporal patterns. Since all behavior is temporal, the primary importance of the temporal aspects of neural activity must be accepted. First, they obviously signal simple temporal aspects of an event, such as its onset and offset. Above this, there is considerable evidence that temporal patterns of activity carry other, more subtle kinds of information. This coding has been treated in the modular and/or population formats. In the modular form, a given temporal pattern encodes an event. In the population format, temporal patterns gain their meaning in the context of the different but concurrent temporal patterns in parallel neurons. Modular temporal coding In the modular view, each temporal pattern can represent a different event, or different events at different times; these would be 'labeled temporal patterns' in analogy with individual neuron 'labeled lines'. Several who take this position include Von der Malsburg (1995); Softky (1995); Stevens and Zador (1995), Llinas and Pare (1996), Gerstner et al. (1997), and Ehert (1997). Following Covey's seminal study in taste, Di Lorenzo and Hecht (1993) showed that an electrically driven temporal pattern per se in the nucleus of the solitary tract can produce sufficient information to identify tastes. Quantification
24 of the information in temporal patterns in V1 and IT cortex was examined by Baddeley et al. (1997). McClurkin et al. (1996) entertained the notion that color and visual pattern are both encoded in the same neurons in V1, V2 and V4. These authors suggest that each of these neurons carry separate, but simultaneous (multiplexed) temporal codes for pattern and color information; therefore, separate temporal modules for different colors and forms coexistent within the same neurons. They aver that these temporal modules take the main, perhaps the only, role in color and pattern vision. Sakurai (1996a, 1998) reviews temporal population coding in memory, including Hebb's 1949 contribution (Sakurai, 1996b). Population temporal coding In distinction to the modular idea of temporal coding, Erickson et al. (1994) established that, at least in taste, there are a few (about three) basic temporal patterns of activity (fuzzy temporal sets 4) which, viewed in population format, help establish the identity of the various stimuli. They do this much as Young's broad NRFs encode color in that each stimulus evokes these few multiplexed temporal patterns to degrees idiosyncratic to each stimulus and neuron. That is, each stimulus will produce one or more of these temporal patterns to various degrees in each neuron. In its variation across neurons, this complex temporal pattern identifies the taste stimulus along with population rate coding (average rate of firing). Friston (1997) finds the potential for information in differences in temporal patterns (not in synchrony) over various neural areas, a population idea.
Binding and temporal patterns Timing of neural activity is seen as a strong contender for solving the 'binding problem'. A common hypothesis is that some kind of temporal synchrony over the neural population accomplishes binding. For example, Farmer (1998) suggests that such " ... binding problems may be solved through transient temporal synchronization of the discharges of populations of neurones." But the binding problem is not solved, or even addressed, by distributed temporal patterns. Given temporal synchrony there is still the problem of how to get the distributed events bound together. Temporal coincidence of increased activity in the neurons involved may hold the events up together for special note (large neural mass) in an otherwise relatively quiescent field. But how this 'binds' has not thus been made more evident. If the idea of population coding is accepted it should be clear that, although good signal level - - as through in-phase temporal synchrony - - is always appreciated, events do not need to be physically brought together to be 'bound'. Others Never shunning the important problems for the convenient, Lashley (1951) emphasized the time dimension as a neglected but important aspect of neural coding, including the issue of temporal organization over long periods of time as in language. Hebb (1949) used neural timing to account for the development of coherent neural organization distributed over cell assemblies. Conclusions
Temporal/spatial patterns changing over time Nicolelis et al. (1995) showed that the representation of a tactile stimulus, leading to a movement, is represented by complex temporal patterns across changing ensembles of neurons; this shifting pattern identifies the stimulus/response event. In place of static topography they suggest that " ... spatiotemporal complexity substitutes for topography as the main strategy for the coding of sensory information."
This small and selective review suggests that the evolution of molar neuroscience can be largely characterized by two general coding ideas, modular and population, but that these ideas have not evolved (changed form) for two centuries. The evolution that has occurred has been in technical methodologies, with our statements of the ideas being recast in terms of these techniques rather than leading to them (see Erickson, 1978). These techniques are inherently modular, from Gall's examinations of skull shape,
25 B e l l a n d M a g e n d i e ' s ( a n d m a n y to f o l l o w ) s t i m u l a t i o n a n d a b l a t i o n m e t h o d s , to n e u r a l r e c o r d i n g t e c h n i q u e s a n d b r a i n i m a g i n g . It s e e m s difficult to t h i n k in t e r m s o t h e r t h a n t h e m o d u l a r d a t a t h e s e t e c h n i q u e s p r o v i d e . S e c o n d , t h e d e v e l o p m e n t o f i d e a s h a s also b e e n c o n s t r a i n e d b y t h e n a t u r e o f o u r l a n g u a g e (Eri c k s o n , 1978) i n w h i c h w e are e n c o u r a g e d to e x p r e s s o u r i d e a s i n m o d u l a r f o r m . T h e o n l y true e x c e p t i o n was that of Young whose very successful effort was to s o l v e t h e b a s i c p r o b l e m o f n e u r a l e c o n o m i c s q u i t e aside from language and techniques. It s e e m s l a r g e l y true t h a t o u r t e c h n o l o g y a n d our verbal nature have guided the evolution of our s c i e n c e . It is s u g g e s t e d t h a t b e f o r e w e c a n b e c o m fortable with conclusions about the nature of brain f u n c t i o n , b e t h e y m o d u l a r or p o p u l a t i o n , t h e e x p e r i m e n t a l t e c h n i q u e s a n d l a n g u a g e w e h a v e u s e d to direct our science should be objects of investigation t h e m s e l v e s - - t h e f a c t t h a t it s e e m s difficult to proceed in any other way does not justify the means. B e i n g t h e b a s i c t o o l s o f o u r trade, w e s h o u l d k n o w w h a t role t h e y p l a y i n u n d e r s t a n d i n g a b r a i n t h a t may not be organized in their terms. This should be a c o n c e r n o f t h e first order.
bution to the code, including 'ensemble', 'across-fiber pattern' (AFP), 'parallel', 'combinatorial', 'assembly', and 'distributed' codes; these all express the same idea, with the AFP model being additionally based on the amounts of activity in each neuron. Alternatively the use of these terms to refer to redundant neural activity given to many neurons spread out over local or large brain areas represent redundancy, not population codes. The terms 'modular', 'focal', 'non-combinatorial', 'singular', 'specific' and 'labeled-line' have been treated in roughly equivalent ways by various investigators, the idea varying only in the number of neurons involved (labeled-line refers to one neuron, module to many neurons), but not in general concept. Therefore, they are reduced to a common idea in this paper. Concerning other terms, within this paper 'function' and 'event' are roughly equivalent (neither very clearly defined). Neural Response Function (NRF) refers to the breadth of tuning of neurons along their relevant dimensions. The form of this tuning takes some simple form, such as a bell-shaped curve for color, vestibular sensitivity, line orientation in 'simple' cells, and motor neurons; other forms include smoothly and monotonically increasing or decreasing curves as in joint position, and s-shaped curves for color-coding beyond the receptor level. An NRF is an explicitly defined and generalized 'tuning curve'. NM refers to neural mass, a combined and positive function of number of neurons involved in the representation of an event, response magnitude of these neurons, mid the time over which integration occurs. It is approximately represented in brain imaging techniques. Absolute changes in neural mass, summed across neurons, are taken as the basis for neural information, and are termed neural mass differences (NMDs) (Erickson, 1986).
Acknowledgements References T h e c r i t i c a l a n d i n s i g h t f u l r e a d i n g s b y Drs. D o n a l d K a t z a n d B r u c e H a l p e r n are g r e a t l y a p p r e c i a t e d .
Appendix. Comments on definitions The proposed equivalence of some terms as presented in this paper is an attempt at parsimony. But do these 'equivalent' terms actually represent the same ideas as suggested herein, or do the different words express truly different ideas? In many cases, it is clear that investigators have not been aware of their predecessors' and peers' terms and ideas. This makes probable the reinvention of an idea when the idea is good - - but described with different words. Certainly this statement will not be acceptable to all the reinventors, and they may be right. So the present definitions are arbitrarily but simply made to encourage sacrifice by any clear, reasonable argument to the contrary. Physics would not have progressed as it has if each worker used the term 'force' in their own, idiosyncratic way, or if that idea were given various names - - in some cases at least the probable situation in molar neuroscience. There is inadvertent mischief in the use of the same term for different events, or a multiplicity of terms for the same idea. Examples follow. As used herein, 'population coding' includes any event whose representation requires the activity in neurons of diverse contri-
Abbott, L.E and Blum, K.I. (1996) Functional significance of long-term potentiation for sequence learning and prediction. Cereb. Cortex, 6: 406-416. Abbott, L.F., Rolls, E.T. and Tovee, M.J. (1996) Representational capacity of face coding in monkeys. Cereb. Cortex, 6: 498505. Adrian, E.D. (1928) The Basis of Sensation. Christophers, London. Adrian, E.D. (1943) Discharges from the vestibular receptors in the cat. J. Physiol., 101: 389. Adrian, E.D. (1956) The action of the mammalian olfactory organ: the Semon Lecture: 1955. J. Laryngol. Otol., 70: 1-14. Adrian, E.D. and Bronk, D.W. (1928) The discharge of impulses in motor nerve fibres. I. Impulses in single fibres of the phrenic nerve. J. Physiol., 66: 81-101. Adrian, E.D., Cattell, M. and Hoagland, H. (1931) Sensory discharges in single cutaneous nerve fibres. J. Physiol., 72: 377-391. Baddeley, R., Abbott, L.F., Booth, M.C., Sengpiel, E, Freeman, T. and Rolls, E.T. (1997) Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Biol. Sci., 264: 1775-1783. Barnes, C.A., Suster, M.S., Shen, J. and McNaughton, B.L.
26
(1997) Multistability of cognitive maps in the hippocampus of old rats. Nature, 388: 272-275. Bernstein, L. (1970) The Infinite Variety of Music, Chapter 2. The New American Library, Inc., New York. Bishop, RO. (1970) Beginning of form vision and binocular depth discrimination. In: EO. Schmitt (Ed.), The Neurosciences: Second Study Program. R.U. Press, New York, pp. 471-485. Booth, M.C. and Rolls, E.T. (1998) View-invarient representations of familiar objects by neurons in the inferior temporal visual cortex. Cereb. Cortex, 8: 510-523. Boring, E.G. (1929) A History of Experimental Psychology. Appleton-Century, New York. Boring, E.G. (1942) Sensation and Perception in the History of Experimental Psychology. Appleton-Century-Crofts, New York. Capuano, U. and McIlwain, J.T. (1981) Reciprocity of receptive field images and point images in the superior colliculus of the cat. J. Comp. Neurol., 196: 13-23. Casey, K. (1999) Forebrain mechanisms of nociception and pain: analysis through imaging. Proc. Natl. Acad. Sci. USA, 96: 7668-7674. Churchland, RS. and Sejnowski, T.J. (1992) The Computational Brain. MIT Press, Cambridge, MA. Cohen, J.D., Perlson, W.M., Braver, T.S., Hystrom, L.E., Noll, D.C., Jonides, J. and Smith, E.E. (1997) Temporal dynamics of brain activation during a working memory task. Nature, 386: 604-608. Covey, E. (1980) Temporal neural coding in gustation. Doctoral dissertation, Duke University, Durham, NC. Covey, E. (2000) Neural population coding and auditory temporal patterns analysis. Physiol. Behav., 69:211-220. Craik, EI.M., Moroz, T.M., Moscovitch, M., Stuss, D.T., Winocur, G., Tulving, E. and Kapur, S. (1999) In search of self: A positron emission tomography study. Psychol. Sci., 10: 26-34. Damasio, A.R., Damasio, H. (1992) Brain and Language. Sci. Am., Sept., 89-95. Desimone, R. (1991) Face-selective cells in the temporal cortex of monkeys, or. Cognit. Neurosci., 3: 1-8. Di Lorenzo, EM. (1989) Across unit patterns in the neural response to taste: vector space analysis. J. Neurophysiol., 62: 823-833. Di Lorenzo, RM. and Hecht, G.S. (1993) Perceptual consequences of electrical stimulation in the gustatory system. Behav. Neurosci., 107: 130-138. Doetsch, G. (2000) S Patterns in the brain: neuronal population coding in the somatosensory system. Physiol. Behav., 69:187201. Edelman, G. and Mountcastle V.B. (1978) The Mindful Brain. MIT Press, Cambridge, MA. Eggermont, J.J. (1998) Azimuth coding in primary auditory cortex of the cat. II.. J. Neurophysiol., 80: 2151-2161. Ehert, G. (1997) The anditury cortex. J. Comp. Physiol. Set. A Sensory Neural Behav. Physiol., 181: 547-557. Eisenmann, L.M. (1974) Neural encoding of sound location: an
electrophysiological study in auditory cortex (AI) of the cat using free field stimuli. Brain Res., 75: 203-213. Erickson, R.R (1963) Sensory neural patterns and gustation. In: Y. Zotterman (Ed.), Olfaction and Taste, Vol. I. Pergamon Press, Oxford, pp. 205-213. Erickson, R.R (1968) Stimulus coding in topographic and nontopographic afferent modalities. Psychol. Rev., 75: 447-465. Erickson, R.R (1974) Parallel 'population' neural coding in feature extraction. In: F.O. Schmitt, EG. Worden (Eds.), The Neurosciences: Third Study Program. MIT Press, Cambridge, MA, pp. 155-169. Erickson, R.R (i978) Common properties of sensory systems. In: R.B. Masterton (Ed.), Handbook of Behavioral Neurobiology, Vol. 1. Plenum, New York, pp. 73-90. Erickson, R.R (1982) The across-fiber pattern theory: an organizing principle for molar neural function. In: W.D. Neff (Ed.), Contributions to Sensory Physiology, Vol. 6. Academic Press, New York, pp. 79-110. Erickson, R.R (1984) On the neural bases of behavior. Am. Sci., 72: 233-241. Erickson, R.E (1986) A neural metric. Neurosci. Biobehav. Rev., 10: 377-386. Erickson, R.R (2000) The evolution of neural coding ideas in the chemical senses. Physiol. Behav., 69: 3-13. Erickson, R.R, Doetsch, G.S. and Marshall, D. (1965) The gustatory neural response function. J. Gen. Physiol., 49: 247263. Erickson, R.R, Di Lorenzo, RM. and Woodbury, M.A. (1994) Classification of taste responses in brain stem: Membership in fuzzy sets. J. NeurophysioI., 71: 2139-2150. Farmer, S.E (1998) Rhythmicity, synchronization and binding in human and primate motor systems. J. Physiol., 509: 3-14. Fernandez, C., Goldberg, J.M. and Abend, W.K. (1972) Response to static tilts of peripheral neurons innervating otolith organs of the squirrel monkey. J. Neurophysiol., 35: 978-997. Fodor, J.A. (1983) The Modularity of Mind: An Essay on Faculty Psychology. MIT Press, Cambridge, MA. Fredrick, R.W. and Korsching, S.I. (1998) Chemotopic, combinatorial, and non-combinatorial odorant representations in the olfactory bulb revealed using a voltage-sensitive axon tracer. J. Neurosci., 23: 9977-9988. Friston, K.J. (1997) Another neural code? Neuroimage, 5: 213220. Gabrieli, J.D.E., Desmond, J.E., Demb, J.B., Wagner, A.D., Stone, M.V., Vaidya, C.J. and Glover, G.H. (1996) Functional magnetic resonance imaging of semantic memory processes in the frontal lobes. Psychol. Sci., 7: 278-283. Georgopoulos, A.R, (1995) Motor cortex and cognitive processing. In: M.S. Gazzaniga (Ed.), The Cognitive Neurosciences. MIT Press, Cambridge, MA, pp. 507-517. Gerstner, W., Kreiter, A.K., Markrarn, H. and Herz, A.V. (1997) Neural codes: firing rates and beyond. Proc. Natl. Acad. Sci. USA, 94: 12740-12741. Geschwind, N. (1979) Specializations of the human brain. Am. Sci., 9: 180-199. Gochin, RM., Colombo, M., Dorfman, G.A., Gerstein, G.L.
27
and Gross, C.G. (1994) Neural ensemble coding in inferior temporal cortex. J. Neurophysiol., 71: 2325-2337. Godefroy, O., Cabaret, M., Petit-Chenal, V., Pruvo, J.R and Rousseaux, M. (1999) Control functions of the frontal lobes. Modularity of the central-supervisory system? Cortex, 35: 120. Goldman-Rakic, ES. (1988a) Topography of cognition: parallel distributed networks in primate association cortex. Annu. Rev. Neurosci., 11: 137-156. Goldman-Rakic, RS. (1988b) Changing concepts of cortical connectivity: parallel distributed cortical networks. In: R Rakic and W. Singer (Eds.), Neurobiology of Neocortex. John WiIey and Sons, New York, pp. 177-202. Gould, S.J. (1994) The persistently fiat earth. Natural History, 3: 12-19. Graziano, M.S.A. and Gross, C.G. (1995) The representation of extrapersonal space: a possible role for bimodal, visualtactile neurons. In: M.S. Gazzaniga (Ed.), The Cognitive Neurosciences. MIT Press, Cambridge, MA, pp. 1021-1034. Hartline, H.K. (1940) The receptive field of the optic nerve fibers. Am. J. Physiol., 130: 690-699. Hebb, D.O. (1949) The Organization of Behavior: A Neuropsycholoical Theory. John Wiley and Sons, New York. Helmholtz, H. von. (1860) Handbuch der physiologischen Optik. Translated from the third German edition as J.RC. Southall (Ed.), Helmholtz's Treatise on Physiological Optics II. 1924, Dover, New York, NY. Helmholtz, H.L.E von (1862) On the Sensations of Tone. Second edition (1885) republished in 1954 by Dover, New York, NY (see pp. 144-149). Henry, G.H., Dreher, B. and Bishop, RO. (1974) Orientation specificity of cells in cat striate cortex. J. Neurophysiol., 37: 1394-1409. Hubel, D.H. and Wiesel, T.N. (1968) Functional architecture of the striate cortex. In: F.C. Carlson (Ed.), Physiologial and Biochemical Aspects of Nervous Integration. Prentice-Hall, Englewood Cliffs, NJ, pp. 153-161. Hubel, D.H. and Wiesel, T.N. (1974) Uniformity of monkey striate cortex: a parallel relationship between field size, scatter and magnification factor. J. Cornp. Neurol., 158: 295-306. Ingle, D. and Crews, D. (1985) Vertebrate neuroethology: definitions and paradigms. Annu. Rev. Neurosci., 8: 457-494. James, W. (1890) Principles of Psychology. Henry Holt, New York, p. 195. Johansson, H., Bergenheim, M., Djupsjobacka, M. and Sjolander, R (1995) A method for analysis of encoding of stimulus separation in ensembles of afferents. J. Neurosci. Methods, 63: 67-74. Johansson, R.S. and Vallbo, A.B. (1980) Spatial properties of the population of mechanoreceptive units in the glaborous skin of the human hand. Brain Res., 184: 353-366. John, E.R. (1972) Switchboard versus statistical theories of learning and memory. Science, 177: 850. Johnson, K.O. and Hsiao, S.S. (1992) Neural mechanisms of tactual form and texture perception. Annu. Rev. Neurosci., 15: 227-250.
Kimble, G. (1996) Psychology: The Hope of a Science. MIT Press, Cambridge, MA, p. 137. Knight, R.T. (1994) Attention regulation and human prefrontal cortex. In: A.M. Thierry, J. Glowinski, R Goldman-Rakic and Y. Christen (Eds.), Motor and Cognitive Functions of the Prefrontal Cortex. Research and Perspectives in Neurosciences. Springer, New York, pp. 160-173. Kundsen, E.I. (1982) Auditory and visual maps of space in the optic rectum of the owl. J. Neurosci., 9:1177-1194. Lancet, D. (1986) Vertebrate olfactory reception. Annu. Rev. Neurosci., 9: 329-355. Lancet, D. and Ben-Aire, N. (t993) Olfactory receptors. Cur~ Biol., 3: 668-674. Lashley, K.S. (1929) Neural mechanisms in adaptive behavior. In: Brain Mechanisms and Intelligence. Univ. Chicago Press, Chicago, pp. 157-163. Lashley, K.S. (1931) Mass action in cerebral function. Science, 73: 245-254. Lashley, K.S. (1942) The problem of cerebral organization in vision. In: H. Kluever (Ed.), Biological Symposia, Vol. VII, Visual Mechanisms. Jaques Cattell Press, Lancaster, pp. 301322. Lashley, K.S. (1951) The problem of serial order in behavior. In: L.A. Jeffress (Ed.), Cerebral Mechanisms in Behavior. John Wiley and Sons, New York, pp. 112-136. Law, I., Svarer, C., Rostrup, E. and Paulson, O.B. (1998) Parieto-occipital cortex activation during self-generated eye movements in the dark. Brain, 121: 2189-2200. Lee, C., Rohrer, L.C. and Sparks, D.L. (1988) Population coding of saccadic eye movements by neurons in the superior colliculus. Nature, 332: 357-360. Llinas, R. and Pare, D. (1996) The brain as a closed system modulated by the senses. In: R. Llinas and R Churchland (Eds.), The Mind-Brain Continuum. MIT Press, Cambridge, MA, pp. 1-18. Llinas, R. and Welsh, J. (1994) On the cerebellum and motor learning. Curt. Opin. Neurobiol., 3: 958-965. Loomis, J.M. and Collins, C.C. (1978) Sensitivity to shifts of a point stimulus: an instance of tactile hyperacuity. Percept. Psychophys., 24: 487-492. Lukashink, A.V. and Georgopoulas, A.R (1993) A dynamical neural network model for motor cortical activity during movement: population coding of movement trajectories. Biol. Cybernet., 69: 517-524. Malnic, B., Hirono, J., Sato, T. and Buck, L.B. (1999) Combinatorial receptor codes for odors. Cell, 96:713-723. Marsolek, CJ. (1999) Dissociable neural subsystems underlie abstract and specific object recognition. Psychol. Sci., 10: ill-118. McClurkin, J.S., Zarbock, J.A. and Optican, L.M. (1996) Primate striate and prestriate cortical neurons during discrimination II. Separable temporal codes for color and pattern. J. Neurophysiol., 75: 496-507. McIlwain, J.T. (1975) Visual receptive fields and their images in superior colliculus of the cat. J. Neurophysiol., 38: 219-230. Mcllwain, J.T. (1986) Point images in the visual system: new interest in an old idea. Trends Neurosci., 9: 354-358.
28
McIlwain, J.T. (1991) Distributed spatial coding in the superior colliculus: a review. Vis. Neurosci., 6: 3-13. Mecklinger, A. (1998) On the modularity of recognition memory for object form and spatial location: a topographic ERP: analysis. Neuropsychologia, 36: 441-460. Merzenich, M.M., Nelson, R.J., Stryker, M.R, Cynader, M.S., Schoppmann, A. and Zook, J.M. (1984) Somatosensory cortical map changes following digit amputation in adult monkeys. J. Comp. Neurol., 224: 591-605. Merzenich, M.M. and deCharms, R.C. (1996) Neural representations, experience, and change. In: R. Llinas and R Churchland (Eds.), The Mind-Brain Continuum. MIT Press, Cambridge, MA, pp. 61-81. Miller, W.H., Ratlliff, F. and Hartline, H.K. (1961) How cells receive stimuli. Sci. Am., Sept., W.H. Freeman, San Francisco, CA, pp. 1-12. Mishkin, M. (1983) Object vision and spatial vision. Trends Neurosci., 6: 414-417. Mountcastle, V.B. (1966) The neural replication of sensory events in the somatic afferent system. In: J.C. Eccles (Ed.), Brain and Conscious Experience. Springer, New York, pp. 85-115. Mountcastle, V.B. (1979) An organizing principle for cerebral function: the unit module and the distributed system. In: F.O. Schmitt and EG. Worden (Eds.), The Neurosciences: Fourth Study Program. MIT Press, Cambridge, MA, pp. 21-42. Mountcastle, V.B. (1980) Sensory receptors and neural encoding: introduction to sensory processes. In: V.B. Mountcastle (Ed.), Medical Physiology. 14th ed., C.V. Mosby, St. Louis, pp. 327346. Mountcastle, V.B., Poggio, G.E and Wemer, G. (1963) The relation of thalamic cell response to peripheral stimuli varied over a intensive continuum. J. Neurophysiol., 38: 908. Mueller, J. (1838) Handbuch der Physiologie des Menschen, Vol. H. Holscher, Coblentz. Nafe, J.R (1929) A quantitative theory of feeling. J. Gen. Psychol., 2: 199-210. Nagai, T., Katayama, H., Aihara, K. and Uamamoto, T. (1995) Pruning of rat cortical taste neurons by an artificial neural network model. J. Neurophysiol., 74: 1010-1019. Nelson, J.I. (1975) Globality and stereoscopic fusion in binocular vision. J. Theol: Biol., 49: 1-88. Nicolelis, M.A.L., Baccala, L.A., Lin, R.C.S. and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, 268: 1353-1358. Perkel, D.H. and Bullock, T.H. (1968) Neural Coding. Neurosciences Res. Prog. Bull., 6: 227-343. Peterson, C.C. and Siegal, M. (1999) Representing inner worlds: theory of mind in autistic, deaf, and normal hearing children. PsychoI. Sci., 10: 126-129. Pfaffman, C. (1941) Gustatory afferent impulses. J. Cell. Comp. Physiol., 17: 243-258. Pilpel, Y. and Lancet, D. (1999) The variable and conserved interfaces of modeled olfactory receptor proteins. Protein Sci., 8: 969-977.
Popper, K. (1962) Conjectures and Refutations. Basic Books, New York. Posner, M.I. (1992) Attention as a cognitive and neural system. Curt. Dir. Neurosci., 1: 11-14. Posner, M.I. and Petersen, S.E. (1990) The attention system of the human brain. Annu. Rev. Neurosci., 13: 25-42. Ray, R.H. and Doetsch, G.S. (1990) Coding of stimulus location and intensity in populations of mechanosensitive nerve fibers of the racoon, II. Across-fiber response patterns. Brain Res. Bull., 25: 533-550. Ressler, K.J., Sullivan, S.L. and Buck, L.B. (1994) Information coding in the olfactory system: evidence for a stereotyped and highly organized epitope map in the olfactory bulb. Cell, 79: 1245-1255. Rolls, E.T., Treves, A. and Tovee, MJ. (1997a) The representational capacity of the distributed encoding of information provided by populations of neurons in primate temporal visual cortex. Exp. Brain Res., 114: 149-I62. Rolls, E.T., Treves, A., Tovee, MJ. and Panzeri, S. (1997b) Information in the neuronal representation of individual stimuli in the primate temporal visual cortex. J. Comput. Neurosci., 4: 309-333. Sakurai, Y~ (1996a) Population coding by cell assemblies-what it really is in the brain. Neurosci. Res., 26: 1-16. Sakurai, Y. (1996b) Hippocampal and neocortical cell assemblies encode memory processes for different types of stimuli in the rat. J. Neurosci., 16: 2809-2819. Sakural, Y. (1998) The search for cell assemblies in the working brain. Behav. Brain Res., 91: 1-13. Schmidt, L.A. (1999) Frontal brain electrical activity in shyness and sociability. Psychol. Sci., 10: 316-320. Schneider, D. (1955) Mikro-electroden registrieren die elecrischen Impulse einzelner Sinnesnervenzellen der Schmetterlingsantenne. Ind. Electron. Forsch. Ferfigung, 3: 3-7. Softky, W.R. (1995) Simple codes versus efficient codes. Curt. Opin. Neurobiol., 5: 239-247. Soodak, R.E., Shapley, R.M. and Kaplan, E. (1987) Linear mechanisms of orientation tuning in the retina and lateral geniculate nucleus of the cat. J. Neurophysiol., 58: 267-275. Sperry, R.W. (1959) The growth of nerve circuits. Sci. Am., 201: 68-75. Sperry, R.W. (1969) A modified concept of consciousness. PsychoI. Rev., 76: 532-536. Spinelli, D.N. (1966) Receptive fields in the cat's retina. Science, 152: 1768-1769. Spinelli, D.N., Pribram, K.H. and Bridgemen, B. (1970) Visual receptive field organization of single units in the visual cortex of monkey. Int. J. Neurosci., 1: 67-74. Stevens, C.E and Zador, A. (1995) The enigma of the brain. Curt. Biol., 5: 1370-1371. Stone, V.E., Baron-Cohen, S. and Knight, R.T. (1998) Frontal lobe contributions to theory of mind. J. Cognit. Neurosci., 10: 640-656. Surmeier, D.J., Honda, C.N. and Willis, W.D. (1986) Responses of primate spinothalamic neurons to noxious thermal stimulation of glaborous and hairy skin. J. Neurophysiol., 56: 328350.
29
Tager-Flusberg, H. and Boshart, J. (1998) Reading the windows to the soul: evidence of domain-specific sparing in Williams syndrome. J. Cognit. Neurosci., 10: 631-639. Teevan, R.C. and Birney, R.C. (Eds.) (1961) Color Vision. Van Nostrand, Princeton, Chapters 1 and 2. Titchener, E.B. (1898) The postulates of a structural psychology. Philos. Rev. VII: 449-465. Tononi, G. and Edelman, G.M. (1998) Consciousness and complexity. Science, 282: 1846-1851. Tower, S.S. (1943) Pain: Definition and properties of the unit for sensory reception. In: Wolff, H.G., Gasser, H.S. and Hinsey, J.C. (Eds.), Pain. Research Publications: Association for Research in Nervous and Mental Disease, Vol. 23, pp. 16-43. Treisman, A.M. and Kanwisher, N.G. (1998) Perceiving visually presented objects: recognition, awareness, and modularity. Curt. Opin. Neurobiol., 8: 218-226. Vogels, R. (1999) Categorization of complex visual images by Rhesus monkeys. Part 2: single-cell study. Fur. J. Neurosci., 1l: 1239-1255.
Vogels, R. and Orban, G.A. (1991) Quantitative study of striate single unit responses in monkeys performing an orientation discrimination task. Exp. Brain Res., 84: 1-11. Waltz, J.A., Knowlton, B.J., Holyoad, K.J., Boone, K.B., Mioshkin, R.S., de Menezes Santos, M., Thomas, C,R. and Miller, B.L. (1999) A system for relational reasoning in human prefrontal cortex. Psychol. Sci., 10: 119-125. Young, M. and Yamane, S. (1992) Sparse population coding of faces in the inferotemporal cortex. Science, 256: 1327-1329. Von der Malsburg, C. (1995) Binding in models of perception and brain function. Curt. Opin. Neurobiol., 5: 520-526. Zeki, S. (1993) A Vision of the Brain. Blackwell, Oxford. Zeki, S. and Bartels, A. (1998) The autonomy of the visual systems and the modularity of conscious vision. Phil. Trans. Roy Soc. Ser. B: Biol. Sci., 353: 1911-1914. Zhang, K. and Sejnowski, T.J. (1998) Neuronal tuning; to sharpen or broaden? Neural Comput., 11: 75-84. Zotterman, Y. (1958) Studies in the nervous mechanism of taste. Exp. Cell Res., Suppl. 5: 520-526.
M.A.L. Nicolelis (Ed.)
Progressin BrainResearch.Vol. 130 © 2001 Elsevier Science B.V. All fights reserved
CHAPTER 3
Overcoming the limitations of correlation analysis for many simultaneously processed neural structures Luiz A. Baccal~i 1,, and Koichi Sameshima 2 1 Telecommunications and Control Engineering Department, Escola Polit~cnica, Av. Prof. Luciano Gualberto, Trav. 3, #158, University of Sdo Paulo, Sdo Paulo, SP, CEP 05508-900, Brazil 2 Disc. Medical lnformatics and Functional Neurosurgery Laboratory, School of Medicine, University of Sdo Paulo, Sdo Paulo, Brazil
Introduction Despite modem methods in molecular biology, neuroanatomy, and functional imaging, monitoring electric signals from neuronal depolarization remains important when evaluating the functional aspects of both normal and pathological neural circuitry. Correlation methods still rank popular and are extensively used to analyze the functional interaction in the electroencephalogram (EEG), the magnetoencephalogram, local field potentials and more recently, in simultaneously recorded single- and multiunit activity of many structures (tens to hundreds at a time). This last item has deserved increasing attention due to its potential in bridging the gap between the study of isolated single neurons and the understanding of encoding and processing of information by neuronal populations (Eichenbaum and Davis, 1998; Nicolelis, 1998). A host of other analytical techniques have emerged, some employing information theoretic rationales by assessing mutual information (Yamada et
* Corresponding author: Luiz A. Baccal~i, Telecommunications and Control Engineering Department, Escola Polit6cnica, Av. Prof. Luciano Gualberto, Trav. 3, #158, University of S~o Paulo, S~o Paulo, SP, CEP 05508-900, Brazil. E-mail:
[email protected] al., 1993; Rieke et al., 1997; Brunel and Nadal, 1998) or interdependence between signal pairs (Schiff et al., 1996; Amhold et al., 1999), while others are extensions of spectral analysis/coherence analysis (Glaser and Ruchkin, 1976; Duckrow and Spencer, 1992; Christakos, 1997; Rosenberg et al., 1998). Despite these advances, a large fraction of neuroscientists still chiefly rely on the cross-correlation between the activity of pairs of neural structures to infer their functionality. Like cross-correlation, all of these methods are in one way or another restricted in their calculations to using just the signal of two structures at a time. In this article, we show that it is not only possible but also desirable to analyze more than two structures simultaneously. Furthermore, we show also that effective structural inference is only possible if simultaneous signals from many (representative) structures are jointly analyzed. To handle many simultaneous structures, we employ the recently introduced notion of partial directed coherence (PDC). This is a novel frequency domain approach for simultaneous multichannel data analysis based on Granger causality that employs multivariate auto-regressive (MAR) models for computational purposes (Baccal~i and Sameshima, 2001). We review PDC in Section 2 and illustrate its usefulness via toy linear models simulating multi-electrode EEG measurements in Section 3, where we contrast
34 it to other techniques (correlation/coherence analysis). We discuss an application to experimental data in Section 4. Further examples of PDC in a single- and multi-unit activity context are available in Sameshima and Baccald (2001). Partial directed coherence The concept of partial directed coherence is the latest development of a number of time series analysis efforts for describing how neural structures are interconnected (Baccal~i and Sameshima, 2001; Sameshima and Baccald, 2001). Its remote origin is the paper by Saito and Harashima (1981) which introduced the notion of directed coherence between the activity of pairs of structures. Their method allows factoring the classical coherence function (the frequency domain counterpart of correlation analysis) of a pair of structures into two 'directed coherences': one representing the feedforward and the other one representing the feedback aspects of the interaction between these two neural structures. Examples of use of pairwise directed coherence in studying the relation between Parkinson's tremor and lack of feedback in motor control are contained in Schnider et al. (1989). In an attempt to generalize directed coherence to a context of analysis of more than two simultaneously processed structures, the so-called method of directed transfer function (DTF) was introduced with several equivalent variants (Franaszczuk et al., 1994; Baccal~i and Sameshima, 1998; Baccal~i et al., 1998). This method was applied to foci determination and to EEG studies in mesial temporal lobe seizure (Franaszczuk et al., 1994). Details on DTF are contained in Appendix A, In their original paper, Saito and Harashima (1981) allude to a possible rationale for their method. This concept is now known as Granger causality (Granger, 1969). According to it, an observed time series x (n) Granger-causes another series y (n), if knowledge of x(n)'s past significantly improves prediction of y(n); this kind of predictability improvement is not reciprocal, i.e. x(n) may Grangercause y(n) without y(n) necessarily Granger-causing x(n). This lack of reciprocity is the basic property behind the determination of the direction of information flow between pairs of structures which,
in turn, is the basis for decomposing classical coherence into directed feedforward and feedback coherence factors. Following that rationale, we investigated how generalizations, like DTF, of directed coherence to N simultaneously processed structures compared to statistical tests of Granger causality for N simultaneous time series (Baccal~i et al., 1998). We realized that DTF provided a physiologically interesting frequency domain picture, yet structural inference based on its computation did not always agree with the result of Granger causality tests (GCT). We could show that this was due to intrinsic aspects of DTF's definition (Baccald and Sameshima, 2001) (see also Appendix A). Because Granger causality is a more fundamental concept than the ad hoc generalization represented by DTF, we went on to introduce the notion of partial directed coherence (Baccal~i and Sameshima, 2001). This new structural connectivity estimator relies on the simultaneous processing of N > 2 time series and is able to expose a frequency domain picture of the feedforward and feedback interactions between each and every pair of structures within the set of N simultaneously processed signals. Perhaps more importantly, PDC reflects Granger causality closely by paralleling the definition of Granger causality test estimators. The main preliminary ingredient of both PDC and GCT (and of DTF as well, but in a fundamentally different way) is their practical use of multivariate autoregressive models as exemplified for N = 3 simultaneously monitored structures in the model
Lxlnl Iallr al2ra3rl x2(n)
=
x3(n)
r=l
×
a21(r)
a22(r)
a23(r)
a31(r)
a32(r)
a33(r)
x2(n -- r) x3(n -- r)
+
w2(n) w3(n)
(1)
In this model, xl(n) depends on its own past values xl ( n - r ) through the coefficients a n (r) while, for example, xl (n)'s dependence on the past values of the other series like xz(n - r) is through the a12(r) coefficients. As such, the time series xz(n) only
35 Granger-causes xl(n) if we can statistically show that al2(r) ~6 0 for some values of r. Or equivalently, rejecting the null hypothesis of aij(r) = 0 means that xj (n) does Granger-cause xi (n). The partial directed coherence from series j to series i, at frequency f can be defined as 7rij(f ) =
~;j(f) x/~j (f)"~j (f)
(2)
Where
+ wl (n)
r=l
p - ~ aij (r)e -jz~rfr, otherwise
Model I xl (n) = 0 . 9 5 ~ x l (n - 1) - 0.9025xl (n - 2)
P
1 - ~ aij (r)e -j2Jrfr, if i = j aij ( f ) =
taneously processing fewer than the N structures important to the dynamics. The first toy model example, mimicking local field potential measurements along hippocampal structures, is represented by the following set of linear difference equations with N = 7 structures:
(3)
r=l
x2(n) = - 0 . 5 x l ( n - 1) + to2(n ) x3(n) = 0.4xl(n -- 4) -- 0.4x2(n -- 2) + w3(n)
and fij ( f ) is the vector
x4(n) = --0.5x3(n -- 1) + 0.25v/2xn(n -- 1)
au(f)
+ 0.25~¢/-2xs(n -- 1) + w4(n) (4)
hj(f) =
aNj(f) Because of its dependence on aij(r) in Eq. 3, the nullity of 7rij(f ) at a given frequency implies lack of Granger causality from xj(n) to xi(n) at that frequency. PDC is, therefore, a direct frequency domain counterpart of GCT. Though further details on PDC are contained elsewhere (Baccald and Sameshima, 2001), a compact summary is available at Appendix A together with its relation to DTE Methods of MAR model fitting are reviewed elsewhere (Marple, 1987). In the next section, we discuss some examples of PDC's application contrasting it to other techniques.
Illustrative simulations To provide objective comparisons of PDC with other techniques, we use time series generated from known linear toy models. In this case, exact theoretical calculations of pairwise cross-correlation, classical coherence, DTF and PDC can be made and allow exposing all the relative methodological merits of each approach while avoiding possible pitfalls of experimental signals collected from neural systems with unknown structure. The use of toy models is further motivated by the desire to investigate possible structural inference impairments when simul-
xs(n) = --0.25~u/2x4(n -- I) + 0 . 2 5 ~ x s ( n -- 1) + ws(n) x6(n) = 0.95V/2x6(n -- 1) -- 0.9025x6(n -- 2) + w6(n) xT(n) = --0.1x6(n -- 2) + w7(n) with wi (n) standing for innovation noises. These equations are designed so that xl(n) behaves as an oscillator driving the other structures, either directly or indirectly, according to the diagram in Fig. 1. Note that the interaction between xl (n) and x3(n) is both via a direct path and via an indirect route through x2(n). The dynamics of the pair xa(n) and x5 (n) is designed so that they jointly represent an oscillator, whose intrinsic characteristics are due to their mutual signal feedback but which are entrained to the rest of the structure via x3(n). The signals x6(n) and x7(n) belong to a totally separate substruc-
Fig. 1. Signal flow diagram for Model I.
36
1 2 ~,:,=;.~
4 5 6
I
(~
'
,
,
,
IOLO0
'
I
I
2000
Time (samples) Fig. 2. Signalsobtainedby simulatingModelI. ture where x6(n) is designed to generate oscillations at the same frequency as xl (n); x7(n) does not feedback anywhere. A sample of the signals produced in this way can be appreciated in Fig. 2. We begin our analysis by inspecting the theoretical pairwise cross-correlation contained in the array of plots in Fig. 3a. Consider just the latencies and lead structures represented by the theoretical correlation maxima of Fig. 3a as summed up in Fig. 3b's diagram whose arrows are labelled with the absolute values of the latencies and originate in the leading structure. In deducing the structural relationships between the signals using this information, we may attempt to trim the diagram in Fig. 3b. This leads to several possible hypothetical structures compatible with the observed latencies such as in Fig. 3c,d. Note that structural ambiguities not only remain and but also that no conceivable trimming of Fig. 3b can possibly produce the correct solution in Fig. 1 because Fig. 3b's relation between x3(n ) and x2(n) turns out inverted with respect to that in Fig. 1. In short, this example shows that correlation information alone leads to ambiguous structural inference when considering several time series measurements simultaneously.
The results of the pairwise interaction using the theoretical DTF is depicted by dark shaded curves along the off-diagonals over a 7 x 7 array layout of plots of Fig. 4a. Along the shaded main diagonal of the array in Fig. 4a, we portray the power spectrum for each time series. Note the spectral similarity that characterizes all signals for this structure. To facilitate comparisons, solid-line graphs along the off-diagonals of the array depict the high pairwise classical coherences among those structures that are interconnected. Fig. 4c's schematic represents a summary of the relations described by DTF in which signal sources are labelled along the x-axis and targets along the y-axis. Thus, for instance, in Fig. 4c, an arrow leaves xl(n) and reaches xs(n) because the first column of Fig. 4a has a significant shaded area in row five. No direct reverse arrow exists as there is no dark shaded area in column 5, row 1 of Fig. 4a. In this and later graphs, thinner/dashed arrows portray weaker connections. This leads to a complex connectivity pattern in the graph describing DTF relationships (Fig. 4c). As in the case of cross-correlation, the only possible inference is, for example, that the signal in xl(n) affects all other nodes without clues as to how or through which pathway this interaction takes place. Using the same
37
(b)
(a)
2
1 2 3 4 5 6 :
',
',
2
3
7 -50
0
1
50
4
(c)
5
6
7
(d)
Fig. 3. The theoretical autocorrelations are shown along the main diagonal and, below it, all the theoretical cross-correlation functions are plotted (a), with the x-axis scale ranging from - 5 0 to 50 sample points, and the y-axis is between - 1 and 1. Directed graph summarizing signal propagation latency information (encoded via arrow labels) contained in the cross-correlation function (b). Two possible simplified structures compatible with the theoretically calculated latencies are shown in (c) and (d). Note that, the graph simplification from (c) to (d), the connection 1 ~ 4 (with propagation latency 5 time units) is removed because it can be explained by the pathway 1 ~ 3 ~ 4 with the same total propagation latency value.
rule for associating source to target in labelling pairwise interaction using PDC, a completely distinct situation emerges in Fig. 4b where PDC calculations (dark shaded curves along the off-diagonal) lead to the correct structure of Fig. 4d (compare with Fig. 1). We next slightly increase the complexity of this example by adding a feedback from xs(n) to Xl (n). This is accomplished by rewriting the first equation in Model I as Xl (n) = 0.95~/2Xl (n -- 1) - 0.9025xl (n -- 2) + 0.5xs(n - 2) + wl(n)
(5)
in accord with the diagram of Fig. 5. As in the previous case, the theoretical DTF is difficult to analyze (Fig. 6a,c), as opposed to PDC (Fig. 6b,d) which clearly reflects the newly added feedback. This pattern of straightforward analysis using PDC carries over to a practical simulation scenario of using 500 data points where the feedback-free situation (Fig. 7a) is easily distinguishable
from that when feedback from xs(n) to xt(n) is present (Fig. 7b). To provide some sense of the potential temporal resolution of the method, we display the time evolution of PDCs involving xl(n) and x5(n) (Fig. 8) while randomly switching the feedback on and off. Each PDC estimate comprises the use of 250 simulated data points with 50% overlap between adjacent data segments. In comparing these examples, note that DTF's graph has arrows connecting almost all structures when the feedback is switched on (Fig. 6c); this is related to the fact that the PDC graph contains pathways (direct or indirect) that connect any two structures. Some arrows in Fig. 6c are missing (x2(n) ~ x l ( n ) , x3(n) ~ x l ( n ) , x3(n) ~ xs(n)). Their presence would have made Fig. 6c's graph fully connected. Though corresponding to existing signal pathways, the missing arrows correspond to small (but theoretically nonzero) DTFs that reflect
38
(a)
(b) DTF
PDC
J I
•
A
i''
^
:
L-- I i
II
I I
(c)
2
3
4
5
6
7
1
2
3
4
5
6
7
(d)
Fig. 4. Comparison between the theoretical DTF (a) (and its inferred structural interactions (c)) and the theoretical PDC results (b) (with its signal flow diagram (d)). In both cases the signal flow graphs are constructed by assigning an arrow from the source structure (x-axis) to the targets (y-axis) when dark shaded areas are significant. The spectral densities for the time series are depicted along the shaded main diagonal of the arrays. The pairwise classical coherences (solid lines) are also depicted. In all plots, the x-axis represents the normalized frequency in the 0 to 0.5 range, while the y-axis for power spectrum plots is scaled between 0 and peak value and values for the other coherence plots lie between 0 and 1. the weakness of the connection strength of the total pathways between structures that are far apart from one another. In fact, D T F can be interpreted as a marker for signal energy 'reachability' (see Remark 3 in A p p e n d i x A) and must be analyzed with care. For example, examine the dip in the D T F from x2(n) to x3(n) in Figs. 4a and 6a. It coincides with the m a x i m u m o f the power spectrum o f both these series. Rather than mean lack of pathway connection at that frequency, this dip exemplifies (by model design) how the energy reaching a structure (x3(n)) from another structure (xl(n)) at one frequency m a y be almost exactly cancelled by the energy coming through another pathway (xz(n)) due to a phase inversion in the signal. To emulate scalp EEG, our second example employed
Model II xl (n) = 1.8982Xl (n - 1) - 0.9025xl (n - 2) + wl (n)
xz(n)
= 0 . 9 x l ( n - 2) + to2(n)
x3(n) = 0.85xz(n -- 2) + w3(n)
X4(n ) = 0.82xl(n -- 2) + 0.6x6(n -- 3) + toa(n) xs(n) = --0.9x6(n -- 2) + 0.4xz(n -- 4) + ws(n) x6(n) = 0.9xs(n -- 2) + to6(n)
Fig. 5. Schematic diagram describing the inclusion of feedback from x5 (n) to Xl (n) into Model I.
39
(a)
(b) PDC
DTF
I L_
I;
L_ L_
II
&
1
2
3
4
5
6
7
(c)
1
2
3
4
5
6
7
(d)
Fig. 6. Comparison between the theoretical DTF (a) (and its inferred structural interactions (c)) and the theoretical PDC results (b) (with its signal flow diagram (d)) after turning on the feedback from xs(n) to xl (n). Spectral densities for the time series are depicted along the shaded main diagonal of the arrays. The pairwise classical coherences (solid lines) are also depicted. Thin/dashed arrows portray weaker connections.
(a)
(b)
PDC without feedback
P D C with f e e d b a c k
mL~"
i
i
e
7 1
2
3
4
5
6
7
1
2
3
4
5
6
7
Fig. 7. Estimated PDC for Model I without (a) and with (b) feedback from xs(n) to xl (n) using 500 simulated points. Note how x6(n) and x7(n) show residual classical coherence with the other time series, despite their lack of direct connection.
where xl (n) is a n oscillator driving, directly or indirectly, all the other structures. In this e m u l a t i o n , the odd n u m b e r e d signals represent the left h e m i s p h e r e
l e a v i n g the other ones to m a p the other h e m i s p h e r e as in Fig. 9. A n e x a m p l e o f the s i m u l a t e d signals is portrayed in Fig. 10.
40
(a)
Feedback Connection
1 10 Hz).
Integration Tactile acquisition by the whiskers is a motorsensory active process. Whiskers are moved by the motor system to acquire information, which is then analyzed by the sensory system. This system operates in a closed loop, since sensory information drives motor circuits at multiple levels (Kleinfeld et al., 1999). As with any other closed loop, there is no starting point, the process is not more sensory-motor than it is motor-sensory. The sensory part of the loop contains two parallel pathways: the lernniscal and paralemniscal systems. Our working hypothesis is that the paralemrtiscal system decodes the low-frequency (whisking-range) temporally encoded information, and that the lemniscal system decodes spatially encoded information and high-frequency (texture-dependent) temporally encoded information. Since temporal decoding by PLLs results in a rate-population code, the integration of all the decoded outputs should be straightforward. Indeed, signs of such integration were observed in layer 2/3 of the barrel cortex (Ahissar et al., 2001). Yet, some anatomical segregation is preserved between these two systems, even at the cortical level (Kim and Ebner, 1999), which suggests
85 that even the sensory-motor control systems operate in parallel, at least up to a later motor stage. From a functional point of view, the vibrissal system includes three parallel sensory-motor processes. These processes can be distinguished by their decoding of sensory cues; one decodes temporally encoded low-frequency horizontal localization cues, another temporally encoded high-frequency textural cues, and the third, spatially encoded vertical cues. The sensory part of the first process is accomplished by the paralemniscal system, whereas those of the other two are implemented by the lemniscal system. Theoretically, each process could be implemented by a separate sensory-motor loop, and could even have a separate muscular system on the mystacial pad. Alternatively, the three processes could share at least some of the motor circuits and muscles. Although the latter scheme seems more plausible, the stage at which the different processes merge is not yet clear. As with the local sensory loops, the operation of each sensory-motor loop is characterized by its loop gain and sensitivity. The gains of the different sensory-motor loops are probably under global control so that the brain can regulate the strength of the sensory-motor coupling of each loop. This regulation can bring a specific sensory-motor process to the 'foreground', while keeping the other two processes in the 'background', during the performance of a specific task.
Abbreviations PLL iPLL ePLL FM RCO PD VPM POm TRN
phase-locked loop inhibitory PLL excitatory PLL frequency modulation rate-controlled oscillator phase detector ventral posterior medial nucleus of the thalamus medial division of the posterior nucleus of the thalamus thalamic reticular nucleus
Non-keyboard characters sensitivity gain
cr (Greek sigma) 13(Greek beta)
phase straddlers
qb(Greek phi) ct, 13, y, ~ (Greek alpha, beta, gamma, delta)
Acknowledgements We wish to thank S. Haidarliu and R. Sosnik for their help with the experiments and data analysis and B. Schick for reviewing the manuscript. This work was supported by the United States-Israel Binational Science Foundation (Israel) Grant 97-222 and the MINERVA Foundation, Germany.
References Ahissar, E. (1995) Conversion from temporal-coding to rate-coding by neuronal phase-locked loops. The Weizmann Institute of Science, Rehovot, Technical Report GC-EA/95-4. Ahissar, E. (1998) Temporal-code to rate-code conversion by neuronal phase-locked loops. Neural Comput., 10(3): 597650. Ahissar, E. and Arieli, A. (1997) Seeing through miniature eye movements: A hypothesis. Neurosci. Lett. Suppl., 48: $2. Ahissar, E. and Vaadia, E. (1990) Oscillatory activity of single units in a somatosensory cortex of an awake monkey and their possible role in texture analysis. Proc. Natl. Acad. Sci. USA, 87: 8935-8939. Ahissar, E., Haidarliu, S. and Zacksenhouse, M. (1997) Decoding temporally encoded sensory input by cortical oscillations and thalamic phase comparators. Proc. Natl. Acad. Sci. USA, 94: 11633-11638. Ahissar, E., Sosnik, R. and Haldarliu, S. (2000) Transformation from temporal to rate coding in a somatosensory thalamocortical pathway. Nature, 406: 302-306. Ahissar, E., Sosnik, R. and Haidarliu, S. (2000) Temporal frequency of whisker movement: II. Laminar organization of cortical representations. J. Neurophysiol., in press. Bengi, H. and Thomas, J.G. (1972) Studies on human ocular tremor. In: R.M. Kenedi (Ed.), Prospectives in Biomedical Engineering. Macmillen, London, pp. 281-292. Bishop, G.H. (1959) The relation between nerve fiber size and sensory modality: Phylogenetic implications of the afferent innervation of cortex. J. Nerv. Ment. Dis., 128:89-114. Bourassa, J., Pinault, D. and Deschenes, M. (1995) Corticothalamic projections from the cortical barrel field to the somatosensory thalamus in rats: a single-fibre study using biocytin as an anterograde tracer. Eur. J. Neurosci., 7: 19-30. Brecht, M., Preilowski, B. and Merzenich, M.M. (1997) Functional architecture of the mystacial vibrissae. Behav. Brain Res., 84: 81-97. Brown, A.W. and Waite, P.M. (1974) Responses in the rat thalamus to whisker movements produced by motor nerve stimulation, J. Physiol., 238: 387-401. Buanomuno, D. and Merzenich, M.M. (1995) Temporal infor-
86
mation transformation into a spatial code by a neural network with realistic properties. Science, 267: 1028-1030. Carr, C.E. (1993) Processing of temporal information in the brain. Annu. Rev. Neurosci., 16: 223-243. Carvell, G.E. and Simons, D.J. (1990) Biometric analyses of vibrissal tactile discrimination in the rat. J. Neurosci., 10: 2638-2648. Chmielowska, J., Carvell, G.E. and Simons, D.J. (1989) Spatial organization of thalamocortical and corticothalamic projection systems in the rat SmI barrel cortex. J. Cornp. Neurol., 285: 325-338. Diamond, M.E. (1995) Somatosensory thalamus of the rat. In: E.G. Jones and I.T. Diamond (Eds.), Cerebral Cortex, Vol. 11. Plenum Press, New York, pp. 189-219. Diamond, M.E. and Armstrong-James, M. (1992) Role of parallel sensory pathways and cortical columns in learning. Concepts Neurosci., 3: 55-78. Diamond, M.E., Armstrong-James, M., Budway, M.J. and Ebner, EE (1992) Somatic sensory responses in the rostral sector of the posterior group (Pore) and in the ventral posterior medial nucleus (Vpm) of the rat thalamus: dependence on the barrel field cortex. J. Cornp. Neurol., 319: 66-84. Eckhorn, R., Frien, A., Bauer, R., Woelbern, T. and Kehr, H. (1993) High frequency (60-90 Hz) oscillations in primary visual cortex of awake monkey. NeuroReport, 4: 243-246. Eizenman, M., Hallett, EE. and Frecker, R.C. (1985) Power spectra for ocular drift and tremor. V~s.Res., 25: 1635-1640. Fanselow, E.E. and Nicolelis, M.A.L. (1999) Behavioral modulation of tactile responses in the rat somatosensory system. J. Neurosci., 19: 7603-7616. Friedberg, M.H., Lee, S.M. and Ebuer, EE (1999) Modulation of receptive field properties of thalamic somatosensory neurons by the depth of anesthesia. J. Neurophysiol., 81: 2243-2252. Gardner, EM. (1979) Phaselock Techniques. Wiley, New York. Ghazanfar, A.A. and Nicolelis, M.A. (1997) Nonlinear processing of tactile information in the thalamocortical loop. J. Neurophysiol., 78: 506-510. Gray, C.M. and McCormick, D.A. (1996) Chattering cells: superficial pyramidal neurons contributing to the generation of synchronous oscillations in the visual cortex. Science, 274: 109-113. Haidarliu, S. and Ahissar, E. (1997) Spatial organization of facial vibrissae and cortical barrels in the guinea pig and golden hamster. Z Cornp. Neurol., 385: 515-527. Haidarliu, S. and Ahissar, E. (2001) Size gradients of barreloids in the rat thalamus. J. Comp. Neurol., 429: 372-387. Von Holst, E. (1973) Relative coordination as a phenomenon and as a method of analysis of central nervous functions. In: R.D. Martin (Ed.), The Behavioral Physiology of Analysis
of Central Nervous Functions of Animals and Man. Selected Papers of Eric yon Holst. University of Miami Press, Coral Gables. Hoogland, EV., Welker, E., Van der Loos, H. and Wouterlood, EG. (1988) The organization and structure of the thalamic afferents from the barrel cortex in the mouse; a PHA-L study. In: M. Bentivoglio and R. Spreafico (Eds.), Cellular Thalarnic Mechanisms. Elsevier, Amsterdam, pp. 152-162.
Jeffress. L.A. (1948) A place theory of sound localization. J. Comp. Physiol. Psychol., 41: 35-39. Johansson, R.S., Landstrom, U. and Lundstrom, R. (1982) Responses of mechanoreceptive afferent units in the glabrous skin of the human hand to sinusoidal skin displacements. Brain Res., 244: 17-25. Jones, M.S. and Barth, D.S. (1999) Spatiotemporal organization of fast (>200 Hz) electrical oscillations in rat vibrissa/barrel cortex. J. Neurophysiol., in press. Kim, U. and Ebner, EE (1999) Barrels and septa: separate circuits in rat barrels field cortex. J. Comp. Neurol., 408: 489505. Kleinfeld, D., Berg, R.W. and O'Connor, S.M. (1999) Anatomical loops and their electrical dynamics in relation to whisking by rat [In Process Citation]. Somatosens. Mot. Res., 16: 69-88. Koralek, K.A., Jensen, K.E and Killackey, H.P. (1988) Evidence for two complementary patterns of thalamic input to the rat somatosensory cortex. Brain Res., 463: 346-351. Lebedev, M.A. and Nelson, R.J. (1995) Rhythmically firing (2050 Hz) neurons in monkey primary somatosensory cortex: Activity patterns during initiation of vibratory-cued hand movements. J. Cornput. Neurosci., 2: 313-334. Lee, S.M., Friedberg, M.H. and Ebner, EF. (1994) The role of GABA-mediated inhibition in the rat ventral posterior medial thalamus. I. Assessment of receptive field changes following thalamic reticular nucleus lesions. J. NeurophysioL, 71: 17021715. Lu, S.M. and Lin, R.C. (1993) Thalamic afferents of the rat barrel cortex: a light- and electron-microscopic study using Phaseolus vulgaris leucoagglutinin as an anterograde tracer. Sornatosens. Mot. Res., 10: 1-16. Nicolelis, M.A.L., Baccala, L.A., Lin, R.C.S. and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, 268: 1353-1358. Perkel, D.H. and Bullock, T.H. (1968) Neural coding. Neurosci. Res. Progr. Bull., 6: 221-248. Perkel, D.H., Schulman, J.H., Bullock, T.H., Moore, G.P. and Segundo, J.P. (1964) Pacemaker neurons: effects of regularly spaced synaptic input. Science, 145: 61-63. Salt, T.E. (1989) Gamma-aminobutyric acid and afferent inhibition in the cat and rat ventrobasal thalamus. Neuroscience, 28: 17-26. Sherman, S.M. and Guillery, R.W. (1996) Functional organization of thalamocortical relays. J. Neurophysiol., 76: 13671395. Shipley, M.T. (1974) Response characteristics of single units in the rat's trigeminal nuclei to vibrissa displacements. J. Neurophysiol., 37: 73-90. Simons, D.J., Carvell, G.E., Hershey, A.E. and Bryant, D.P. (1992) Responses of barrel cortex neurons in awake rats and effects of urethane anesthesia. Exp. Brain Res., 91: 259-272. Sosnik, R., Haidarliu, S. and Ahissar, E. (2001) Temporal frequency of whisker movement: I. Representations in brainstem and thalamus. J. Neurophysiol., in press. Talbot, W.H., Darian-Smith, I., Kornhuber, H.H. and Mountcastie, V.B. (1968) The sense of flutter-vibration: comparison of
87
the human capacity with response patterns of mechanoreceptive afferents from the monkey hand. J. Neurophysiol., 31: 301-334. Welker, E., Hoogland, P.V. and Van der Loos, H. (1988) Organization of feedback and feedforward projections of the barrel cortex: a PHA-L study in the mouse. Exp. Brain Res., 73: 411--435. Welker, W.I. (1964) Analysis of sniffing of the albino rat. Behaviour, 22: 223-244. Williams, M.N., Zahm, D.S, and Jacquin, M.E (1994) Differential foci and synaptic organization of the principal and spinal trigeminal projections to the thalamus in the rat. Eur. J. Neurosci., 6: 429-453. Wilson, M.A. and Bower, J.M. (1989) The simulation of largescale neural networks. In: C. Koch and I. Segev (Eds.), Meth-
ods in Neuronal Modeling: from Synapses to Networks. MIT Press, Cambridge, MA, pp. 291-333. Wineski, L.E. (1983) Movements of the cranial vibrissae in the golden hamster (Mesocritus auratus). J. Zool. Lond., 200: 261-280. Woolsey, T.A. (1997) Barrels, vibrissae and topographic representations. In: and G. Adelman and B. Smith (Eds.), Encyclopedia of Neuroscience Vol. I. Elsevier, Amsterdam, pp. 195199. Zacksenhouse, M. (2001) Sensitivity of basic oscillatory mechanisms for pattern generation and decoding. Biol. Cybern., in press. Zucker, E. and Welker, W.I. (1969) Coding of somatic sensory input by vibrissae neurons in the rat's trigeminal ganglion. Brain Res., 12: 138-156.
M.A.L. Nicolelis (Ed.)
Progressin BrainResearch,Vol. 130 © 2001 Elsevier Science B.V. All fights reserved
CHAPTER 7
Thalamocortical and corticocortical interactions in the somatosensory system Miguel A.L. Nicolelis 1,2,3,, and Marshall Shuler 1 1 Department of Neurobiology, Duke University, Durham, NC 27710, USA 2 Department of Biomedical Engineering, Duke University, Durham, NC 27710, USA 3 Department of Psychology: Experimental, Duke University, Durham, NC 27710, USA
Introduction Until recently, neurophysiological theories aimed at accounting for the exquisite tactile perceptual capabilities of mammals have been dominated by the notion that the somatosensory system relies primarily on feedforward computations to generate a broad spectrum of sensations (e.g. fine touch, thermo sensation, pain, etc.) (Mountcastle, 1957, 1974; Dykes, 1983; Johnson et al., 1995). Despite the widespread acceptance of this view, over the last three decades, considerable anatomical, physiological, and behavioral evidence has been put forward to challenge a pure feedforward view of touch. Accordingly, more than ever, the potential contribution of some of the main 'building blocks' of this model of touch, concepts such as the classic receptive field, independent parallel pathways from the periphery to the cortex, cortical columns, and static somatotopic maps, have been the subject of considerable debate (Merzenich et al., 1983; Purves et al., 1992; Ghazanfar and Nicolelis, 1999). The anatomical organization of the somatosensory system provides the first hint that tactile in-
*Corresponding author: Miguel A.L Nicolelis, Department of Neurobiology, Box 3209, Bryan Research Building, Room 333, 101 Research Drive, Durham, NC 27710, USA. Tel. +1-919-684-4580; Fax: +1-919-684-5435; E-mail:
[email protected] formation processing involves more than just feedforward interactions (Fig. 1). On their way to the neocortex, ascending somatosensory pathways converge on neurons located in a series of subcortical nuclei in the spinal cord, brainstem, and thalamus (Kaas and Pons, 1988). In addition to these parallel feedforward pathways that convey information from the peripbery to the cortex, neurons located at cortical and subeortical regions that define the different processing levels of the somatosensory system receive convergent input from multiple descending feedback pathways, originating in several cortical areas (Kaas, 1990). These feedback projections define extensive thalamocortical and corticocortical loops that are largely ignored by the classical feedforward model of touch. Part of the reason for this omission is the inherent experimental difficulty of measuring the potential effects of feedback projections, which has contributed to a scarcity of information and conflicting data regarding the physiological role of recurrent circuits. Research on artificial neural networks suggests that the main computational advantage of a neural system with highly recurrent projections is that such networks do not have to synthesize a new view of the world every time new raw information is sampled (as suggested by a pure FF model). Instead, a recurrent system can use previously learned experiences to generate an 'internal' model of the world (Mumford, 1994), and take advantage of this model to generate expectations and predictions every time an
90
Fig. 1. Schematic diagram of the rat trigeminal somatosensory system. Whiskers on the rat's snout are labeled according to the row and column in which they are located. Whisker columns are labeled from 1 to 5, caudal to rostral, while whisker rows are labeled A to E, dorsal to ventral. Peripheral nerve fibers innervating single whisker follicles have their cell bodies located in the trigeminal ganglion (Vg). Here, only the projections from Vg neurons to two main subdivisions of the trigeminal brainstem complex, the principal trigeminal nucleus (PrV), and the spinal trigeminal nucleus (SpV), are illustrated. Proponents of the feedforward model of touch usually divide these projections into rapidly adapting (RA) and slowly adapting (SA) fibers, according to their physiological responses to tactile stimuli (see text). Each of these categories contains further subdivisions, which are not described here. Neurons located in these two brainstem nuclei give rise to parallel excitatory projections to the ventroposterior medial nucleus of the thalamus (VPM). Neurons in VPM give rise to projections to layer IV of the primary somatosensory cortex (SI). A collateral of these thalamocortical projections reach the reticular nucleus (RT), whose neurons provide the main source of GABAergic inhibition to the VPM. Descending excitatory corticothalamic projections, originating in layer VI of the SI cortex, reach the VPM and the reticular nucleus of the thalamus (RTn). The assumed topographic arrangement of these projections in the VPM and the RT are illustrated in the scheme, Feedback corticofugal projections originated in layer V of the SI cortex also reach the trigeminal brainstem complex, targeting primarily the SpV subdivision.
V0
_
,
exploratory tactile behavior is planned. In the case of the somatosensory system, the vast network o f corticocortical connections and massive corticofugal pathways to the thalamus, brainstem and spinal cord would provide the anatomical substrate for the dissemination o f predictions o f an internal touch model of the world across multiple cortical and subcortical areas (Grossberg, 1999). The interaction between these 'expectations' and raw tactile information provided by feedforward pathways could then define the computations needed to allow animals to accurately perceive the nature o f any given tactile stimulus as well as provide a continuous update o f the internal model. This would be accomplished through dynamic interactions between descending and ascending projections that reciprocally connect populations of cortical and thalamic neurons.
The recent introduction of electrophysiological methods that allow one to simultaneously record the activity of large populations of single neurons, located in multiple cortical areas and subcortical nuclei while carrying out selective and reversible pharmacological inactivation of selective regions o f the cortex has provided new insights on the functional contribution o f corticocortical and thalamocortical loops in tactile information processing. Here, we review some physiological results obtained in our laboratory regarding two o f these recurrent circuits in the rat somatosensory system: the thalamocortical loop between the primary somatosensory cortex (SI) and the ventral posterior medial (VPM) nucleus of the thalamus; and the loop formed by the reciprocal callosal connections between the SI cortices.
The potential role of corticothalamic feedback projections in tactile information processing Like all other m a m m a l i a n sensory systems, the somatosensory system contains massive feedback cor-
91 ticocortical and corticothalamic projections, which define closed loops between cortical areas and between the cortex and the thalamus. For instance, in primates, feedforward somatosensory pathways terminate in four distinct somatosensory areas located in the anterior parietal cortex (areas 3a, 3b, 1 and 2). Projections from the anterior parietal cortex also reach motor cortical areas in the frontal lobe, the secondary somatosensory cortex, and the somatosensory and multi-modal cortical areas of the posterior parietal cortex (Kaas et al., 1983). Somatosensory cortical areas in the parietal cortex are reciprocally connected through feedforward and feedback projections. In addition, cortical neurons located in some somatosensory and motor cortical areas are also connected by massive corticocortical feedback projections. Another source of massive corticofugal projections, which originate in the infragranular layers of the primary and higher order somatosensory cortical areas, project to all intermediary subcortical relays (i.e spinal cord, brainstem, and thalamic nuclei) of the somatosensory system (Kaas et al., 1983). Indeed, once the full domain of these feedback projections is considered, the somatosensory system can only be defined as a highly recurrent network, in which multiple feedback projections are intertwined with several parallel feedforward pathways. The importance of corticothalamic projections can be illustrated by a brief description of the anatomical organization and the physiological effects mediated by these pathways in the somatosensory system of rodents. As in every other mammalian species (Andersen et al., 1972; Adams et al., 1997; Deschenes et al., 1998; Sherman and Guillery, 1996), feedback projections from several somatosensory (e.g. SI, SII, PV, etc) cortical areas (Chmielowska et al., 1989; Bourassa et al., 1995) converge on neurons located in primary and secondary thalamic nuclei (e.g. VPM, POM, and ZI) of the trigeminal system of rodents. Studies in mice (Hoogland et al., 1987, 1991) and rats (Bourassa et al., 1995; Deschenes et al., 1994) have shown that these corticothalamic projections terminate primarily in the distal dendrites of these thalamic neurons (Pinault et al., 1997). In the case of the ventral posterior medial (VPM) nucleus, the primary thalamic relay of the trigeminal system, these corticothalamic
projections seem to be organized in a topographic manner (Hoogland et al., 1987; Bourassa et al., 1995; Deschenes et al., 1998; Zhang and Deschenes, 1998) (see Fig. 1). In this arrangement, corticothalamic projections originating from layer VI neurons, which are located under a particular cortical 'barrel' (Woolsey and Van der Loos, 1970) (e.g. barrel D1 in layer IV), terminate on thalamic neurons located across the thalamic barreloids (Van der Loos, 1976) that define the representation of a whisker arc or column (e.g. barreloids A1, B1, C1, D1, El) in the VPM (see Fig. 1). Corticothalamic projections from layer VI also reach the reticular nucleus (RT) of the thalamus (Pinault et al., 1995, 1997), the main source of GABAergic inhibition in the rat VPM (Pinault and Deschenes, 1998). Anatomical evidence suggests that these cortical-RT projections are also organized in a topographic arrangement, which seem to be orthogonal to that observed in the VPM nucleus (Hoogland et al., 1987, 1988). Thus, axons from layer VI neurons located under a given cortical 'barrel' (e.g. C1) target neurons located across the representation of a whisker row (e.g. C1, C2, C3, C4) in the RT nucleus (Hoogland et al., 1987, 1988). Neurons located in secondary somatosensory thalamic nuclei, such as the posterior medial nucleus (POM), also receive corticothalamic terminals (Hoogland et al., 1987, 1991), albeit these are primarily derived from pyramidal neurons located in layer V of the somatosensory cortex. The morphology of corticothalamic terminals also varies according to whether they terminate in the primary (e.g. VPM) or secondary thalamic relay (e.g. POM) nuclei (Hoogland et al., 1988, 1991). Physiological studies have shown that corticothalamic projections are primarily excitatory and likely employ glutamate as their main neurotransmitter (Turner and Salt, 1998, 1999). The glutamate released from these corticothalamic terminal acts on AMPA, NMDA, and metabotropic receptors located in the distal dendrites of thalamic neurons (McCormick and von Krosigk, 1992; Salt and Eaton, 1996; Turner and Salt, 1999). Activation of metabotropic receptors by in vitro stimulation of corticothalamic axons produces long-lasting, slowrising EPSPs in the thalamus (Salt and Turner, 1998; Turner and Salt, 1998). Based on some of these findings, corticothalamic-mediated activation
92 of metabotropic receptors has been suggested to produce the modulation of neuronal firing in the VPM nucleus (Salt and Turner, 1998; Turner and Salt, 1998). For instance, it is conceivable that the slowly rising depolarization produced by activation of corticothalamic projections could allow thalamic neurons to reach firing threshold in the presence of subthreshold synaptic input. In addition, corticothalamic afferents could also contribute to the slow activation of a low-threshold calcium conductance that underlies the production of bursts of action potentials by thalamic neurons (Sherman and Guillery, 1996). Despite a wealth of anatomical, pharmacological, and in vitro physiological information, the role played by corticothalamic projections in tactile information processing has remained elusive. For instance, penicillin-induced epileptic discharge in the cat somatosensory cortex (Ogden, 1960) and cortical spreading depression in the rat cortex (Albe-Fessard et al., 1983) were found to induce a depression of sensory evoked responses in the thalamus. In another series of experiments, carded out in both anesthetized and awake preparations, Yuan et al. (1985, 1986) reported that lidocaine-induced inactivation of SI cortex resulted in reduced thalamic responses to electrocutaneous stimulation without any effect on the spontaneous activity, stimulus threshold, response latency, or receptive fields of the same thalamic neurons. Other studies, however, have reported a facilitatory influence of SI cortex on evoked thalamic discharges (Andersen et al., 1967, 1972) using cortical spreading depression (Waller and Feldman, 1967) or electrical stimulation (Anderson et al., 1964), It is likely that part of the confusion in the literature arises because corticothalamic pathways can mediate both a monosynaptic excitatory and a dysynaptic inhibitory (via RT nucleus) postsynaptic potential in the thalamus. Thus, depending on how the cortex is stimulated or blocked, a variety of facilitatory and inhibitory responses effects could be induced in the thalamus. This hypothesis is supported by the observation that microstimulation of small territories of the SI cortex can lead to a range of thalamic effects, in addition to an overall suppressive influence of thalamic sensory responses, depending upon the relative topographic location of the stimulus and neurons in the ventral posterior nucleus of the thalamus (Shin and Chapin, 1990c).
In our hands, pharmacological block of SI cortical activity by focal infusion of the GABAA agonist muscimol, and the consequent silencing of pools of cortical neurons that give rise to corticofugal projections to the thalamus and brainstem, produced a series of physiological effects in the rat VPM (Krupa et al., 1999). First, we observed that blocking cortical activity altered both the short and long-latency components of the tactile responses of VPM neurons. The end result of these modifications was the demonstration that corticofugal projections contribute to the definition of the complex spatiotemporal structure (Krupa et al., 1999) of the RFs of VPM neurons. These results were obtained by using traditional single whisker stimuli. When more complex tactile stimuli were employed in our experiments, we observed that the ability of VPM neurons to integrate complex tactile stimuli (e.g. multi-whisker deflections) in a non-linear way was also significantly reduced by a pharmacological block of cortical activity. Both supraand sublinear summation of multi-whisker stimuli (Ghazanfar and Nicolelis, 1997) was reduced in these experiments (Ghazanfar et al., 1997). Overall, these findings not only support the hypothesis that corticothalamic projections may mediate both facilitatory and suppressing effects on thalamic neurons, but they also suggest that the action of these corticofugal projections may also depend on the type of tactile stimulus provided to the somatosensory system. As described below, there is direct evidence that the physiological contribution of these descending pathways to tactile information processing may also depend on the behavioral state of the animal. The functional relevance of corticofugal projections in the rat somatosensory system was investigated in studies carried out in our laboratory t o evaluate the contribution of corticofugal projections to the ability of subcortical neurons to express unmasking of novel tactile responses following a peripheral deafferentation (Krupa et al., 1999). This reorganization process, which we dubbed 'immediate or acute plasticity', is known to trigger a system-wide reorganization of the somatotopic maps located at cortical, thalamic, and brainstem levels (Faggin et al., 1997). The most conspicuous effect of this immediate reorganization is the shifting of receptive fields of individual neurons away from the
93 deafferented region due to the unmasking of neuronal tactile responses that were not present before the peripheral block. Interestingly, such unmasking tends to occur almost simultaneously in the brainstem, thalamus, and cortex (Faggin et al., 1997). In a recent series of experiments, we observed that blocking neuronal activity in the infragranular layers of the SI cortex, a procedure that silences the projecting neurons that give rise to corticobulbar and corticothalamic feedback projections, reduces by almost 50% the number of VPM thalamic neurons that exhibit unmasking of tactile responses following a partial and reversible peripheral deafferentation (Krupa et al., 1999). Although plastic reorganization in the VPM nucleus is still observed after cortical inactivation, its spatial extent is reduced significantly. These findings have been confirmed and extended further by the recent demonstration that the immediate, but not the late phase of plastic reorganization in the ventral posterior lateral nucleus (the thalamic relay for somatosensory fibers from the rest of the body), are reduced or eliminated by removal of corticofugal projections (Parker and Dostrovsky, 1999). Further support for the functional relevance of descending corticofugal projections comes from the observation that these projections have been demonstrated to affect the physiological properties of several other subcortical relays of the somatosensory system. For instance, block of neuronal activity in the SI cortex has been reported to eliminate most of the tactile responses of neurons located in the POM nucleus of the thalamus (Diamond et al., 1992). In addition, corticobulbar projections have also been shown to influence the physiological properties of neurons located in the brainstem nuclei that relay ascending somatosensory information to the thalamus (Jacquin et al., 1990b). For instance, removal of corticofugal projections in rats increases the responsiveness of neurons in spinal trigeminal brainstem complex to whisker stimuli (Jacquin et al., 1990b). Overall, the results reviewed above make a compelling case for the need to incorporate recurrent corticofugal projections as an integral part of a comprehensive and realistic model of touch. Indeed, the recurrent nature of the somatosensory system further strengthens our hypothesis that the mammalian somatosensory system relies on highly distributed neu-
ronal interactions, which emerge from the dynamic interplay of multiple ascending and descending pathways, to represent tactile information (Nicolelis et al., 1993b, 1995, 1996, 1997, 1998a,b; Nicolelis and Chapin, 1994; Nicolelis, 1996). Although the concept of distributed processing is not new, and many investigators have proposed schemes based on population coding (Hebb, 1949; Erickson, 1968, 1986; Georgopoulos et al., 1986; Sejnowski et al., 1988; Mumford, 1992; Deadwyler and Hampson, 1997), this encoding scheme has recently attracted the attention of neuroscientists because of the successful application of artificial neural networks in pattern recognition problems (Grossberg, 1976; Grossberg, 1988; Bishop, 1995. In a distributed coding scheme, divergent neural connections ensure that specific units of information are not held in single or small groups of neurons, but instead are widely distributed, or 'encoded' by large neural ensembles located at multiple cortical and subcortical levels of the system (Hebb, 1949). Consequently, each neuron contributes in some way to processing of most of the information handled by the network. In line with this hypothesis, a series of studies in our and other laboratories (Nicolelis et al., 1993a, 1995, 1998b; Kleinfeld and Delaney, 1996; Masino and Frostig, 1996; Moore and Nelson, 1998; Ghazanfar and Nicolelis, 1999; Polley et al., 1999) have begun to re-examine traditional views of information encoding by the somatosensory system. Anatomical evidence in favor of a distributed model includes the fact that ascending feedforward (FF) somatosensory pathways that carry information from the periphery to the SI cortex exhibit different degrees of divergence (Lu and Lin, 1986; Rhoades et al., 1987; Chmielowska et al., 1989; Jacquin et al., 1990a; Lin et al., 1990; Chiaia et al., 1991; Lu and Lin, 1993; Pinault and Deschenes, 1998; Veinante and Deschenes, 1999), which contribute to the large multi-whisker RFs observed in the VPM and SI (Nicolelis and Chapin, 1994; Ghazanfar and Nicolelis, 1999). Thus, the effects of even small but incremental changes at each processing level of the pathway (e.g. from brainstem to thalamus) would tend to multiply through successive relays and could be markedly amplified by the time they reached the cortex. In addition, wide-field sensory inputs, such as high-threshold mechanical and noxious stimuli,
94 which are transmitted through paralemniscal pathways, could also converge on cortical neurons. These effects could be further amplified by corticocortical connections within the SI and between the SI and other cortical areas (Chapin et al., 1987; Fabri and Burton, 1991; Nicolelis et al., 1991). In this context, the existence of massive divergent corticofugal feedback projections to all subcortical somatosensory relay nuclei provide almost unlimited opportunity for increasing the ultimate radius of influence from a single sensory event (Mumford, 1991). In this model, single neurons would not serve as the functional unit of the system. Instead, neurons would work as part of ensembles that are capable of representing and processing multiple tactile attributes of a given complex stimulus simultaneously. Massive corticofugal projections, that reach somatosensory relay structures located in the thalamus, brainstem, and the spinal cord, could offer the anatomical substrate for the definition of such multitasking networks. Such distributed and recurrent networks could be formed by somatosensory, motor, limbic, and association cortical areas, and influence the activity of neurons located in subcortical centers, even before mechanoreceptors in the skin were activated by a tactile stimulus. According to this view, corticofugal feedback projections could incorporate subcortical nuclei into the computational processes required for the emergence of tactile percepts. Although rarely discussed in the literature, reciprocal loops between cortical and thalamic nuclei could also mediate a different type of corticocortical communication, in which thalamic networks combine convergent signals from one or more cortical areas and then disseminate the resulting signals to vast cortical territories. Such an interactive view of the somatosensory system would predict that top down-influences would be capable of modulating the activity of subcortical neurons during different behavioral states. But is there evidence for the existence of such top-down influences in the somatosensory system? In the next section we describe a well-known phenomenon that may provide the key for unraveling the physiological role played by corticofugal feedback on tactile perception and, hence, serve as the basis for mounting a formidable challenge to the FF model of touch.
A potential physiological role for corticothalamic pathways in tactile processing: sensory gating of neural responses during active tactile exploration A number of studies carried out in many species indicate that during different exploratory behaviors the magnitude and latencies of tactile responses as well as the manner in which the brain responds to complex tactile stimuli, can change considerably. Thus, in rats, reductions in responses to tactile stimuli during motor activity have been observed in SI (Chapin and Woodward, 1981, 1982a,b; Shin and Chapin, 1990b), the ventral posterior lateral thalamus (VPL) (Shin and Chapin, 1990a,b), and the dorsal column nuclei (DCN) (Shin and Chapin, 1989). Similarly, in cats, medial lemniscus sensory responses elicited by stimulation of the radial nerve are reduced in magnitude during limb movement (Ghez and Lenzi, 1971; Coulter, 1974). Primates also show modulations in SI (Nelson, 1984, 1987; Chapman et al., 1988) prior to and during motor movement. Alterations in sensory responses during movement have also been observed in human, evoked potential studies (Coquery, 1971; Lee and White, 1974; Cohen and Starr, 1987). These observations imply that the nervous system is capable of dynamically altering how cortical and subcortical neurons respond to a tactile stimulus, depending on the behavioral context in which such a stimulus is presented to the animal. The crux of this argument, therefore, lies in the hypothesis that the emergence of the broad spectrum of natural tactile sensations experienced by mammals results from a much more intimate association between the somatosensory and motor systems than postulated before by previous neurophysiological theories of touch. But what is the significance of endowing the somatosensory system with the capability of altering the type of tactile information that can reach the cortex during motor activity? First, the alterations in response magnitude may allow tactile and proprioceptive information pertinent to execution or completion of the movement to be selectively enhanced. Thus, during the execution of a planned motor act, reciprocal interactions between the motor and the somatosensory cortices would ensure that certain types of input are gated 'in' while others are gated 'out'. This may be necessary in order to allow
95 the movement to occur as planned without interference from extraneous sensory feedback. In support of this idea, Chapin and Woodward (Chapin and Woodward, 1982a) showed that tactile responses, across the cortical and subcortical relays of the somatosensory system, can be inhibited or enhanced at different epochs of the step cycle in locomoting rats. According to these authors, irrelevant tactile responses, such as those caused by the movement itself, would be selectively gated out at certain times during the movement, while sensory information that would describe certain movement epochs (e.g. foot fall) would be enhanced. There is strong evidence in the literature supporting the hypothesis that this selective modulation of tactile responses is mediated by descending efferent activity from the motor cortex or other central motor nuclei. The most relevant experimental finding supporting the occurrence of centrally mediated gating of tactile information is the observation that sensory responses across the somatosensory pathway can be reduced as much as 100 ms prior to the initiation of a given movement. These findings, which have been obtained in both monkeys (Nelson, 1987; Chapman et al., 1988) and cats (Coulter, 1974; Ghez and Lenzi, 1971) strongly suggest that reductions in tactile responses in the cortex and subcortical relays of the somatosensory system do not result from alterations in tactile or proprioceptive feedback generated by the movement itself. Instead, they may be related to the central motor command that is generated hundreds of milliseconds prior to the movement onset. Further support for this view comes from the observation that microstimulation of the motor cortex can reduce tactile neuronal responses throughout the somatosensory system. For example, Shin and Chapin showed that stimulation of the forepaw region in the MI cortex, prior to electrical stimulation to the forepaw, led to a 43% suppression of tactile responses in the thalamus (Shin and Chapin, 1990b) and 8% reduction in the dorsal column nuclei (Shin and Chapin, 1989). Another set of experiments has demonstrated that the amount of tactile response modulation varies across the different intermediary relays of the ascending somatosensory pathways. For instance, in rats, SI and VPL tactile responses to forepaw stimulation have been shown to decrease 71 and 31%, respectively, when the stimuli are delivered during
the animal's locomotion (Chapin and Woodward, 1981; Shin and Chapin, 1990b). These findings illustrate the general observation that the magnitude and frequency of the somatosensory gating effect tends to increase as one ascends through the somatosensory system. As such, this observation further supports the hypothesis that modulatory signals derive from central neural networks responsible for generating the motor command. Despite robust evidence favoring the existence of a central mechanism for modulating tactile neuronal responses, in some cases one can also demonstrate that feedforward mechanisms contribute to the alteration of cortical and subcortical tactile responses during the execution of exploratory behavior. For example, Schmidt et al. (1990b) have shown that when anesthesia was applied to one or more sensory nerves of the hand, inhibition of tactile stimulation that normally occurred when subjects moved the finger being stimulated, was reduced (i.e. there was less gating of the response) by up to 70%. This led to the conclusion that a significant portion of the gating effect, but not all, was caused by afferent sensory stimulation generated in the periphery by the movement itself. Support for the existence of peripheral mechanisms of gating has also been provided by Chapman et al. (1988), who reported that tactile responses in SI to stimulation of the medial lemniscus or the thalamus were not reduced prior to movement, but only during the execution of the movement. These authors reported that somatosensory gating occurring at the level of the dorsal column nuclei (DCN) was caused by central modulation, since it occurred prior to the movement. However, in their hands any additional gating at higher levels of the somatosensory system was caused by motor-induced peripheral afferent activity. Another series of experiments partially supported this view by showing that while passive movements do not cause tactile gating in the DCN, they can induce a certain degree of gating in the thalamus (VPLc) and SI (Chapman et al., 1988). Thus, even though there is substantial evidence for centrally mediated modulation of tactile responses, one cannot discard the possibility that proprioceptive and tactile afferent signals, generated during the execution of movements, contribute to the gating of tactile responses observed at higher levels of the somatosensory system.
96 It is important to emphasize that noradrenergic, serotoninergic, and cholinergic projections, which originate in different locations of the brainstem and diencephalon and target all intermediary relays of the somatosensory system, could also contribute to the central modulation of tactile neuronal responses during different behavioral states (McCormick and Pape, 1990; Waterhouse et al., 1994) as they do in other sensory systems (McLean and Waterhouse, 1994). Over the last three decades, one of the most elegant examples of multi-disciplinary research in neuroscience has indicated that some of these modulatory systems play a fundamental role in the control of the ascending flow of nociceptive information from the periphery that is used for the perception of pain (Fields and Heinricher, 1985). The demonstration that physiological or pharmacological activation of these descending modulatory projections can block the ascending flow of nociceptive information through the spinothalamic system and produce maintained analgesia has revolutionized our understanding of pain perception. Since pain belongs to the spectrum of tactile sensation that all mammals experience, these observations offer more experimental support for our contention that top-down influences cannot be ignored by any theory aimed at describing the neurophysiological basis of tactile perception. In line with this hypothesis, recent studies in the trigerninal system of awake, freely moving rats have corroborated and extended our conviction that top-down influences, such as those mediating the phenomenon of 'somatosensory gating', play a crucial role in the emergence of tactile perception. In these experiments, simultaneous, multi-site chronic recordings were employed to monitor the activity of large populations of single cortical, thalamic, and brainstem somatosensory neurons, while rats moved freely in a behavioral box. Initially, these experiments allowed us to investigate how the expression of different behaviors (e.g. awake immobility, active whisking, moving without whisker movements) could influence the physiological properties of populations of cortical and subcortical neurons in freely behaving animals (Fanselow and Nicolelis, 1999). Subsequently, the same experimental paradigm was used to measure how similar tactile stimuli are processed under different behavioral conditions across the rat somatosensory system.
We observed that complex and dynamic corticothalamic interactions tend to precede any active tactile discrimination in freely behaving rats. For instance, as awake rats assume an immobile posture (i.e. standing on the four paws without producing any whisker or other major body movements), most of the neurons in the SI cortex and VPM thalamus start producing rhythmic bursts of action potentials, which are translated into 7-12 Hz rhythmic oscillations (Nicolelis et al., 1995; Fanselow and Nicolelis, 1999). In the vast majority of the analyzed events, these 7-12 Hz rhythmic oscillations initiate in the whisker area of the rat SI cortex (the barrel fields) (Fig. 2). After a few tens of milliseconds, these osi
i
i
i i
S, CORTEX S, CORTEX
't
VP~
....
VPM
~A~^A~^,dAik,
1 . . . . ^At,.d~A~,AAAAI.^~A, dAA~,
1 SEC I
i
i[
I t iii i
i
!
CORTEX
SI CORTEX VPM
~
~
THALAMUS
t lilttnm~
t
t
t
,
,
tt|S~
c
t
I
i
Sl CORTEX ~ ! S[ CORTEX
VPM THALAMUS
~ I
THALAMUS
1 SEC Fig. 2. Simultaneous field-potential recordings in VPM and SI during Ix rhythm reveal that 7-12 Hz oscillations tend to appear first in the SI cortex and only later are evident in the sornatosensory thalamus.
97 SpV 1
10 0 10 0
VPM
1
A
20 A
Alk
0 .'-",."T'",'"',"'-,"-'r-",'","r"T'"r'",
2°I
A
A
A
,o
2oo 0"
0 r...r...l-..-r.-.r-.-r-.T--.r-..r-.-,..-T...r.-.~
~.
°
30
t
r
40
i
0 |....r.-.r-.:,-.--,-.-.r.-T.--,----,.-..r--T...,...., I
30
7--r--l---T--T---r--
i
-
-
0 ....,...T...,...-,....,...T...r...r_r..T...r.. , Sl 50 0 5
10"
O-
O.
s~
1o I
Io
0 T'"'r "'r"'Ilt'lr'"~'"'l'"'r'l'll'"'r"T"T "''l
0.1
0
0.1
0
0.1
0
0.1
Tlmo lag (s) Fig. 3. During whisker twitching movements, activity in SI and SpV phase leads activity in VPM. Recordings were made from microelectrodes chronically implanted in SpV, VPM and SI in awake rats. This figure depicts cross-correlograms (CCs) for neurons in these three areas during 7-12 Hz whisker twitching movements, which were accompanied by i~-oscillations. The CCs are centered on the VPM neuron depicted by the arrow, and the numbers above the CCs show the number of milliseconds by which a given SpV or SI neuron phase-led the VPM reference neuron. It can be seen that activity in SI and SpV phase led activity in VPM.
cillations appear in the VPM nucleus and later on they can be observed in the spinal nucleus (but not in the principal) of the trigeminal brainstem complex (Figs. 2 and 3). Importantly, these oscillations were never detected in the rat trigeminal ganglion, suggesting that they are generated centrally. Further analysis revealed that these 7-12 Hz thalamocortical oscillations usually precede, by hundreds of milliseconds, the initiation of small amplitude rhyth-
mic facial whisker twitching (WT) movements in the same frequency range (Nicolelis et al., 1995). We also observed that the initiation of WT movements modulated these oscillations (Nicolelis et al., 1995). Thus, soon after the onset of WT movements, rats invariably started to produce slower (4-6 Hz) rhythmic whisker protractions, which had much larger amplitudes than the WT movements. Previous behavioral studies have indicated that rats use these large rhyth-
98 mic whisker movements to discriminate the tactile attributes of objects (Carvell and Simons, 1990). In fact, 'whisking' is present in most rodents and is considered as an important exploratory behavior of rats. In our experiments, we also documented that as soon as the animal started to produce these slower and larger whisker movements, the 7-12 Hz thalamocortical oscillations disappeared (Nicolelis et al., 1995). Altogether, these observations are very reminiscent of a similar phenomenon originally described by Gastaut in human scalp EEG recordings carded out in the 1950s (Gastaut, 1952). Since its original discovery, both EEG and magnetoencephalographic recordings have been used to demonstrate the occurrence of widespread 10 Hz oscillations, originating in the hand representation of the primary somatosensory cortex of the vast majority of healthy human subjects and non-human primates (Niedermeyer, 1993). In the EEG literature, these oscillations were named motor (Ix) rhythm, since its main characteristic is to appear during awake immobility and disappear as soon as the subject starts any hand movement, a key tactile exploratory behavior of primates. The existence of these similarities led us to postulate that the 7-12 Hz oscillations that precede and are modulated by whisker movements in rats are equivalent to the Ix rhythm of primates. The functional role of Ix oscillations in primates and rodents is still unclear. Although MEG recordings have clearly indicated that this preparatory rhythm is present in most normal human subjects, no clear consensus has been reached regarding the potential physiological role played by these oscillations. Because rats use rhythmic whisker movements as their main tactile exploratory behavior, the presence of the IX rhythm in this species led us to propose that these thalamocortical oscillations could prepare the somatosensory system for the imminent onset of a cycle of tactile exploration. According to this hypothesis, during the occurrence of 7-12 Hz oscillations, tactile information would continue to flow from the VPM to the somatosensory cortex. However, since during these oscillations a significant percentage of VPM neurons are producing bursts, it is conceivable that these neurons would have difficulty in faithfully transmitting complex spariotemporal patterns of tactile information, which are likely to be generated when rats use their whiskers to actively explore an ob-
ject. Instead, we proposed (Nicolelis et al., 1995) that these 7-12 Hz oscillations could be used to enhance or even maximize the ability of the somatosensory system to detect the presence of tactile stimuli, either during awake immobility or during the production of whisker twitching movements. In other words, these 7-12 Hz oscillations could represent an 'expectation' signal, a template that could be produced by the rat somatosensory system in anticipation of whisking. In our view, this 'expectation' signal could be generated as part of central motor program and be disseminated to most of the somatosensory system through corticofugal projections. We speculate that once the presence of a tactile stimulus is detected by an immobile rat (or during WT movements), larger amplitude and slower (46 Hz) rhythmic whisker protractions are initiated so that a more detailed tactile exploration of objects by the arrays of vibrissae can be accomplished. As the animals behavior changes, so does the physiological setting of the thalamocortical loop. During the execution of these rhythmic large amplitude whisker movements, thalamocortical oscillations vanish and VPM neurons switch to a tonic firing mode. In this physiological state, VPM neurons have higher spontaneous firing rates and can faithfully represent and transmit complex spatiotemporal patterns of tactile inputs to the SI cortex. It is important to emphasize, however, that during this active tactile exploration, corticothalamic projections can still mediate important interactive computations at the level of the thalamus. In this interactive view of the somatosensory system, the thalamus is no longer considered as a simple, passive relay of tactile information from the periphery to the cortex. Instead, through its reciprocal interactions with different cortical areas, the somatosensory thalamus, (which includes the tactile portion of the reticular nucleus), could participate in a variety of computations, such as non-linear summarion of tactile stimuli (Ghazanfar and Nicolelis, 1997; Shigemi et al., 1999), signal segmentation through resonant interactions, template matching, and error generation. Similar to the recurrent models of Grossberg (1999) and Mumford (1994), the somatosensory thalamus could function as the site where incoming afferent tactile information is compared with cortically stored templates that resume the previous tactile experience of the animal. As feedfor-
99 the same animals engaged in behaviors that did not include the whisker movements (e.g. movement of the head or body). These findings corroborate work in cats (Coulter, 1974), and in humans (Schmidt et al., 1990a Schmidt et al., 1990b), in which decreases in sensory responsiveness were most robust when the sensory stimulus was applied to the part of the body engaged in a tactile exploration, as compared to adjacent digits or contralateral limbs. Thus, as previously suggested by many authors, the phenomenon of somatosensory gating appears to be fairly topographically specific, since it occurs only during motor activity used for active tactile exploration, rather than following any action involved in increasing the general arousal level of the animal. Previous studies have shown that the presence of one stimulus can alter the ability of cortical and subcortical neurons to respond to a subsequent stimulus for a period of time (Simons, 1985; Simons and Carvell, 1989). Evidence from our experiments and other studies (Castro-Alamancos and Connors, 1996a,b) suggest that the ability of one tactile response to modulate the magnitude of a subsequent one is substantially decreased during motor activity (Fig. 5). For example, when two tactile stimuli were presented with an inter-stimulus interval of 2575 ms in the absence of any whisker movements (i.e. awake immobility), the response to the second
ward and feedback projections may be required for the definition of the complex spatiotemporal structure of receptive fields in the rat VPM (Krupa et al., 1999), ensembles of these neurons could participate in template-matching operations and other computations on afferent tactile signals. In order to test some of these assumptions, another series of experiments was carded out in our laboratory. In these experiments, multi-site chronic recordings were carded out while a nerve cuff electrode was used to provide consistent stimulation to the infraorbital (IO) nerve, the nerve that carries tactile information from the vibrissae to the central nervous system, as rats switched between a series of behavioral states (Fanselow and Nicolelis, 1999). In the first series of experiments, individual electrical stimuli that produced neuronal responses that mimic those obtained by mechanical stimulation of multiple facial whiskers were delivered to the IO nerve while rats were immobile, or when they produced the two different types of whisker movements described above. The cortical (SI) and thalamic (VPM) sensory responses elicited by the electrical stimuli were then compared. As predicted by previous studies, the magnitudes of the neural responses in SI and VPM neurons were substantially reduced during the production of rhythmic whisker movements (Fig. 4). Interestingly, this reduction was not observed when
VPM
SI
20
o~,~-
0
~ 20 ~
o
-50
0
w
i
20 50
100
150
200
-50
0
50
100
150
200
ms post stimulus
Fig. 4. Activity levels in VPM and SI following peripheral stimulation differ depending on the behavioral state of an animal. Individual electrical pulses were presented to the infraorbital nerve in awake, freely moving rats and responses to this stimulation were recorded from chronically implanted microwires in VPM and SI. When the animal was in a state of quiet immobility, the initial excitatory response was followed by a period of suppressed firing, during which activity went below pre-stimulus baseline levels (dotted line). This period of suppressed firing was followed by a late excitatory component at approximately 125 post-stimulus. In contrast, during exploratory whisking behavior, the period of suppressed firing was substantially shorter in VPM and non-existent in SI, and there was no late excitatory component in either area. Error bars represent ±SEM. The initial excitatory peaks have been clipped in order to show the other components of the traces more clearly.
100
VPM Quiet
Whisking
150
100 or)
5O
to
~r
|,iiliii|
0 lstl25 50 75 100 t25 150 175200 I stimulus" time to second stimulus in ms
lstl25 50 75 100 125 150 175200 stimulus time to second stimulus in ms
E T'-
Sl Quiet
Whisking
if}
150 0c-
'OSO0 Oi ,,i/i i lstl25 50 75 100 125 150 175200 stimulus time to second stimulus in ms
I,iiiiil!
lstl25 50 75 100 125 150 175 200 stimulus ' time to second stimulus in me
Fig. 5. Responses to peripheral stimulation differ depending on the behavioral state of an animal. Recordings were made from multiple chronically implanted microwires in awake, freely moving rats. Stimulation was provided to a nerve cuff electrode implanted around the infraorbital nerve. Stimuli were presented in pairs with interstimulus intervals (ISI) ranging from 25-200 ms. This figure demonstrates two effects we observed by looking at the responses to these stimuli during two different behaviors, quiet immobility and exploratory whisking. First, during whisking, responses to the first stimulus in the pairs were smaller than those during the quiet state. This indicates that during the whisking state there is an overall gating of responses to ascending stimuli. The second effect was that when animals were in a state of quiet immobility, responses to the second stimulus in a pair was suppressed if the ISI was 25-75 ms. In contrast, during active, exploratory whisking behavior, the responses were not significantly suppressed for any ISI, compared to the response to the first stimulus in the pair. Error bars represent +SEM.
stimulus was significantly reduced (Fig. 5). However, during periods in which the same rats produced whisker movements, the response of VPM and SI neurons to the second stimulus was not statistically
different in magnitude from the first at any interstimulus interval tested (Fig. 5). Further examination of these results indicated that these effects paralleled a change in the amount of post-excitatory inhibition
101 that follows the first IO stimulus in different behavior states (immobility vs. whisker movements). Thus, in the absence of any movement of the whiskers, we observed the occurrence of a long period of reduced firing following the presentation of the first tactile stimulus (Hellweg et al., 1977; Simons, 1985; Simons and Carvell, 1989). However, this period is substantially shorter (in VPM) or non-existent (in SI) during the presence of exploratory whisker movements (Fig. 5), suggesting that motor-activity related changes in post-stimulus inhibition could account for the differential responses to paired stimuli we observed in different behavioral conditions. Overall, the results of these experiments suggest that during different behavioral states, different types of thalamocortical transmission may occur (in this case, awake immobility versus 'whisking') and that these different modes of transmission may serve different perceptual purposes. Thus, differences in cortical and subcortical tactile response characteristics, from periods of whisker immobility to periods of whisker movements, suggest that the somatosensory system can shift from a state of high-sensitivity for detecting individual punctate stimuli (i.e. during immobility and thalamic bursting), to a state in which the system can process with high-fidelity the complex incoming tactile afferent information that are generated by the active tactile exploratory behavior employed by the animal to probe its surrounding environment. Although there is no definitive proof that corticofugal projections are responsible for either the recruitment of VPM neurons into 7-12 Hz oscillations during awake immobility, or the switch of VPM neurons from bursting to tonic firing mode, several indirect observations can be used to build a strong case in favor of this hypothesis. First, in the vast majority of our recordings, IXoscillations clearly initiate in the SI cortex and only later appear in the VPM thalamus. Likewise, in all other species in which the Ix rhythm has been reported it was found to originate at the cortical level. The hypothesis that corticofugal projections provide the anatomical substrate for recruiting the thalamus into a massive wave of synchronous activity is also supported by the observation that removal of corticothalamic projections significantly reduces or completely abolishes the synchronization of neuronal firing across the thalamus (Contreras et al., 1996).
The potential contribution of corticothalamic projections to the switching in firing mode of thalamic neurons is also supported by several indirect observations. Since corticothalamic axons terminate in the distal dendrites of VPM neurons, and exert their direct excitatory effects through metabotropic receptors, they could provide the type of slow depolarization synaptic events that are required for activating the low-threshold calcium conductance that endows thalamic neurons with the ability to fire in bursts. De-inactivation of this calcium conductance, which requires hyperpolarization of VPM neurons, could also be achieved by corticothalamic projections acting through the reticular nucleus which provides GABAergic innervation to thalamic relay neurons. Though many more experiments are required to fully demonstrate the computations carded out by the interplay of corticofugal and ascending somatosensory pathways, the central assumption of our argument remains valid. The type of dynamic thalamocortical interactions described above cannot be explained by a simple feedforward description of the somatosensory system. As seen above, changes in behavioral state significantly alter the responses to tactile stimuli across cortical and subcortical levels of the somatosensory system. These studies have demonstrated that neuronal response properties can be altered on the order of seconds, as animals switch from one behavioral state to the next. The possibility of altering the manner in which somatosensory neurons respond to the same tactile stimuli under different circumstances confers a high degree of adaptability to the animals since it may allow them to filter information in different ways, as required by the situation in which they are involved. This rapid, behavior-dependent adaptation may also provide the somatosensory system with more flexibility for detecting a wider range of stimuli, or allow preferential detection of certain types of stimulation under different circumstances. Corticocortical loops as the substrate for the integration of bilateral whisker information in the rat barrel cortex The second loop investigated in our studies of the rat somatosensory system is the one defined by reciprocal callosal connections between both rat SI cortices,
102 The experimental evidence reviewed in this section suggests that this loop plays a fundamental role in integrating left and right side whisker information required for the formation of bilateral tactile percepts of the environment. In these studies, the role the cerebral cortex plays in the integration of bilateral tactile information was investigated by inferring from extracellular recordings the temporal and spatial transformations performed by cortical neurons on convergent subcortical and corticocortical input. Study of cortical processes in the barrel cortex is facilitated in part by an ability to exploit an orderly topography of connections found throughout the whisker-barrel axis that reflects the arrangement of contralateral whiskers at the periphery (Woolsey and Van der Loos, 1970; Killackey, 1973). It is upon this anatomical topography that the classical hypothesis regarding barrel cortical function emerged, which postulates that activity within a given barrel cortical column directly relates to the attributes of a stimulus applied to a corresponding contralateral whisker. Incorporating more recent findings, contemporary theories of barrel cortical function have evolved to emphasize the role the barrel cortex plays in integrating information across whiskers to form behaviorally relevant percepts, rather than in extracting information from individual whiskers. Experiments using condition-test paradigms have provided a basic understanding of the temporal and spatial nature of integration between barrel regions corresponding to pairs of whiskers (Simons, 1985; Simons and Carvell, 1989; Brumberg et al., 1996). These experiments have provided evidence that excitation of a region of the barrel cortex is followed by a prolonged period of inhibition, which attenuates over time and acts to diminish the probability of a second, stimulus-evoked response. The magnitude of inhibition appears to be maximal in the region corresponding to the whisker stimulated, with the spatial distribution of inhibition decreasing as a function of distance from this center. However, such studies are limited by the fact that observed cortical responses are not solely functions of cortical processes, but instead, also may reflect the processes of subcortical structures along the whisker-barrel axis. Whatever the nature of subcortical integration along the whiskerbarrel axis, corticocortical integration may still be addressed by exploiting the role the corpus callo-
sum plays in integrating sensory information that is lateralized subcortically. In this context, if the rat is to create a perception of the environment regarding both sides of its face if it is to generate an appropriate behavioral response to bilateral stimuli - - then this information must so too be integrated. That rats are capable of navigating complex terrain even in absolute darkness by virtue of their whiskers seems to dictate that they make comparisons between groups of whiskers. As rats bilaterally and synchronously whisk objects they encounter, to successfully detect the orientation of an obstacle or the width of an aperture they must gather and integrate information regarding the distance of objects from the whisker pads to either sides of the face. The most logical place to identify and characterize how bilateral information is integrated, therefore, is at the level left and right side whisker information first converge (Fig. 6). As ascending whisker-related pathways are fully crossed subcortically, this conver-
-
Barrel Cortex
Barrel Cortex
W h i s k e r Pad
Fig. 6. Schematicof whisker-barrel anatomyillustrating convergence of ascending contralateral, whisker-relatedpathways with ipsilateral, whisker-relatedcallosal pathways. Neurons innervating the whiskerpad terminate in the principle trigeminalnucleus (PrV) and spinal trigeminalnuclei (SpV). Projectionsfrom these nuclei then decussate and terminate in two thalamic nuclei: the ventroposteriormedial nucleus (VPM) and the posteriormedial nucleus (PoM), which in turn project to the barrel cortex. The barrel cortex also receivescallosal input regardingthe ipsilateral whisker pad from a portionof neurons in supra- and infragrandular layers of the oppositebarrel cortex.
103 gence is thought to occur at the level of the SI barrel cortices, as they are interconnected via the corpus callosum in a roughly homotopic manner (White and DeAmicis, 1977; Koralek et al., 1990; Olavarria et al., 1992; Cauller et al., 1998). This presumption is also supported by evidence collected by Pidoux and Verley (1979), who provided local field potential recordings indicating that ipsilaterally evoked responses exist in the barrel cortex and are mediated by the corpus callosum. In support of this view, we have recently provided direct evidence that indicates layer V barrel cortical neurons integrate not only contralateral whisker information, but whisker information from both sides of the face, and further postulate that such interactions underlie the formation of bilateral tactile percepts. By combining methods for creating bilateral, multi-whisker stimuli with multi-electrode recordings we addressed whether single barrel cortical neurons respond to both contra- and ipsilateral whisker stimuli, characterized ipsilaterally evoked response properties, and determined the spatial and temporal aspects of interaction evoked by bilateral whisker stimulation (Shuler et al., 2001). Pharmacological inactivation of the opposite barrel cortex corroborated the proposition that the source of ipsilateral input is the opposite barrel cortex. We further demonstrated that during inactivation, not only were ipsilaterally evoked responses abolished, but the ensuing inhibitory influence such responses exerted on subsequent contralaterally evoked activity was also shown to be removed. Lastly, by designing a behavioral task that requires the cooperativity of the barrel cortices, we determined that rats are, in fact, capable of forming bilateral tactile percepts. What constitutes an effective ipsilateral whisker stimulus was addressed by varying the number and location of whiskers stimulated ipsilateral to cortically implanted electrode arrays. The arrangement of 16, independently drivable whisker deflectors allowed single whiskers, as well as all possible combinations of two, three, and four whiskers to be stimulated in each of four whisker columns (or arcs) tested. The result of such a stimulus regime was the characterization of ipsilaterally evoked responses in 72% of neurons recorded, with an average probability of evoked response of 21.8 ± 13% (mean -tSD). Compared to an 11 ± 3 ms minimal latency
for responses elicited by contralateral stimuli, the average minimal latency for a response elicited by an ipsilateral stimulus was 23 -4- 4 ms. Numerous instances of 'supra-linear' responses were detected as combinations of simultaneously deflected whiskers frequently were capable of eliciting ipsilateral responses when no responses were elicited from the individual whiskers that defined the ipsilateral stimulus. Although response probabilities were also found to increase as the number of whiskers deflected increased, this increase was decidedly sublinear for neurons that responded to the constituent parts of the stimulus when given alone. Therefore, not only is the barrel cortex responsive to ipsilateral stimulation, but the proportions of neurons and their underlying firing probabilities are nonlinearly effected by multi, ipsilateral whisker stimuli. Given the presence of ipsilateral responses in the barrel cortex, we next addressed the impact such responses may have on contralaterally evoked activity, and vice versa. The nature of bilateral whiskerevoked interactions within the barrel cortex was investigated by using a condition-test paradigm that varied the spatiotemporal attributes of left and right side whisker stimuli (Fig. 7). Three parameters of bilateral whisker stimuli were varied: (1) the hemispheric sequence (ipsi- then contralateral, or contrathen ipsilateral stimulation); (2) the inter-stimulus interval (ISI); and (3) the spatial location of condition stimuli. Variation of these three factors tested the null hypotheses that the hemispheric sequence, ISI, and spatial relationship between bilateral whisker stimuli do not change the firing probabilities of barrel cortical neurons as compared to responses evoked when ipsi- or contralateral stimuli are given alone. These attributes of bilateral stimuli were all shown to significantly impact the evoked response probabilities of barrel cortical neurons, demonstrating that the barrel cortices are capable of integrating bilateral whisker information. Furthermore, the change in response probability caused by bilateral interactions could not be explained by postulating that barrel cortical neurons simply were less likely to fire to the test stimulus on trials that neurons had fired to the condition stimulus. This result indicates that ipsilaterally, as well as contralaterally evoked suprathreshold activity is followed by an epoch of inhibition of an even greater spatial extent. This
104 conclusion is further supported by noting that even for neurons without identifiable ipsilateral responses, contralaterally evoked responses to test stimuli were significantly impacted by prior ipsilateral stimulation. Therefore, bilateral interactions give rise to hemispheric and spatial differences in recovery of subsequent responses, potentially reflecting a differential activation of inhibitory networks. To determine the source of ipsilateral input, the opposite barrel cortex was pharmacologically inactivated by infusion of the GABAA agonist, muscimol, prior to bilateral stimulation. Not only did inactivation remove ipsilaterally evoked responses in the intact hemisphere, but the observed effects of prior ipsilateral stimulation on contralaterally evoked responses were also negated. Such results indicate that the barrel cortices provide one another with ipsilateral whisker information. Considering that callosal connections are thought to be excitatory, these results suggest that ipsilaterally evoked suprathreshold activity subsequently activates local inhibitory networks, rather than indicating that such inhibition derives from yet another source. A central question raised by these physiological results is whether rats can make use of bilateral tactile cues to discriminate objects. Surprisingly, though a number of behavioral studies have address how rats use their whiskers to discriminate tactile features of the environment (Hutson and Masterton, 1986; GuicRobles et al., 1989; Barneoud et al., 1991; Pazos et al., 1995; Brecht et al., 1997), no study to date has directly addressed the ability of rats to compare bilateral tactile features. To test the hypothesis that rats can form bilateral percepts, we developed a discrimination task in which rats learn to compare the relative distance of two walls, one to each side of the face, using only the facial whiskers. Of eight rats trained on this task, all learned to associate equidistant or non-equidistant bilateral stimuli with
water reward made available at one of two reward windows, respectively. These results provide the first evidence that rats can indeed combine information from both whisker pads (Shuler et al., 2000). The anatomical constraints of the whisker-barrel system makes it ideal for studying cortical integration of independent sources of sensory input. We exploited this anatomy by investigating cortical integration of contralateral whisker-evoked activity ascending via thalamocortical pathways with that of callosally converging ipsilateral activity. Ipsilaterally evoked responses, as well as interactions due to the hemispheric, temporal, and spatial attributes of bilateral stimuli strongly contradict the classical notion that the rat barrel cortex solely represents stimuli delivered to the contralateral whisker pad. Such interactions evoked by multi-whisker stimuli do not support the hypothesis that cortical profiles of activity result from the topographic, linear superposition of individual responses. Furthermore, such interactions cannot be explained by postulating the existence of superimposed contra- and ipsilateral topographic maps. To propose such a coding scheme, a countless number of topographies would be required to uniquely describe all possible permutations of bilateral whisker stimuli. Rather we propose that ascending thalamic as well as converging transhemispheric input differentially excite the barrel cortex, subsequently initiating a wake of spreading inhibition. Spatial and temporal asymmetries in activating excitatory and inhibitory elements of the network result in spatiotemporally unique profiles of cortical activity, allowing each hemisphere to render unambiguously the attributes of a bilateral stimulus. In conclusion, we propose that such bilateral interactions are fundamental to forming bilateral tactile percepts, allowing rats to discriminate ethologically meaningful stimuli, such as the orientation and diameter of apertures.
Fig. 7. Evoked responses of two, layer V barrel cortical neurons to bilateral whisker stimulation. Simultaneous, single unit recordings from neurons in the left (neuron 1) and fight (neuron 2) hemispheres were obtained while stimulating left and fight whisker arcs (whiskers b3, c3, d3, e3). Solid vertical lines centered at time 0 denote delivery of test stimuli, while dashed vertical lines denote delivery of condition stimuli. Six stimulus conditions are shown for neurons 1 and 2. The top two rows depict responses to the test stimuli (left whisker arc, L; fight whisker arc, R) when given alone. The bottom four rows depict responses under condition-test stimulation; left then right whisker arcs with an ISI of 35 ms (L-R 35 ms), left then fight whisker arcs with an ISI of 175 ms (L-R 175 ms), fight then left whisker arcs with an ISI of 35 ms (R-L 35 ms), and finally,fight then left whisker arcs with an ISI of 175 ms (R-L 175 ms). The y-axis is in spike counts per 1 ms bin of time (300 presentations of each stimulus configurationwere given).
t05
401
Neuron 1
Neuron 2
2o30 lps L
Contra
3O 20 10
l
10
o .i,J,.,,hi,,-,,~,.L~£.
0
L'L"JL "kL "L ' LLaA' 'u" t'r'"'L~ ,,JllJlvl,l,,,,,,,
''''''''1'''''''''"''''''''1
20301
Contra
'''''''''1
4°1
Ipsi
30 20
10
10
0 ~'dlL""'~l"J"~'" t'"' ,,,,,,,,I,,,,,,*,,
0
I,,,,,,,,,
I
JI1,
"11"11'u I'LIIII t ""1 *ill / L -- "i'l ',,,,,,ll,Jll,l,,, ,,,,,,,T,,,
I
Ipsi
203040110 C°ntra I
L'-R 20 35ms 10
t-
.m
0 J,.,,,ll.,J-..~.hJ.,
0
,,,
,~,lllllllllllllJl
301 ~lpsi L'-R 20 175ms 10 0
Contra
20
35ms
lO 0
Contra
4°1
30 Ipsi 20 10 0 .,, L.,Ld,I,J,,,I,~
401]
L~k~,~.l..~..~,L ~•
I'lllIJlltJlllJll
R,-L 20 175ms 10
Contra
Ipsi
30 20 10
Ipsi
,.,
0
/ i, I"'JllJllJll,,ll,,,
°ll t ,
-0.1 0 Time (sec)
0.1
i
Ipsi
,
h'''''llillllllll
~si
IlrllllliTi
4°~1 ° ~ 30 20 0
Contra R,~L
f--~
& d., JL.,,Jlk~.. ,..,,.,,!1
0
o
00
- -
I c;~:r: lif,,i,,,,ij
i Contra
'l'''''''''l
-0.1 0 Time (sec)
0.1
106
Conclusions Although the classic feedforward model of touch has provided a fundamental blueprint for the development of somatosensory research in the last five decades, a variety of experimental findings and theoretical arguments demonstrate that this model no longer offers an accurate description on how tactile perception emerges in the mammalian brain. Instead, anatomical, physiological, and computational arguments favor the hypothesis that tactile perception emerges through interactive and recurrent interactions between multiple cortical and subcortical levels that define the mammalian somatosensory system. Central to this recurrent model of touch is the experimental demonstration that the massive corticofugal projections, that originate in the neocortex and reach most of the subcortical structures that form the somatosensory system, may play as relevant a role in tactile information processing as the parallel feedforward pathways of this system.
Acknowledgements This chapter describes research supported by grants from DARPA-ONR (N00014-98-1-0676), NSF IBN99-80043, and NIH DE-11121-01 to M.A.L.N. and an NRSA (1 F31 MH12570-01A1)to M.S.
References Adams, N.C., Lozsadi, D.A. and Guillery, R.W. (1997) Complexities in the thalamocortical and corticothalamic pathways. Eur. J. Neurosci., 9: 204-209. Albe-Fessard, D., Condes-Lara, M., Kesar, S. and Sanderson, P. (1983) Tonic cortical controls acting on spontaneous and evoked thalamic activity. In: G. Macchi, A. Rustioni and R. Spreatico (Eds.), Somatosensory Integration in the Thalamus: a Reevaluation Based on New Methodological Approaches. Elsevier, Amsterdam. Anderson, P., Eccles, J.C. and Sears, T.A. (1964) Cortically evoked depolarization of primary afferent fibers in the spinal cord. J. NeurophysioL, 27: 63-77. Andersen, P., Junge, K. and Sveen, O. (1967) Cortico-thalamic facilitation of somatosensory impulses. Nature, 214(92): 1011-1012. Andersen, P., Junge, K. and Sveen, O. (1972) Cortifugal facilitation of thalamic transmission. Brain Behav. Evol., 6: 170184. Barneoud, P., Gyger, M., Andres, E and Van der Lots, H. (1991)
Vibrissa-related behavior in mice: transient effect of ablation of the barrel cortex. Behav. Brain Res., 44: 87-99. Bishop, C.M. (1995) Neural Networks for Pattern Recognition. Clarendon Press, Oxford. Bourassa, J., Pinault, D. and Deschenes, M. (1995) Corticothalamic projections from the cortical barrel field to the somatosensory thalamus in rats: a single-fibre study using biocytin as an anterograde tracer. Eur. J. Neurosci., 7(1): 19-30. Brecht, M., Preilowski, B. and Merzenich, M.M. (1997) Functional architecture of the mystacial vibrissae. Behav. Brain Res., 84: 81-97. Brumberg, J.C., Pinto, D.J. and Simons, D.J. (1996) Spatial gradients and inhibitory summation in the rat whisker barrel system. J. Neurophysiol., 76(1): 130-140. Carvell, G.E. and Simons, D.J. (1990) Biometric analyses of vibrissal tactile discrimination in the rat. J. Neurosci., 10: 2638-2648. Castro-Alamancos, M.A. and Connors, B.W. (1996a) Short-term synaptic enhancement and long-term potentiation in neocortex. Proc. Natl. Acad. Sci. USA, 93(February): 1335-1339. Castro-Alamancos, M.A. and Connors, B.W. (1996b) Spatiotemporal properties of short-term plasticity in sensorimotor thalamocortical pathway of the rat. J. Neurosci., 16: 2767-2779. Cauller, L.J., Clancy, B. and Connors, B.W. (1998) Backward cortical projections to primary somatosensory cortex in rats extend long horizontal axons in layer I. J. Comp. Neurol., 390: 297-310. Chapin, J.K. and Woodward, D.J. (1981) Modulation of sensory responsiveness of single somatosensory cortical cells during movement and arousal behaviors. Exp. Neurol., 72(1): 164178. Chapin, J.K. and Woodward, D.J. (1982a) Somatic sensory transmission to the cortex during movement: gating of single cell responses to touch. Exp. NeuroL, 78(3): 654-669. Chapin, J.K. and Woodward, D.J. (1982b) Somatic sensory transmission to the cortex during movement: phasic modulation over the locomotor step cycle. Exp. Neurol., 78(3): 670-684. Chapin, J.K., Sadeq, M. and Guise, J.L.U. (1987) Corticocortical connections within the primary somatosensory cortex of the rat. J. Comp. Neurol., 263: 326-346. Chapman, C.E., Jiang, W. and Lamarre, Y. (1988) Modulation of lemniscal input during conditioned arm movements in the monkey. Exp. Brain Res., 72(2): 316-334. Chiaia, N.L., Rhoades, R.W., Bennett-Clarke, C.A., Fish, S.E. and Killackey, H.P. (1991) Thalamic processing of vibrissal information in the rat. I. Afferent input to the medial ventral posterior and posterior nuclei. J. Comp. Neurol., 314(2): 201216. Chmielowska, J., Carvell, G.E. and Simons, D.J. (1989) Spatial organization of thalamocortical and corticothalamic projection systems in the rat SmI barrel cortex. J. Comp. Neurol., 285: 325-338. Cohen, L.G. and Start, A. (1987) Localization, timing and specificity of gating of somatosensory evoked potentials during active movement in man. Brain, 110(Pt 2): 451-467. Contreras, D., Destexhe, A., Sejnowski, T.J. and Steriade, M. (1996) Control of spatiotemporal coherence of a thalamic
107
oscillation by corticothalamic feedback. Science, 274(5288): 771-774. Coquery, J.M. (1971) Changes in somaesthetic evoked potentials during movement. Brain Res., 31(2): 375. Coulter, J.D. (1974) Sensory transmission through lemniscal pathway during voluntary movement in the cat. J. Neurophysiol., 37(5): 831-845. Deadwyler, S.A. and Hampson, R.E. (1997) The significance of neural ensemble codes during behavior and cognition. Annu. Rev. Neurosci., 20: 217-244. Deschenes, M., Bourassa, J. and Pinault, D. (1994) Corticothalamic projections from layer V ceils in rat are collaterals of long-range corticofugal axons. Brain Res., 664(1-2): 215-219. Deschenes, M., Veinante, P. and Zhang, Z.W. (1998) The organization of corticothalamic projections: reciprocity versus parity. Brain Res. Rev., 28(3): 286-308. Diamond, M.E., Armstrong-James, M., Budway, M.J. and Ebner, F.F. (1992) Somatic sensory responses in the rostral sector of the posterior group (POm) and in the ventral posterior medial nucleus (VPM) of the rat thalamus: dependence on the barrel field cortex. £ Comp. Neurol., 319: 66-84. Dykes, R.W. (1983) Parallel processing of somatosensory information: a theory. Brain Res. Rev., 6: 47-115. Erickson, R.P. (1968) Stimulus coding in topographic and nontopographic afferent modalities: on the significance of the activity of individual sensory neurons. Psychol. Rev., 75(6): 447-465. Erickson, R.P. (1986) A neural metric. Neurosci. Biobehav. Rev., 10: 377-386. Fabri, M. and Burton, H. (1991) Ipsilateral cortical connections of primary somatic sensory cortex in rats. J. Comp. Neurol., 311: 405-424. Faggin, B.M., Nguyen, K.T. and Nicolelis, M.A. (1997) Immediate and simultaneous sensory reorganization at cortical and subcortical levels of the somatosensory system. Proc. Natl. Acad. Sci. USA, 94(17): 9428-9433. Fanselow, E. and Nicolelis, M. (1999) Behavioral modulation of tactile responses in the rat somatosensory system. J. Neurosci., 19: 7603-7616. Fields, H.L. and Heinricher, M.M. (1985) Anatomy and physiology of a nociceptive modulatory system. Phil. Trans. R. Soc. Lond. B, 308: 361-374. Gastaut, H. (1952) Etude electrocorticographique de la reativite des rhytmes rolandiques. Rev. Neurol. (Paris), 87: 176-182. Georgopoulos, A.P., Swartz, A.B. and Ketter, R.E. (1986) Neuronal population coding of movement direction. Science, 233: 1416-1419. Ghazanfar, A.A. and Nicolelis, M.A.L. (1997) Nonlinear processing of tactile information in the thalamocortical loop. J. Neurophysiol., 78(1): 506-510. Ghazanfar, A.A. and Nicolelis, M.A.L. (1999) Spatiotemporal properties of layer V neurons of the rat primary somatosensory cortex. Cereb. Cortex, 9: 348-361. Ghazanfar, A.A., Krupa, D.J. and Nicolelis, M.A.L. (1997) Tactile processing by thalamic neural ensembles: the role of cortical feedback. Soc. Neurosci. Abstr., 1797. Ghez, C. and Lenzi, G.L. (1971) Modulation of sensory trans-
mission in cat lemniscal system during voluntary movement. Pflugers Arch. Eur. £ Physiol., 323(3): 273-278. Grossberg, S. (1976) Adaptive pattern classification and universal recording: II Feedback, expectation, olfaction, illusions. Biol. Cybern., 23: 187-202. Grossberg, S. (1988) Nonlinear neural networks: principles, mechanisms and architectures. Neural Networks, 1: 17-61. Grossberg, S. (1999) The link between brain, learning, attention, and consciouness. Consciousness Cognit., 8: 1-44. Guic-Robles, E., Valdivieso, C. and Guajardo, G. (1989) Rats can learn a roughness discrimination using only their vibrissal system. Behav. Brain Res., 31(3): 285-289. Hebb, D.O. (1949) The Organization of Behavior: a Neuropsychological Theory. John Wiley and Sons, New York. Hellweg, EC., Schultz, W. and Creutzfeldt, O.D. (1977) Extracellular and intracellular recordings from cat's cortical whisker projection area: thalamocortical response transformation. J. Neurophysiol., 40(3): 463-479. Hoogland, P.V., Welker, E. and Van der Loos, H. (1987) Organization of the projections from barrel cortex to thalamus in mice studied with Phaseohis vulgaris-leucoagglutinin and HRP. Exp. Brain Res., 68: 73-87. Hoogland, P.V., Welker, E., Van der Loos, H. and Wouterlood, EG. (1988) The organization and structure of the thalamic afferents from the barrel cortex in the mouse; a PHA-L study. In: M. Bentivoglio and R. Spreafico (Eds.), Cellular Thalamic Mechanisms. Elsevier Science, Amsterdam, pp. 151-161. Hoogland, P.V., Wouterlood, EG., Welker, E. and Van der Loos, H. (1991) Ultrastructure of giant and small thalamic terminals of cortical origin: a study of the projections from the barrel cortex in mice using Phaseolus vulgaris leuco-agglutinin (PHA-L). Exp. Brain Res., 87: 159-172. Hutson, K.A. and Masterton, R;B. (1986) The sensory contribution of a single vibrissa's cortical barrel. J. Neurophysiol., 56(4): 1196-1223. Jacquin, M.E, Chiaia, N.L., Haring, J.H. and Rhoades, R.W, (1990a) Intersubnucleus connections within the rat trigeminal brainstem complex. Somatosens. Motor Res., 7(4): 399-420. Jacquin, M.E, Wiegand, M.R. and Renehan, W.E. (1990b) Structure-function relationships in rat brain stem subnucleus interpolaris. VIII. Cortical inputs. J. Neurophysiol., 64(1): 327. Johnson, K.O., Hsiao, S.S. and Twombly, I.A. (1995) Neural mechanisms of tactile form recognition. In: M.S. Gazzaniga (Ed.), The Cognitive Neurosciences. MIT Press, Cambridge, MA, pp. 253-267. Kaas, J.H. (1990) Somatosensory system. In: G. Paxinos (Ed.), The Human Nervous System. Academic Press, San Diego, pp. 813-844. Kaas, J.H. and Pons, T.P. (1988) The somatosensory system of primates. In: H.D. Steklis and J. Erwin (Eds.), Comparative Primate Biology, Vol. 4. Alan R. Liss, New York, pp. 421468. Kaas, J.H., Merzenich, M.M. and Killackey, H.P. (1983) The organization of the somatosensory cortex following peripheral nerve damage in adult and developing mammals. Annu. Rev. Neurosci., 6: 325-356.
108 Killackey, H.P. (1973) Anatomical evidence for cortical subdivisions based on vertically discrete thalamic projections from the ventral posterior nucleus to cortical barrels in the rat. Brain Res., 51: 326-331. Kleinfeld, D. and Delaney, K.R. (1996) Distributed representation of vibrissa movement in the upper layers of somatosensory cortex revealed with voltage-sensitive dyes [published erratum appears in J. Comp. NeuroL (1997) 378(4):594]. J. Comp. Neurol. , 375(1): 89-108. Koralek, K.A., Olavarria, J. and Killackey, H.P. (1990) Areal and laminar organization of corticocortical projections in the rat somatosensory cortex. J. Comp. Neurol., 299: 135-150. Krnpa, D.J., Ghazanfar, A.A. and Nicolelis, M.A.L. (1999) Immediate thalamic sensory plasticity depends on corticothalamic feedback. Proc. Natl. Acad. Sci. USA, 96: 8200-8205. Lee, R.G. and White, D.G. (1974) Modification of the human somatosensory evoked response during voluntary movement. Electroencephalogr. Clin. NeurophysioL, 36(1): 53-62. Lin, C.S., Nicolelis, M.A., Schneider, J.S. and Chapin, J.K. (1990) A major direct GABAergic pathway from zona incerta to neocortex [see comments]. Science, 248(4962): 1553-1556. Lu, S.-H. and Lin, R.C.S. (1993) Thalamic afferents of the rat barrel cortex: a light- and electron-microscopic study using Phaseolus vulgaris Leucoagglutinin as an anterograde tracer. Somatosens. Motor Res., 10: 1-16. Lu, S.M. and Lin, C.S. (1986) Cortical projection patterns of the medial division of the nucleus posterior thalami in the rat. Soc. Neurosci. Abstr., 12: 1434. Masino, S.A. and Frostig, R.D. (1996) Quantitative long-term imaging of the functional representation of a whisker in rat barrel cortex. Proc. Natl. Acad. Sci. USA, 93: 4942-4947. McCormick, D.A. and Pape, H.-C. (1990) Noradrenergic and serotonergic modulation of a hyperpolarization-activated cation current in thalamic relay neurones, J. Physiol., 431: 319-342. McCormick, D.A. and von Krosigk, M. (1992) Corticothalamic activation modulates thalamic firing through glutamate metabotropic receptors. Proc. Natl. Acad. Sci. USA, 89: 2774. McLean, J. and Waterhouse, B.D. (1994) Noradrenergic modulation of cat area 17 neuronal responses to moving visual stimuli. Brain Res., 667: 83-97. Merzenich, M.M., Kaas, J.H., Wall, J.T., Nelson, R.J., Sur, M. and Felleman, D.J. (1983) Topographic reorganization of somatosensory cortical areas 3b and 1 in adult monkeys following restricted deafferentation. Neuroscience, 8(1): 33-55. Moore, C.I. and Nelson, S.B. (1998) Spatio-temporal subthreshold receptive fields in the vibrissa representation of rat primary somatosensory cortex. J. Neurophysiol., 80(6): 2882-2892. Mountcastle, V. (1957) Modality and topographic properties of single neurons of cats' somatic sensory cortex. J. Neurophysiol., 20: 408-434. Mountcastle, V. (1974). Neural mechanisms in somesthesia. In: V. Mountcastle (Ed.), Medical Physiology, Vol. I. C.V. Mosby, St. Louis, pp. 307-347. Mumford, D. (1991) On the computational architecture of the neocortex. I. The role of the thalamo-cortical loop. Biol. Cybern., 65(2): 135-145.
Mumford, D. (1992) On the computational architecture of the neocortex. II. the role of corticocortical loops. Biol. Cybern., 66: 241-251. Mumford, D. (1994). Neuronal Architectures for pattern-theoretic problems. In: D. Koch (Ed.), Large-Scale Neuronal Theories of the Brain. MIT Press, Cambridge, MA, pp. 125152. Nelson, R.J. (1984) Responsiveness of monkey primary somatosensory cortical neurons to peripheral stimulation depends on 'motor-set'. Brain Res., 304(1): 143-148. Nelson, R.J. (1987) Activity of monkey primary somatosensory cortical neurons changes prior to active movement. Brain Res., 406(1-2): 402--407. Nicolelis, M.A.L. (1996) Beyond maps: a dynamic view of the somatosensory system. Braz. J. Med. BioL Res., 29: 401--412. Nicolelis, M.A. and Chapin, J.K. (1994) Spatiotemporal structure of somatosensory responses of many-neuron ensembles in the rat ventral posterior medial nucleus of the thalamus. J. Neurosci., 14(6): 3511-3532. Nicolelis, M.A.L., Chapin, J.K. and Lin, R.C.S. (1991) Ontogeny of corticocortical projections of the rat somatosensory cortex. Somatosens. Motor Res., 8(3): 193-200. Nicolelis, M.A., Lin, R.C., Woodward, D.J. and Chapin, J.K. (1993a) Dynamic and distributed properties of many-neuron ensembles in the ventral posterior medial thalamus of awake rats. Proc. Natl. Acad. Sci. USA, 90(6): 2212-2216. Nicolelis, M.A.L., Lin, R.C.S., Woodward, D.J. and Chapin, J.K. (1993b) Induction of immediate spatiotemporal changes in thalamic networks by peripheral block of ascending cutaneous information. Nature, 361: 533-536. Nicolelis, M.A., Baccala, L.A., Lin, R.C. and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system. Science, 268(5215): 1353-1358. Nicolelis, M.A.L., Oliveira, L.M.O., Lin, R.C.S. and Chapin, J.K. (1996) Active tactile exploration influences the functional maturation of the somatosensory system. J. Neurophysiol., 75: 2192-2196. Nicolelis, M.A., Fanselow, E.E. and Ghazanfar, A.A. (1997) Hebb's dream: the resurgence of cell assemblies. Neuron, 19(2): 219-221. Nicolelis, M,A., Katz, D. and Krupa, D.J. (1998a) Potential circuit mechanisms underlying concurrent thalamic and cortical plasticity. Rev. Neurosci., 9(3): 213-224. Nicolelis, M.A.L., Ghazanfar, A.A., Stambaugh, C.R., Oliveira, L.M.O., Laubach, M., Chapin, J.K., Nelson, R.J. and Kaas, J.H. (1998b) Simultaneous encoding of tactile information by three primate cortical areas. Nature Neurosci., 1(7): 621-630. Niedermeyer, E. (1993) The normal EEG of the waking adult. In: E. Niedermeyed and E Lopez da Silva (Eds.), Electroencephalography. Basic Principles, Clinical Applications and Related Fields. Williams and Wilkins, Baltimore, MD, pp. 131-152. Ogden, T.E. (1960) Cortical control of thalamic somato-sensory relay nuclei. Electroenceph. Clin. Neurophysiol., 12: 621-634. Olavarria, J.F., DeYoe, E.A., Knierim, J.J., Fox, J.M. and Van Essen, D.C. (1992) Neuronal responses to visual texture pat-
109
terns in middle temporal area of the macaque monkey. J. Neurophysiol., 68(1): 164-181. Parker, J. and Dostrovsky, J. (1999) Cortical involvement in the induction, but not expression, of thalamic plasticity. J. Neurosci., 19: 8623-8629. Pazos, A.J., Orezzoli, S.L., McCabe, P.M., Dietrich, W.D. and Green, E.J. (1995) Recovery of vibrissae-dependent behavioral responses following barrelfield damage is not dependent upon the remaining somatosensory cortical tissue. Brain Res., 689(2): 224-232. Pidoux, B. and Verley, R. (1979) Projections on the cortical somatic I barrel subfield from ipsilateral virbrissae in adult rodents. Electroencephalogr. Clin. Neurophysiol., 46: 715726. Pinault, D. and Deschenes, M. (1998) Projection and innervation patterns of individual thalamic reticular axons in the thalamus of the adult rat: a three-dimensional, graphic and morphometric analysis. J. Comp. NeuroL, 391(2): 180-203. Pinanlt, D., Bourassa, J. and Deschenes, M. (1995) The axonal arborization of single thalamic reticular neurons in the somatosensory thalamus of the rat. Eur. J. Neurosci., 7(1): 3140. Pinault, D., Smith, Y. and Deschenes, M. (1997) Dendrodendritic and axoaxonic synapses in the thalamic reticular nucleus of the adult rat. J. Neurosci., 17(9): 3215-3233. Polley, D.B., Chen-Bee, C.H. and Frostig, R.D. (1999) Varying the degree of single-whisker stimulation differentially affects phases of intrinsic signals in rat barrel cortex. J. Neurophysiol., 81(2): 692-701. Purves, D., Riddle, D.R. and LaMantia, A.-S. (1992) Iterated patterns of brain circuitry (or how the cortex gets its spots). Trends Neurosci., 15(10): 362-368. Rhoades, R.W., Belford, G.R. and Killackey, H.P. (1987) Receptive-field properties of rat ventral posterior medial neurons before and after selective kainic acid lesions of the trigeminal brain stem complex. J. Neurophysiol., 57(5): 1577-1600. Salt, T.E. and Eaton, S.A. (1996) Functions of ionotropic and metabotropic glutamate receptors in sensory transmission in the mammalian thalamus. Prog. NeurobioL, 48: 55-72. Salt, T.E. and Turner, J.P. (1998) Modulation of sensory inhibition in the ventrobasal thalamus via activation of group II metabotropic glutamate receptors by 2R,4R-aminopyrrolidine-2,4-dicarboxylate. Exp. Brain Res., 121(2): 181-185. Schmidt, R.E, Schady, W.J. and Torebjork, H.E. (1990a) Gating of tactile input from the hand. I. Effects of finger movement. Exp. Brain Res., 79(1): 97-102. Schmidt, R.E, Torebjork, H.E. and Schady, W.J. (1990b) Gating of tactile input from the hand. II. Effects of remote movements and anaesthesia. Exp. Brain Res., 79(1): 103-108. Sejnowski, T.J., Koch, C. and Churchland, P.S. (1988). Comput. Neurosci. Sci., 241: 1299-1306. Sherman, S.M. and Guillery, R.W. (1996) Functional organization of thalamocortical relays. J. Neurophysiol., 76(3): 13671395. Shigemi, S., Ichikawa, T., Akasaki, T. and Sato, H. (1999) Temporal characteristics of response integration evoked by
multiple whisker stimulations in the barrel cortex of rats. J. Neurosci., 19: 10164-10175. Shin, H.C. and Chapin, J.K. (1989) Mapping the effects of motor cortex stimulation on single neurons in the dorsal column nuclei in the rat: direct responses and afferent modulation. Brain Res. Bull., 22(2): 245-252. Shin, H.C. and Chapin, J.K. (1990a) Modulation of afferent transmission to single neurons in the ventroposterior thalamus during movement in rats. Neurosci. Lett., 108(1-2): 116-120. Shin, H.C. and Chapin, J.K. (1990b) Movement induced modulation of afferent transmission to single neurons in the ventroposterior thalamus and somatosensory cortex in rat. Exp. Brain Res., 81(3): 515-522. Shin, H.-C. and Chapin, J.K. (1990c) Mapping the effects of SI cortex stimulation on somatosensory relay neurons in the rat thalamus: direct responses and afferent modulation. Somatosens. Motor Res., 7: 421--434. Shuler, M.G., Krupa, D.J. and Nicolelis, M,A.L. (2000) Discrimination of bilateral whisker stimuli in the freely behaving rat.
Soc. Neurosci. Abstr. Shuler, M.G., Krupa, D.J. and Nicolelis, M.A.L. (2001) Bilateral integration of whisker information in the primary somatosensory cortex of rats, submitted for publication. Simons, D.J. (1985) Temporal and spatial integration in the rat SI vibrissa cortex. J. Neurophysiol., 54(3): 615-635. Simons, D.J. and Carvell, G.E. (1989) Thalamocortical response transformation in the rat vibrissa/barrel system. J. Neurophysiol., 61(2): 311-330. Turner, J.E and Salt, T.E. (1998) Characterization of sensory and corticothalamic excitatory inputs to rat thalamocortical neurones in vitro. J. Physiol. (Lond.), 510(Pt 3): 829-843. Turner, J.E and Salt, T.E. (1999). Group III metabotropic glutamate receptors control corticothalamic synaptic transmission in the rat thalamus in vitro [In Process Citation]. J. Physiol. (Lond.), 519 Pt 2, 481-491. Van dcr Loos, H. (1976) Barreloids in mouse somatosensory thalamus. Neurosci. Lett., 2: 1-6. Veinante, E and Deschenes, M. (1999) Single- and multi-whisker channels in the ascending projections from the principal trigeminal nucleus in the rat. J. Neurosci., 19(12): 50855095. Waller, H.J. and Feldman, S.M. (1967) Somatosensory thalamic neurons: effects of cortical depression. Science, 157: 10741077. Waterhouse, B.D., Border, B., Wahl, L. and Mihalloff, G.A. (1994) Topographic organization of rat locus coeruleus and dorsal raphe nuclei: distribution of cells projecting to visual system structures. J. Comp. Neurol., 336: 345-361. White, E. and DeAmicis, R. (1977) Afferent and efferent projections of the region in mouse SmI cortex which contains the posteromedial barrel subfield. J. Comp. NeuroL, 175: 455482. Woolsey, T.A. and Van der Loos, H. (1970) The structural organization of layer IV in the somatosensory region (SI) of mouse cerebral cortex: the description of a cortical field composed of discrete cytoarchitectonic units. Brain Res., 17: 205-242.
110
Yuan, B., Morrow, T.J. and Casey, K.L. (1985) Responsiveness of ventrobasal thalamic neurons after suppression of SI cortex in the anesthetized rat. J. Neurosci., 5: 2971-2978. Yuan, B., Morrow, T.J. and Casey, K.L. (1986) Cortifugal influences of S 1 cortex on ventrobasal thalamic neurons in the
awake rat. J. Neurosci., 6:3611-3617. Zhang, Z.-W. and Deschenes, M. (1998) Projections to layer VI of the postermedial barrel field in the rat: a reappraisal of the role of corticothalamic pathways. Cereb. Cortex, 8: 428-436.
M.A.L. Nicolelis (Ed.)
Progress in Brain Research, Vol. 130 © 2001 Elsevier Science B.V. All rights reserved
CHAPTER 8
Synchronization and assembly formation in the visual cortex Winrich A. Freiwald 1,2,,, Andreas K. Kreiter 1 and Wolf Singer 2 1 Institute for Brain Research, University of Bremen, FB2, P.O. Box 330440, D-28334 Bremen, Germany 2 Max-Planck-lnstitutefor Brain Research, Deutschordenstrasse 46, D-60528 Frankfurt~Main, Germany
Neural assemblies The main challenge for information processing by the brain's sensory systems is the complexity of natural environments. This complexity is of a combinatorial nature: while elementary features at very different levels of physical organization reappear and may be quite limited in number, the space of possible feature combinations is virtually unlimited. Since the unlikely, the new and even the physically impossible can be seen with ease, the coding schemes used by the visual system are adapted to provide representations for all possible feature constellations and not only likely ones. For this reason, covariance of feature appearance (e.g. the co-occurrence of the color green and the form of leaves) does not reduce the principal demands for visual information processing. What mechanisms then does the visual system employ to cope with combinatorial complexity? Donald Hebb's cell assembly concept (Hebb, 1949) may be conceived as the proposal to let the problem be the solution. In the same way as natural outside world objects are composed of elementary features, internal neural representations are generated by combinations of elementary neural responses. The presence of a feature is signaled by the * Corresponding author: Wim'ich A. Freiwald, Institute for Brain Research, University of Bremen, FB2, P.O. Box 330440, D-28334 Bremen, Germany. Tel.: +49-421218-9095; Fax: +49-421-218-9004; E-mail:
[email protected] activity of a specific neuron or a set of neurons, the presence of an object whole is signaled by a spatially distributed and cooperatively interacting population of neurons, termed a cell assembly. Thus, while neurons are outnumbered by possible feature constellations (Sejnowski, 1986), cell assemblies are not, because they are of combinatorial nature. For this theoretical reason, single 'gnostic units' at the tip of a processing hierarchy (Konorski, 1967) can hardly be the only neural correlate of object representations (Sherrington, 1941; Barlow, 1972b; Harris, 1980), even though they could serve to encode frequently occurring highly familiar constellations of features (Miyashita, 1988; Sakai and Miyashita, 1994; Vogels and Orban, 1994; Logothetis et al., 1995; Kobatake et al., 1998; Gauthier et al., 1999). The combinatorial nature of neural assemblies yields two more desirable coding properties. First, even though assembly coding seems energetically expensive in terms of the number of active neurons, it is economical in terms of the overall number of neurons necessary (Hinton, 1981; Field, 1994), because neurons can be used in different stimulus contexts by recombination with other neurons to form assemblies. Second, this coding scheme is very flexible, because variations in stimulus constellations are met by variations of neural activity patterns. By the same token, newly encountered stimuli can be represented by new patterns of already existing building blocks (Braitenberg, 1978; Grossberg, 1980; Edelman, 1987; Gerstein et al., 1989; Engel et al., 1997). Furthermore, cooperativity, the second
112 key feature of assembly codes, allows for pattern completion of only partly visible stimuli, generalization over similar stimuli and fault tolerance (Palm, 1982; Rumelhart and McClelland, 1986; Hopfield and Tank, 1991). Finally, assembly codes are reliable due to redundancy of neural population responses. Since neural systems can adapt to the statistics of different environments and behavioral demands, very different aspects of an object may serve as elementary features. Yet commonly co-occurring features appear to be represented by hard-wired neural responses that are then grouped together with other responses representing different features. Thus, the concept of assembly coding is open to incorporate diverse response selectivities of neurons. However, combinations of highly abstract cardinal neurons alone (Barlow, 1972a) are insufficient for the representation of the detailed structure of natural objects (Kreiter and Singer, 1996a). Distributed coding in the visual cortex The quintessential principle of assembly coding is the generation of complex internal activity patterns for the representation of outside world objects. Internal representational complexity is already reflected in the structural organization of the visual cortex (Tononi et al., 1994), which has been revealed by anatomical and more recently functional imaging techniques. More than 30 cortical areas have been described in, the macaque visual system (Livingstone and Hubel, 1988; Zeki and Shipp, 1988; Felleman and van Essen, 1991; Bullier and Nowak, 1995), a number likely to be paralleled by other primate species, including humans. Visual areas are typically further subdivided into smaller compartments (Kaas and Krubitzer, 1991; Krubitzer, 1995). This structural segmentation is accompanied by functional specialization. Response properties of neurons within areas and in areal subcompartments are more similar than responses of cells in different compartments. Thus, the many different compartments operate as functionally specialized modules for the analysis of the visual scene. Every visual stimulus will therefore activate modules in very different parts of the visual cortex (Zeki, 1993), leading to a largely distributed representation, as documented by functional imaging studies. Distributed representations also exist within
each module, because most neurons respond like broadly tuned filters (van Essen et al., 1992; Martin, 1994), and thus, many neurons will be activated by any feature relevant for a particular module. While these considerations show that responses are distributed within the visual system, they do not by themselves provide evidence for true parallel processing within the visual system. This conclusion can however be reached by considering the anatomical coupling of cortical areas. Based on laminar patterns of inter-areal connections that correlate with inputoutput functions (Rockland and Pandya, 1979), the various cortical areas can be placed into hierarchical schemes (Felleman and van Essen, 1991; Hilgetag et al., 1996; Crick and Koch, 1998), but several cortical areas have to be placed at every level of the hierarchy. Thus, visual information is processed in parallel at every level of the hierarchy including its final stages. The existence of rich and typically reciprocal coupling between areas, including feedforward, lateral and feed-back connections (Lamme et al., 1998), further suggests that this processing is not accomplished independently in each cortical area, but rather through cooperative interactions. In agreement with this view, strong effects of feed-back connections on primary visual cortex responses have been described recently (Hup6 et at., 1998). Distributed coding and neural interactions For the reasons discussed in the previous paragraphs, the concept of distributed coding has become almost commonplace. However, less agreement exists on the properties of these distributed representations. Issues in question are the population size, with a scale of possibilities ranging from coarse coding (see e.g. Rumelhart and McClelland, 1986; Eurich and Schwegler, 1997) to sparse coding (see e.g. Young and Yamane, 1992) and the strategies used for stimulus encoding and decoding of population activity (Churchland and Sejnowski, 1992; Snippe and Koenderink, 1992; Sanger, 1996; Brown et al., 1998; Zhang et al., 1998; Eurich and Wilke, 1999; Oram et al., 1999; Zhang and Sejnowski, 1999). This second issue raises the question of how population signals are internally organized. As stated above, a central aspect of the Hebbian cell assembly concept is the cooperativity of its members. These
113 interactions of neurons result in temporally correlated activity (Aertsen et al., 1986; Johannesma et al., 1986; Gerstein et al., 1989; Singer et al., 1990, 1997), and for this reason synchronous firing of neurons has been considered as an operational definition for the experimental detection of cell assemblies. In contrast, influential models of population coding, especially vector coding (see e.g. Georgopoulos, 1990) have assumed independence of cell firing. The rationale of these concepts is the Ergodic hypothesis, namely that the nervous system can take population averages over neurons to achieve reliable stimulus coding in the same way as the experimenter equipped with a single electrode can determine the single-neuron's exact response property by averaging over trials of stimulus presentations (Gerstein and Gochin, 1992). Within this framework, covariation of neural firing has been conceived as a limiting factor for effective sampling over populations due to the introduction of redundancies (Britten et al., 1992; Zohary et al., 1994). Contrary to this point of view, however, the potential usefulness of temporally coordinated activity for information processing by neural populations has recently been demonstrated by both theoretical arguments and experimental findings. First, information theoretical considerations have shown that correlations of neural firing, both stimulus-dependent and stimulus-independent ones, can markedly enhance the accuracy of population responses, increase information rates and therefore allow for increased stimulus discriminability (Snippe and Koenderink, 1992; Richmond and Gawne, 1998; Abbott and Dayan, 1999; t r a m et al., 1999; Panzeri et al., 1999). Thus, synchrony does not necessarily imply redundancy for stimulus encoding but can be the source of synergy. Second, synergy by synchrony has been found experimentally in the visual system. Synchronous spike pairs can carry extra information not available from the individual spikes elicited by two neurons independently. This finding was obtained for visual cortical and retinal cell pairs by demonstrating the difference between the receptive fields (RFs) of the individual neurons and the socalled 'bicellular receptive fields' reconstructed from synchronous events (Ghose et al., 1994; Meister et al., 1995; Meister, 1996) and by an information theoretic analysis of neural pairs in the lateral geniculate
nucleus of the thalamus (Dan et al., 1998). Third, in visual, auditory, motor and premotor cortex and hippocampus, changes of synchronization in relation to external or internal events have been observed without concomitant changes of firing rates (Vaadia et al., 1995; deCharms and Merzenich, 1996; Kreiter and Singer, 1996b; Riehle et al., 1997; Sakurai, 1999), indicating that information about these events was available to the nervous system by the relative timing of spikes but not by activity levels. In the motor cortex, in addition, it has been shown that extra information about the direction of a forthcoming arm movement was provided by the synchronous firing of cells beyond that available from the rates or rate changes (Hatsopoulos et al., 1999; Maynard et al., 1999). Taken together, these theoretical arguments and experimental findings show that coordinated neural interactions, a key element of the cell assembly concept, may indeed play an important role in neural population coding.
The temporal binding hypothesis If correlated activity can be the signature of neural assemblies, it may be essential in the context of multiple object encoding (vonder Malsburg, 1981; von der Malsburg and Schneider, 1986; Singer, 1990a, 1993). The basic argument, illustrated in Fig. 1, is the following. Assume for the moment, a cell assembly in the visual cortex is solely defined as a population of cells activated by an object in the environment. The several objects which are typically contained within a natural visual scene, will therefore activate several assemblies. Since, by definition, assemblies are distributed representations, i.e. are not confined to a specific cortical locus, they will superpose in cortical space. This is especially evident for objects in close proximity or with partial overlap in visual space since their representations will even overlap within processing modules with precise topographical mapping and high spatial resolution, e.g. within areas V1 and V2, even more so due to cells' non-classical RF properties (see e.g. vonder Heydt et al., 1984, Allman et al., 1985 and Lee et al., 1998). This overlap of multiple representations (A, B, C . . . . ) is a serious challenge for all distributed coding schemes, coined the 'superposition problem', because any part of a full representation
114
÷°-) ~ Oc~~i
II ,, ,II I
Assembly
i
Fig. 1. Schematic illustration of multiple-object encoding by neural assemblies. The visual scene considered here contains two complex objects, a mouse and an apple. Each object is represented by a distributed group of cells which are responding to the object's features. These two cell assemblies representing the two objects are shown in the middle of the diagram. For four cells, the receptive field properties are indicated. These are intended to match the moderate selectivity found for neurons within the inferotemporal cortex (Tanaka, 1996), showing that a distributed representation is also needed in this final stage of the ventral path processing hierarchy (Felleman and van Essen, 1991). Since the two objects overlap in visual space, the two assemblies are overlapping in cortical space. Due to this superposition, the membership relation of active cells is ambiguous. It is not clear which cells belong to the same and which belong to different cell assemblies. By the same token, the relationships between the represented features are lost. To avoid this so-called superposition catastrophe, a label is needed that uniquely identifies constituents of the same object and distinguishes between members of different assemblies. This label could be provided by the synchronous firing of neurons of the same assembly and asynchronous firing of cells in different assemblies (shown on the right-hand side). Extending the classical Hebbian definition of a cell assembly from mere activation to synchronous activity preserves the essential property of cooperativity of assemblies, because synchronization need not be provided by an external source, but can be brought about by neural groups in a self-organizing manner (see e.g.: yon der Malsburg and Schneider, 1986; Sompolinsky et al., 1990; Wang et al., 1990; Grossberg and Somers, 1991; K6nig and Schillen, 1991; Sporns et al., 1991; Neven and Aertsen, 1992; yon der Malsburg and Buhmann, 1992; Chawanya et al., 1993; Ritz et al., 1994; Schillen and K0nig, 1994; Kappen, 1997; Lumer et al., 1997; Eckhorn, 1999a).
A, say Ai, might as well belong to any other part of representations B, C . . . . as to other parts of A, Aj, Ak and so forth. Thus, the relationships between neurons may be confounded, and by the same token, the relationships between features in the outside world remain ambiguous. Not even the number of objects present can be deduced from the activity of neurons alone. Clearly, a mechanism is needed which unambiguously identifies the members of the same assembly and distinguishes members of different assemblies. A solution to this so-called 'binding problem' has been suggested by vonder Malsburg (1981, 1986), Abeles (1982a) and, in a preliminary form, by Milner (1974). According to this proposal, cells belonging to the same assembly fire action potentials synchronously with a precision of a few milliseconds, and cells belonging to different assemblies
fire asynchronously. This way, relationships between features are made explicit and multiple objects can be represented within the same cortical space. This concept, which we will refer to as the 'temporal binding hypothesis', is an attractive solution to the binding problem, because it preserves the essential features of assembly coding, which we have discussed above. Essentially, the notion of neural interactions in the Hebbian concept is made more explicit: synchrony with a precision of a few milliseconds is the label for assembly membership. The temporal binding hypothesis requires cortical neurons or microcircuits to act as coincidence detectors (Abeles, 1982b; Softky, 1994; K6nig et al., 1996), a proposal for which ample direct and indirect experimental evidence has accumulated in the last years (Alonso et al., 1996; Matsumara et al., 1996;
115 Castelo-Branco et al., 1998; Margulis and Tang, 1998; Prut et al., 1998; Rager and Singer, 1998; Stevens and Zador, 1998; Volgushev et al., 1998; Azouz and Gray, 1999; Larkum et al., 1999). Different alternative solutions to the binding problem have been suggested, which have been critically discussed in detail by Singer (1990b), Singer and Gray (1995), Kreiter and Singer (1996a), Engel et al. (1997) and Roelfsema (1998). Briefly, the proposal to introduce binding units, i.e. neurons which are selective for feature conjunctions, has to be rejected, because it re-introduces the problem which assembly codes had actually been designed to avoid, i.e. scaling neural numbers with the number of feature combinations. Mechanisms based on a spotlight of attention (Treisman, 1986, 1996; Olshausen et al., 1993) certainly help to distinguish neural responses to spatially separate objects, but do not provide a general solution to the binding problem, since multiple objects can be included within the spotlight. Furthermore, selective attention is based on the results of pre-attentive segregation and binding processes working in parallel within the whole visual field which by definition are not attentive. Thus, synchronous activity within the millisecond time range may be the signature of neural assembly formation. In this review, we will present experimental data from the mammalian visual cortex related to this proposal. Therefore, we do not aim to provide a full review of temporal coding within the visual system, but rather restrict the scope of this paper in several ways. First, since most of the results presented have been obtained using the cross-correlation technique (Gerstein and Perkel, 1969) (which is briefly explained in the legend of Fig. 3) and the Joint-PSTH (Aertsen et al., 1989), we will refer to the term synchrony as a significant excess amount of coincident events within a time window of a few milliseconds, as detected with these techniques and thereby avoid the problem of finding a general definition for synchrony (Aertsen and Arndt, 1993). Second, we will focus on temporally precise forms of spike time coordination. It should be noted, however, that in the visual cortex very different ranges of precision of synchronization have been found, ranging from a single millisecond to several hundreds of milliseconds (see e.g. Gochin et al., 1991, Nowak et al., 1995, 1999, Lampl et al., 1999 and Bair,
1999 for a review). All of these forms may be of functional relevance in the same way as firing rates within temporal intervals of different duration are. Third, we will concentrate on synchronization in the context of assembly formation, also recently termed emergent synchrony (Usrey and Reid, 1999). Other forms of synchrony in the visual system, caused by anatomical divergence, have recently been reviewed (Usrey and Reid, 1999). Finally, relative spike timing of neurons is but one form of temporal coding. Within the temporal binding hypothesis, it is not specified in which format a single neuron encodes features of an object. For the sake of simplicity and congruence with common belief, we may assume that a feature is encoded in the neuron's firing rate, but this does not necessarily have to be the case. Binding by synchrony is compatible with very different single-cell encoding schemes: a cell which preferentially responds to temporally structured input patterns, may itself use both a temporal code for representation of features and employ relative timing to express relationships between features. Thus, while at a conceptual level, single-cell and assembly codes may be treated independently, the necessity for coincidence detection by individual neurons or small neural circuits may indicate that these two issues are intimately intertwined at a mechanistic level. These are important issues, and we will therefore refer to three recent reviews on the general topic of temporal coding in the visual system (Bair, 1999; Gawne, 1999; Victor, 1999).
Evidence for spike synchronization in the visual cortex Temporally correlated activity of individual neural pairs within the visual cortex has been investigated in many laboratories starting in the early 1980s (Toyama et al., 1981a,b; Michalski et al., 1983; Ts'o et al., 1986; Volgushev, 1988; Aiple and Krtiger, 1988; Hata et al., 1988; Kriiger and Aiple, 1988; Ts'o and Gilbert, 1988; Gochin et al., 1991; Hata et al., 1991, 1993; Schwarz and Bolz, 1991; Liu et al., 1992; Roe and Ts'o, 1992; Aarnoutse et al., 1997; Albus et al., 1998; Alonso and Martinez, 1998; Molotchnikoff and Shumikhina, 1998; Shumikhina et al., 1998) most often with a motivation to reveal structural coupling between cells. This approach to
116 functional anatomy had been methodologically and conceptually outlined by Perkel et al. (1967), Moore et al. (1970), Kirkwood (1979), Aertsen and Gerstein (1985) and Surmeier and Weinberg (1985) and successfully applied to the invertebrate nervous system (Bryant et al., 1973). The occurrence of centered or shifted peaks and troughs in the cross-correlograms was interpreted as evidence for the presence of excitatory or inhibitory connections either directly connecting one cell to the other or providing common input to both recorded cells. The results of these many different studies taken together clearly demonstrated the presence of temporally precise synchronization within the cat and macaque visual cortex. While this conclusion is in general agreement with the temporal binding hypothesis, it remained to be shown that not only individual pairs of cells, but large groups of cells would fire synchronously in response to a visual stimulus.
Local synchronization and gamma oscillations Evidence has indeed been obtained that many neurons within a column of cat visual cortex can engage in a state of highly synchronous activity in response to an optimally oriented moving light bar (Gray and Singer, 1987; Gray et al., 1989). Oscillatory activity in the g a m m a frequency range was observed in both multi-unit activity (MUA), i.e. spike sequences elicited by clusters of neighboring cells which were not further subdivided into contributions of individual cells, and local field potential (LFP) signals, which m a y be thought of as a the local EEG signal of a cortical column (Fig. 2, and cf. Schillen et al., 1992). LFP oscillations can only be observed when many neurons fire in synchrony, since otherwise the individual neurons' electric fields would simply cancel out. Furthermore, the occurrence of high frequencies demonstrates that local synchrony is generated with high temporal precision, which is also indicated by the M U A responses. Thus, whole groups of neighboring neurons were shown to discharge synchronously in response to the same visual object, an important conclusion for any concept of assembly coding which could not have been reached by recordings from pairs of individual cells. Furthermore, this finding is in general agreement with the hypothesis that neighboring cells
r
r rI
12oo, @
T) = P ( x > T / a ) = ~ _l _ / of r ee e -x2/2 dx
where x = V / a , and e r f c is the complementary error function (equivalent to the dark shaded areas in
a) 0.45
0.30
"E 0.15
1 _e_V2 2o.2
p ( V ) = ~2rc---Y
/
(6) /
where V is measured relative to Vre~t, and a is the standard deviation of V. At any given time. the probability that V is above threshold T, is given by:
(7)
= erfc(T/~/2a)/2
/
/
0J -3 -2 0.5( ~ 0
b)
/
/
-1
0
V/(~
1
2
3
0.4 A
I,,,, 0.3 A
Fig. 4. Analysis of the firing probability of a neuron with a threshold, T; a membrane voltage, V. with standard deviation. ~7: and EPSPs of asynchronous inputs, V*A and VA,*. (a) Probability density function of membrane voltage. Abscissas: normalized membrane voltage, V/a, in dimensionless units. The area of the dark shaded, region, where V > T, is proportional to the firing rate of the neuron. The area of the light shaded region. where T - VA* < T, is proportional to the average efficacy of asynchronous inputs, A*, in driving the neuron. (b) The probability that the membrane voltage is greater than threshold versus normalized voltage, V/a. This is equivalent to the probability that one spike fires in any given 3 ms period (see text). Graph illustrates the ealcnlafion yielding coincidence advantage (Abeles, 1982). Horizontal line from T: baseline probability of firing. O, probability of firing (0.5) given 9 synchronous EPSPs of size ]/A.,*-X, cumulative additional probability of firing (above baseline) for 9 asynchronous EPSPs of size VA*..The coincidence advantage is the ratio: (O minus baseline)/(X minus baseline). (c) Same plot as (b), but over a narrower range. Graph illustrates calculation yielding synergy ratio (see text). O, probabifity of firing given 2 synchronous EPSPs of sizes VA* and VA,*. X, cumulative addit]ort~ pro~abilit,j of firing (above baseline), for two asynchronous EPSPS of sizes VA* and VA,*. Synergy ratio is the ratio: (O minns basetine)/(X minus baseline).
0. 0.2 0.1
~ . , T . V ~,
2
0
c)
0.1-
3
V/~
0.08 A
I~ 0.04-
0.02 O~
1.4
1.7
V/~
2
152 Fig. 4a; plotted in Fig. 4b,c). As argued by Abeles, the mean firing of a neuron under these conditions is approximately P ( V > T ) / A t , where At is the interval over which the neuron will tend to fire exactly one spike if V is above threshold. That is, At is somewhere between the absolute refractory period and the relative refractory period. For our purposes, we can take At = 3 ms. Although this is clearly an approximation, it allows T / a to be estimated as the unique value that satisfies: P ( x > T / a ) = erfc(T/~/2a)/2 = )~At
(8)
where )~ is the mean firing rate. Given this scenario, if a spike from presynaptic neuron A depolarizes the postsynaptic neuron by an amount VA, this shifts the distribution of V to the fight by VA. For our purposes, this is equivalent on average to shifting the threshold to T - VA. This increases the probability of firing by the marginal increase in area under the Gaussian distribution (Fig. 4a, light shaded region). This increase in area is clearly not a linear function of Va, as further illustrated in Fig. 4b,c. Although we cannot determine VA from an extracellular recording, we can determine the average efficacy of the synapse by cross-correlation. Again, the efficacy is the marginal increase in firing probability during the monosynaptic peak (again, defined as 3 ms for the thalamocortical synapse; Reid and Alonso, 1995; Alonso et al., 1996). Therefore, VA/a is the unique value such that: P ( x > T / a - gA/ff)
=
erfc(T/x/2a
- V A / V ~ a ) / 2 = XAt + efficacy
(9)
For the cortical neuron, B, illustrated in Fig. 3, the baseline firing rate was -~10 spikes/s and the efficacies of the non-synchronous inputs, A* and A'* (defined above: second-order analysis: interactions between inputs), were 1.71% and 1.41%, respectively (Alonso et al., 1996). Substituting these values into Eqs. 8 and 9 yields: T / a = 1.88, VA./a = 0.21, and VA,*/a = 0.18 (10) By assumption, the synchronous activation of both inputs yields a depolarization that is the sum of the two individual depolarizations: VA&A,/~ = 0.39
(11)
Finally, substituting this value into Eq. 9 yields a value of 3.8% for the predicted efficacy of the synchronous input: A&A'. Under the simple model of a Ganssian distribution of membrane potential and linear summation of synaptic inputs, therefore, one would have predicted a synergy ratio of: Synmodel -- 3.8%/(1.71% + 1.41%) = 1.22 (12) This point can be made graphically. The lines tangent to the curves in Fig. 4b,c indicate the expected efficacy of two or more inputs if efficacy added linearly. The concavity of the curve (defined in Eq. 7) assures that the actual efficacy is greater than the linear prediction. Given this relatively modest synergy ratio for two synchronous inputs, it is instructive to use the same model to examine the coincidence advantage, as defined by Abeles (1982), for a larger number of synchronous inputs. The coincidence advantage is defined in terms of the number of identical synaptic inputs required to overcome the threshold, T (i.e., equivalent to shifting the threshold all the way to zero in Fig. 4a). For the stronger of the two asynchronous inputs to B, A*, this number, termed the synchronous attenuation, is given by: As = T~ VA. = 9.0
(13)
Substituting this value into Eq. 9, we get a coincidence advantage, 'CA', equal to: CAmodel = 0.5(1/1.7%)/9.0 = 3.27
(14)
This means that the net effect of nine synchronous inputs is 3.27 times stronger than if the inputs arrived asynchronously. For the weaker asynchronous input, A'*, the coincidence advantage is slightly higher: CAmodel = 3.40. In summary, given the assumptions of linear summation of EPSPs and a simple threshold model of firing, the effect of many synchronous thalamic inputs would be strongly synergistic, but two inputs would only be slightly synergistic. From a past study (Alonso et al., 1996), however, we have found that two thalamic inputs sum more synergistically than would be expected from this simple model: the synergy ratio had a mean value of 1.71 and a median of 1.50. In this study, an important control was that the synergy ratio was high not only for pairs of synchronized inputs (as in Fig. 3), but also for uncorrelated
153
a visual stimulus is strong and thus driving the retina to a N g h ratei Second, Several groups have suggested that t h e preferential transmission of synChronous thalamic inputs might help in the cortical processing e f certain classes of visual stimuli (Sillito et a l , 1994; K6nig et al., 1996; Neuenschwander and Singer, 1996).Finally, preferential cortical responses to synchronous inputs m a y help in the transmission of visual information. For pairs of synchronized L G N cells, if Synchronous spikes are considered as a separate spike train ( A & A ' , See above), informationtheoretic maalvsis with the stimulus reconstruction technique (Rieke et al., 1997) yields more information about the stimulus t h a n if synchrony were ignored. Thus the ability to 'read-off' s y n c ~ o n o u s spikes might help in transmitting information from retina to cortex, through the potential bottleneck of the thalamus.
Acknowledgements This work was supported by NIH grants EY10115 and EY12196. I thank Pamela Reinagel, Mark Andermann, and John Reppas for careful readings of the manuscript.
References Abetes. M. (19821 Role of the cortical neuron: integrator or coincidencedetector? [sr. J. Med. Sci., 18: 83-92.
Abeles, M. (1991) Corticonics: Neural Circuits o f the Cerebral Cortex. CambridgeUniversityPress, London. Alonso. J.M., Usrey, W.Mi and Reid, R,C. (1996) Precisely correlated firing in cells of the lateral genieulate nucleus. Nature. 383: 815-819. Bernander, O,, Koch, C. and Douglas, R.J. (1994) Amplification and lineafization of distal synaptic input to cortical py~rnidal cells. J. NeurophysioL. 72: 2743-2753. Berry II, MJ. arid Meister, M. (1998) Refractoriness and neural precision. J. Neurosci., 18: 2200-22tl. Berry, M.J., Warland.D.K. and Meister, M. (1997)The structure and precision of retinal Spike trains. Proc. Natl. Acad. Sci. USA. 94! 5411-5416. Borg-Graham, LJ., Monier, C. and Fr6gnac, Y. (1998)Visual input evokes transient and strong shunting inhibition in visual cortical neurons. Nature, 393: 369-373. Brivanlou, I.H., Warland, D.K. and Meister, M. (1998) Mechanisms of concertedfiring among retinal gangli0ncells. Neuron. 20: 527-539. Cash, S. and Yuste, R. (1999) Linear summation of excitatory inputs by CA1 pyramidal neurons. Neuron, 22: 383-394. CMand, B.G. (1986) The dorsal lateral geniculate nucleus of the cat. In: J.D. Pettigrew, K.S. Sandersonand W.R. Levick (Eds.), Visual Neuroscience. CambridgeUniversityPress, London, pp. 111-120. Cleland, B.G. and Lee, B.B. (1985) A comparison of visual responses of cat lateral geniculate nucleus neuroneswith those of ganglion cells afferent to them. J Physiol., 369: 249-268. Cleland. B.G.. Dubin. M.W. and Levick, W.R. (1971) Simultaneous recording of input and output of lateral genienlate neurones. Nat.-New Biol.. 231: 191=192. Dan. Y., Alonso, J.M., Usrey, W.M. and Reid, R.C. (1998) Coding of visual information by precisely correlated spikes in the lateral geniculatenucleus. Nat. NeuroscL, 1: 501-507. Eckhorn, R., Bauer. R., Jordan, W., Brosch, M., Kruse. W.. Munk, M. and Reitboeck, H.J. (1988) Coherent oscillations: a mechanism of feature linking in the visual cortex? Multiple electrode and correlation analyses in the cat. BIOL Cybern., 60: 121-130. Ferster, D. (1994) Linearity of synaptic interactions in the assembly of receptive fields in cat visual cortex. Curt. Opin. Neurobiol., 4: 563-568. Fetz. E.E. and Gustafsson. B. (1983) Relation between shapes of post-synapticpotentials and changes in firing probability of cat motoneurones.J. Physiol., 341: 387-410. Frien, A., Eckhorn. R., Bauer, R.. Woelbern, T. and Kehr, H. (1994) Stimulus-specific fast oscillations at zero phase between visual areas V1 and V2 of awake monkey.Neuroreport, 5: 2273-2277. Gerstein, G.L., Perkel, D.H. and Dayhoff, J.E. (1985) Cooperative firing activity in simultaneously recorded populations of neurons: detection and measurement.J. Neurosci., 5: 881-889. Ghose, G.M., Ohzawa. I. and Freeman, R.D. (1994) Receptivefield maps of correlated discharge between pairs of neurons in the cat's visual cortex, J. NeurophysioL, 71: 330-346. Gray, C.M., K6nig, P., Engel, A.K. and Singer, W. (1989) Oscillatory responses in cat visual cortex exhibit inter-columnar
154 synchronization which reflects global stimulus properties. Nature, 338: 334-337. Henry, G.H., Harvey, A.R. and Lund, J.S. (1979) The afferent connections and laminar distribution of cells in the cat striate cortex. J. Comp. Neurol., 187: 725-744. Hirsch, J.A., Alonso, J.M.. Reid, R.C. and Martinez. L.M. (1998) Synaptic integration in striate cortical simple cells. J. Neurosci. 18: 9517-9528. Jagadeesh, B., Gray, C.M. and Ferster, D. (1992) Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science, 257: 552-554. Kaplan, E., lharpura. K. and Shapley, R.M. (1987) Contrasl affects the transmission of visual information through the mammalian lateral geniculate nucleus. J. Physiol., 391: 267288. Ktnig, P., Engel, A.K. and Singer, W. (1996) Integrator or coincidence detector? The role of the cortical neuron revisited. Trends Neurosci., 19: 130-137. Kreiter, A.K. and Singer, W. (1996) Stimulus-dependent synchronization of neuronal responses in the visual cortex of the awake macaque monkey. J. Neurosci.. 16: 2381-2396. Lampl, I., Reichova, I. and Ferster, D. (1999) Synchronous membrane potential fluctuations in neurons of the cat visual cortex. Neuron, 22: 361-374. Langmoen. I.A. and Andersen, P. (1983) Summation of excitatory postsynaptic potentials in hippocampal pyramidal neurons. J. NeurophysioL, 50: 1320-1329. Levick, W.R., Cleland. B.G. and Dubin. M.W. (1972) Lateral geniculate neurons of cat: retinal inputs and physiology. Invest. Ophthal., 11:302-311. Livingstone, M.S. (1996) Oscillatory firing and interneuronal correlations in squirrel monkey striate cortex.. J. Neurophysiol., 75: 2467-2485. Mastronarde, D.N. (1983a) Correlated firing of cat retinal ganglion cells, I. Spontaneously active inputs to X- and Y-cells. J. Neurophysiol., 49: 303-324. Mastronarde, D.N.: (1983b) Correlated firing of cat retinal ganglion cells. H. Responses of X- and Y-cells to single quantal events. J. Neurophysiol.. 49: 325-349. Mastronarde. D.N. (1983C) Interactions between ganglion cells in cat retina. J. Neurophysiol., 49: 350-365. Mastronarde, D.N. (1989) Correlated firing of retinal ganglion cells. Trends Neurosci.. 12: 75-80. Mastronarde, D.N. (1987) Two classes of single-input X-cells in cat lateral geniculate nucleus. II. Retinal inputs and the generation of receptive-field properties. J. Neurophysiol., 57: 381-413. Meister, M. (1996) Multineuronal codes in retinal signaling. Proc. Natl. Acad. Sci. USA, 93: 609-614. Meister. M., Lagnado, L. and Baylor, D.A. (1995) Concerted signaling by retinal ganglion ceils. Science, 270: 1207-1210. Mel, B.W. (1993) Synaptic integration in an excitable dendritic tree. J. Neurophysiol., 70: 1086-1101. Neuenschwander, S. and Singer, W. (1996) Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature. 379: 728-732.
Rall, W. :(1964)Theoretical significance of dendritic trees for neuronal input-output relations. In: R.F. Reiss 0Eds.), Neural Theory and Modeling, Stanford University Press, Palo Alto, pp. 73-97. [Reprinted in Rail, W. (1995), I. Segev, J. Rinzel and G.M. Shepherd (Eds.), The Theoretical Foundation of Dendritic Function: Selected papers of Wilfrid Rail with commentaries. MIT Press, Cambridge, MA, pp. 122-146. Reich~ D.S., Victor, J.D., Knight, B.W., Ozaki, T. and Kaplan, E. (1297) Response variability and timing precision of neuronal spike trains in vivo. J. Neurophysiol., 77: 2836-2841. Reid,: R.C. and Alons0, J,M. (1995) Specificity of monosynapfic connections from thalamus to visual cortex. Nature, 378: 281284. Reid, R.C., Victor, J.D. and Shapley, R.M. (1997) The use of m-sequences in the analysis of visual neurons: linear receptive field properties. Vis. Neurosci., 14: 1015-1027. Reinagel, P. and Reid, R.C. (2000) Temporal coding of visual information in the thalamus. J. Neurosci., 20: 5392-5400. Rieke; F., Warland, D., de Ruyter van Steveninck, R. and Bialek, W. (1997) Spikes: Exploring the Neural Code. MIT Press, Cambridge, MA. Sillito, A.M., Jones, H.E., Gerstein, G.L. and West, D.C. (1994) Feature-linked synchronization of thalamic relay cell tiring induced by feedback from the visual cortex. Nature, 369: 479-482. Softky, W. (1994)Sub-millisecond coincidence detection in active dendritic trees. Neuroscience, 58: 13-41. Tanaka, K. (1983) Cross-correlation analysis of geniculostriate neuronal relationships in cats. J. Neurophysiol., 49: 13031318. Tanaka, K. (1985) Organization of geniculate inputs to visual cortical cells in the cat. Vis. Res., 25: 357-364. Toyama, K.i Kimura, M. and Tanaka, K. (1981) Cross-correlation analYsis of intemeuronal connectivity in cat visual cortex. J. Neurophysiol., 46: i91-214. Ts'o, D.Y., Gilbert, C.D. and Wiesel, T.N. (1986) Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis. J. Neurosci., 6: 1160-1170. Usrey, W.M. and Reid, R.C. (1999) Synchronous activity in the visual system. Annu. Rev. Physiol., 61: 435-456: Usrey, W.M., Reppas, J.B. and Reid, R.C. (1998) Paired-spike interactions and synaptic efficacy of retinal inputs to the thalamus. Nature, 395: 384-387. Usrey, W.M., Reppas, J.B. and Reid, R.C. (1999) Specificity and strength of retinogeniculate connections. J. Neurophysiol., 82: 3527-3540. Usrey, W.M., Alonso, J.-M. and Reid, R.C. (2000) Synaptic interactions between thaiamic inputs to simple cells in cat visual cortex. J. Neumsci., 20: 5461-5467. Vaadia, E., Haalman, I., Abeles, M., Bergman, H., Prut, Y., Slovin, H. and Aertsen, A. (1995) Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature, 373: 515-518. Wandell, B. (1993) Foundations of Vision. Sinaner Associates, Sunderland, MA.
M.A.L. Nicolelis (Ed,)
Progressin BrainResearch. Vol, 130 © 2001 Elsevier Sciet~ce B.V. All rights reserved
CHAPTER 10
Comparative population analysis of cortical representations in p ~ tric spaces of visual field and skin: a unifying role for nonlinear interactions as a basis for active information processing across modalities Hubert R. D i n s e * and Dirk Jancke 1 Institute for Neuroinformatics. Theoretical Biology, Ruhr-University Bochum, Bochum, Germany
Introductory From a phenomenological point of view, the concept of population :analysis is a rather straightforward and inescapable Consequence of the observation that a huge number o f broadly tuned neurons is activated after even the simplest form of sensory stimulation or in relation to motor outputs. This mass activity ineludes both spiking and suprathreshold activity. Our approach outlined in this chapter was developed in order to accotmt for population activity recorded in early sensory cortices at a spiking level. Our goal was to Visualize and to analyze cortical activity distributions in the coordinates of the respective stimulus space to explore cooperative processes (Dinse et al., 1996; Jancke et al., 1996; Kalt et al., 1996; Erlhagen et al, 1999; Jancke et al., t999). The basic assumption is that neuronal interactions are an intricate part of cortical info~ation processing generating
* Corresponding author: Hubert R. Dinse, Institute for Neuroinformatics; Theoretical Biology, Ruhr-University Bochum, Bochum, Germany. Tel.: +49=234-32-25565; Fax: +49-234-32-14209; E-mail: hubert @neuroil~formatik.rmhr-uni-bochum.de t Present address: The Weizmann Institute of Science, Rehovot, Israel.
internal representations of the environment beyond simple one-to-one mappings of the input parameter space. Using this approach, we can demonstrate that the spatio-temporal processing of sensory stimuli is characterized by a delicate, mutual interplay between stimulus-dependent and interaction-based strategies contributing m the formation of widespread cortical activation patterns.
Why populations? In 1972, Barlow published the well-recognized article 'Single units and sensation: a neuron doctrine for perceptual psychology'. He proposed that "active high-level neurons directly and simply cause the elements of our perception" (Barlow, 1972). This work articulated the prevailing conceptual framework of that time and had a great impact on research of sensory information processing in early cortical areas. In fact, during the late fifties and sixties, single-cell recordings, the monitoring of extraceltular potential changes, had become feasible routine in every laboratory. It is tempting to speculate, in how far purely technical aspects of that type boosted the conceptual framework of single-cell analysis. While this approach became dominant during the next decades, at the same time. it became more and more evident that there might be more
156 to higher brain processes than revealed by single-cell recordings. It should be stressed that the emphasis on distributed population activity, instead of a single cell, does not imply the underestimation of the performance of single cells. On the contrary, there is more and more experimental evidence that axons, passive and excitable dendrites and spines play a possibly underestimated role in signal transfer and processing (Segev and Rall, 1998). Anatomical analysis of cortical networks revealed the enormous richness of connectivity and interconnectedness of cortical networks (for review see Braitenberg and Schtiz, 1991). According to their minute analysis, a single cortical cell receives on average synaptic inputs in the magnitude of 105. However, the proportion of direct sensory afferents is only 20% even in layer 4 that provides most of the sensory inputs. The degree of interconnectedness is best illustrated by calculations made by Braitenberg and Schtiz (1991), according to which 1 mm 3 cortical volume contains about 150,000 neurons, 3 km axonal fibers and 450 m dendritic branches. Consequently, cortical networks are characterized by densely coupled widespread arborization of dendritic and axonal connections. From a theoretical point of view, these anatomical constraints have been interpreted as an ideal substrate for parallel processing based on recurrent loops, in which lateral interactions and nonlinearities play a key role. From a functional point of view, there is abundant evidence that is fully in line with the outlined theoretical and anatomical considerations: (1) Cortical point-spread functions are broad. It is well established that widespread patterns of cortical activation are evoked by even very small and simple, i.e. 'point-like', stimuli. This is true for visual, auditory and somatosensory cortical areas. It simply implies that whatever the stimulus is, large populations of many thousands of neurons are invoked. It is interesting to note that most recent developed techniques to record neural activity do in fact measure equivalents Of the cortical point spread function, as is the case for PET, fMRI and optical imaging of intrinsic or dye-coupled signals. (2) Cortical processing is active. Neurons in striate visual cortex have been characterized with re-
spect to physical key features, such as visual field location, orientation, motion direction, ocular dominance, and spatial frequency. These approaches allowed to analyze neural representations within parameter spaces that are explicitly defined by physical stimulus attributes. However, dependent on stimulus context, a large number of visual illusions, e.g. the perception of illusory contours (Kanizsa, 1976; Von der Heydt et al., 1984; Ramachandran et al., 1994; Sheth et al., 1996; Mendola et al., 1999), indicate that the visual system must contain additional mechanisms leading to representations within parameter spaces which have no physical counterpart. This is in line with the observation that single neurons exhibit complex, non-predictive behavior dependent on stimulus context (for review see Gilbert et al., 2000). Accordingly, this complex spatio-temporal response properties can be modified by stimulation displaced from the receptive field-center or even from outside the classical receptive field (Allman et al., 1985; Dinse. 1986; Gilbert and Wiesel, 1990; Sillito et al., 1995). This can be rewritten by stating that if interaction contributes significantly to neural activation in the visual cortex, then representations of the visual environment will differ from a simple feedforward remapping of visual space. (3) Behavioral performance is superior to singlecelt performance. Except for a few~ examples, the performance inferred from single-celt, tuning characteristics is not sufficient to explain the performance seen at a behavioral level There are usually ,two responses to that: (a) neurons at higher (possibly unknown) stages of processing will show the required characteristics; and (b) the required performance will be generated as soon as a 'pool' of neurons is taken into consideration. A famous example is 'hyperacuity', the threshold of which is several fold beyond that of single cells (Westheimer, 1979). The 'coarse coding' framework is an attempt to explain how high resolution can be easily obtained with broadly tuned elements (Hinton et al., 1986). Taken together, in order to address the implications listed above, the conceptual consideration of neural populations, their recording, analysis and understanding appears straightforward. A less straightforward question is how to accomplish this goal.
157 Emergence of de novo representations? As each single neuron is p ~ of a population, a single neuron's'- activity is based on the entire network activity and vice versa, the network activity is dependent on the contributing single neurons. It has in fact been shown that the activity of a single neuron reflects the actual state of the entire neural network [Arieli et at., 1996; Tsodyks et al., 1999). Yet, an important question remains whether a population is able to create de novo "qualia' neither explicitly present at the single-cell level nor in the input (Lehky and Sejnowski, i999). The most prominent example might be the sensation of 'white' arising from the trichromatic color vision system (Young, 1802) by the joint activation of a population of retinal receptors tuned to different wavelengths. There are a number of recent experimental findings suggesting the population-based creation Of de novo properties (Diesmann et al., 1999; Jancke, 2000; Thief et al., 2000). Why parametric In principle, when investigating the visual system, a distribution of population activity within the parametric space of the visual field is equivalent to activities recorded in fimctional imaging studies such as fMRI or optical imaging assuming a clean retinotopy. There are a number of differences, however. The main problem arises from the fact that the retinotopy is far from coming close to a clean representation of the visual field. This is particularly obvious at a spatial scale that differentiates between visual angles iess than 1° apart (Hubel and Wiesel, I962; All'us, 1975). The main constrains arise from the considerable scatter of RF position that is larger or in the same range than the required systematic Shifts due to a topographic gradient in the map, Again, this holds true for other modalities in an analogous way. At a larger scale of several degrees, a clear retinotopic gradient is present, though distorted. Yet at this scale, other factors complicate the aspect of a clean topography. As extensively studied in the Visual system, the retinotopic gradient of the cortical map of the visual field is overlaid by so-called: functional maps'. Functional maps contain an orderly arrange-
ment of certain stimulus attributes in a repeated way for certain portions of the reSpectiVe retinal locations (Hubener et al., 1997; Kim et al., 1999; Swindale, 20001). At present, functional maps have been established for orientation of moving gratings (Blasdel and Salama, 1986; Swindale et ai., 1987; ]3onhoeffer and Grinvald, 1991), direction of motion (Weiiky et al., 1996) and the spatial frequency of the moving grating (Shoham et al., 1997; Kim et al., 1999). In addition, maps exist for the inputs of the two eyes (ocular dominance maps,, Wiesel et al., 1974; LeVay et al., 1978) and disparity (Burkitt et al., 1998). There is, in fact, evidence that the retinotopic map contains discontinuities to account for the discrete organization according to certain stimulus attributes (Das and Gilbert, !997). Taken together, the requirement for 'cleanness' of the m a p i s not fulfilled at either spatial scale. For a discussion of multiple functional maps in auditory cortex, see (Schreiner, 1995). Taken together, our parametric population approach takes into account that: (1) neurons are broadly tuned, e.g. covering large ranges of parameter values; and it enables (2) to analyze their common responses within the metrics of given stimulus attributes. In addition, the construction o f distributions of population activity that are defined in physical metrics can help to find underlying neural transformation strategies that map sensory stimulus parameters onto the cortical anatomy. Which metrics ? One fundamental question arises when discussing the metrics within which population of neurons should be studied. Probably the main advantage of the conventional single-cell receptive field (RF) approach was to describe neural activity within the metrics of the stimulus space, and not in the mettics of the anatomical connections, e.g, the dendritic branching of the cell. This simple remapping of activity made it possible to study the cell's firing as a function of any possible stimulus attribute in terms of its parameter space. As detailed below, our approach similarly consists of a systematic remapping of population activity from their cortical coordinates back into the parametric space (Fig. 1). Accordingly, constructing
158
population representation in cortical coordinates
0.5 mm
visual hemifield
nlnnmall mm ;, wan
1 population representation in stimulus coordinates
nllMk JlPjnn lnilllm
• = stimulus 0.4 °
Fig. 1. Population representation in different coordinates exemplified for visual cortex studies. A stimulus is presented at a fixed location in the visual field (bottom, left). Recording of evoked activity results in a cortical activation corresponding to the cortical point spread function (top, right). Shown is an intrinsic optical imaging map evoked by a small square of light. Extracellular recordings of single cell responses (top, right) were used to determine the receptive fields (RFs) defined in the visual field coordinates (bottom, left, colored RF outlines). In contrast to conventional methodologies, we pursue a non-centered approach, in which a stimulus is kept at a fixed position independent from the location of the RFs being studied (bottom, left: relation of black square to RF outlines in blue and green. see also Fig. 2). The definition of single cell activity in parameter space allows a systematic investigation of the cells activity as a function of variation of stimulus parameter. Our approach accomplishes a transformation of population activity in parameter space which corresponds to a 'population RF' (see also Fig. 2). It should be noted that recording of activity across the cortical surface as obtained by optical imaging equivalently refers to a non-centered field approach. In contrast, a distribution of population activity in parameter space can be regarded as the inverted cortical point-spread function ('cortical spread-point function'. As a consequence, this procedure allows to investigate population distributions and their interactions in the physical metrics of the stimuli.
a parametric distribution of population activation can b e r e g a r d e d as a ' p o p u l a t i o n r e c e p t i v e f i e l d ' . W e w i l l d e m o n s t r a t e t h a t t h i s a p p r o a c h is h i g h l y s u i t a b l e to
reveal insight into processing principles, including n e u r a l i n t e r a c t i o n s t h a t g o b e y o n d t h o s e d e f i n e d at the level of classical single-cell approaches.
159
Two types of averaging Implication of the non-centered field approach Our population approach is based on two different types of averages. Here we discuss the averaging across different spatial locations within the: RFs. In the conventional RF approach, stimuli are applied to the RF center: Accordingly, as a first step, the approx~aate shape and center of an RF has to be determined. Once that is done, stimuli of various types are presented at the center or along a centered orientation in order to study possible dependencies of the firing rate from stimulus variations, We :call this procedure RF centered approach i in contrast, as our main goal is to: study a population response to a given stimulus, i.e. the contributions of all neu-
tion. curvature, length, motion direction et cetera (see also above 'overlaying maps'). To characterize the contribution of each neuron to the representation of a given stimulus, one might conceive of the highdimensional space spanned by its different parameters. Each neuron could be thought of as a point in this parametric space. This point corresponds to a set of preferred values for all represented parameters. By asking only how the neuron's firing rate depends on visual field position, the contributions of all neurons are averaged, although their preferred parameter set may be different along other dimensions. In this sense, the population distribution is a projection from a potentially high dimensional space onto a common neuronal space representing only visual field :position (Jancke et al~, 1999). In this way, the p0pulation receptive field can be regarded as the inverse of the cortical point-spread function ('cortical spread-point function').
Reconstruction of information
across RFs. In our view, this way of stimulus presentation and averaging is crucial for an understanding of how complex scenes are represented in the visual Cortex. A similar approach has been pursued by van Essen and coworkers, who investigated the activity. Of visual cortex cells under natural viewing conditions (Gallant et al., 1998). In a way, this procedure cotTesponds to a systematic shift of a stimulus throughout an RF (cf. Szulborski and Palmer, 1990). Instead; we do not shift the stimulus, but sample the contributions of P,Fs of many neurons shifted randomly across the stimulus. Multidimensional spaces, subp0pulations, and the contributing neuron A second important averaging is performed across many different cell types. Neurons in area 17 contribute potenfiaUy tothe representations of many different parameters, such as retinal position, orienta-
There is agreement that physical attributes of sensory stimuli are encoded as activity levels in populations of neurons. Reconstruction or decoding describes the inverse problem in which the physical attributes are estimated from neural activity. Reconstruction methods have been regarded useful first in quantifying how much information about the physical attributes is present in a neural population and, second, in providing insight into how the brain might use distributed activity (Nicolelis, 1996; Zhang and Sejnowski, 1999; Doetsch. 2000). However. given that: (a) an optimal reconstruction method is utilized; (b) the population is of sufficient size, i.e. it contains a sufficient number of neurons: and (c) the stimuli are within the range of behavioral relevance and resolution, i.e. belong to a stimulus that is represented in the brain, we argue that the mere reconstruction does not yield much additional information. Of course, one of the problems behind this is the question who reads the code. In the case of optimal reconstruction, an implicit assumption is that the brain is able to perform a comparable analysis. Ways to prove this assumption are to compare population data with psychophysical data of performance, thresholds, discrimination abilities, reaction times etcetera. An ultimate control consists of execution of behavior.
160
I
r~
B
Fig. 2. Schematic illustration of stimulation and construction of population distributions exemplifiedfor the visual cortex. (A) Illustration of the non-centered field approach. Stimuli, indicated by the black square, were presented independent of the locations of the receptive fields (RFs) of the measured neurons (schematically illustrated by the ellipses). The frame with the cross-hair (gray) illustrates the analyzed portion of the visual space (2.8 x 2.0). (B-D) Illustrations of the Gaussian interpolation method to construct the distribution of populatiorl activity. (B) The location of the RF center of each neuron as determined by response plane techniques was weighted with its firing rate, illustrated as vertical bars of varying length at various locations. (C) The distribution of population activity was obtained by Gaussian interpolation (width = 0.6°). (D) View of the distribution of population activation using gray-levelsto indicate activation. The location of the stimulus is indicated by the square outlined in white together with the stimulus frame. In the results section, activity distributions are shown as color-codedcontour plots. as exemplified in the study by Chapin et al. (1999), where rats were trained to position a robot arm to obtain water by pressing a lever. :Mathematical transformations were used tO convert multineuron signals into 'neuronal population functions' that accurately predicted lever trajectory and w e r e used by the animals as a substitute o f executed behavior to position the robot arm and obtain water. More generally, during recent years, it became evident that a critical step for the investigation of how distributed cell assemblies process behaviorally relevant information is the introduction of methods for data analysis that could identify functional neuronal interactions within high dimensional data sets (cf. Nicolelis, 1999).
What is gained: neural interaction information Our approach seeks an altemative in comparing different states of population activity evoked by different classes of stimulL Instead of asking how accurately the parameter of for example stimulus location
can be reconstructed or decoded, we primarily were interested in analyzing interaction-based deviations o f population representations dependent on defined variations of stimulus configurations. Accordingly, there is an important point of departure from the interest we share with aspects relating to estimation theory. Instead, our analysis aimed to investigate how the representation of position is affected by interaction among neurons. The impact of interaction on human perception has been repeatedly shown. Repulsion effects between neighboring stimuli have been described in humans. Errors incurring when subjects estimate the visual distance between two spots of light depend systematically on the retinal distance of the stimuli. Small separations are underestimated, large distances are overestimated (Hock and Eastman, 1995). Similar results have been obtained for estimation of the orientation of stimuli (Westheimer, 1990). In the experiments described below, we introduce similar paradigms contrasting population actlw
161 e l e m e n t a r y stimuli
-I
c o m p o s i t e stimuli
!
.................
,:
:
1 i[f I I I ////
. . . . .
..............
0.4 d e g
---I . . . . . . . . . . . . . . .
I=-I ...........
,
..... I
I - - !
•
. . . . . . . . . . . . .
.........
. . . . . . . _ • ......... • . . . . .
•
• • ~,
separation
r¸ •
-I
...... -I
-I
........... I--
o
t
.......... -----I-
2.4 deg
2.8 deg tem porat
nasal
site 1
~
j
7 mm
site 2 site 3
0-0-
~-......~____~--
12 mm
j
20 mm
~._.____j~
29ram
~
site 4
e-site 5
Fig. 3. Comparison of stimulation configurations used for the investigation of nonlinear interaction in the visual and the somatosensory cortex, stimuli employed were small squares of light (Fat visual cortex) and small tactile stimuli (light taps) applied to the glabrous skin of the hindpaw (rat somatosensory cortex), which were denoted as elementary stimuli. Top row: in the visual cortex, the elementary stimuli (squares of light, 0.4 x 0.4°) were presented at seven horizontally shifted positions within the central visual field representation. Bottom row: in the somat0sensory cortex, elementary stimuli (1.5 mm in diameter) were presented at five positions along the distalproximal axis of the hindpaw. Both studies had in Commonthat composite stimuli were assembled from combinations of the elementary stimuli. In the visual cortex, they were presented at six different separation distances of 0.4-2.4 ° (top row). The left stimulus component was always kept at a fixed nasal position. In the somatosensory cortex, the composite stimuli were presented at four different separation distances betw~n 7 and29 mm Skin surface (bottom row). The most distal stimulus component (site 1) was always kept fixed. ity resulting from so-called 'simple' or 'elementary' sti_muli with composite stimuli that were assembled f r o m the elementary ones using different separation distances (Fig, 3), In detail, we extract the contribution of neurons to the representation of the,location of small squares o f light which we called elementary' stimuli. We then project the neural responses to 'composite' stimuli assembled from the two elementary stimuli o f varied separations onto this subspace by analyzingi:population distributions weighted with the responses to composite stimuli. If nonlinear interactions contribute significantly to neural activation in the visual cortex, then the representations o f composite stimuli will systematically:differ ~ o m t h e superposifion of two elementary, Insight into neural interactions of the analysis of dis-
tance-dependent deviations o f the distributions from additivity, i.e. how interactions distort the distributions of activation. Such interactions may arise from recurrent connectivity within the cortical area as well as from recurrency within the network providing the sensory input. Methodological
considerations
and
population
construction
Construction of a distribution of population activation Our general idea behind constructing a population distribution is to extract the contributions of neurons to the representation of a particular stimulus param-
162 eter. To obtain entire distributions that are defined for sensory field location, two types of analysis were applied (for details see Appendix): (1) Based on the measured RF profiles, the calculated RF centers served to construct two-dimensional distributions of population activity by interpolating the normalized firing rates of each contributing neuron with a Gaussian profile. Calculation of population representations in the somatosensory cortex based on an analogous Gaussian interpolation (2) To minimize the reconstruction error for the elementary stimulus conditions, we extended the Optimal Linear Estimator (OLE) (Salinas and Abbott, 1994) resulting in one-dimensional distributions of population activity. Data acquisition: visual cortex We recorded responses of single units in the central visual field representation in area 17 of the left hemisphere of anesthetized cats. Stimuli were always presented to the contralateral eye. Recordings were performed simultaneously with two or three glasscoated platinum electrodes (resistance between 3.5 and 4.5 MOhm. Thomas-Recording, Germany). The bandpass-filtered (500-3000 Hz) electrode signals were fed into spike sorters based on an on-line principle component analysis (Gawne and Richmond, NIH, USA). Visual stimulation Stimuli were displayed on a PC-controlled 21-inch monitor (120 Hz, non-interlaced) positioned at a distance of 114 cm from the animal. An identical set of common stimuli was presented to all neurons: (1) elementary stimuli (Fig. 3), small squares of light (size 0.4 x 0.4°), were flashed at one of seven different horizontally contiguous locations within a fixed reference frame; and (2) composite stimuli, two simultaneously flashed squares of light, were separated by distances that varied between 0.4 and 2.4 ° . Each stimulus was flashed for 25 ms. The interstimulus interval (ISI) was 1500 ms. There was a total of 32 repetitions of each stimulus, arranged in pseudo-random order across the different conditions. Stimuli had a luminance of 0.9 c d / m 2 against
a background luminance of 0.002 c d / m 2. The retinal position of these common stimuli was constant, irrespective of the RF location of individual neurons. The profile of each individual RF was assessed quantitatively with a separate set of stimuli, consisting of small dots of light (diameter 0.64 °) which were flashed in pseudo-random order (20x) for 25 ms (ISI 1000 ms) on the 36 locations of an imaginary 6 x 6 grid, centered over the hand-plotted RF. To control for eye drift, RF profiles were repeatedly measured during each recording session. Data acquisition: somatosensory cortex Single unit activity was extracellularly recorded in layer I I I - I V of the primary somatosensory cortex of anesthetized rats at depths of 700-750 I~m using glass micro-electrodes filled with concentrated NaCl (1-2 MOhm). The bandpass-filtered (500-3000 Hz) electrode signals were fed into a window discriminator. The output TTL-pulses were stored on a PC with a time resolution of 1 ms. Raw analog recordings were displayed on oscilloscopes and on audio monitors. Digitized neural responses were displayed as post-stimulus time-histograms (PSTHs) on-line during the recording sessions. Data were analyzed offline in the IDL graphical environment (RSI, USA). Tactile stimulation Penetrations were usually placed 200-300 ~ m apart to map the entire spatial extent of the hindpaw representation. The location and extent of RFs on the glabrous skin of the hindpaw was determined by hand-plotting (Merzenich et al., 1984). RFs were defined as those areas of skin at which just visible skin indentation evoked a reliable neural discharge. Other studies have shown that just-visible indentation is in the range of 250-500 l~m, which is in the middle of the dynamic range of cutaneous mechanoreceptors (Johnson, 1974; Gardner and Palmer, 1989). Cells responding either to high threshold stimuli, joint movements or deep inputs were classified as non-cutaneons and were excluded from further evaluation. The size (area of skin in m m 2) of cutaneous RFs was quantitatively analyzed by planimetry. For the analysis of neural population representations, an identical set of computer-controlled so-
163 called 'con~non stimuli' was presented to all neurons recorded independent from their RF location
Here we s e recent t merit findings, in which we explore the rote Of neural into ral interactions for the reprosentation of senSoi~ ~ imuii ii iinearly cortical areas. In detail, we desCr~he the dis :he diStance-dependent interactions for c Site stimuli timuli 0bsereed in the visual and the somatoSer~sory: cortex ~ortex that share substantial similarities.
Distance-dependent interactions for composite stimuli observed:in cat visual cortex We constructeddistributions of pop~ation activity in response to a set: of:slmali squ~es of light (so±coiled elementary s t i m u i i ' ) ~ ¢ h : d in their position along a v i ~ a l l ~talline (Figs. 3 and 4). The distributions were defined in the visual space and were based on single-cell responses from 178 neurons recorded in the central visual field representation of cat area I7. In order to obtain these distributions; we used a two-dimensional Gaussian
interpolation procedure, in which the RF centers were weighted with the normalized firing rate of responses obtained during the responses were analyzed (30onset). The width of the Gausiformly to 0.6 ° to match the ~daverage RF profile of all neual., 1999). N addition, based mat the representation of visual sidered as a function of acti~pace, we minimized the error edimensional distributions ustr estimator (OLE) procedure. ml in the sense that it extracts tion from the firing rates under 1st square fit. Both approaches sults. For the OLE-dehved reesolved approach that captured :on responses and the analysis cs see Jancke et al. (1999). ig. 4, the interpolation derived were monomodal and centered visual field position (indicated by white squ~es). The spatial arrangement of activity of these distributions implies that neurons in primary visual cortex contribute as an ensemble to the representation of visual field location although each neuron's RF might be broadly tuned to Stimulus location, i e is characterized by RF sizes several fold larger than the stimuli employed. As discussed above, the mere construction of these representations does not provide much information about ongoing processing m e c h ~ s m s ; We therefore asked the question in how far:the rep: resentation of composite stimuli consisting of two elementary stimuli can be predicted from the representations of the elementary ones, thereby addressing the question of neural interactions within the population representation. If there were no interacfions within the population, then the distributions of the composite stimuli would be predicted to be the linear superpositions of the distrit~utions of the com, ponent elementary stimuli. To test this hypothesis, we build distributions based on the same estimator used for elementary stimuli, but now weighting each cell's contribution with the firing rate observed in response to the composite stimuli. Fig. 5 illustrates the distributions of composite stimuli and their super-
164
X
E
~
"
.~~',~
~.~ ~:~
E
~
~
~ . ~ E- ,
'o
--~ , ~
r~
2
Z,--~
165 Fig. 5. The measured twmdimensional activity distributions recorded in 'the vlsUal, cortex o f the six composite stimuli (left. from mp to bottom', 0.4-2.4 ° separation, cf. Fig, 3) were compared to the superpositions of their component elementary stimuli(right), The activity distributions were based on spike activity of 178 cells averaged over the ~ e interval from 30 to 80 ms after stimulus onset. Same conventions as in Fig. 4, the colorscale was normalized to peak activation separately tbr each row. For small stimulus separation, note the remarkably reduced !evel of activation for the measured as compared t o t h e superimposed responses. This behavior could not be explained by saturation effects, see text). A transitior/from monomodal to bimodal distributions was found between 1.2 and 1,6° separation. The bimedal distribution recorded for the largest stimulus separation comes close to match the superposition. However, inhibitory interaction can still be observed. An asymmetry in the shape and amplitudes between the representations of the left and the right stimulus component was present for the measured as compared to the superimposed distributions, specifically for stimulus separations of 1,2 and 1.6°. Reproduced, with premission, from Jancke et al.. 1999. Copyright ~999 by the Society for Neuroscience.
100 90
Y
80 ff
70 •-
f
J
J
iv"
60 0,4
0,8
1.2
1,6
2
2,4
separation distance (degree) Fig. 6. Interaction based suppression of the population activity recorded in cat visual cortex induced by composite stimuli as a function of separation between the two component stimuli. The total activation in the distribution was expressed as a percentage of the total activation in the superposition. Inhibition was strongest for zero distance (0.4%eparation) and decreased almost monotonically with increasing distance, but was still presem at the largest separation tested (2.4°).
2,8 (3)
positions. Both, the measured and the superimposed dis~butions of population activity were monomodal for small, and bimodal for large stimulus separations, the transition occurring at around 1.6° separation. The most striking deviation from the linear superposition was a reduction of activity compared to the measttred responses, which was particularly strong for small stimulus separations. This reduction was not due to a saturation of population activity since it was also observed for composite stimuli of larger separations, where the distributions ~,~ere bimodal and had little overlap (for a more extended discussion, see Jancke etal.. 1999). A quantitative assessment of this suppressive interaction allowed to uncover its dependence on stimulus distance. The total activation in the population distribution was computed as the area under the distribution and was expressed as a percentage of the total activation contained in the superposition. This percentage was always below 100%, confirming the inhibitory effect as a consequence of using composite stimuli, :Suppression was strongest for small distances and decreased with increasing distances (Fig. 6). To quantitatively assess the accuracy with which the distributions o f population activity represent location, we compared the position of the maximum
-o v
2,4
,.0
2 b0 ."
1,6
J
o
__o
1,2
~6 2 0,4
°
4
~0 cO o
-0,4 -0,4
0
0,4 0,8 1,2 1,6
2
2,4 2,8
stimulus locations (deg) Fig. 7. Constructed versus real position of the elementary stimuli using the averaged spike activity. The positions of the maximum of the population distributions are shown for the seven elementary stimuli as a function of the real stimulus position. The dotted line indicates the perfect match between estimated and real stimulus position. Stimulus position can be accurately estimated; however, there is a systematic deviation in localization for all locations that might reflect a true mislocalization as described recently for human perception.
of each distribution to the respective stimulus position. Fig. 7 plots these constructed positions against the real stimulus positions. For all positions tested, there was a systematic deviation of on average 0.20-4- 0.11°. Interestingly, in a recent psychophysical
166 study, briefly presented stimuli have been found to be mislocalized. When observers were asked to localize the peripheral position of a probe with respect to the midposifion of a spatially extended comparison stimulus, they tend to judge the probe as being more toward the periphery than is the midposifion of the comparison stimulus (Mtisseler et al., 1999). It might therefore be speculated that the systematic error is not due to a bias in sampling RFs or due to errors in reconstruction, but instead reflect a true mislocalization in the representation.
Fig. 8. Population representations of the five elementary stimuli (cf. Fig. 3) recorded in rat somatosensory cortex depicted as two-dimensional activity distributions (top) over the glabrous skin surface of the hindpaw (cf. Figs. 1 and 2). The construction was based on the activity of 206 neurons. Activity distributions were computed in the time interval between 16 and 30 ms after stimulus onset corresponding to the neural peak responses. The activation level is shown in a color-scalenormalized to maximal activation separately for each stimulus. The grid overlying the distribution is intended to facilitate localization of activity. Red indicates high levels of activation. Bottom: figurines of the hindpaw indicate schematically position of the elementary stimuli. Note that for each stimulus, the focal zone of activation is fairly centered on each stimulus location.
Distance -dependent interactions for composite stimuli observed in rat somatosensory cortex Similar to our procedure utilized for the exploration of interaction effects in cat visual cortex, we constmcted distributions of population activation in response to a set of small tactile stimuli which differed in their position along the distal-proximal extension from the tip o f digit 3 to the palm. In analogy to the visual cortex study, these stimuli were termed 'elementary stimuli'. The distributions of population activity were derived from single-cell responses from 206 neurons recorded in the hindpaw representation of rat somatosensory cortex. They were obtained after backprojection of cortical activity onto stimulus space and can be regarded as a population receptive field defined in skin space. Similar to the visual cortex study, we used a two-dimensional Ganssian interpolation procedure, in which the RF centers were weighted with the normalized firing rate of each neuron. As the distribution of all recorded RFs was not homogenous, a higher number of neurons w e r e sampled at t h e distal part of the paw representation, the population activity was normalized for sampling density. The resulting color coded activity distributions of the neural population are depicted in :Fig. 8. The corresponding stimulus configuration is indicated in a schematic drawing. T h e areas with maximum population activity matched fairly well the sites of tactile stimulation, except for the representation of the proximal hindpaw of stimulus site 5, where we observed a fairly broad distribution without a sharp peak of activity. The shape of the distributions showed a substantial degree of variability, a finding not that evident for visual representations. The activity distri,
bntions for stimulation of sites 2 and 4 were rather distinct compared to the population representations for stimulation sites 1 and 3. The activity distribution was scaled to cover the whole color table in order to illustrate the shape of the population distributions. Quantitative assessment revealed that the population response amplitudes varied only by about 15% between sites 1 and 4. Despite the density normalization, activity at site 5 reached only 55% of the maximum activity elicited at site 4. We therefore assume that the weak population response of this site might reflect a genuine difference of the cortical representation of this very proximal part of the hindpaw. To address the question of neural interaction dynamics within the population representations, we compared the activity distribution obtained for composite stimuli to their superposifions. As in the visual cortex study, evidence for interactions was inferred from deviations of the population representations of the composite stimuli from the linear superpositions of the component elementary stimuli. We therefore build population representations based on the same estimator used for elementary stimuli, but now weighting each cell's contribution with the firing rate observed m response to the composite stimuli. In Fig. 9, the population activity distribution for the measured and the superimposed distributions are illustrated. The measured distributions were monomodal for smaller stimulus separations (stimuli 1-2 and 1-3, 7 and 12 turn). Bimodal distributions were only found for larger separations of 20 and 29 mm (stimuli 1-4 and 1-5). Similar to what we had found in the visual cortex, the most striking devia-
167
~
__imm!
tion from the linear superposition was a reduction of activity compared :to tiae measured responses. Again, this suppressive effect was particul~ly strong for smaller stimulus separations between 7 and 12 mm. The distance-dependent Suppression was quantified
Fig. 9 The measured two-dimensional activity distributions recorded in rat somatosensory cortex of the four composite stimuli itopi from left to right: 5-24 mm skin surface separation, cf. Fig. 3)were compared to the superpositions of the representations oJ their component elementary stimuli (middle). The activity distt[bufions were based on spike activity 0f206 ceils averaged oi er the time interval from 16 to 30 ms after stimulus onset, S mac conventions as in Fig. 8, the cot0r-scale was normalized to :oak activation separately for each column. For small stimuius ~ iparation, note the remarkably reduced Ievel of activation for the measured (top) as compared to the superimposed (middle) sponses. The measured distributions were monomodal for sn ll-er stimulus separations (stimUli 1-2 and 1-3, 7 mm and t2 ~). Bimodal distributions were only found for larger separatioan~ of 20 and 29 mm (stimuli 1-4 and i-5). The bimodal distribution recorded for the largest stimulus separation comes close to match the superposition. As in the case of the visual cortex study, inhibitory interaction can still be observed. An asymmetry in the shape and amplitudes between the representations was present for the measured as compared tO the superimposed disttbutions, with the tendency of an attraction of activity towards the distal aspects Of the hindpaw.
Q
168 "-', 10o
40-
j,
L~ 9o "~
7 °
E E v
35-
g
30
°
0
80
25-
f
0D
t.p
,° /
..go 20-
70
15 •-
60 0
5
10
15
20
25
30
separation distance (ram skin) Fig. 10. Interaction based suppression of the population activity recorded in the somatosensory cortex induced by composite stimuli as a function of separation between the two component stimuli. Similar to our findings in the visual cortex (cf. Fig. 6), the total activation in the distribution was expressed as percentage of the total activation in the superposition. Inhibition was strongest for the smallest separation tested and decreased almost monotonically with increasing distance, but was still present at the largest separation tested (29 mm skin surface).
by expressing the integrated population activity in the measured distributions to those obtained from superposition. As shown in Fig. 10, we found a clear distance dependence very similar to that obtained in the v~sual cortex. ' We also found a substantial a s y m m e t r y in the distributions for composite stimuli (Fig. 9) that was even more pronounced than that observed in the Visual cortex. This asymmetry consisted in an attraction of activity evoked from composite stimuli towards the distal aspects of the hindpaw, i.e. in the direction of the digit representation. It is well known that the cortical map of the hindpaw is dominated by the representation of the digits. In contrast, a similar asymmetry in the visual field representations at the scale of our stimuli is not present. In single-cell recordings f r o m monkey area 3b, a systematic bias in t h e spatial distribution of inhibitory surrounds of tactile RFs towards the fingertip was reported (DiCarlo et al., 1998). :We therefore conclude that the substantial asymmetry found in SI probably reflects the representational constraints present in the somatotopic representation. In order to illustrate the accuracy with which the population distributions represent locations within the skin o f the hindpaw, the coordinates along the distal to proximal dimension of the locations with maximal activity levels were plotted as function of
2
10
tO
5 0 0
5
10
15
20 25
30
35
40
s t i m u l u s l o c a t i o n s (mm)
Fig. 11. Constructed versus real position of the elementary stimuli using the averaged spike activity. The positions of the maximum of the population distributions are shown for the five elementary stimuli as a function of the real stimulus position. The dotted line indicates the perfect match between estimated and real stimulus position. Stimulus position can be accurately estimated, however, fluctuations appear to be more random. the real stimulus location Points on the diagonal indicate an ideal representation of the stimulus location. The distance of the points from the diagonal is a measure for the discrepancy o f the reconstructed with the real stimulus locations. As shown in Fig. 11, the reconstructed locations were usually shifted to the proximal portions of the paw. Only in the case of stimulation at site 4, the reconstructed stimulus location is situated more distally compared to the actual position. We are not aware of any psychophysical evidence for a systematic mislocalization towards the tip of the hands, although there are many reports of significant mislocalizations after plastic reorganizations (Sterr et al., 1998; Braun et al., 2000). Accordingly, it remains an open question as to how far the observed errors in localization reflect shortcomings in RF sampling. Using a population of 206 neurons, the stimulus position could be reconstructed with an accuracy of 4-3.39 ram, given an average RF size of 56 m m 2 (cf. Fig. 11). Recently, it had been reported that simultaneous multi-site neural ensemble recordings in three areas of the primate somatosensory cortex (areas 3b. SII and 2) consisting of small neural ensembles ( 3 0 40 neurons) of broadly tuned somatosensory neurons were able to identify correctly the location of a sin-
169 8T
A~
©
2
E 0
50
100
150
200
250
number of neurons in population 0.40 "
~
~
t
B
$ o.3~ x/
0.50
=
o
0.25
Z'
~
E 0.15. 0.10
. 20
.
50
.
.
I O0
150
"-
neuron number
Fig. 12. (A) Average deviation of the activity distributions from an ideal localization of the stimulated sites as a function of neuron numbers in the somatosensory cortex. There is a fairly strong reduction in the deviation from ideal localization when the number of neurons is increased from 30 to 40 cells. Further increase of neuron number yields only little further increase in performance of localization. (B) Average deviation of the activity distributions from an ideal localization of the stimulated sites as a function of ndm'on numbers in the visual cortex. Similar to the data from som0xosensory cortex, the most prominent gain in localization is obtained for a fairly small size of population in the range of 40-50 neurons.
gle tactile stimulus on a single trial (Nicolelis et al., 1998). We performed atype of bootstrap analysis, in which neurons froln the entire population were randomlv selected m generate subpopulations of 33.40. 50 and 67 neurons. Re-analysis of the data showed that the localization error was significantly reduced when we increased the number of neurons from 33 to 40, while further increase yielded little further improvement in localization (Fig. 12A). Interestingly, a similar analysis performed in the visual cortex revealed a comparal~le dependency on neuron number (Fig. 1 2 B ) .
Similarities and differences of population representation of simple elementary Stimuli and resulting interactions from composite stimuli Our goal was to explore the general relevance of active, cooperative processes in the representation o f simple sensory stimuli by comparing neural interaction effects recorded in primary visual and somatosensory areas. We therefore constructed sets of stimuli that combined aspects of simplicity with aspects of comparability. In our view, small squares of light displayed in the visual field at contiguous locations along a virtual horizontal line might be maximal equivalent to small taps of cutaneous stimuli presented at adjacent locations along the distalproximal axis of the glabrous skin of the paw. Most importantly, in both cases, the composite stimuli were assembled from these elementary stimuli to yield a set of 'complex' stimuli of variable separation distances (cf. Fig. 3). Under these assumptions, we could demonstrate considerable similarities of the overall properties of population activity recorded in the visual and the somatosensory cortex: (1) Population activity in the parametric space of the visual field or the hindpaw glabrous skin could be Characterized by s h @ l y focused distributions centered around the location of the stimulus. (2) The distributions evoked by the composite stimuli could not be predicted from the superpositions of the distributions obtained for the elementary ones. (3) Accordingly, in both modalities, nonlinear interactions played an important role in the generation of 'complex' representations. (4) The effects of interactions consisted of a substantial suppression of response that was dependent on separation distance between the stimuli. (5) The amount of inhibition seen and the slope of the distance-response function were similar. indicating that the metrics that govern the interactions are comparable, despite their enormous differences in physical attributes. (6) Analyzing the number of neurons necessary to yield a given performance resulted in a comparable population size. However, we also observed a number of dissimilarities, The observed differences mostly deal with aspects of asymmetry within the representation.
170 Most notably, while population activity in the parametric space of the visual field was fairly similar in amplitude and shape across the seven elementary stimuli, we observed a gradient in amplitude in the representations of the tactile stimuli, with the representation of the most proximal aspects of the hindpaw being considerable weaker and broader. Embedding parametric maps and interaction ranges in cortical anatomy
It remains to be clarified in how far the similarities of distance-dependence of interactions translate into the metrics of their respective cortical representations. In order to provide a: first approximation, we take into account the mapping experiments of Tusa and coworkers in cat visual cortex (Tusa et al., 1978). According to their data. 2.8 ° of the central visual field corresponds roughly to an area of 3 mm cortical surface. In contrast, the dimensions of the rat hindpaw map in primary somatosensory cortex is in the range of 1-1.5 mm (Chapin and Lin, 1984). This comparison implies that comparable interaction mechanisms might operate on spatial scales of cortical coordinates that differ by a factor of two. It has been speculated that long-range horizontal connections are instrumental in providing a substrate mediating the interaction effects. Wide-spread horizontal connections have been described for visual cortex spanning several millimeters (Gilbert and Wiesel. 1985; L6wel and Singer, 1992: Kisvarday et al., 1997; Sur et al., 1999). In addition, short range interactions have been shown to be involved in contextual modulations (Das and Gilbert, 1999). It is conceivable that the spatial ranges of short- and long-range connections are different in rat cortex (cf. Hickmott and Merzenich, 1998; Cauller et al., 1998), thereby providing a species- and modality-specific adaptational scaling to the needs of interaction dimensions (Dinse and Schreiner, 2001).
evoked by composite stimuli built from simple ones. This approach was applied in studies of the visual and somatosensory cortex to address the question about a possible modality and area-independent role of neural interactions (Dinse and Schreiner, 2001). We found that the population activity to elementary and composite stimuli shared a number of substantial similarities. Most notably, we found a comparable distance-dependence of nonlinear suppressive effects. These data raise the question of a modality-specific adaptational scaling of the spatial ranges of short- and long-range connections to the needs of interaction dimensions. We are currently extending our approach to the analysis of moving stimuli and the role of stimulus history for the establishment of a moving trajectory of cortical activity. In addition, we initiated studies to explore the impact of plastic reorganizations on nonlinear interactions. An ultimate goal would be to record neural activity simultaneously, which additionally allows insight into the ongoing dynamics of cooperative processes. Combining this technique with real-time optical imaging can provide insight into the cortical layout of the spatial interaction ranges and their implementation by anatomical connections.
Acknowledgements This work was supported by the Deutsche Forschnngsgemeinschaft and by the Institute for Neuroinformatics, D.J. holds a Minerva stipend. We thank our colleagues for extensive and extended discussions: Drs: Amir Akhavan, Wolfram Erlhagen, Martin Giese, Gregor Sch6ner, Axel Steinhage and Werner yon Seelen. We also thank Luls Tissot for help in data analysis. We gratefully acknowledge the cooperation of Dr. Thomas Kalt in the studies of rat somatosensory cortex and for providing instructive material,
Appendix Conclusions We used a population coding approach to visualize and to investigate representations of simple sensory stimuli in terms of their parametric spaces. We provided evidence that nonlinear interactions are crucially involved in the generation of representations
Constructing two-dimensional distributions of population activity by Gaussian interpolation
For each locationon the 6 x 6 grid, an averageresponse strength was determined for each cell by averaging the firing rate in the time interval between 40 and 65 ms after stimulus onset corresponding to the peak responses in the PSTHs. RF profiles
171
t a Gaussian e RF of each at part of the rate. timu~us humtime interval ~ver 32 stimmated as the 5als. For the on, the firing :ing rate, mn, ~s and during any single lO*ms bin in the time interval from stimulus onset to 100 ms after stimulus onset, This normalized firing rate f , ( s t) -- bn F, (s, t) = - m~z - b~
(1)
was always welt defined and positive. The normalized firing rates, /~;~(s, t), were depicted at the position of each neuron's calculated RF center. For interpolation of the data points, the width of the Gaussian profile was chosen equal to 0.6° in the visual space (approximately corresponding to the average RF width of all ~eurons recorded (Fig. 2A)). To correct for uneven sampling of visual space by the limited number of RF centers, the distribution was normalized by dividing by a density function, which was simply the sum of unweighted Gaussian profiles (width = 0.64°) centered on all RF centers. Calculation o f population representations in the somarosensory cortex To obtain population distributions that are defined for the parameter skin field location, the area of the hand-plotted RF was con~?oiuted with a Gaassian, normalized to amplitude of unity. To be independent of this particular type of normalization, in a second approach, the same procedure was followed by normalizing the Gaussian distribution to integral of unity. To construct tWOdimensional activity distributions across the skin the calculated Gaussians were summed. To correct for uneven sampling, the distribution wag normalized by dividing by a density function, which was the sum of the unweighted Gaussian profiles. The resuiting population representation reflects the local distribution of neuronal activation with highest amplitude at the actual location of a common ~timutus encoded by the entire population.
References
Albus, K (1975)A quantitative study of the projection area of the central and the paracentral visual field in area 17 of the cat. I. The precision of :the topography. Exp. Brain Res.. 24: 159-179. Allman, J., Mieziu. E and McGuiness, E.L. (1985) Stimulus specific responses from beyond the classical receptive field. /~nnu. Rev. Neurosci.. 8: 407-430. Arieli, A., Sterkin, A., Gfinvald, A. and Aertseu, A. (1996) Dynamics of ongoing activity: explanation of the large variability in evoked cortical responses. Science. 273: 1868-1871.
Barlow. H.B. (1972) Single units and sensation: a neuron doctrine for perceptual psychology? Perception. 1: 371-394. BlasdeL G.G. and Salama, G. (1986) Voltage-sensitive dyes reveal a modular organization in monkey striate cortex. Nature. 321: 579-585. Bonhoeffer. T. and Grinvald, A. (1991) lso-orientation domains in cat visual cortex are arranged in pinwheel-like patterns. Nature. 353: 429-431. Braitenberg, V. and SchiJz, A. (1991) Anatomy of the Cortex. Swinger, New York. Braun. C., Schweizer, R,, Elbert, T., Birbanmer. N. and Taub. E. (2000) Differential activation in somatosensory cortex for different discrimination tasks. J. Neurosci.. 20: 446-450. Burkitt, G.R., Lee. J. and Ts'o, D.J. (1998) Functional orgamzation of disparity in visual area V2 of the macaque monkey. Soc. Neurosci. Abstr., 24: 1978. Canller, L.J., Clancy, B. and Connors, B.W. (1998) Backward cortical projections to primary somatosensory cortex in rats extend long horizontal axons in layer I. J. Comp. Neurol.. 390: 297-310. Chapin; J.K. and Lin, C.S. (1984) Mapping the body representation in the SI cortex of anesthetized and awake rats. Comp. NeuroL, 229: 199-213. Chapin, J.K., Moxon. K.A.. Markowitz, R.S. ~and Nicolelis. M.A. (1999) Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex. Nat. NeuroscL. 2: 664670. Das. A. and Gilbert, C.D. (1997) Distortions of visuotopic map match orientation singularities in primary visual cortex. Nature. 387: 594-598. Das, A. a n d Gilbert. C.D. (1999) Topography of contextual modulations mediated by short-range interactions in primary visual cortex. Nature, 399: 655-661. DiCarlo, JJ.. Johnson. K.O. and Hsiao, S.S. (1998) Structure of receptive fields in area 3b of primary somatosensory cortex in the alert monkey. J. Neurosci.. 18: 2626-2645. Diesmann. M.. Gewaltig, M.O. and Aertsen, A. (1999) Stable propagation of synchronous spiking in cortical neural networks. Nature. 402: 529-533. Dinse. H.R. (1986) Foreground-background-interaction Stimulus dependent properties of the cat's area 17, 18 a n d 19 neurons outside the classical receptive field. Perception. 15: A6. Dinse, H.R. and Jancke. D. (2001) Time-varient processing in VI: from microscopic (single cell) to mesoscopic (population) levels. Trends Neurosci.. 24: 203-205. Dinse, H.R. and Schreiner. C.E. (2001) Do primary sensory areas play homologous roles in different sensory modalities? In: A. Schtiz and R. Miller (Eds.), Cortical Areas: Unity and Diversity: Conceptual Advances in Brain Research. Harwood. m press. Dinse, H.R., Jancke, D., Akhavan. A.C., Kalt, T. and Sch6ner. G. (1996) Dynamics of population representations o f visual and somatosensory cortex based on spatio-temporal stimulation. In: S.I. Amari. L. Xu, L.W. Chan, I. King and K.S. Leung (Eds.), Progress in Neural Information Processing. Imerna-
172
tional Conference on Neural In~tbrmation Processing. ICONIP '96. Springer, Singapore, pp. 1285-1290. Doetsch. G.S. (2000) Patterns in the brain. Neuronal population coding in the somatosensory system. Physiol. Behav., 69: 187201. Erlhagen. W., Basfian, A., Jancke. D., Riehle. A. and Sch6ner, G. (1999) The distribution of neuronal population activation (DPA) as a tool to study interaction and integration in cortical representations. J. Neurosci. Methods, 94: 53-66. Gallant, J.L.. Connor. C.E. and van Essen, D.C. (1998) Neural activity m areas V1, V2 and V4 during free viewing of natural scenes compared to controlled viewing. NeuroReporr, 9: 2153-2158. Gardner, E.P. and Palmer, C. (1989) Simulation of motion on the skin. I. Receptive fields and temporal frequency coding by cutaneous mechanoreceptors of OtrI'ACON pulses delivered to the hand. J. Neurophysiol., 62: 1410-1436. Gilbert, C.D. and Wiesel, T.N. (1985) Intrinsic connectivity and receptive field properties in visual cortex. Vision Res.. 25: 365-374. Gilbert, C.D. and Wiesel. T.N. (1990) The influence of contextual stimuli on the orientation selectivity of cells in primary visual cortex of the cat. Vision Res., 30: 1689-1701. Gilbert, C., Ito, M., Kapadia. M. and Westheimer, G. (2000) Interactions between attention, context and learning in primary wsual cortex. Vision Res., 40: 1217-1226. Hickmott. P.W. and Merzenich, M.M. (1998) Single-cell correlates of a representational boundary in rat somatosensory cortex. J. Neurosci., 18: 4403-4416. Hinton, G.E., McClelland, J.L. and Rmnelhart. D.E. (1986) Distributed representations. In: J.A. Feldman. P.J. Hayes and D.E. Rumelhart (Eds.), Parallel Distributed Processing. Exploration in the Microstrucmre of Cognition. Volume 1: Foundations MIT Press. Cambrige, MA, pp. 77-109. Hock. H.S. and Eastman. K.E. (1995) Context effects on perceived position: sustained and transient temporal influences on spatial interactions. Vision Res., 35: 635-646. Hubel, D.H. and Wiesel, T.N. (1962) Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. Z Physiol., 160: 106-154. Hubener. M., Slioham, D.. Grinvald, A. and Bonhoeffer. T. (1997) Spatial relationships among three columnar systems in cat area 17. J. Neurosci.. 17: 9270-9284. Jancke. D. (2000) Orientation formed by a spot's trajectory: A two-dimensional population approach in primary visual cortex. J. Neurosci. 20: RC86. Jancke. D., Akhavan, A.C., Erlbagen, W., Giese, M., Steinhage, A., Sch6ner, G. and Dinse. H.R. (1996) Population coding in cat visual cortex reveals nonlinear interactions as predicted by a neural field model. In: C. yon der Malsburg, W. yon Seelen. J.C. Vorbrtiggen and B. Sendhoff (Eds.), Artificial Neural Networks ICANN '96. Springer, pp. 641-648. Jancke, D.. Erlhagen, W., Dinse, H.R., Akhavan, A.C., Giese, M., Steinhage, A. and SchSner, G. (1999~ Parametric population representation of retinal location: neuronal interaction dynamics in ca! primary visual cortex. J. Neurosci., 19: 90169028.
Johnson, K.O. f1974) Reconstruction of population response to a vibratory stimulus in quickly adapting mechanoreceptive afferent fibre population innervating glabrous skin of the monkey. J. Neurophysiol.. 37: 48-72. Kalt, T.. Akhavan. A.C., Jancke. D. and Dinse, H.R. (1996) Dynamic population coding in rat somatosensory cortex. Soc. Neurosci. Abstr., 22: 105. Kanizsa, G. (1976) Subjective contours. Sci. Am.. 234: 48-52. Kh-m D.S., Matsuda. Y., Ohki, K., Ajima, A. and Tanaka, S. (1999) Geometrical and topological relationships between multiple functional maps m cat primary visual cortex. NeuroReporr. 10: 2515-2522. Kisvarday, Z.E, Toth, E., Ransch, M. and Eysel, U.T. (1997) Orientation-specific relationship between populations of excitatory and inhibitory lateral connections in the visual cortex of the cat. Cereb. Cortex. 7: 605-618. Lehky, S.R. and Sejnowski, T.J. (1999) Seeing white: qualia in the context of decoding population codes. Neural Comput.. 11: 1261-1280. LeVay, S.. Stryker. M.E and Shatz, C.J. (1978) Ocular dominance columns and their development in layer IV of the cat's visual cortex: a quantitative study. J. Comp. Neurol., 179: 223-244. L6wel. S. and Singer, W. (19921 Selection of intrinsic horizontal connections in the visual cortex by correlated neuronal activity. Science. 255: 209-212. Mendola. J.D., Dale. A.M., Fischl, B., Liu, A.K. and Tootell. B.H. (1999) The representation of illusory and real contours in human cortical visual areas revealed by functional magnetic resonance imaging. J. Neurosci., 19: 8560-8572. Merzehich, M.M., Nelson. R.J., Stryker, M.P.. Cynader, M.S.. Schoppmann. A. and Zook. J.M. (1984) Somatosensory cortical map changes following digit amputation in adult monkeys. J. Comp. Neurol.. 224: 591-605. Mtisseler. J.. Van der Heijden, A.H.C., Mahmud, S.H., Deubel, H. and Ertsey, S. (1999) Relative mislocalization of briefly presented stimuli in the retinal periphery. Percept. Psychophys., 61: 1646-1661. Nicolelis, M,A.L. (1996) Beyond maps: a dynamic view of the somatosensory system. Bra~ J. Med. Biol. Res., 29: 401-412. Nicolelis, M.A.L. (Ed.) (1999) Methods in Neural Ensemble Recordings. CRC Press, New York. Nicolelis, M.A.. Ghazarffar, A.A., Stambaugh, C.R., Oliveira. L.M., Laubach, M., Chapin, J.K., Nelson, R.J. and Kaas, J.H. (1998) Simultaneous encoding of tactile information by three primate cortical areas. Nat. Neurosci.. 1: 621-630. Ramachandran, V.S., Ruskin, D.. Cobb, S. and RogersRarnachandran, D. (1994) On the perception of ilhtsory contours. Vision Res., 34: 3145-3152. Salinas, E. and Abbott. L.E (1994) Vector reconstruction from firing rates. J. Comp. Neurosci., 1: 89-107. Schreiner, C.E (1995) Order and disorder in auditory cortical maps. Curr. Opin. Neurobiol., 5: 489-496. Segev, I. and Rail, V~ (1998) Excitable dendrites and spines: earlier theoretical insights elucidate recent direct observations. Trends Neurosci., 21: 453-460. Sbeth, B.R., Sharma, J., Rao. S.C. and Sur, M. (1996) Often-
173
tatiou maps of subjective contours in visual cortex. Science. 274: 2110-2115. Shoham D., Huhener, M., Schulze, S,, Grinvald, A. and Bonhoeffer T. (1997) Spatio-temporal frequency domains and their relation to cytochrome oxidase staining in cat visual cortex. Nature. 385: 529-533. Sillito, A.M, Ca-ieve, K.L., Jones, H.E., Cudeiro, J. and Davis, J. (1995) Visual cortical mechanisms detecting focal orientation discontinuities. Nature, 378: 49g-496. Sterr, A., Mullel', M.M., Etbert, T., Rockstroh, B. Pantev, C. and Taub, E. (1998): Perceptual correlates of changes in cortical representation of fii~gers in blind multifinger Braille readers. J. Neurosci.. 18: 441714423. Sur, M., Angelucci, A. and Sharma, J. (1999) Rewiring cortex: the role 0f patterned activky in development and plasticity of neocortical c i t e r s . J. Neurobiol., 41: 33-43. Swhadale, N.V. (2000) How many maps are there in visual cortex? Cereb. Cortex, 10: 633-643. Swindale0 N.V.. Matsubara. LA. and Cynader. M.S. (1987) Surface organization of orientation and direction selectivity in cat area 18. J. Neurosci.. 7: 1414-1427. Szulborski R G and Palmer L.A. (1990) The two-dimensional spatial structm'e .of nonlinear subunits in the RFs of complex ceils. Vision Res., 30: 249~-254. Thier, E. Dicke: RW., Haas. R. and Barash. S. (2000) Encoding of movement time by populations of cerebellar Purkinje cells,
Nature. 405: 72-76. Tsodyks. M., Kenet, T., Grinvald, A. and Arieli, A. (1999) Linking spontaneous activity of single cortical neurons and the underlying functional architecture. Science. 286: 1943-1946. Tusa, R.J., Palmer. L.A. and Rosenqnist, A.C. (1978) 1'he retinotopic organization of area 17 ~striate cortex) in the cat. J. Comp. Neurol.. 177: 213-235. Von der Heydt, R., Peterhans. E. and Baumgarmer, G. (1984) Illusory contours and cortical neuron responses. Science. 224: 1260-1262. Weliky, M., Bosking, W.H. and Fitzpatrick, D. (1996) A systematic map of direction preference in primary, visual cortex. Nature. 379: 725-728. Westheimer. G. (1979) Cooperative neural processes involved in stereoscopic acuity. Exp. Brain Res., 36: 585-597. Westheimer, G. (1990) Simultaneous orientation contrast for lines in the human fovea. Vision Res.. 30:1913-1921. Wiesel, T.N., Hubel. D.H. and Lain, D.M. (1974) Autoradiographic demonstration of ocular-dominance columns in the monkey striate cortex by means of transneuronal transport. Brain Res.. 79: 273-279. Young, T. (1802) II. The Bakerian Lecture. On the theory of light and colors. Phil. Trans. R. Soc. Lond. 91: 12-48, Zhang, K. and Sejnowski. T.J. (1999) A theory of geometric constraints on neural activity for natural three-dimensional movement. J. Neurosci.. 19: 3t22-3145.
M.A.L. Nieolelis (E&)
Progressin Brain Research,VoL 130 © 2001 Elsevier Science B.V. All rights reserved
CHAPTER 11
Coordinate transformations in the visual system: how to generate gain fields and what to compute with them Emilio Salinas 1,, and L.E Abbott 2 z Howard Hughes Medical Institute, Computational Neurobiology Laboratory, The Salk Institute for Biological Studies, 10010 North Torrey Pines Road, La Jolla, CA 92037. USA 2 Volen Center for Complex Systems and Department of Biology, Brandeis University, Waltham, MA 02454-9110, USA
Introduction
Studies of population coding, which explore how the activity of ensembles of neurons represent the external world, normally focus on the accuracy and reliability with which sensory information is represented. However, the encoding strategies used by neural circuits have undoubtedly been Shaped by the way the enc0ded information is used. The point of encoding sensory information is, after all, to generate and gnide behavior. The ease and efficiency with which sensory information can be processed to generate motor responses must be an important factor in determining the nature of a neuronal population code. In other words, to understand how populations of neurons encode, we cannot overlook how they compute. Gain modulation, which is seen in many cortical areas, is a change in the response amplitude of a neuron that is not accompanied by a modification of response selectivity. Just as population coding is a ubiquitous form of information representation, gain mod~afion appears to be a widespread mechanism of neuronal computation. In particular, it allows information from different sensory and cognitive modalRies to be combined. Gain modulated *CorreSponding author: Emilio Salinas, Computational Neurobiology Laborat0ry, The Salk Institute, 10010 North Torrey Pines Road, La Jolla, CA 92037, USA. E-mail:
[email protected] neurons simultaneously represent multiple forms of information in a population code. The responses of ensembles of neurons are necessary to understand what the population is representing, no single neuron is sufficient. The distributed, multi-modal representations of gain modulated neurons are ideally configured to facilitate certain kinds o f computations, namely coordinate transformations. Functionally, the gain-modulated population code forms a distributed substrate for both information representation and processing. Cortical areas that process visual information are subdivided functionally and anatomically into two pathways. The 'where' pathway runs dorsally from primary visual cortex into posterior parietal cortex, and the 'what' pathway runs ventrally from primary visual cortex into inferotemporat cortex (Ungerleider and Mishkin, 1982; Goodale and Milner. 1992). Parietal cortex is involved in the spatial analysis necessary for motor planning and for the localization of external objects (Andersen, 1989; Andersen et al., 1997), whereas inferotemporal (IT) neurons are important for object recognition (Goodale and Milner. t992; Gross, 1992). In spite of their distinct functional roles, neuronal populations in the two streams are subject to similar forms of gain modulation. Gaze direction provides a strong gain control signal in the dorsal stream, while attention provides a similar signal in the ventral stream. Although gaze-dependent and attention-dependent gain modulation act preferentially on separate processing streams and, to a
176 good extent, independently of each other, they seem to serve the same purpose, the computation of coordinate transformations. This functional interpretation of a widespread neuronal modulatory mechanism is the subject of this chapter. We review experimental evidence revealing gain modulatory processes in the dorsal and ventral visual pathways, focusing on two questions that have been addressed through analytical and simulation methods: (1) How can gain modulation be implemented by cortical microcircuitry? and (2) How can gain modulation be used to perform behaviorally useful computations?
Retinocentered receptive fields Neurons that respond to sensory stimuli are typically characterized by their selectivity, which is expressed in terms of a receptive field. In this context, we use a somewhat expanded definition o f a receptive field. For example, the receptive field of a visually responsive neuron defines not only the location in the visual field where an image must be placed to trigger a response, but also the specific image pattern that elicits the maximal response at a given stimulus intensity. The receptive field thus defines both the preferred location and preferred visual stimulus for a given neuron. These two receptive field attributes change progressively as information flows more centrally in the visual system, so that receptive field sizes tend to increase while preferred images become more complex. The receptive fields of neurons early in the visual pathway, such as those of retinal ganglion ceils and ceils in the lateral geniculate nucleus of the thalamus (LGN), are best described in retinal coordinates. This is because the locations of the receptive fields of these neurons are fixed to the eye or, equivalently, are always the same relative to the direction of gaze. Neurons in primary visual cortex or V1 are usually described in retinal coordinates as well, although recent reports suggest a more complex description (Guo and Li. 1997: Trotter and Celebrini, 1999; but see Sharma el al., 1999). A retinocentric receptive field is schematized in Fig. 1. At this level of processing the actual gaze direction, which is determined by a combination of eye and head positions with respect to the body, does not influence neuronal activity by itself. If an image is shown at
(b)
@ 4-
(c)
*+
[
Fig. 1. Early visual neurons operate in retinocentered coordinates. Thalamic visual neurons respond to contrasting center-surround patterns. The bars on the fight represent the expected firing rate of a hypothetical thalamic neuron in response to the images shown on the left. The cross indicates the fixation point, the location to which gaze is directed. (a) The image is aligned with the receptive field, and the neuron fires rapidly. (b) If the pattern is shown at the same physical location, but at a different position with respect to the fixation point, the neuron does not fire. (c) If the pattern is moved to the original location with respect to the fixation point the neuron responds again, regardless of the actual gaze angle. In other words, when the gaze direction changes, the receptive field moves with the eyes.
two different locations in visual space, the neuronal response does not change systematically as long as these locations correspond to the same position on the retina.
Gain modulation in parietal cortex Richard Andersen, initially working with Mountcastle (Andersen and Mountcastle, 1983) and later in collaboration with others (Andersen et al., 1985, 1990; Andersen, 1989; Brotchie et al., 1995), showed that, in contrasl to thalamic or retinal ganglion neurons, visual responses in parietal cortex depend on both the retinal location of a visual stimulus and on gaze direction. In these experiments, parietal neurons responded to spots of light located at various places within the visual field, ff gaze direction is held fixed and the response is plotted as a function of
177
(a)
(b)
::o@o
(c)
(d)
- IOO
{ '°°-I
80
'5.
".~
6o
> k~ O
-
_Fz ILl E3
100
J
J ILl 0
0 -40
'20
0
Elevation 1500 co .J J w 0
1000
o w
m
500
z -40
-20
Elevation Fig. 6. Distribution of corticopontine neurons in cat visual area 18. The PN was injected with large amounts of wheat germ agglutinin horseradish peroxidase and the retrogradely labelled neurons an the cortex were quantitatively mapped. The histograms show the distribution of corticopontine neurons in equally sized blocks of the lower visual field close to the vertical meridian (azimuth 0°-20°). Upper left: Densities of corticopondne neurons (cells/ram 2 cortex) decrease from the representation of the lower peripheral visual field (elevation - 5 0 °) to the central visual field representation (elevation 0%. Lower left: The number of corticopontine neurons devoted to each equally sized block of the visual field (cells/ram 2 cortex x m m 2 cortex/visual field block) is higher for blocks close to the central visual field representation. The perimeter chart shows the relative strength of the corticopontine projection from different parts of the visual field represented in area 18, based on quantitative data exemplified in the lower right histogram. Modified from Bjaalie (1985).
the parallel fiber to Purkinje cell synapse, can be applied to complex input patterns. Without such a recoding scheme perceptrons cannot learn to distinguish patterns that are not linearly separable (Minsky and Papert, 1969). Albus' paper contains a nice example where he shows how the recoding of mossy fiber input by the granular layer, which is in effect a combinatorial expansion by about two orders of magnitude, can circumvent this problem. In these theories the inhibitory Golgi cells control the activation threshold of granule cells, thereby keeping the number of active parallel fibers relatively small and constant over large variations in the number of active mossy fibers (Marr. 1969). This control over the number of active parallel fibers enhances the performance of the perceptron learning rule. A1bus (1971) used the term 'automatic gain control' to describe the role of the feedback inhibition by
Golgi cells. Overall this would restrain the number of active parallel fibers contacting a single Purkinje cell to 1% (Albus, 1971) or 0.3 to 6% (Marr, 1969). We have recently criticized the proposed gain control function of Golgi cells (De Schutter et al., 2000) and will not repeat our arguments in detail here. Instead we will focus on our recent modeling and experimental work, which suggests another function for cerebellar Golgi cells: the control over the timing of granule cell spikes (Maex and De Schutter, 1998a; Vos et al.. 1999a).
Golgi cells fire synchronously along the parallel fiber beam Our modeling studies of cerebellar cortex indicate that the cerebellar granular layer is highly prone to synchronous oscillations (Maex and De Schut-
287
o
g
8 D
o
i
D c 8
8
~ °
B
D
8
c
g
Q
e
5
g
m
o
a
8
5
c
O
n
8
~
3~
8
E
~°
o
o
200 ms
N
o ~
Fig. 7. Raster plot showing spike timing of 10 Golgi cells (upper part) and 300 granule cells (lower part). Initially the network is not activated and Only the Golgi cells fire occasionally. At the time indicated by the vertical line homogeneous 40 Hz mossy fiber input is applied and the complete circuit starts firing synchronously at a regular rhythm of about 20 Hz. Simulation of the standard network configurationdescribed in Maex and De Schutter (1998b) but with a more dense packing of the units (30 Gotgi cells, 21555 granule cells and 2160 mossy fibers~. ter. 1998b). A typical example is shown in Fig. 7. The raster plots show the activity in a large onedimensional network simulation, where all units are positioned along the parallel fiber axis. Initially, no
m o s s y fiber input is provided and the spontaneous activity of Golgi cells results in a l o w rate o f desynchronized firing. W h e n the simulated mossy fibers are activated, however, all Golgi cells synchronize
288 immediately and start firing rhythmically. In comparison, the granule cells show more complicated behavior. While they are also entrained in the synchronous oscillation, they fire less precisely and often skip cycles. The differences in behavior of individual granule cells in Fig. 7 can be explained by the randomization of connectivity and intrinsic excitability parameters (Maex and De Schutter, 1998b), indicating that such relative small sources of variability can generate complex activity patterns within the overall regular oscillation. The appearance of synchronous oscillations is explained by the intrinsic dynamics of the pure feedback inhibition circuit (Fig. 1). This can be easily understood by first considering the subcircuit consisting of a single Golgi ceil and its many postsynaptic granule cells. Inhibitory neurons exert a strong influence on the timing of action potentials in their target neurons (Lytton and Sejnowski, 1991', Cobb et al., 1995), The simulated granule cells will fire when inhibition is at its lowest, which is just before the next Golgi cell spike. Consequently the large population of granule cells postsynaptic to one Golgi cell will fire at about the same time. The loosely synchronous granule cell activity then excites the same Golgi cell and causes it to fire immediately, lea~ng to the establishment of a synchronized oscillation within this subcircuit, with granule spikes shortly preceding the Golgi celi spike. The long parallel fibers which are typical for the structure of the Cerebellar cortex couple many of these oscillatory subcircnits together. Common parallel fiber input will cause Golgi and granule cells located along the same transverse axis to fire (almost) synchronously. This is a dynamic property of the cerebellar circuitry; once the granular layer ~s activated sufficiently the most stable form of spiking is a synchronous oscillation (Maex and De Schutter, 1998b). The accuracy of this synchronization increases with increased mossy fiber activity, which also leads to an increased Golgi cell firing rate (Maex and De Schutter, 1998b), Consequently we expect to find a firing-rate dependency of the synchronization (Maex et al., 2000). As seen in Fig. 7 the synchronization is immediate upon activation; there is no delay due to the slow parNlel fiber conduction velocity. Finally, Golgi and granule cell populations are synchronized over the complete extent of the transverse
axis where mossy fibers are activated, even if this is much longer than the mean parallel fiber length of 4.7 mm (Pichitpomchai et al.. 1994). Because both cell populations fire in loose synchrony Golgi cell activity can be used to estimate the timing of granule cell spikes, though individual granule cells may skip cycles of the oscillation (Fig. 7). This is important as one cannot isolate single granule cells in vivo, while it is relatively easy to isolate Golgi cells (Edgley and Lidierth, 1987; Van Kan et al., 1993). These modeling predictions were confirmed using multi-single.unit recordings of spontaneous Golgi cell activity in the rat cerebellar hemispheres (Vos et al., 1999a). A total of 42 Golgi cell pairs in 38 ketamine-xylazine anesthetized rats were recorded. Of these, 26 pairs were positioned along the transverse axis (i.e. along the same parallel fiber beam), while the other 16 pairs were located along the sagittal axis (no common parallel fiber input). All transverse pairs except one showed a highly significant coherence measured as the height of the central peak in the normalized cross-correlogram. An example of such a cross-correlogram obtained from a pair of Golgi cells along the transverse axis is shown in Fig. 8. Conversely, in 12 out of 16 sagittal pairs no synchrony could be found~ The remaimng four sagittal pairs showed low levels of coherence, but in each of these pairs the Golgi cells were located within 200 ~m from each other. We assume that in these latter four pairs the cells were so close to each other that their dendritic trees overlapped (Dieudonn6, 1998), allowing them to sample common mossy and/or parallel fiber input despite their parasagittal separation. These findings confirmed the main prediction of the network simulations: Golgi cells along the parallel fiber beam fire indeed synchronously. Additionally, as predicted, the accuracy of synchronization, evidenced by higher and sharper central peaks in the cross-correlogram, increased with the Golgi cell firing rate (fig. 1 of Vos et al., 1999a), This indicates that synchronization may be much more accurate in awake animals, compared to the loose synchrony observed in the anesthetized rat. The only data presently available from awake animals are field potential recordings (Pellerin and Lamarre, 1997; Hartmann and Bower, 1998). These studies have also demonstrated the presence of oscillations in the gran-
289 7
6
iI
5
82
1 0
...........
-1 -2
.--t----
-250 -200 -150 -100 -50
0
50
100
150
200
250
Time lag (ms) Fig. 8. Cross-correlationof spontaneous activityof two Golgi cells receivingcommonparallel fiber input. The highly significantcentral peak (the cross-correlogramhas been normalizedfor firing frequency)is indicativeof synchronousfiring which is not veryaccurate as the peak is wide. This is the same Golgi cellpair as of Fig. 4. See Vos et al. (1999a) for experimentaland statistical procedures. ular layer that may correspond to those predicted by the model.
The effect of spatially localized mossy fiber input In the previous section we considered the synchromzation of Golgi c~lls in response to a spatially homogeneous mossy fiber input in the network simulations and compared this to experimental data obtained without any stimulation. This is of course a rather artificial assumption. Considering the patchy receptive fields i n the granular layer (Fig. 2) one expects stimulation to cause spatially heterogeneous mossy fiber activation. Recent modeling work in our laboratory demonstrates that a similar behavior can be obse~ed when a patchy mossy fiber input is applied. Specifically, if two patches ~ activated by comparable levels of mossy fiber input they will synchronize immediately, even if separated by a few millimeters of only lowly activated granular layer (Franck et al., 2001)~ This can be observed in Fig. 9A, which shows the spike trains of the model Golgi cells and of a subset of granule cells in a 0he-dimensional model with strong ~'eedback ~ b i f i o n . Activation of two small patches (200 t~m diameter c o n t ~ n g about 50 mossy fibers and 500 granule celts each, 1 mm separation) leads tO the immediate synchronization of all Golgi cells along about 6 mm of the parallel fiber beam overly-
ing the patches. As parallel fibers are 5 mm long in the network model, this means that all Golgi cells receiving input from the two patches become entrained in the rhythm, though the synchronization is clearly less robust than that evoked by homogeneous input (Fig. 7). The Other Golgi cells in the network fire iesS than before or are hardly affected at all. The picture looks somewhat different for granule cells. In Fig. 9 only a small subset of granule cell traces can be shown, so the borders of the patches are not represented. It can nevertheless be seen that they are activated inside the patches only. Between the two patches and at the outer borders of the patches they are actively inhibited by the activated Golgi cells. The granule cell activity within and between the two patches is highly synchronized. Like for the fully activated network (Fig. 7), granule cells spike together with Golgi cells, but sometimes skip cycles. In addition granule cells sometimes fire bursts of two spikes. The latter behavior was even more pronounced when the network parameters from previous studies (Maex and De Schutter, 1998b) were used (to diminish bursting in Fig. 9A the synaptic strengths of parallel fiber and Golgi cell synapses have been raised, making the feedback inhibition loop stronger). The possible importance of granule cell bursting for induction of synaptic plasticity at the parallel fiber to Purkinje cell synapse (Linden and Connor, 1995: Finch and Augustine. 1998) has
290
1
©
o
.I r~
r~
-SJ ---~q i
.
i
-
0
...,-
q
_
L
!q
:"
Y
,,,,1
"1
~-~
q -
3), such as the multiple visual features represented concurrently in the ventral stream of primate visual system, more information can be coded per neuron by broader tuning. Although sharpening makes individual neurons appear more informative, it reduces the number of simultaneously active neurons, a factor that dominates in higher dimensions where neighboring tuning functions overlap more substantially. On the other hand, sharpening can always improve the Fisher information coded per spike and thus energy efficiency for spike generation (Zhang and Sejnowski, 1999a). The universal scaling law in Eq. 16 still holds when the firing rate fluctuations of different neurons are weakly correlated. The result is
J -~ o(yD-2 ( -Aq-- B =
.
which depends only on the Euclidean distance to the center c. Here o- is a parameter that describes the tuning width, which scales the tuning function without changing its shape. This general formula includes all radial symmetric functions, of which gaussian tuning is a special case. The probability P ( n I x, r) for n spikes to occur within a time interval of length ~: can be an arbitrary function of the mean firing rate f ( x ) . These general assumptions are strong enough to prove that the total Fisher information has the form: J oc O~rD-2.
q
(15)
(16)
+
B
)
,
(17)
ignoring contributions from terms slower than linear with respect to the population size, where A and B are constants independent of the correlation strength q (Zhang and Sejnowski, 1999a). Here the noise covariance between neurons i and j is given by the average:
Cij -~"E [ ( n i
-
IJ.i)(rtj -/~j)]
JC 2
i f / = j,
| CiCj
otherwise,
(18)
where Ci = ~(/xi) is an arbitrary function of the mean number of spikes #i = E[nj] within certain
337 time interval. The spike statistics are assumed to follow a multivariate gaussian distribution Thus the noise correlation does not affect the scaling factor t/o-p-2, and the proportional constant is affected by the factor 1/(t - q), which slightly increases the Fisher information when there is positive correlation (q > 0). An intuitive derivation of the universal scaling factor ~o-D-2 is as follows. The average square error for estimating the encoded variable from the activity of a population of neurons should scale as: 0-2 E [ 8 2] (X
.
(19)
N"
where N ' is the total number of activated neurons for each fixed value of the encoded variable. The factor t / N ' may be justified by the square root law for using a large: number of independent neurons. The factor 002 arises because of the dimensionality reqmrement: t f the length scale of the encoded variable is changed, b o t h the value of the tuning width and the square error shoutd change accordingly, leading to the factor 002. Because each neuron is tuned with width 00 in each of the D dimensions, the number of activated neurons in response to a fixed value of the encoded variable should be proportional to: N t c< ~cr D
(20)
where r~ is the density of neuron as before. Substituting Eq. 20 into Eq. 19 yields the average square error:
1
E[e 2] e~
D-~' (21) ~Ta Using Eq. 5, we see that the Fisher information should scale as ~TaD-2, which is the same as Eq. 16. More generafly, if the tuning function is not radially symmetric but has different widths for different dimensions, then Eq: 20 should be replaced by (22)
N ' cv ~7001cr2 . . - 00D,
so that the average square error for the i-th dimenstun should be: E[g 2] o( 00~ o( N'
o.2
.
(23)
r ] o t 0°2 • • • 00D
A formal derivation of this result using Fisher information is given in Eurich and Wilke (2000). If
the width o.i in dimension i is sharpened while the widths of all other dimensions are kept constant, the error should decrease in proportion to 00i, consistent with the previous result for a one-dimensional problem. When the tuning widths of different dimensions are identical: o.1 = o-2 . . . . . 00, the factor ~/o.D-2 is recovered.
Origin of cosine tuning function In the above, different shapes of the tuning functions are directly used without considering why a particular shape exists. There is a general argument for why the cosine tuning curves are so widespread in the sensory and motor systems (Zhang and Sejnowski. 1999b). A cosine tuning function implies a dot product between a fixed preferred direction and the actual movement direction (Georgopoulos et al., 1986; Schwartz et al.. 1988). This suggests a linear relation with the movement direction, although the actual coding may have various forms (Mussa-Ivaldi. 1988; Sanger, 1994). The general argument below exploits the inherently low dimensionality of natural movements and shows that neuronal activity tuned to movement often obeys simple generic rules as a first approximation, insensitive to the exact sensory or motor variables that are encoded and the exact computational interpretation (Zhang and Sejnowski. 1999b). Consider the reaching problem and assume that the mean firing rate of a neuron relative to its baseline is proportional to the time derivative of an arbitrary unknown function of hand position in space during stereotyped reaching movement. As a linear approximation, the firing rate can always be written as f
= f o + p v cos oe,
(24)
where f0 is the baseline rate. p is a constant, v is the instantaneous reaching speed of the hand. and ol is the angle between the preferred direction and the instantaneous reaching direction. A similar linear approximation applies to the visual system. In response to a three-dimensional object moving at translational speed v and angular speed w, the mean firing rate of a motion-sensitive neuron should be given by f = f0 + p v c o s ~ t q w c o s f l ,
(25)
338 where f0 is the baseline rate, p and q are constants, o~is the angle between the instantaneous translational velocity and a fixed preferred direction, and fi is the angle between the instantaneous angular velocity and a fixed preferred rotation direction. The assumption here is that the firing rate is proportional to the time derivative of an arbitrary function of the position and orientation of the object in three-dimensional space (Zhang and Sejnowski, 1999b). Thus, given a particular view of a particular object, the response above baseline is predicted to be the sum of two components, one translational and one rotational, each with cosine tuning and multiplicative modulation by speed and angular speed, respectively, In the motor system, broad cosine-like tumng curves have been observed in many brain areas, including the primary motor cortex, premotor cortex, parietal cortex, cerebellum, basal ganglia, and somatosensory cortex. The tuning rule in Eq. 24 can account for various experimental facts besides the cosine shape, including multiplicative speed modulation, trajectory reconstruction by the population vector, curvature power law. elbow position, and the linear relation between the spike count and reaching distance. In the visual system, the predicted tuning rule in Eq. 25 has not been tested with realistic moving three-dimensional objects. A partial confirmation of a special case of this tuning rule is provided by neurons selective to wide-field spiral visual motion with broad cosine tuning in monkey medial superior temporal area (MST), the ventral intraparietal area (VIP), and the parietal area 7a. because such motions may be generated plausibly by a large moving planar object facing the observer. A population of such neurons tuned to three-dimensional object movement might be useful for updating static-view representations in the ventral visual stream (Zhang et al., 1998).
Optimal decoding by Bayesian method Various methods have been used to 'decode' or read out information from the spike trains of a population of neurons. Within a probabilistic framework, the theoretically optimal methods are based on the Bayes rule (F6ldifik, 1993; Sanger, 1996; Zhang et al., 1998: Brown et al.. 1998).
A popular reconstruction method is called the population vector scheme, first used in motor cortex, where the average firing rate of a given neuron is maximal when the arm movement is in a particular direction, known as the preferred direction for that neuron (Georgopoulos et al.. 1986). The population vector method estimates the direction of arm movement by summing the preferred direction vectors weighted by the firing rate of each neuron. A more general approach to reconstruction is to allow the neurons to represent more general basis functions of the physical variables (Girosi and Poggio, 1990; Pouget and Sejnowski, 1997), Each neuron contributes a basis function in this space of variables whenever it fires, and the best estimate of the physical variables can be computed from the sum of these functions weighted by the number of spikes occurring in each neuron during a time interval. An alternative method for decoding a population code is based on Bayesian reconstruction or maximum likelihood estimation. These optimal probabilistic methods take into account prior probabilities and attempt to reconstruct the entire probability distribution. Instead of adding together the kernels, as in the basis function method, the probabilistic approach multiplies them, assuming that the spikes in each neuron are independent. An example of how the timing of spikes in a population of neurons can be used to reconstruct a physical variable is the reconstruction of the location of a rat in its environment from the place cells in the hippocampus of the rat. The place cells are silent most of the time, and they fire maximally only when the animal's head is within a restricted region in the environment called its place field (Muller et al., 1987: Wilson and McNanghton, 1993). The reconstruction problem is to determine the rat's position based on the spike firing times of a few dozen simultaneously recorded place cells. Examples of the Bayesian method and the direct basis function method applied to the place cell data are shown in Fig. 2. The Bayesian reconstruction method works as follows. Assume that a population of N neurons encodes several variables (xl, x2 . . . . ), which will be written as vector x. Here x is the position of the animal in the maze. From the number of spikes
339 Time = 1 sec
Time = 0 see
Time
=
2 sec
True Position
L
Probabilistie Method
Direct,~sis Method
Fig. 2; Predictlng the. position Of a freely moving rat from the spike trains of 25 simultaneously recorded hippocampal place cells. distribution density for the position Of the rat are compared with its true position, which occupied a single pixel on the 64 x 6 4 grid, corresponding to 111 x I 11 cm in real space. The probabilistic method or Bayesian method often yielded a sharp distribution with a single peak by combining the place fields multiplicatively, whereas the direct basis, method typically led to a broader distrib~ation with multiple peaks by combining the place fields additively. Taken from Zhang et al. (1998) with premission.
SnapshOtSbftfie:reconstmcted
n = (nl, n2~.. : , nA0 .fired by the N neurons within a time interval ~:, we w a n t t o estimate the value o f x using the Bayes rule for conditional probability: P ( x l n ) = P ( n [ x)P(x)/P(n),
(26)
assuming independent Poisson spike statistics of different neurons. The final Bayesian estimate is: P ( x ~ n)
prior probability for the animal to visit position x, which can be estimated from the data, and fi(x) is the empirical tuning function, i.e. the average firing rate of neuron i for each position x. Examples of the probability P (x n) for the probable position of the rat is shown in Fig. 2. The most probable value of x can thus be obtained by finding the x that maximizes P ( x n), namely, = argmax P ( x I n).
= kP(x)
'
\i=1
3}(x) nl /
exp - r \ i=1
3~(X )
, / (27)
where k is a norma_lization constant. P(x) is the
(28)
By sliding the time window forward, the entire time course of x can b e reconstructed from the time varying-activity of the neural population.
340 A comparison of different reconstruction methods for this problem shows that the Bayesian reconstruction method was the most accurate (Zhang et al., 1998). As the number of neurons included in the reconstruction is increased, the accuracy of all the methods increased. The best mean error using 25 30 simultaneously recorded cells was about 5 cm, in the range of the intrinsic error of the infrared position tracking system. Alternative formulas were derived from Bayes rule assuming a gaussian model of place field and updating the estimated position only when a spike occurs (Brown et al.. 1998), although its accuracy seemed to be slightly lower than that of the Bayesian method above (Chan et al., 1999). There are thousands of place cells in the hippocampus of a rat that respond in any given environment. However, it is not known how this information is used by the rat in solving spatial and memory problems.
Synaptic learning for Bayesian decoding Although these reconstruction methods may be useful for telling us what information could be available in a population of neurons, it does not tell us what information is actually used by the brain. In particular. it is not clear whether these reconstruction methods could be implemented with neurons. Pouget et al. (1998) show how maximum likelihood decoding can be performed using the highly recurrent architecture of cortical circuits, and thus demonstrate that the theoretical limit corresponding to the Fisher information is achievable. Zhang and Sejnowski (1999b) show how a feed:forward network with one layer of weights could in principle read out a Bayesian code. Thus, optimal decoding is well within the capability of the network mechanisms known to exist in the COrtex.
How a feedforward network can implement the Bayesian decoding method is shown in Fig. 3. The first layer contains N cells tuned arbitrarily to a variable of interest x, and the tuning function f/(x) of cell i describes its mean firing rate. The cells in the second layer represent the value of the encoded variable x (discretized if it is continuous) by their locations in the layer. Let function ~i(x) be the synaptic connection weight from cell i in the first layer to cell x in the second layer. Given the numbers of spikes hi, n2 . . . . . nN from the N cells in the first
Neurons in Layer 2
1
2
1
2
3
i
3 Neurons in Layer 1
N
Fig. 3. The Bayesian decoding method can be implementedby a feedforward network whose synaptic strength can be learned by a Hebbian learning rule that is proportionalto the presynaptic firing rate on a log scale, suggesting that the biological system might possibly be able to team to use the theoretically optimal Bayes rule for reading out information contained in a neuronal population. layer within a unit time interval, the distribution of activity in the second layer computes the sum N
ni qSi(X).
(29)
i=1
To reconstruct the true value of x, the cell with the highest activity in the second layer should be chosen with a winner-take-all circuit. Different reconstruction methods correspond to different basis functions 4)i(x), as shown in more details in Zhang et al. (1998). A constant bias term A(x) that is independent of the activity may be added to implement the Bayesian rule in Eq. 27. We only need the basis function: Oi (X) (3( log
fi (X),
(30)
because by taking logarithm of Eq. 27. the only term depending on the spikes /iti is the sum Y~Ni=1 ni log fi(x). Thus, the synaptic weight should be proportional to the logarithm of the tuning function. Here we show that a simple Hebb rule is sufficient to establish the weights needed for the Bayesian method. We propose that the synaptic weight should change according to:
A W o~ Post x log(Pre)
(31)
where for the presynaptic cell, Pre = firing rate. and for the postsynaptic cell. Post may be taken as
341
a binary variable, During training, the activation at the second layer is confined to a single unit corresponding to the true Value of the encoded variable, which sho.uld vary, sampting all possible values unifomfly. It can be shown ihat the final outcome of this learning is the synaptic weight patterns that are proportional to the average firing rate of the presynaptic cell in Iog units. R a t is, the requirement in Eq. 30 is satisfied after the training. Thus, the Bayesian decoding method considered in the preceding section can be implemented by a feedforwatd network and the synaptic weights can be learned with a synaptic weight plasticity proportional to presynaptic firing rate at logarithm scale. The bias or prior in the Bayesian method should be imptement~ as a tonic input independent of the activation of the cells in the first layer, so that when the context changes, the tonic input should switch accordingly: The learning rule shows how to pool a bank of existing feature detectors with Poisson spikes to quickly develop new detectors at the next stage that are optimal. In particular, this may be used to develop detectors for complex patterns. such as a "grandmother cell'. Overall it does not seem to take much to implement Bayesian formula to achieve optimal performance in the special cases considered above, even though an explicit readout of a population code may not be needed until the final common pathway of the motor system since projections between cortical areas may simply perform transformations between different population codes, References Abbott. L.F. ,and Dayan, R (1999) The effect of correlated variabitity on the accuracy of a population code. Neural CompuL. 11: 91-101. Baldi. R and Heiligenberg, W. (1988) How sensory maps could enhance resolution through ordered arrangements of broadly tuned receivers. Biol. Cybernet.. 59: 313-318. Brown. E.N.. Frank. L.M.. Tang, D., Quirk. M.C. and Wilson, M.A. (1998) A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J. Neurosci.. 18:74117425. Brunel, N. and Nadal, J.-R (1998) Mutual information. Fisher information, and population coding. Neural Comput.. 10: 17311757. Chart. K.-L.. Zhang~ K.-C.. Knierim. J.J.. McNaughton. B.L. and Sejnowski, T.J. (1999) Comparison of different methods for
position reconstruction from hippocampal place cell recordings. Soc. Neurosci. Abstr.. 25: 2166. Cover, T.M. and Thomas. J.A. (1991) Elements of Information Theory. Wiley, New York. Eurich. C.W. and Wilke. S.D. (2000) Multi-dinlensional encoding strategy of spiking neurons. Neural Compur., 12: 15191529. F61difik, R (1993) The 'ideal humunculus': Statistical inference from neural population responses. In: E Eeckman and J. Bower (Eds.), Computation and Neural Systems 1992. Kluwer Academic. Norwell, MA. Frieden, B.R. (1998) Physics from Fisher Information: A Unification. Cambridge University Press. Cambridge. Georgopoulos, A.E. Schwartz. A.B. and Kettner. R.E. (1986l Neuronal population coding of movement direction, Science. 233: 1416-1419. Girosi. E and Poggio, T. (19901 Networks and the best approximation property. Biol. Cybernet.. 63: 169-176. Hinton. G.E.. McCMland. J.L. and Rumelhart. D.E. (1986) Distributed representations. In: D.E. Rumelhart and J.L. McClelland (Eds.). Parallel Distributed Processing, Vol. 1, MIT Press. Cambridge, MA, pp. 77-109. Lehky, S.R. and Sejnowski, T.J. (1990) Neural model of stereoacuity and depth interpolation based on a distributed representation of stereo disparity. J. Neurosci.. 10: 2281-2299. Muller, R.U.. Kubie. J.L. and Ranck Jr.. 1.B. (1987) Spatial firing patterns of hippocampal complex-spike cells in a fixed environment. J. Neurosci.. 7: 1935-1950. Mussa-Ivaldi. EA. (1988) Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosct. Lett., 91: 106-111. Paradiso. M.A. 11988) A theory for the use of visual orientation information which exploits the columnar structure of striate cortex. Biol. Cybernet.. 58: 35-49. Pouget, A. and Sejnowski. TJ. (1997) Spatial u'ansformations in the parietal cortex using basis functions. J. Cognit. Neurosci.. 9: 222-237. Pouget. A.. Zhang, K.-C.. Deneve, S. and Latham. RE. (1998) Statistically efficient estimation using population code. Neural Comput.. 10: 373-401. Rieke. E. Warland. D.. de Ruyter van Steveninck. R. and Biatek, W. (1997) Spikes: Exploring the Neural Code. MIT Press, Cambridge. MA. Sanger. T.D. (1994) Theoretical considerations for the analysis of population coding in motor cortex. Neural Compur.. 6: 12-21. Sanger, T.D. (1996) Probability density estimation for the interpretation of neural population codes. J. NeurophysioL. 76: 2790-2793. Schwartz. A.B.. Kettner. R.E. and Georgopoulos, A.R (1988) Primate motor cortex and free ann movements to visual targets in three-dimensional space. I. Relations between single cell discharge and direction of movement. J. Neurosci., 8: 29132927. Seung, H.S. and Sompolinsky, H. (1993) Simple models for reading neuronal population codes. Proc. Natl. Acad. Sci. USA. 90: 10749-10753.
342 Snippe, H.E (1996) Parameter extraction from population codes: a critical assessment. Neural Comput., 8:511-529. Snippe, H.E and Koenderink, J.J. (1992) Discrimination thresholds for channel-coded systems. Biol. Cybernet., 66: 543-551. Wilson, M.A. and McNaughton, B.L. (1993) Dynamics of the hippocampal ensemble code for space. Science, 261: 10551058. (Corrections in Vol. 264, p. 16). Wu, S., Nakahara, H., Murata, N. and Amari, S. (2000) Population decoding based on an unfaithful model. In: S.A. Solla, T.K. Leen and K.R. Muller (Eds.), Advances in Neural Information Processing Systems, Vol. 12. M1T Press, Cambridge, MA, pp. 192-198. Yoon, H. and Sompolinsky, H. (1999) The effect of correlations on the Fisher information of population codes. In: M.S. Kearns, S.A. Solla, and D.A. Cohn (Eds.), Advances in Neural Information Processing Systems, Vol. 11, MIT Press, Cambridge, MA, pp. 167-173.
Zhang, K.-C. and Sejnowski, T.J. (1999a) Neuronal tuning: to sharpen or broaden? Neural Comput., 11: 75-84. Zhang, K.-C. and Sejnowsld, T.J. (1999b) A theory of geometric constraints on neural activity for natural three-dimensional movement. J. Neurosci., 19: 3122-3145. Zhang, K.-C., Ginzburg, I., McNaughton, B.L and Sejn0wsld, T.J. (1998) Interpreting neuronal population activity by reconstruction: unified framework with application to hippocampal place cells. Z Neurophysiol., 79: 1017-1044. Zohary, E. (1992) Population coding of visual stimuli by cortical neurons tuned to more than one dimension. BioL Cyberner, 66: 265-272. Zohary, E., Shadlen, M.N. and Newsome, W.T. (1994) Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 370: 140-143.
M.A.L. Nicolelis ~Ed.)
Progressin BrainResearch_Vol.
130 © 2001 Elsevier Science B.V. AII rights reserved
CHAPTER 22
What ensemble recordings reveal about functional hippocampal cell encoding Robert E. Hampson *, John D. Simeral and Sam A. Deadwyler Departrr~,m of Physiology and Pharmacology, Wake Forest University School of Medicine, Winston-Salem, NC 27157-1083, USA
Introduction
One of the key advantages of recording from neuronal ensembles in the hippocampus, or any brain area. is the potential for deciphering behavioral and cognitive correlates. Single neuron recordings have been made in all major subfields in the hippocampus. and cell identification via firing signature is not a problem i n m o s t cases (Fox and Ranck Jr.. 1981: Christian and Deadwyler, 1986) (Freund and Buzsaki, 19%). However. with the exception of place field firing, recordings of single hippocampal neurons reveal encoding o f only a 'pieces' of the information puzzle required to be encoded d ~ g ongoing behavioral performance fHampson and Deadwyler, 1996a). In order to extract the complete 'code' it is therefore necessary to record from ensembles of hippocampal neurons and employ population or multivariate analyses to extract such Codes (Hampson and Deadwyler, 1998, 1999). There are several means of constructing ensembles of neurons: either from combined single neuron recordings ('Serial' reconstruction) or from simultaneous recordings of multiple neurons ('parallel' recording). 'Serial' reconstruction of ensembles by combining single neuron recordings has previously
* Corresponding author: Robert E. Hampson, Department Of Physiology and Pharmacology, Wake Forest University School of Medicine. Winston-Salem. NC 27157-1083. USA. Tel.: +1-336-716-854t: Fax: +1-336-716-8501: E-mail:
[email protected] been used correlate behavioral correlates with ensemble activity lMiller et al., 1993; SChultz et al.. 1993; Gochin et al., 1994). However. such techniques applied to ensembles of hippocampal neurons have failed to reveal encoding any more than simultaneously recorded neurons (Hampson and Deadwyler. 1996a). One reason for this result is that serially recorded data increases uncorrelated noise, and it may not be possible to isolate some of the ensemble information code from this elevated noise level. Another reason may be that serial recording of single neurons does not account for 'local circuits' within the hippocampal anatomy which may influence the activity of neurons recorded in different locations. In other words, the activity of one neuron in a circuit influences the activity of other neurons within the circuit, however, when only a single neuron is recorded, this covariance cannot be determined: Despite differences in the neural 'codes' determined using serial vs. parallel recording techniques. The single neuron and ensemble codes are likely mutually dependent. Once a code is identified within the ensemble, it should be possible to determine how single neurons contribute to the ensemble pattern. Identification of robust single neuron encoding may only be possible within the context of the ensemble code. thus revealing codes that would be missed by serial recording of single neurons. Multineuron recordings using a recording electrode 'array' (Deadwyler et al., 1996) with electrode tips to record positioned electrodes along specific anatomic projections can be used to simultaneously record ensembles of neurons within hippocampus. Once a population of neurons is
346 recorded, ensemble analyses can proceed using multiple cross-correlations, population vector analysis, or linear discriminant analyses. Given these factors, it should be possible to determine how single neuron activity within hippocampus circuits is integrated within ensemble codes for behavioral and cognitive events. Identifying information encoded by neural ensembles As stated in a previous report (Hampson and Deadwyler, 1999), cross-correlation analyses are useful in examining functional connectivity between neurons. In an ensemble recorded from a patterned array of electrodes, this may be useful in defining neural circuits across the hippocampus. However, since crosscorrelations can only be practically calculated for pairs of neurons, this technique is not suited to extracting simultaneous firing across ensembles of neurons. In addition, recent reports by Gerstein (Bedenbaugh and Gerstein, 1997) have shown that incomplete separation of multiple neuron recordings from single electrodes result in a contamination of crosscorrelations between neurons recorded from different electrodes. Alternatively, population vector analyses can take advantage of correlated firing between neurons to extract patterns of firing across the whole ensemble. Vector analyses have been successfully applied by Georgopoulos, Schwartz and others (Georgopoulos et al.. 1989; Moran and Schwartz, 1999a,b; Georgopoulos, 1995; Pellizzer et al., 1995) to extract ensemble codes for directional and velocity of motor movements. Since the population vector treats instantaneous firing rate of all neurons in the ensemble as a single vector in multidimensional space. different behavioral 'states' can also be represented by unique population vectors (i.e. vectors pointing in different 'directions') (Chapin et al.. 1989). One weakness of both correlational and vector analyses is that they do not provide a means of identifying codes for multiple pieces of information that are represented within the ensemble at the same time. Identification and separation of multiple overlapping codes requires a multivariate analysis, such as linear or canonical discriminant analysis (CDA). This type of analysis subdivides the sources of variance within the ensemble firing into components which can be used to discriminate the behavioral events
encoded by the ensemble activity (Deadwyler et al., 1996; Hampson and Deadwyler, 1998). The CDA used extensively in our laboratory is a form of linear discriminant analysis which has allowed the identification of variance sources corresponding to three orthogonal dimensions of a delayed-nonmatch-to-sample (DNMS) behavioral task (task phase, response position and trial performance; see Deadwyler et al., 1996: Hampson and Deadwyler, 1996a). The analysis revealed that the ensemble activity s i m u l t a n e o u s l y encoded different types of information that was critical to performance of the task (Hampson and Deadwvler. 1996b; Deadwyler and Hampson. 1997; Hampson et al., 1999). Although the CDA operated on fixed temporal intervals corresponding to behavioral events within the trial, later analyses revealed that the task-specific information appeared at other times during the task as well (Hampson and Deadwyler, 1999). Where does ensemble information come from? Was the above successful determination of an ensemble code for behavior due in large part to the simultaneous nature of the ensemble recordings? In other words, was the code broadly distributed across neurons? or did each neuron encode a portion of the code with multiple neurons adding up to a complete code? Although these two questions appear similar. in fact, they imply two different underlying processes. If the ensemble code is broadly distributed across neurons then examination of each neurons' firing would reveal only a small fraction of firing related to behavioral events. Only when a large number of such small fluctuations occur synchronously would a coherent pattern (i.e. 'code') be detected. On the other hand. if each neuron encoded a distinct portion of the complete code. it would be possible to detect that for a single cell (even if encoding were incomplete). The synchronized successive firing of all neurons, each encoding different aspects of each behavioral event, would be revealed across the ensemble activity as a complete code for the event. The above scenario leads to two possible means of deciphering the means by which information is encoded by neural ensembles: a top down model implies that information is d i s t r i b u t e d a c r o s s neurons, while a bottom-up model suggests that n e u r o n s
347
ehse/nblecOde! .. Un y,. i f the inforrnatior~ within neural ensembles .is~b0th distrib~ted iind locally encoded. it: may .be ~ c u l t to identify the specific aspects Of single rteuron, firing that Critically correspond to the enCoded.:~enf,-This is the case with ensembles of h,ippoc~pal neurons, where prior investigations have r e p o ~ both :iocal (Hamps0n et al.,: 1993; Heyser et~aL].-t993)ias well as distributed encoding (Deadwyiet et ai., i996; Hampson and Deadwyler, 1996a,: !999;Deadwyler and Hampson, 1997). One metals of bridgiag:.~s problem is to first extract v~ance C0m~0nents :fr0m the entire ensemble firing pattern¢ ther1?i~ntify ~pecific neurons that contribute tO those, v ~ a n c e components. I n this manner, both local :and d~s~butrdencoNng of information is address.~, within .the same ,ensemble: using statistic~ constraints ':that: a~?oid over-interpretation of 'weak firing correlates.
A recent refinement ~of .the CDA has been the imp!ementati0n .of.;a 'sli:ding analysis window', which allows exami.nation o f ensemble encoding for all temporal segments: :throughout all phases the task (Hampson -andDeadwyler, 1998, 1999). The CDA is iterativety ~ealent:ated throughout segments of prescribed duration by sliding the analysis '.window' of discri~nanf ftinction',calculations across successive temporaI, segments., By using the same coefficients of the discriminant..functions derived :from; key events •
i
within the trial (Fig. t), the state of.extracted vailance sources can be determined.throughout the.trial. Dfita was :cOllected'from 23 ~Nals"hsing the electrode array system (Fig. 1A) duriag..performance of a DNMS task. Each animal (mate L0ng,Evans rats) performed the task by pressing .~. leftilor right presented lever in the sample phase,., noseP01dng into a lighted device on the opposite wall-'fbr the duration of a random 1-40 s Delay phase, then.,pressing .the opposite lever during the nonmatch phase (Hampson et al., 1993; Deadwyler et al., 1996). The CDN was applied successively to the intertrial interval (ITI), sample response (SR), last nosepoke (LNP) in the delay internal, and nonmatch response (Nit). events (Fig. 1B). Five sets of canonical discriminant functions were derived, with the first three corresponding to task phase and type of responses and the fourth and fifth corre~pon~ng io"levers pressed in the sample and nonm~ch i'~phaseS.(cf. ~Deadwyler : et-at~,.:199)5). By reassessing discriminant functions derix~ed from one aspectof the taskat all other:thins Wittfin the trial, patterns o f ensemble.firing Corresponding to these factors predictably varied in accordance with the behaviorai contingenciesof ihe Nsk (Hampson and Deadwyler, 1999). The plot in Fig. 1C shows discriminant scores throughout the entire trial, for three of the discriminant functions: CAN1 which tracked task phase (38.% of total variance), CAN4 which, tracked lever response position (10% of Variance), and CAN5 which tracked the type of trial (8% of varianceS. Each function shows firing changes during the trial which decreased across the delay interval and were increased during sample and nonmatch phases. Note that CAN1 and CAN4 showed 'crossover' from.sam, ple to nonmatch in accordance with either the-different phase (CAN1), or the opposite lever response (CAN4) required by the DNMS task. There was no difference in CAN1 variance for left and right lever trials as determined by sample lever position. However. variances for CAN4 and CAN5 exhibited mirror image differences between the two .trim types defined by the encoded sample lever. While CAN1 did not discriminate position, discriminant scores were quite, significantly different (F(1,1213) = 11.2. P < 0.00D for sample vs. nonmatch events, but not for lever position (F(1,1213) = 1.9, P = 0.16). Scores for CAN4 were
348
C.
A,
CANt- Trial Phase
3
QQ
~
2 1
¢
u}
1
n"
0
-2 -3
d u" sK" ;R" "; "i0 "~;" ?0" i ; ;0" ~; "iN;"NR"~l'"
B.
1
S D
o
m
-1
m'" "s;'" "s~'";" "~;""~;"~" "2;""~" "3;'i.'; "N;i",g"
Hz
,,=,
o
,.. z
SR
10
20 30 Delay (sec)
LNP NR
k~
-1
~.
-2 ITI
SP
SR
5
10
15
20
25
30
35 L ~
NR
m
Trial Phase
Fig. 1. Ensemble recording and analysis during delayed-nonmatch-to-sample (DNMS) task. (A) Example of a electrode array for hippocampal recording. Array consisted of two rows of eight microwires (28-56 Ixm diameter) at 200-~m intervals oriented to traverse longitudinal axis of htppoeampus. Rows were fixed 800 txm apart, with 1200 ~,m difference in length to allow simultaneous placement in CA1 tdepth approximately 2.8-3.2 mm) and CA3 (depth approximately 4.0-4.4 ram) (Deadwyler et al.. 1996). (B- Ensemble histogram from a single animal showing averaged firing distribution of all cells (n = 16) during the entire DNMS trial. SR sample presentation: SR. sample response; LNR last nosepoke during delay; NR, nonmatch response. The canonical discriminant analysis (CDA) was calculated using ensemble firing within a 3-second interval around SR. LNP. NR, and a point 2 s prior to the start of the trial during the intertrial interval (ITI ~. The 'sliding' CDA consisted of applying the discriminant functions to 3-s intervals throughout the trial fcross-hatched and shaded band) to calculate discriminant scores for the complete trial. (C) Discriminant scores for three canonical discriminant functions (CAN1. CAN4, CAN5) obtained from the CDA (see text). For each function, discriminant scores were calculated at3-s intervals through DNMS trials, sorted by trial, and averaged within each trial type. Filled circles represent CDA scores on left trials (initiated by a left sample responses, while unfilled circles represent scores from right trials (initiated by a right sample response). Scores are shown for CANI. which discriminated sample from nonmatch phase responses; CAN4, which discriminated left from right responses irrespective of phase: and CAN5 which discriminated all responses for a given trial type irrespective of phase and actual response position). s i g n i f i c a n t l y d i f f e r e n t (E1.1213~ = 7.4. P < 0 . 0 1 ) f o r l e f t vs. r i g h t trials, a n d as e x p e c t e d , w e r e sign i f i c a n t l y d i f f e r e n t (FcH213, = 9.6, P < 0 . 0 0 1 ) f o r s a m p l e vs. n o n m a t c h r e s p o n s e s as r e q u i r e d b y t h e ' D N M S ' c o n t i n g e n c y . S c o r e s for C A N 5 w e r e s i g n i f i c a n t l y d i f f e r e n t (F(1,1213~ = 9.1, P < 0 . 0 1 ) t h r o u g h o u t all p h a s e s o f left vs. r i g h t trials, i n c l u d -
i n g t h e delay, b u t w e r e n o t s i g n i f i c a n t l y d i f f e r e n t
(F(1,1213) --~ 2.7, P = 0 . 1 1 ) f o r s a m p l e vs. n o n m a t c h r e s p o n s e s w i t h i n t h o s e s a m e trials. T h i s f u n c t i o n ( C A N 5 ) t h e r e f o r e e n c o d e d trial type, b u t d i d n o t d i s c r i m i n a t e i n d i v i d u a l e v e n t s w i t h i n t h e trial (i.e. s a m p l e vs. n o n m a t c h or left vs. r i g h t r e s p o n s e s ) . Other derived CDA components behaved in a simi-
349 larly predictable manner in other portions of the trial. primarily encoding lever pressing vs. nosepoking and within-trial vs. intertrial events. Employing the CDA in this manner, as a 'sliding window' provided not only profiles from which altered individual neuronal activity could be extracted, but also 'benchmarks' regarding the level of such activity as well.
Using the canonical discriminant functions to indicate which neurons and temporal intervals were critical to encoding specific DNMS events, hippocampal cells could be categorized as firing differentially and significantly to the position of lever response (left or right), or phase of the task (sample or nonmatch), and were cias:sifiedas position- or phase-" ' cells. Perievent histograms were constructed around each behavioral event (within temporal domains identified by the canonical discriminant functions) and used to classify individual cell types. A cell was considered
to encode the event if its z-score ([Maximum Baseline] + SD of Baseline) exceeded 3.19 (equivalent to a significance of P > 0.001). Further, the cell was considered to differentially encode the event, if one or more peaks were at least 3.19 (z)larger than the remaining event peaks. Of the complete set of neurons (n = 263), only 20 could not be identified with differential event encoding that conformed to the above three categories. Examples of these 'functional cell types' (FCTs) are illustrated by the rastergrams in Fig. 2A.B. The same task phase information extracted by CAN1 was represented by the phase-only functional cell types. Fig. 3 illustrates an 'ensemble' constructed of six sample-only and six nonmatchonly neurons selected from the complete set of 80 such neurons. The firing of the same 12 neurons is shown in both Fig. 3A and B - - for left and right sample trials, respectively. The firing patterns are identical throughout the trial for sample-only cells (foreground) or nonmatch-only cells (background) (Fig. 3A,B). This is further illustrated by tlae non-
~.
~.
Contribution of different cell types to CDA patterns
Position
..~'P~
,~
.
.
o
llo|
•
'',
g
.. ~ ..º.~,~.aCiP'.' ~.' ' "
u i
,,"~?.i~:'; ' .',
, ..
~ ®|
.
| '°L m
o
"T..':.'./.' "i:~a
.eo | .
"'
C,
"I
" .'..
j
I
i I
'~ j •
. ., . . -
-1.0
i~ •
t
.oo
.I D II
~ ~
Osec
Sample
I
a!
, g 01~
. . . . ;'
•
~,,',° |I0 I -
+1,0
i
"
-
I
-
*
I /I '' u
-1,0
i I
.
F
.'
'I
'1
I
•
"r
i
•
• ,,,
,
"•''
I
I
Osec
Nonmatch
I,
-
~
+1,0
I
' ~ II .I
i
II I
I
I
I l n l l m JmI | , I
i, , ,,. .,'.'~r'
]
' .
,'1'I ,,. ',.~'" .
..
' , ',°
I
I
lJlCo, i ',,,.
.
" .
..,.~,~
"
.
""'," ....:' "'•
I
Trial Type I
Iii I
• I I | I
"
i
II |
I
|
....
I
II
i||e
I1~1 i I
II
', t ,,P';'¢.I.
"" II
''
iiii
•
|
° i gI
I
I hill
" ,. . . .
....
.,. r~ '~,:!~-~r,'d"~,'...' . , "
s Iit
i / ml
,'.,~
v., I
"
D.
.
e o m
L
II
'
I
,
Conjunctive
"
Phase
:'
IiIll
t
i
',: ,'
.,J ~'.,"~r':...'.:.
'
i.i
,
I I ~1
"
~llm ~.
,,I.,,
"
'
"
"
"
...... .
r,
,
,,~,,,~
•
-1.0
0
se
Sample
+1.0
-1.0
0 Sec
+1.0
Nonmatch
Fig. 2. Single,trial cell tiring records shown in 'raster' form (each row represents a single trial) for four different functional cell types (FCTs) of hippocampal cells. Each vertical tick mark represent a single action potential occurrence. Tick marks within a row represent all action potentials tired within the 3-s interval for a given trial. Ten trials are shown for each condition. Vertical line indicates lever response during Sampte or nonmatch phase on the left (L) or right (R) lever, thus for each FCT, four sets of rasters are depicted, one for each combination of phase and response position. (A) Position cell fires only to responses at the left (L) and not the right (R) lever. (B) Phase ceil fires during response in the nonmatch (not sample) phase irrespective of response position. (C) Example of a Conjunctive cell that fires only to the combination of left (L) response in the nonmatch phase. (D) Trial type cell fires in sample and nonmatch phases, but to one trial contiguration fi.e. right sample and left nonmatch. RS-LN) and not the other.
350
3.
A,
P h a s e Cell I
I
s
is
l
II
II
aI I I
I
|l
1!
| iiii
e ii
!
o
e I ii
II
,,
i
,,
| ,
I |
|
".'" , :;'
| |
, i~
|.°.
I el Bill l i t I IIIII I I • I • IllO • I I iI lil|l II | I I I III | I Illl I loll I II Rill
Osec
I
='n|,~om]='=
u,'
•
'
'• .... ~l"t'.,, "4,., ,,,..,-,,.l.;., '', ,,',,,
a
,o,~
a|''|°"
+i.O '
•
"
-1.0
Sample
D ,
I
I I I loll I I II I •
...."'t ' '';"
"
"-1.1)
| Ill I
/
"
!
"
0 sec +1.0 Nonmatch
CAN1
-
Trial Phase
B 2
1 m
0
o
-1
~;)
-2
.
u~
ITI
SP
SR
5
10
15
20
25
30
35 LNP NR ITI-
Trial Phase Fig. 3. Examples of phase-only cells. An 'ensemble' was constructed of six sample-only and six nonmatch-only neurons selected from 80 phase-only neurons. (A) Firing of the 12 neurons during a left trial (i.e. initiated with left sample response). Axis at left represents individual neurons, and separates the sample-only cells (right or foreground) from nonmatch-only cells (left or background). Axis at right depicts time throughout the DNMS trial, with the sample response (SR) at left and nonmatch response (NR) at right (see Fig. 1B for comparison). Note firing of sample-only and nonmatch-only cells during the appropriate phases. (B) Firing of the same 12 neurons during a right trial (i.e. initiated with right sample response). Axes and order of neurons is the same as m A. Note that there was no change in the pattern of neuron firing. (C) Rastergram las in Fig. 2) depicting firing of a single nonmatch-only cell. (D) 'Sliding' CDA discriminant scores for CAN1 (phase) throughout DNMS trial. Filled circles, CDA discriminant scores averaged over left trials: unfilled circles. CDA scores averaged over right trials. The CDA shows the same discrimination of phase (irrespective of position] as the phase-only cells, with the exception of the difference in sign between phases. This difference is likely contributed by the fact that there are two different subpopulations of sample-only and nonmatch-only cells within the phase-only cells.
match-only cell rastergram for a nonmatch-only cell type (Fig; 3C). Thus the trial-dependent reciprocal discriminant scores extracted by CAN1 (Fig. 3D) result in part from the activity of two distinct populations of phase.only firing cells, one of which encodes only sample, and the other only nonmatch. The position-only cell firing corresponded to the same information represented by CAN4 (response position) in a manner similar to that described for CAN1 and phase-only cells described above. There
were 33 cells identified with left-only or right-only firing. Fig. 4 illustrates the firing of six of each cell type in response to left or right trials (Fig. 4A,B). The left-only cells (foreground) fired only during the sample phase, while the right-only cells (background) fired only during the nonmatch phase. Fig. 4 also shows that some cells fired not only at the respective left or right responses, but also during the Delay phase prior to an appropriate nonmatch response. For example, left-only cells increase fir-
351
C.
A.
Position
L
Cell
', ', ',"''-"--~tt']',',' ! ' , %. , . . . ,, , ,,,"t,,~}: gS' ' ' ¶ II I ~'&"~' ' i I ','~,,,,I Ir~#JI,"', ,', ' " ', I ',''~',',~I,,ll',ll, '~'.~",i' '' ,. ' ' ,,",'¢~'d~,' ~ ' , '
'
ii
el
'
•
!
"e
,| !' '
I
""':
•
I|*Q
", , 0' ...I .... . '. :." . . .I . I • . • I •
R
-1.0
0 sec
B.
I1",
,, ,,:..
•
" ''&" '' "' i ,, ; ''[I " "
';" !
+1.0
-1.0
0 sec
Sample
D,
. •. . .S
Im
-
+1.0
Nonmateh
CAN4
- Lever
Position
~:Z:
n-
1
l
~3
I
U~
0
°
-1
I
--o--. . . . ,,,,1
ITI
, , , 0 ~
SP
RIGHT
TRIAL
0 , , , , 0 ° , , , , , , , , , o , , ° ,
SR
5
10
15 Trial
20
25
. . . .
30
°,
35 LNP NR
°
°,
•
,
ITI
Phase
Fig. 4, Examples of position-only cells. An ensemble similar to that shown in Fig. 3 was= constructed of six left-only and six right-only neurons selected fi-om 33:position-only neurons. (A) Firing of the 12 neurons during a left trial. Axes are the same as in Fig. 3A, except that the six right-only ceils have been placed to the right (foreground~ of the neuron axis. and the six left-only cells were placed to the left ~backgro~md) 0 f the neuron axis. (B) Firing of the same 12 neurons during a right trial. Axes and order of neurons is the same as in A. NOte that position-only cells fire in different phases on different trial types in accordance with the nonmatch demands of the DNMS task, (C) Rastergram (as in Fig. 2) depicting firing of a single left-only ceil. The cell fired in both sample and nonmatch phases as long as the response was in the left (L) position. (D) 'Sliding' CDA discriminant scores for CAN4 (position) throughout DNMS trial shows the same discrimination of position. Note that again, the difference in sign (negative vs. positive) likely results from the two subpopulations of left-only arid right-only cells.
ing during the delay preceding a left nonmatch. while right~on!y cells increase firing during the delay preceding :a right nonmatch, The rastergram for a representative left:only cell in Fig. 4C confirm thal the cell exhibited nearly identical firing in sample or nonmatch phase, :if the response position was the same (left), Hence the reciprocity in CAN4 during the trial (Fig: 4D) also restdted from a combination of two subpopulations of position-only cells, one encoding left responses, and the other encoding right responses. The)e were considerably fewer positiononiy cells (n = 33) than phase-on}y cells (n = 80)
identified in the complete data set, and this was reflected in the discriminant function for position (CAN4) which accounted for less ensemble firing variance (10%) than the discriminant function for phase (CAN1, 38%). Other cells could be further differentiated and classified with respect to conjunctions of the above two features (i.e. right sample, left sample, right nonmatch or left nonmatch), and these were called conjunctive cells (Table 1, Fig. 2C, see also Hampson and Deadwyler, 1999). A total of 101 Conjunctive cells were identified, with nearly equal numbers of
352 TABLE 1 Functional cell types: frequencies of occurrence
Position No Position
Left
Right
Totals
No Phase
No Correlate n= 20 (7%)
Left-Only n= 14 (5%)
Right-Only n= 19 (7%)
53
Sample
Sample-Only n= 38 (14%)
85
Nonmatch Nonmatch-Only n= 42 (16%) Trial Type Cells I Totals
100
94 Left Trial n= 17 (6%)
Right Trial n= 12 (5%)
29
79
84
263
Conjunctive cell types are shown in the shaded area.
cells corresponding to each of the four possible conjunctions (Table 1, gray shading). Each cell type fired only during the appropriate conjunction or combination of events, hence a left-sample cell would fire only during the sample response at the left lever, but not during any nonmatch or right sample responses. This population of conjunctive cells thus encodes only one of the four possible responses that comprise a DNMS trial, and can thus contribute to the encoding o f these events by t h e ensemble as extracted by both CAN1 and CAN4. Since equal numbers o f cells were identified for each conjunction, the number o f neurons that encode any events according to phase is increased from 80 to 181, and the number of neurons encoding position from 33 to 134 neurons, consistent with a higher percentage of total variance in ensemble variance being accounted for by phase than position factors. Yet another cell type was identified which could be classified as firing specifically to the two events that make up a trial type (i,e. right sample and left nonmatch), but not to the events that make up the other trial type (i.e. left sample and fight nonmatch), and hence were designated trial type cells (Fig. 2D). Only 29 trial'type cells were identified, whose firing was shown to correspond to the same information extracted as CAN5 (Fig. 2) across the DNMS trial. Fig. 5A and B depicts six left-trial, and six righttrial cells that fired throughout the respective DNMS
trial type. The left-trial cells (foreground) that only fired on trials initiated with a left sample response (Fig. 5A), and were essentially silent during trials initiated by a fight sample response (Fig. 5B). Thus this functional cell type fired 'disjunctively' on opposite levers in different phases, but always the same combination ttrial) of phases and levers (Fig. 5C). The initial peak, abrupt decrease, and slow 'ramping' of firing during the delay prior to the nonmatch response (Fig. 5A,B) was similar to the behavior of the discfiminant scores for CAN5 (Fig. 5D). This form of delay-phase firing did not occur in phaseonly or conjunctive cells for left or fight sample or consistently in any other cell type. As with CAN1 and CAN4, the appropriate conjunctive cells may have contributed to the ensemble firing patterns represented by CAN5. but only the 29 trial-type cells unambiguously encoded a given trial type.
Functional hippocampal cell types in ensembles The relative frequencies of functional cell types (FCTs) are provided in Table 1. Of the 263 cells analyzed in 21 animals, only 20 did not show a discrimination of some aspect trial phase or response position, and were excluded from the analvsis (Table 1. 'No Correlate'). phase-only cells accounted for 30% of cells, while position-only cells accounted for only 12% of cells. The largest population of
353 A.
C.
Trial-Type Cell
?'1'
,
"
,
,
t
I
a ,
¢ i
• ii
i
i
. ' .,. B
•
u
-1.0
•
I
-
ID
I
!
||
,ola
u
•
• me •
+1.0
o !
~u'
•
-1.0
oa o
~ Inao
'=
Jo o l l o u | ' o
I I
lull li O
o i~ u |; t! B,
, '=
', 0',.I
-
0 sec
!
iii
am
io,lj~ o
• i|! ~ | to ° e°al Ii I ~ a .n
=o~
|m
~,, ,,
I. ' M r . '
O ~l
"
,
i i ,',~La.......
t
,, o, ao
i
.
oD
.t~t,,
|
i
o
i
o
, .
0 see
|||
,u|
v'-,,
aa|o
•
..
|
+1.0
Nonmatch
Sample
D. CAN5
B.
F--1O
2
(
1
- Trial Type
O o
co
/
5}
~5 J
]
I ILL UJ -J
-2
I
----o--
LEFT TRIAL ---o--- RIGHT TRIAL o . . . . . . . . . . . . . . . . . . . . o
ITI
SP
SR
5
10
. . . . . o . . . . . . . . . . . ° .
15
20
25
30
35 LNP NR ITI
Trial Phase Fig. 5. Examples of trial-type cells. An ensemble of six left-trial and six fight-trial cells (out of 29 trial-type neurons) was constructed as in Figs. 3 and 4. (A) Firing of the 12 neurons during a left trial. Axes are the same as in Fig. 4 except for placement of neurons. Right-trial neurons were:placed to the right of the neuron axis. while left-trial neurons were placed to the left of the neuron axis. (B) Firing of the same neurons during a right trial. Note that the opposite trial-type cells are virtually silent during the inappropriate trial. (C) Rastergrarns of a right-trial cell show firing at the right sample and left nonmatch responses appropriate to a right trial, but no firing at the responses (left sample, right nonmatch) appropriate ~o a left trial. (D) 'Sliding' CDA scores for CAN5 (trial type) show the same discrimination as the trial-type cells. Note that the scores are reduced immediately after the sample response (SR) but are 'ramped up' during the delay interval. This same pattern is also exhibited by the delay firing of trial-type cells (A,B).
TABLE 2 Functional ceil types and DNMS task-specific information
Phase-only Position-only Conjunctives Trial-type
Proportion of DNMS information encoded per cell
Number of cells (%)
1/4 1, 4 1/8 1/2
80 33 101 29
cell types were the conjunctive cells, accounting for 38%, with trial-type cells accounting for the smallest group at 11% of the total. These proportions were used to create a-prototypical' ensemble consisting
(30%) (12%) (38%) (11%)
of 20 neurons encompassing all the FCTs, as shown in Fig. 6. The firing of this prototype ensemble during left and fight trials illustrates that some neurons fired at the same times on different trials (i.e.
354
5 Hz
R
5 Hz
L~
Fig. 6. Simulated ensembles of 18 neurons containing a proportional mixture of functional cell types (FCTs). The simulated ensemble was constructed with one left-only, three sample-only, two left-nonmatch, two right-nonmatch, one right-trial, one left-trial, two right-sample, two left-sample, three nonmatch only, and one right-only cells. Neurons are ranked in this same order from left-to-right along the neuron axis. Time axis is the same as in Figs. 3-5. (A) Firing pattern of simulated ensemble on left trial. (B) Firing of the same ensemble on a right trial. Note that although one-third of the cells fire the same in both trials (phase-only cells) the overall pattern of firing is distinct for the two trials. Conjunctive cells, which were not assigned to CAN1, CAN4, or CAN5 (Figs. 3-5) are included here, and contribute to the overall discrimination of phase and position.
phase-only cells), while others fired at specific times during the trials (i.e. position-only cells). The above FCTs only accounted for 42% of the population of cells, the remaining 58% of the population were conjunctive cells ,and trial-type cells which fired at unique combinations of phase and position. In terms of the information required to complete the DNMS task, each phase-only and position-only cell could encode at most only 25% of the information required to represent a trial, while each conjunctive cell could uniquely encode half of that information (Table 2). The trial-type cells encoded nearly all of the information required to represent a given trial,
but each cell could only encode one of the two possible DNMS trial types (Table 2). FCTs that encoded the highest proportion of information about DNMS events occurred with the lowest frequencies, whereas those that encoded less information occurred more frequently. The results in Table 2 suggest that as few as eight neurons with the appropriate FCTs would sufficiently encode all information required to perform the DNMS task; however, since recorded hippocampal ensembles rarely contained an 'ideal' mixture of cells types, larger ensembles would generally be required to encode all DNMS events (Hampson and Deadwyler, 1996a).
355 DNMS
If single neuron FCTs encode so much information about DN~vIS events, then do hippocampal ensembles exhibit distributed or local encoding? Prior analyses have shown the presence of both distributed and local encoding in hippocampat ensembles: place cells exhibit distinct l ~ a l coding in the form of place fields, yet the ~presentation of a complete spatial environment is distributed over a large number of neurons (WiIson a n d McNaughton, 1993; McHugh et al., 1996~ Bedenbaugh and Gerstein, 1997; Brown et al., i998). Likewise. spatial navigation has been
show distinct receptive fields (Georgopoulos et at., t986; Miller et al., i993; Lee et al., 1998), but encode considerably more information as a populations (Pellizzer et al.. 1995: Nicolelis et al.. 1995. 1998; Chapin et al., 1999). Encoding of DNMS events was previously shown to be distributed across neurons (Hampson and Deadwyler, 1996a; Deadwyler and Hampson, 1997), yet an anatomic or procedural segregati0n of information (Hgmps0n and Deadwyler, 1999) was observed when ensembles were examined for single neuron contributions, The prototypical histograms in Fig. 6 illustrate one mechanism by which ,the ,hippocan~al FCTs may provide a procedural map of a DNI~S trial. Du~ng sample phase, only the respective sample and position cells are active, falling silent at the beginning of the delay phase. Meanwhile the nonmatch-only and opposite position cells remain silent Until the later delay phase. Only trialtype cells remain active throughout the trial. As the delay progresses in the nonmatch phase, the nonmatch and appropriate position cells fire (Fig 6)
In this manner, hippocampal cells would fire in sequence through a 19NMS trial, encoding task-specific information as much :by firing sequence, as by the individual neural correlates. Thus the encoding of the DNMS trial is distribuWd across the ensemble, but relies on the local encoding revealed by the FCTs. If single neurons encoded DNMS :events with the precision revealed by the FCTs, what additional information did ensemble analysis contribute? The answer lies in the manner in which FCTs were identified. T h e canonical discriminant analysis (CDA) of the ensembles was essential to determining the FCTs. Prior analyses of single neurons (Hampson et al., 1993; Heyser et al.~ 1993) showed that each neuron's firing varied in response to several events. Ensembles constructed of non-simultaneously recorded neurons contained too much noise uncorrelated with task events, to accurately encode all DNMS events (Hampson and Deadwyler. 1996a). After the total variance of the simultaneously-recorded neurons were analyzed as an ensemble via CDA. the discriminant functions were used to identify firing patterns and temporal intervals, and search for FCTs. Thus the single neuron FCTs were not identified until afwr the ensemble analysis, and relied on the CDA for segregation of variance associated with DNMS event encoding. Neural ensemble recording revealed a feature of DNMS event encoding by hippocampal neurons that was not immediately apparent from single neuron recordings, yet single neurons exhibited features that contributed to the ensemble code. Previous reports revealed both simple and conjunctive firing patterns of single hippocampal neurons (Otto and Eichenbaum, 1992; Hampson et al., 1993; Schoenbaum and Eichenbaum, 1995b; Young et al., 1997), but did not relate that firing to ensemble encoding. The above results demonstrate that both local and distributed encoding occur within the hippocampus. Similar findings have been reported for other brain areas (Nicolelis et al., 1998). The presence of neurons with distinctly tuned visual or sensory receptive fields is often cited as evidence for local encoding of information (cf. Georgopoulos et al., 1986; Gawne and Richmond. 1993). However, even with distinct receptive fields, the 'tuning' of a given field is often not precise enough to solely account for the precision of the code by ensembles (Salinas and Abbott.
356
1994; Georgopoulos, 1995; Fitzpatrick et al., 1997). Thus it is possible to have highly tuned receptive fields which encode only very specific information, similar to the event specificity exemplified by conjunctive cells, yet still have a code distributed across neurons. Single cell and ensemble encoding are mutually dependent processes, and both analyses reveal characteristics that could not be obtained alone: ensemble analyses revealed task-specific information that was encoded across neurons, yet portions of the code could be represented by single neurons. However, it was only after the ensemble firing patterns were extracted, that the corresponding single neuron firing patterns could be effectively identified. Thus both techniques together showed how DNMS events were represented within the ensemble (across neurons), as well as how complete DNMS task-specific information Could be represented in ensembles of only 10-20 neurons.
Acknowledgements The authors thanks D.R. Byrd, J.K. Konstantopoulos, J. Brooks and T. Bunn for technical support. This work was supported by NIH Grants DA08549 to R.E.H. and DA03502 and DA00119 to S,A.D.
References Bedenbaugh, E and Gerstein, G.L. (1997) Multiunit normalized cross correlation differs from the average single-unit normalized correlation: Neural Comput:, 9: 1265-1275. Brown, E.N., Frank, L.M., Tang, D., Quirk, M.C. and Wilson, M.A, (1998) A statistical paradigm for neural spike train decoding applied to position prediction from ensemble firing patterns of rat hippocampal place cells. J. Neurosci., 18:74117425. Chapin, J., Nicolelis, M:A., Yu, C.-H. and Sollot, S. (1989) Characterization of ensemble properties of simultaneously recorded neurons in somatosensory (SI) cortex. Soc. Neurosci. Abstr., 15: 312. Chapin, J.K., Moxon, K.A:, Markowitz, R.S. and Nicolelis, M.A. (1999) Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex [see comments]. Nat. Neurosci., 2: 664-670. Christian, E:E and Deadwyler, S.A. (1986) Behavioral functions arid hippocampal cell types: evidence for two nonoverlapping populations in the rat. J. Neurophysiol., 55: 331-348. Deadwyler, S.A. and Hampson, R.E. (1997) The significance of neural ensemble codes during behavior and cognition. In: W.M. Cowan, E.M. Shooter, C.E Stevens, and R.F. Thomp-
son (Eds,), Annual Review of Neuroscience, Vol. 20. Annual Reviews. Palo Alto. CA. pp. 217-244. Deadwyler. S.A.. Bunn, T. and Hampson. R.E. (1996) Hippocampal ensemble activity during spatial delayed-nonmatch-to-sample performance in rats. J. Neurosci., 16: 354372. Fitzpatrick. D.C.. Batra. R., Stanford, T.R. and Kuwada, S. (1997~ A neuronal population code for sound localization. Nature, 388: 871-874, Fox. S.E. and Ranck Jr.. J.B. (1981) Electrophysiological characteristics of hippocampal complex-spike cells and theta ceils. Exp. Brain Res.. 41: 399-410. Freund. T.E and Buzsaki, G. (1996) Internenrons of the hippocampus. [Review] [757 refs]. Hzppocampus, 6: 347-470. Gawne, TJ. and Richmond. B.J. (1993) How independent are the messages carried by adjacent inferior temporal cortical neurons? J. Neurosci., 13: 2758-2771. Georgopoulos. A.E (1995) Current issues in directional motor control. Trends Neurosci.. 18: 506-510. Georgopoulos. A,Eo Schwartz, A.B. and Ketmer, R.E. (1986) Neuronal population encoding of movement direction. Science. 233: 1416-1419. Georgopoulos. A,R, Lurito, J.T.. Petrides. M., Schwartz. A.B. and Massey, J.T. (1989) Mental rotation of the neuronal population vector. Science, 243: 234-236. Ghazanfar. A.A. and Nicolelis. M.A. (1997J Nonlinear processing of tactile information in the thalamocortical loop. J. NeurophysioL, 78: 506-510. Gochin. EM., Colombo. M.. Dorfman, G.A., Gerstein, G.L. and Gross. C.G. 11994) Neural ensemble coding in inferior temporal cortex. J. Neurophysiol.. 71: 2325-2337. Hampson. R.E. and Deadwyler, S.A. (1996a) LTP and LTD and the encoding of memory in small ensembles of hippocampal neurons. In: M. Baudry and J. Davis (Eds.), Long-Term Potentiation. Volume 3. MIT Press, Cambridge, MA. pp. 199214. Hampson. R.E and Deadwyler, S.A. (1996b) Ensemble codes involving hippocampal neurons are at risk during delayed performance tests. Proc. Natl. Acad. Sci. USA, 93: 1348713493. Hampson. R.E. and Deadwyler, S.A. (1998) Methods, results and issues related to recording neural ensembles. In: H. Eichenbaum and J. Davis (Eds.), Neuronal Ensembles: Strategies for Recording and Decoding. Wiley, New York, pp. 207-234. Hampson. R.E. and Deadwyler, S.A. (1999) Pitfalls and problems in the analysis of neuronal ensemble recordings during performance of a behavioral task. In: M. Nicolelis (Ed.), Methods for Simultaneous Neuronal Ensemble Recordings. Academic Press. New York. pp. 229-248. Hampson, R.E., Heyser, C.J. and Deadwyler. S.A. (1993) Hippocampal cell firing correlates of delayed-match-to-sample performance in the rat. Behav. Neurosci.. 107: 715-739. Hampson. R.E.. Simeral, J.D. and Deadwyler, S.A. (1999) Distribution of spatial and nonspatial information in dorsal hippocampus. Nature, 402: 610-614. Heyser, C.J., Harnpson, R.E. and Deadwyler, S.A. (1993) The effects of delta-9-THC on delayed match to sample performance
357 in rats: alterations in short~term memory produced by changes in task specific firing of hipp6campal neurons. J. Pharmacol. Exp. The~:,264: 294-307. Lee; D,, Pert, N.P., Kruse. W. and Georgopoulos, A.R (1998) Neuronal population coding: multielectrode recording in primate cerebral cortex. Im H. Eichenbaum and J. Davis (Eds.).
Neu/onal Eesembles: St~'awgiesfor Recording and Decoding. Wiley. New York. McHugh. T.J.. Blum. K.I, Tsien, J.Z., Tonegawa. S. and Wilson. M.A. (1996) Impaired hiptmcampal representation of space in CAl-specific NMDAR1 knockout mice [see comments]. Cell, 87: t 339-1349 Miller, E.K., Li, L. and Desimone, R. (1993) Activity of neurons in anterior inferior temporal cortex during a short-term memory task. J. Neurosci., 13: 1460-1478. Moran, Diw. and Schwartz, A.B. (1999a) Motor cortical activity during drawing movements: population representation during spiral tracing. J. Neurophysiot., :82: 2693-2704. Moran, D.W. and Schwartz, A.B. (1999b~ Motor cortical representation of speed and direction during reaching. J. Neurophysiol.. 82: 2676-2692. Muller, R.U, Stead, M. and Pach, J. (1996) The hippocampus as a cognitive graph [Review] [108 refs] J. Gen Physiol i 107 663-694. Nic01elis. M.A.. Baccala. L.A.. Lin. R.C and Chapin, J.K. (1995) Sensorimotor encoding by synchronous neural ensembIe activity at multiple levels of the somatosensory system. Science, 268: 1353-1358. NicoMis M.A., Ghazanfar A.A., Statabaugh, C.R. Oliveira L.M.. Lanbach. M.. Chapin, J.K.. Nelson, R.J. and Kaas. J.H. (1998) Simultaneous encoding of tactile information by three primate cortical areas, Nat. Neurosci. t: 621-630.
Otto. T. and Eichenbaum, H (1992) Neuronal activity in the hippocampus during delayed non-match to sample performance in rats: evidence for hippocampal processing in recognition memory. Hippocampus. 2: 323-334. Pellizzer. G., Sargent, E and Georgopoulos. A.E H995) Motor cortical activity in a context-recall task. Science. 269: 702705. Redish. A.D. (1999) Beyond the Cognitive Map. MIT Press. Cambridge, MA. Salinas, E. and Abbott, L.E (1994) Vector reconstruction from firing rates. J~ Comput. Neurosci., 1: 89-107. Samsonovich. A. and McNaughton, B.L. (I997) Path integration and cognitive mapping in a continuous attractor neural network model. J. Neurosci.. 17: 5900-5920. Schoenbaum, G. and Eichenbanm. H. (1995a} Information coding in the rodent prefrontal cortex. II. Ensemble activity in orbitofrontal cortex. J. Neurophysiol., 74: 751-762. Schoenbaum. G. and Eichenbaum, H. (1995b) Information coding in the rodent prefrontal cortex. I. Single-neuron activity in orbitofrontal cortex compared with that in pyriform cortex. J. Neurophysiol., 74: 733-750. Schultz. W., Apicella. R and Ljungberg, T. (1993) Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. J. Neurosci., 13: 900-913. Wilson. M.A. and McNanghton, B.L. (1993) Dynmmcs of the hippocampal ensemble code for space. Science. 261: 10551058. Young, B.J.. Otto. T., Fox, G.D. and Eichenbaum. H. (1997) Memory representation within the parahippocampal region. J. Neurosci.. 17: 5183-5195.
359
Subject Index
rhythm, 98 7-12 Hz rhythmic oscillations, 96 accuracy of neurat coding, 333 across-fiber patterning. 14 acute plasticity, 92 adaptive spike mechanism. 197 Adrian, 4, 1 t AMPA receptors, 91 amplitude-modulated stimuli, 221 animat, 56 arbitrary mappings, 261 area 3b, 63, 168 area IT, 182 area LIP. 182 area SII, 168 artificial neural networks, 89, 93, 246 artificial olfactory system, t 96 audition. 16 auditory cortex, 221 auditory feedback. 326 auditory pathway, 221 auditory place code. 210 auditory system, 206. 322 auditory-motor interactions. 324 backpropagation, 180 Backpropagation Through Time. 247 Barlow. Horace. 4 barretoids, 91 barrels, 91,284 bats, 206 Bayesian method. 338 behavioral neurophysiology, 245 binaural response, 221 binding hypothesis, 23, 113, 114 bird song system, 320 Bishop. 17 brain plasticity, 65 brain slices. 54 brainstem auditory pathways. 208
calcium imaging, 54 canonical discriminant analysis. 346 cell assembly. 17, 111 central motor program, 98 cerebellar coding, 279 cerebellar cortex. 279. 297 cerebellum. 299 characteristic frequency, 223 chemical sensor array, 196 chronically implanted microwires, 235 cluster analysis, 301 coarse coding, 6, 20 cochlear nucleus, 208 cognitive maps, 17 coherence, 130 coherence analysis, 33 common neural input, 143 condition-test paradigms, 102 connectionist models, 246 convergence, 142, 212, 268 coordinate transformations, 182 corpus callosum, 102 correlated discharge, 235 correlated firing, 113. 227 correlated noise, 335 correlation methods, 33 correlations, 271 correlograms, 150, 274 cortical point-spread functions. 156 cortical processing, 156 corticocortical loop. 89 corticomotoneuronal control, 268 corticopontine pathway, 282 corticospinal axon, 268 corticospinal model, 25 l corticothalamic projections, 91 cosane tuning function, 337 cross-correlation, 33, 66, 142, 235, 288 cross-correlation techniques. 236 cultured neuronal networks, 49
360 delay lines, 78 directed coherence, 34 directed transfer function, 34 discriminant analysis, 301,303, 346 dissociated neural cultures, 53, 54 dissociated neuronal networks, 49 distributed coding, 3, 191 distributed coding in the visual cortex, 112 distributed encoding of sound frequency, 222 distributed encoding of sound localization, 226 distributed network interactions, 305 distributed processing, 93 distributed representations, 319, 321 divergence, 142 EEG, 33 EEG recordings, 98 ensemble codes, 235 EPSP, 91,216 error generation, 98 expectation, 98 face-selective neurons, 186 feed-back connections, 112 feed-forward connections, 112 feedback, 142 feedback pathways, 89 feedforward, 142 feedforward computations, 89 fine coding, 20 fine tuning, 20 Fisher information, 333 fMRI, 126 fractured somatotopy, 280 frequency encoding, 222 frequency-modulated stimuli, 221 GABAergic innervation, 101 GABAergic transmission, 222 gain field, 177 gain modulation, 175 Gall, 9 gamma frequency range, 116 gating effect, 95 Ganssian interpolation procedure, 163 Gaussian probability density, 151 geniculocortical correlations, 149 Gestalt, 17
glial fibrillary acidic protein, 66 glomemlus, 194 Golgi cell, 280 grandmother cell, 233 Granger causality, 34 Granger causality tests, 34 guatation, 15 Hartline, 16 Hebb, Donald, 17, 111,227 Hebbian. theory, 130 Helmholtz, 10 hidden unit, 248 hierarchical schemes, 112 hippocampal ensemble codes, 347 hippocampus, 52, 345 horizontal connections, 119 Hubel and Wiesel, 11 HVc, 321 hyperacuity, 20, 21, 156 inferior colliculus, 207 inferior olivary neurons, 299 inferotemporal cortex, 17 inter-areal synchronization, 117 interaural time differences, 227 interhemispheric cross-correlations, 68 intracellular recordings, 207 intracortical correlations, 145 joint position, 16 Joint-PSTH, 115 Knudsen, 16 Kuffler, Stephen, 5 labeled-line hypothesis, 4, 11 Lashley, 17 lateral correlations, 144 lateral geniculate nucleus, 21,117, 141, 176 Lettvin, 4 local field potential, 116 long-range synchronization, 117 long-term depression; 279 look-up table, 261 Mtiller's law, 5 mappings, 246 memory, 17
361 mental rotation. 258 Merzenich, 15 metabotropic receptors. 91 Michigan system, 64 micro-elec~ode arrays. 50 microwires, 64 mitral cell, 14, 194 modular coding, 11 modularity, l0 Monte Carlo technique, 240 mossy fibers. 280 motoneurons. 267 motor cortex. 234, 245 motor preparatory activity, 268 motor systems, 16, 208. 267 Mountcasfleo I5 MT. 123 multi-electrode at-ray culture dishes, 50 multi-single-unit recordings, 288, 345 multi-site chronic recordings, 99 multi-unit activity, 1 t6, 118 multichannel neuronal acquisition processor, 64 multielectrode recording, 141. 305 multiple electrodes, 245 multiple-input network. 256 muscle sense, t6 mutual information. 33 network computational modeling, 328 neural assemblies. 111. 227. 346 neural ensemble recording, 355 neural mass, 22 neural mass differences. 18 neural network. 246 neural rewesentation. 205 nenrochip, 51 neuromagnetic responses. 131 neuronal population functions. 160 neuronal prosthesis. 297 NMDA receptors. 91 noclceptive information, 96 non-centered RF approach. 159 non-monotonic relationship. 214 nonlinear interactions, 169 nonlinear summation of tactile stimuli, 98 olfaction. 14 olfactory code, 195
olfactory receptor proteins. 193 olfactory sensory neurons. 191 optical imaging, 54. 196 optical recordings, 55 optimal linear estimator, 163 organotypic synaptic connections, 49 oscillation-based circuits. 78 oscillations. 141. 195 parallel afferent pathways, 77 parallel descending information. 267 parallel processing, 18, 211. 267 parietal cortex, 175 partial directed coherence, 33 perceptron learning, 279 peripheral deafferentation. 92 peripheral encoding, 75 Pfaffman, Carl, 4 phase-locked activity, 22! phase-locked loop, 79 pheromone, 19 piriform cortex, 195 pitch perception, 210 plastic reorganization. 93 population analysis. 155 population coding, 3, 13.24. 175, 196. 205. 234 population receptive field, 158 population recordings, 192 population vector. 245 population vector analyses, 346 population vector average, 234 premotor areas. 246 premotor cortex, 274 proprioceptive information, 299 Purkinje cell, 279, 298 RA neurons, 320 rat auditory cortex. 222 rat somatosensory cortex. 166 rat vibrissal system. 75 rate-population code, 82 read-in, 297 readout. 12, 19. 206, 297 receptive fields. 5, 15, 65, 157. 176. 183,281,355 reconstruction method, 338 recurrent connections. 178 respiratory centers, 52 reticular nucleus of the thalamus. 91
362 retina, 17, 52 retinogeniculate, 143 RF centered approach, 159 rhythmic bursts, 96 second-order analysis, 149 sensorimotor interactions, 320 sensorimotor mapping, 325 serial processing, 267 Shannon mutual information, 333 signal segmentation, 98 silicon-based array, 235 simple cells, 21 simultaneous recording, 245 single best neuron, 239 single-cell activity, 245 single-cell recordings, 155 single-compartment neuron, 197 sleep, 325 somatosensory cortex, 63, 93, 166 somatosensory gating, 96 somatosensory oscillators, 78 somatosensory system, 89, 284 somatotopic maps, 89 somesthesis, 15 song learning, 320 sound intensity, 221 sound localization, 226 sound-source location, 227 spatial code, 16 spatial encoding, 75 spatio-temporal coding, 291 spatio-temporal pattern, 198, 227,298, 319 spatio-temporal response, 156 spectral analysis, 33 spectrotemporal dynamics, 225 Sperry, 15 spinal cord, 267 spinal interneurons, 268 standard mapping, 246 stereopsis, 16 summation, 149 superior colliculus, 5, 16
I
superposition catastrophe, 234 superposition problem, 113 supplementary eye field, 261 suprachiasmatic nucleus, 52 synaptic learning, 340 synaptic potentials, 207 synchronous activity, 67, 115, 149, 237, 270, 302 synchronous oscillations, 288 synergy, 149 tactile stimulus, 94 template matching, 98 temporal coding, 75, 292, 319 temporal decoding, 78 temporal relations, 141, 211 temporal-to-rate code transformation, 83 temporally correlated activity, 115 thalamic implants, 64, 69 thalamocortical loop, 89 thalamocortical system, 208 tonotopic organization, 209 top-down influences, 94 topographic, 20 unmasking of novel tactile responses, 92 Utah system, 64 V-shaped response, 221 vector coding, 113 ventroposterior nucleus, 69 vertebrate olfactory, 191 vestibular sense, 16 visual coding, 14 visual cortex, 112 visual oscillations, 78 visuomotor behavior, 246 whisker twitching, 97 whisking, 75 whole-cell patch-clamp, 212 Young, Thomas, 3, 9, 12, 157 zebra finches, 324
