Progress in Brain Research Volume 144 The Roots of Visual Awareness - A Festschrift in Honour of Alan Cowey

List of Contributors

J.L. Barbur, Applied Vision Research Center, City University, Northampton Square, London EC1V 0HB, UK G. Berlucchi, Dipartimento di Scienze Neurologiche e della Visione, Sezione Fisiologia Umana, Universita` di Verona, Strada Le Grazie 8, I-37134 Verona, Italy C.M. Butter, Department of Psychology, University of Michigan, Ann Arbor, MI 481091109, USA G. Campana, Dipartimento di Psicologia Generale, Universita` degli Studi di Padova, Via Venezia 8, Padova, Italy and Department of Experimental Psychology, University of Oxford, South Parks Road, OX1 3UD, UK G.G. Cole, Department of Psychology, Science Laboratories, South Road, Durham DH1 3LE, UK A. Cowey, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, OX1 3UD, UK M. Da Silva Filho, Department of Physiology, Biological Science Center, Federal University of Para´, 66075-900 Bele´m, Para´, Brazil P. Dean, Department of Psychology, University of Sheffield, Western Bank, Sheffield, S10 2TP, UK R. Edwards, School of Psychology, University of St. Andrews, St. Andrews, KY16 9JU, UK P. Foldia´k, School of Psychology, University of St. Andrews, St. Andrews, KY16 9JU, UK M.A. Goodale, Department of Psychology, University of Western Ontario, London, ON N6A 5C2, Canada C.G. Gross, Department of Psychology, Green Hall, Princeton University, Princeton, NJ 08544, USA R.L. Gregory, Department of Experimental Psychology, University of Bristol, 8 Woodland Road, Bristol BS8 1TN, UK C.A. Heywood, Department of Psychology, Science Laboratories, South Road, Durham DH1 3LE, UK A. Hurlbert, Henry Wellcome Building for Neuroecology, School of Biology, Framlington Place, Newcastle upon Tyne NE2 4HH, UK C.-H. Juan, Department of Psychology, Vanderbilt University, 301 Wilson Hall, Nashville, TN 37240, USA R.W. Kentridge, Department of Psychology, Science Laboratories, South Road, Durham DH1 3LE, UK C. Keysers, School of Psychology, University of St. Andrews, KY16 9JU, UK B.E. Kilavik, Department of Experimental Ophthalmology, University of Tu¨bingen, D-72076 Tu¨bingen, Germany J. Kremers, Department Experimental Ophthalmology, University of Tu¨bingen, D-72076 Tu¨bingen, Germany

v

vi

B.B. Lee, SUNY Optometry, New York, NY 10036, USA and Max Planck Institute for Biophysical Chemistry, Department of Neurobiology, D-3400 Go¨ttingen, Germany C.A. Marzi, Department of Neurological and Visual Sciences, University of Verona, 8 Strada Le Grazie, 37134 Verona, Italy R.D. McIntosh, Cognitive Neuroscience Research Unit, Wolfson Research Institute, University of Durham, Queen’s Campus, University Boulevard, Stockton-on-Tees, TS17 6BH, UK A.D. Milner, Cognitive Neuroscience Research Unit, Wolfson Research Institute, University of Durham Stockton Campus, Stockton-on-Tees, TS17 6BH, UK A. Minelli, Department of Neurological and Visual Sciences, University of Verona, 8 Strada Le Grazie, 37134 Verona, Italy T. Moore, Department of Psychology, Green Hall, Princeton University, Princeton, NJ 08544, USA D.I. Perrett, School of Psychology, University of St. Andrews, St. Andrews, KY16 9JU, UK V.H. Perry, CNS Inflammation Group, University of Southampton, SO16 7PX Southampton, UK L. Pessoa, Laboratory of Brain and Cognition, National Institute of Mental Health, NIH, Building 49, Room 1B80, 49 Convent Drive, Bethesda, MD 20892-4415, USA J. Porrill, Department of Psychology, University of Sheffield, Western Bank, Sheffield, S10 2TP, UK B.E. Reese, Neuroscience Research Institute, Department of Psychology, University of California at Santa Barbara, Santa Barbara, CA 93106-5060, USA H.R. Rodman, Department of Psychology and Yerkes RPRC, Emory University, 532 N. Kilgo Circle, Atlanta, GA 30322, USA E.T. Rolls, Department of Experimental Psychology, University of Oxford, Oxford OX1 3UD, UK C.A. Saito, Department of Physiology, Biological Science Center, Federal University of Para´, 66075-900 Bele´m, Para´, Brazil S. Savazzi, Department of Neurological and Visual Sciences, University of Verona, 8 Strada Le Grazie, 37134 Verona, Italy L.C.L. Silveira, Department of Physiology, Biological Science Center, Federal University of Para´, 66075-900 Bele´m, Para´, Brazil S. Soloviev, Department of Neurology, Harvard Medical School, 75 Francis Street, Boston, MA 02215, USA M.A. Sommer, Laboratory of Sensorimotor Research, National Eye Institute, N.I.H., Building 49, Room 2A50, MSC 4435, 9000 Rockville Pike, Bethesda, MD 20892-4435, USA P. Stoerig, Heinrich Heine University, Institute of Experimental Psychology, Universita¨tstrasse 1, 40225 Du¨sseldorf, Germany J.V. Stone, Department of Psychology, University of Sheffield, Western Bank, Sheffield, S10 2TP, UK L.G. Ungerleider, Laboratory of Brain and Cognition, National Institute of Mental Health, NIH, Building 10, Room 4C104, 10 Center Drive, Bethesda, MD 20892-1366, USA L.M. Vaina, Brain and Vision Research Laboratory, Boston University, Department of Biomedical Engineering, College of Engineering, 44 Cummington Street, Room 315, Boston, MA 02215, USA V. Walsh, Institute of Cognitive Neuroscience, University College London, 17 Queen Square, London WC1N 3AR, UK

vii

L. Weiskrantz, Department of Experimental Psychology, University of Oxford, Oxford, OX1 3UD, UK D.A. Westwood, School of Health and Human Performance, Dalhousie University, Halifax, NS B3H 3J5, Canada F. Wilkinson, Centre for Vision Research, CSB, York University and Toronto Western Research Institute, 4700 Keele Street, Toronto, ON M3J 1P3, Canada K. Wolf, Henry Wellcome Building for Neuroecology, School of Biology, Framlington Place, Newcastle upon Tyne NE2 4HH, UK R.H. Wurtz, Laboratory of Sensorimotor Research, National Eye Institute, N.I.H., Building 49, Room 2A50, MSC 4435, 9000 Rockville Pike, Bethesda, MD 20892-4435, USA D. Xiao, School of Psychology, University of St. Andrews, St. Andrews, KY16 9JU, UK E.S. Yamada, Department of Physiology, Biological Science Center, Federal University of Para´, 66075-900 Bele´m, Brazil A. Zeman, Department of Clinical Neurosciences, University of Edinburgh, Wesern General Hospital, Edinburgh EH4 2XU, UK

Foreword

The rewards of teaching come delayed, in successes of gifted students, and are immediate in the questions they raise and their challenges to the teacher’s knowledge and beliefs. The fresh eyes of students make us look again. I had the privilege of supervising (as we called it) Alan Cowey at Cambridge in the early 1950s. He went on to do his Ph.D. with Larry Weiskrantz, and I well remember how focused he was on monkey perimetry and measuring their eye movements. This was demanding, difficult work which Alan grasped in both hands, succeeding where previous attempts had largely failed. One can see this now as the basis of Alan’s creative career. He was fortunate in his Ph.D. supervisor, and others around, then and somewhat later, especially Nick Humphrey and his monkey Helen. Alan has well-developed foveal and peripheral mental vision, for he combines lengthy and for most people tedious experiments with philosophical speculations, as on consciousness. He is, indeed, an experimental philosopher, looking for specific experimental results to uncurl question marks. This is the most exciting and rewarding kind of science. This is what made seventeenth century physics so exciting with its mental and physical tools for its investigations — and now the tools of micro-electrode recording and brain imaging, which with cognitive concepts for interpreting their data allows not only brain but mind to be probed, revealing secrets in our heads. Although gifted students soon take off on their own adventures, where they start must be helpful — or the opposite. I had begun to think that many perceptual phenomena, especially varieties of illusions, can have adequate explanations from so to speak the strategy of the physiology. Thus a General can lose or win battle by how his forces are deployed. Of course there must be forces, but to see or control their effects, strategies are necessary. Historians may know the strategies top-down or deduce them bottom-up from observations. I hope this approach was not a hurdle for my students to overcome. I also expressed doubts on the apparent simplicity of extirpation and brain recording experiments, for (having fooled around with radar and communication circuits in the war) it seemed to me that relations between parts of a machine and its functions are far from simple. I used to think of removing components from a radio, asking: How do you know what the component was doing, from the symptoms of its loss? If the radio howls — was it a ‘howl suppresser’? This by no means follows (I would say). And how is it possible to localize functions if we don’t understand a circuit, or the brain, to know what functions give the output performance? How could one localize an oscillator in a radio if one didn’t know it had an oscillator, or how heterodyning works? Such comments were annoying at the time, especially when seen as attacking brain science aims and dreams. Possibly there was something in this (one defends one’s fort by at least pretending to attack potential invaders), but it seemed to me that interpreting data is as important as the data themselves — and errors due to misinterpretations can be greater than observational errors. I would now say that both statistical significance and conceptual significance are necessary for science. Conceptual errors are far more widely misleading, so really important for the teacher–student relation. But I have described philosophers (and no ix

x

doubt teachers of psychology) as ‘Guardians of Semantic Inertia.’ No doubt Alan’s intelligent selective attention protected him from my perhaps dubious philosophy, and Larry’s great expertise in experimenting led him to question and learn from nature. At that time two of the now most discussed issues were taboo: attention, and consciousness. Attention was a word simply not allowed in the Quarterly Journal of Experimental Psychology, as it had no, or too little operational definition. Consciousness was a non-starter for any budding or fully flowered author to submit for consideration. Alan has made highly significant contributions to both. Fancy getting monkeys to tell us whether they are aware of seeing! Wittgenstein would be astonished by this. The papers in this volume honoring Alan Cowey represent new ways of thinking and experimenting. We as his teachers may or may not have suggested good directions but it is he who homed in on wonderful questions and exciting answers. This, not by wild and (comfortably) woolly speculation, but by decades of exceptionally careful and detailed hard work. So in his turn Alan has become a force to be reckoned with, an inspiration for students and a take-off platform for new research. As he combines both statistical and conceptual significance he is far from a guardian of inertia: Alan gives momentum for progress into the future. R.L. Gregory Department of Experimental Psychology University of Bristol

Contents

List of Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v

Foreword by R.L. Gregory (Bristol, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

Preface by C. Heywood (Durham and Oxford, UK) . . . . . . . . . . . . . . . . . . . . . .

xi

Section I. Visual Pathways 1. Developmental plasticity of photoreceptors B.E. Reese (Santa Barbara, CA, USA) . . . . . . . . . . . . . . . . . . . . .

3

2. Morphology and physiology of primate M- and P-cells L.C.L. Silveira, C.A. Saito, B.B. Lee, J. Kremers, M. da Silva Filho, B.E. Kilavik, E.S. Yamada and V.H. Perry (Para´, Brazil; New York, NY, USA, Go¨ttingen, and Tu¨bingen, Germany and Southampton, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

3. Identifying corollary discharges for movement in the primate brain R.H. Wurtz and M.A. Sommer (Bethesda, MD, USA) . . . . . . . . . .

47

4. Visual awareness and the cerebellum: possible role of decorrelation control P. Dean, J. Porrill and J.V. Stone (Sheffield, UK) . . . . . . . . . . . . .

61

Section II. Cortical Visual Systems 5.

Some effects of cortical and callosal damage on conscious and unconscious processing of visual information and other sensory inputs G. Berlucchi (Verona, Italy) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

79

6. Consciousness absent and present: a neurophysiological exploration E.T. Rolls (Oxford, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

95

7. Rapid serial visual presentation for the determination of neural selectivity in area STSa P. Fo¨ldia´k, D. Xiao, C. Keysers, R. Edwards and D.I. Perrett (St. Andrews, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

107

xiii

xiv

8. Cortical interactions in vision and awareness: hierarchies in reverse C.-H. Juan, G. Campana and V. Walsh (Nashville, TN, USA, Oxford and London, UK and Padova, Italy) . . . . . . . . . . . . . . . . . . . . . . .

117

9. Two distinct modes of control for object-directed action M.A. Goodale, D.A. Westwood and A.D. Milner (London, ON and Halifax, NS, Canada and Stockton-on-Tees, UK) . . . . . . . . . . . . .

131

Section III. Perception and Attention 10. Color contrast: a contributory mechanism to color constancy A. Hurlbert and K. Wolf (Newcastle upon Tyne, UK) . . . . . . . . . .

147

11. The primacy of chromatic edge processing in normal and cerebrally achromatopsic subjects R.W. Kentridge, G.G. Cole and C.A. Heywood (Durham, UK) . . .

161

12. Neuroimaging studies of attention and the processing of emotion-laden stimuli L. Pessoa and L.G. Ungerleider (Bethesda, MD, USA) . . . . . . . . . .

171

13. Selective visual attention, visual search and visual awareness C.M. Butter (Ann Arbor, MI, USA) . . . . . . . . . . . . . . . . . . . . . . .

183

14. First-order and second-order motion: neurological evidence for neuroanatomically distinct systems L.M. Vaina and S. Soloviev (Boston, MA, USA) . . . . . . . . . . . . . .

197

15. Reaching between obstacles in spatial neglect and visual extinction A.D. Milner and R.D. McIntosh (Stockton-on-Tees, UK) . . . . . . .

213

Section IV. Blindsight and Visual Awareness 16. Roots of blindsight L. Weiskrantz (Oxford, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17.

‘Double-blindsight’ revealed through the processing of color and luminance contrast defined motion signals J.L. Barbur (London, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

229

243

18. Stimulus cueing in blindsight A. Cowey and P. Stoerig (Oxford, UK and Du¨sseldorf, Germany) .

261

19. Visually guided behavior after V1 lesions in young and adult monkeys and its relation to blindsight in humans C.G. Gross, T. Moore and H.R. Rodman (Princeton, NJ and Atlanta, GA, USA) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

279

xv

20. Is blindsight in normals akin to blindsight following brain damage? C.A. Marzi, A. Minelli and S. Savazzi (Verona, Italy) . . . . . . . . . .

295

21. Auras and other hallucinations: windows on the visual brain F. Wilkinson (Toronto, ON, Canada) . . . . . . . . . . . . . . . . . . . . . .

305

22. Theories of visual awareness A. Zeman (Edinburgh, UK) . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

321

Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

331

SECTION I

Visual Pathways

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 1

Developmental plasticity of photoreceptors Benjamin E. Reese* Neuroscience Research Institute and Department of Psychology, University of California at Santa Barbara, Santa Barbara, CA 93106-5060, USA

Abstract: During development, retinal ganglion cells undergo conspicuous structural remodeling as they gradually attain their mature morphology and connectivity. Alterations in their dendritic organization and in their axonal projections can also be achieved following early insult to their targets or their afferents. Other retinal cell types are thought not to display this same degree of developmental plasticity. The present review will consider the evidence, drawn largely from recent experimental studies in the carnivore retina, that photoreceptors also undergo structural remodeling, extending their terminals transiently into inner plexiform layer before retracting to the outer plexiform layer. The determinants of this transient targeting to the inner plexiform layer are considered, and the role of cholinergic amacrine cells is discussed. The factors triggering this retraction are also considered, including the concurrent maturational changes in outer segment formation and in the differentiation of the outer plexiform layer. These results provide new insight into the life history of the photoreceptor cell and its connectivity, and suggest a transient role for the photoreceptors in the circuitry of the inner retina during early development, prior to the onset of phototransduction.

Introduction

sculpting, and retraction that has been described for ganglion cell dendrites and axonal projections (Frost et al., 1979; Frost, 1984; Dann et al., 1987, 1988; Ramoa et al., 1987, 1988; Langdon and Frost, 1991; Bodnarenko et al., 1995, 1999). Recent studies have shown, however, that photoreceptors initially project beyond their normal target territory (Johnson et al., 1999), much like retinal ganglion cells that overshoot the superior colliculus and invade the inferior colliculus before retracting to form their normal target innervation within the superior colliculus (Cooper and Cowey, 1990a,b). Likewise, much as these early exuberant retinofugal projections are modulated in response to the time-dependent availability of their normal and alternative targets (Perry and Cowey, 1979, 1982), so the exuberant photoreceptor projection is transiently controlled by the presence of an alternative target during development (Johnson et al., 2001a). This review will consider the major features of photoreceptor development before examining the evidence for this developmental plasticity of the rods and cones within the ferret’s retina.

Our visual abilities arise from the capacity of photoreceptor cells to transduce a photic stimulus into a neural response and to transmit this message to second-order neurons. Effective transmission of that signal is dependent on processes acting during development that orchestrate the formation of the normal retinal architecture and circuitry. The cellintrinsic and environmental factors controlling the morphological differentiation of the photoreceptor outer segment and its associated functional maturation are beginning to be understood, but relatively little is known about the developmental mechanisms responsible for establishing the connectivity of these cells. Photoreceptors, like most other retinal cells besides the retinal ganglion cells, are generally believed to differentiate and form synaptic connections in a targeted manner, avoiding the elaborate overgrowth, *Corresponding author. Tel.: þ1-805-893-2091; Fax: þ1-805-893-2005; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14400-1

3

4

Photoreceptor differentiation The vertebrate photoreceptor is a uniquely polarized nerve cell, with a bipolar-shaped soma giving rise to an apical specialization for phototransduction and a basally directed process for transmitting the neural response to second-order neurons within the retina. Numerous studies have charted the formation and developmental time course of these two specializations, including the ultrastructural appearance of the outer and inner segments, and the establishment of synaptic ribbons associated with the rod spherules and cone pedicles. In general, such studies have shown that these features of the mature rods and cones are acquired gradually and progressively, with little remodeling or transdifferentiation through intermediate morphologies. For example, studies of photoreceptor differentiation reveal the formation of an inner segment, a cilium and ballooning outer segment, followed by the appearance of membranous disks within the outer segment. Subsequently, both the inner and outer segments increase in length until they achieve their adult size (Olney, 1968; Feeney, 1973; McArdle et al., 1977; Vogel, 1978; Tucker et al., 1979; Morrison, 1983; Usukura and Obata, 1995). Likewise, the formation of connectivity within the outer plexiform layer (OPL) proceeds by the appearance of presynaptic densities, or ‘ribbons’, in the basally directed processes of the rods and cones presaging synaptic terminals, followed by the apposition and invagination of lateral processes from horizontal cells. These dyadic complexes are subsequently modified by the invagination of bipolar cell dendrites to create the synaptic arrangements characteristic of mature photoreceptor terminals (Olney, 1968; Weidman and Kuwabara, 1968; Blanks et al., 1974; McArdle et al., 1977; Vogel, 1978; Maslim and Stone, 1986; Rapaport, 1989).

but low levels of protein are occasionally found in cellular compartments for which there is no apparent function. For example, rod opsin protein is detectable not only within the disks of the outer segment but is also found within the plasma membrane of the entire cell (Jan and Revel, 1974; Nir and Papermaster, 1986; Jansen et al., 1987; Usukura and Bok, 1987; Bowes et al., 1988; Hicks et al., 1989; Lewis et al., 1991; Edward et al., 1993). During development, the onset of protein expression is widely assumed to parallel morphological differentiation (Colombaioni and Strettoi, 1993; Timmers et al., 1993), but some studies examining rod opsin expression contradict this view. For instance, rod opsin has been shown to be present in the plasma membrane of the rods well before these cells assemble their outer segments (Hicks and Barnstable, 1987; Bowes et al., 1988; Treisman et al., 1988; Watanabe and Raff, 1990; Saha and Grainger, 1993; Dorn et al., 1995; Jasoni and Reh, 1996). Likewise, the expression of SNARE complex proteins has been regarded as indicative of synaptogenesis (Devoto and Barnstable, 1989; Voigt et al., 1993; Kapfhammer et al., 1994; Dhingra et al., 1997), but there is also evidence that some of these synaptic vesicle proteins and presynaptic membrane proteins are present well before ultrastructurally identifiable synapses can be detected within the retina (Hering and Kro¨ger, 1996; West Greenlee et al., 2001). Antibodies to synaptic vesicle proteins label the entirety of the developing outer nuclear layer (ONL) during the period preceding the emergence of the OPL (Reese et al., 1996); thereafter, these proteins become progressively restricted to the OPL during the period of synapse formation (Greiner and Weidman, 1981). Unfortunately, relatively little is understood about the assembly of synaptic ribbons within photoreceptor terminals; further study of their plasticity in maturity may shed light upon the mechanisms that assemble them during development (Vollrath and Spiwoks-Becker, 1996).

Protein trafficking during development These morphological specializations associated with the apical and basal extensions of photoreceptor cells contain various proteins mediating their visual transduction-related and synaptic functions. In general, these proteins are selectively targeted to the outer segment or the synaptic terminal, respectively,

Environmental determinants of differentiation and connectivity Environmental signals for some of the maturational milestones associated with photoreceptor differentiation are beginning to be defined. For example, the

5

differentiation of an outer segment containing organized stacks of membranous disks requires close association or contact with cells of the retinal pigment epithelium (Hollyfield and Witkovsky, 1974; Spoerri et al., 1988; Stiemke et al., 1994; Pinzo´nDuarte et al., 2000; Bumsted et al., 2001). The outgrowth and targeting of a process from the opposite, basal, pole of the cell body to the future OPL is also presumed to involve interactions with other cells in the local environment, by way of cellsurface or secreted molecules, but little is known about the processes of initial outgrowth and subsequent target recognition. Growth factors released locally by retinal neurons and glia are known to activate receptors expressed on neighboring cells, controlling not only cell survival but also differentiation (Ary-Pires et al., 1997). For example, fibroblast growth-factor (FGF) receptors expressed on photoreceptors are thought to mediate the effects of bFGF on rod fate determination, differentiation, and resistance to injury (Hicks and Courtois, 1992; LaVail et al., 1992; Unoki and LaVail, 1994; Blanquet and Jonet, 1996; Gao and Hollyfield, 1996; Carwile et al., 1998). The glial cell-line-derived neurotrophic factor (GDNF) is also expressed in the retina, where it not only promotes ganglion cell differentiation and survival, but has also been shown to preserve the functional status of photoreceptors in vitro (Norsat et al., 1996; Klocker et al., 1997; Carwile et al., 1998; Yan et al., 1999). The neurotrophins NT-3, NT-4 and brain-derived neurotrophic factor (BDNF), and their receptors trkA, trkB, trkC, and p75 also play important roles in the morphogenesis of the visual system via paracrine mechanisms (von Bartheld, 1998). For example, dopaminergic amacrine cells increase their soma size and innervation density after BDNF application (Cellerino et al., 1998), while ganglion cells modulate their process elongation and arborization in response to BDNF and NT-4 (Bosco et al., 1993; Bosco and Linden, 1999). BDNF may also contribute to photoreceptor development. The avian photoreceptor layer expresses trkB mRNA, although it has gone undetected in the same layer of mammals (Jelsma et al., 1993; Okazawa et al., 1995; Perez and Caminos, 1995). Still, BDNF has been shown to have a clear protective effect on mammalian photoreceptors (LaVail et al., 1992; Unoki and LaVail, 1994; Perez and Caminos, 1995; LaVail et al., 1998),

and trkB knockout mice show delayed rod maturation and defective rod signaling with inner retinal neurons (Rohrer et al., 1999). These actions of BDNF signaling through trkB may be mediated indirectly by other retinal cells, particularly since application of BDNF activates intracellular-signaling pathways in inner retinal neurons and Mu¨ller glia but not in photoreceptors (Wahlin et al., 2000, 2001). A substantial fraction of the BDNF in the inner retina is derived from local sources (rather than from the optic tectum via retrograde transport), including the amacrine and ganglion cells, which synthesize and secrete BDNF, as well as express trkB (Zanellato et al., 1993; Rickman and Brecha, 1995; Ugolini et al., 1995; Cohen-Cory et al., 1996; Cellerino and Kohler, 1997; Herzog and von Bartheld, 1998). Given the intimacy shared between Mu¨ller glia and photoreceptors in vivo (Robinson and Dreher, 1990), and their role as a preferred substrate for neuritic extension by photoreceptors in vitro (Kljavin and Reh, 1991), any indirect neurotrophic effect upon photoreceptors should therefore be mediated through the Mu¨ller glia. While such a plausible neurotrophic action may contribute to the outgrowth and morphological differentiation of photoreceptors during their development, no direct evidence for a role in terminal outgrowth or target recognition has emerged to date.

Transient retinal circuitry How retinal cells communicate during early development, prior to the establishment of the mature circuitry, has become a major focus of attention recently, as has the question of the functional significance of such precocious communication (Catsicas and Mobbs, 1995; Copenhagen, 1996; Feller, 1999; Wong, 1999). In the developing inner retina, neighboring ganglion and amacrine cells display correlated ‘spontaneous’ neural activity, well before photoreceptors can respond to light. This activity has been shown to originate at a location on the retina and then propagate as a wave of activity before dissipating, after which another such wave will materialize elsewhere (Meister et al., 1991; Wong et al., 1993; Feller et al., 1996). While the full significance of these waves of activity remain to be

6

defined, there is increasing evidence that such correlated activity in the discharge patterns of the retinal ganglion cells plays a critical role in the establishment of ocular segregation and ON–OFF lamination within the lateral geniculate nucleus (Cramer and Sur, 1997; Penn et al., 1998; Muir-Robinson et al., 2002; Stellwagen and Shatz, 2002), and may prove to contribute to the formation of retinotopic maps (Eglen, 1999). The mechanisms driving this neural activity are undefined, but recent studies indicate that synaptic and gap-junctional connectivity within the inner plexiform layer (IPL) permits this activity to be propagated across the retina (Penn et al., 1994; Wong et al., 1995; Feller et al., 1996; Zhou, 1998; Singer et al., 2001). Ganglion and amacrine cell dendrites are synaptically connected during this early developmental stage, but the other main constituent of the IPL, the axon terminal of the bipolar cell, normally the driving force of neural activity within the mature IPL, develops postnatally, after this spontaneous activity is already present (Miller et al., 1998). Curiously, cells in the outer retina show receptormediated increases in intracellular calcium concentrations during these early developmental stages (Wong, 1995), suggesting that they participate in the perinatal retinal circuitry, but their identity and function remain to be determined. One possibility is that these cells are developing photoreceptors that initially overextend their terminals into the developing IPL.

Photoreceptor affinity for the inner retina That photoreceptors may have some affinity for inner retinal cells, the amacrine and ganglion cells, is not without precedent. For example, retinas from humans with retinitis pigmentosa contain surviving photoreceptors that extend their terminals into the inner retina, where they contact amacrine cells (Fariss et al., 2000), while dissociated retinal cell cultures have been shown to contain regenerating photoreceptor neurites that preferentially contact amacrine and ganglion cells over their normal target cells (Sherry et al., 1996). Further, in the zebrafish mutant, cannonball, rod photoreceptors project directly into the IPL (Brian Link, personal communication). These

examples are all drawn from anomalous developmental or degenerative conditions, but this relationship between the photoreceptors and inner retinal cells is also present during normal development.

Immature rods and cones project to the inner plexiform layer In the developing ferret’s retina, immature photoreceptors project directly to the inner plexiform layer, well before the OPL has formed (Johnson et al., 1999). By using antibodies to rod opsin, a narrow row of immunoreactive cells occupying the neuroblast layer can be identified on the day of birth, extending apically directed processes to the ventricular surface and basal processes through the neuroblast layer and beyond a layer of postmitotic amacrine cells, reaching the IPL (Fig. 1). These projections typically end in a single terminal expansion, occasionally branching within the IPL. The abundance of these projections to the IPL indicates that their outgrowth is not some rare ectopic event: at least 80% of the rod opsinimmunoreactive cells on postnatal day 1 (P-1) extend such processes (Johnson et al., 1999). Using antibodies to the cone opsins, by contrast, no immunoreactive cone photoreceptors can be detected prior to P-22 (Johnson et al., 2001b). A similar delay in cone opsin expression relative to rod opsin expression has been reported in the monkey and rat retinas (Watanabe and Raff, 1990; Sze´l et al., 1994; Dorn et al., 1995; Jasoni and Reh, 1996; Bumsted et al., 1997), despite the fact that cones are known to be generated before rods in primates, carnivores, and rodents (Young, 1985; LaVail et al., 1991; Johnson et al., 1999), suggesting that cones differentiate later than rods. Antibodies to recoverin, however (a calcium-binding protein found in adult rods and cones; Dizhoor et al., 1991), label two populations of cells in the outer parts of the neuroblast layer on the day of birth: one population is relatively faintly labeled, and can be doublelabeled with antibodies to rod opsin. The other more intensely labeled population consists of immature cone photoreceptors. Like the population of rod opsin-immunoreactive cells, the entirety of these more intensely labeled recoverin-immunoreactive cells is labeled, including apical and basal processes.

7

generated (Zimmerman et al., 1988), and already occupy an intermediate position within the developing neuroblast layer, anticipating the future level of the OPL (Greiner and Weidman, 1981; Reese et al., 1996). These horizontal cells have already begun to differentiate laterally oriented, neurofilamentimmunoreactive processes (Fig. 1), yet despite the presence of this postsynaptic target for photoreceptor cells, the projections of the latter extend well beyond this level (Johnson et al., 1999). The neuroblast layer must therefore already contain positional information specifying the level of the future OPL, yet the developing photoreceptor terminals do not respond to it.

Immunopositive cells are not proliferating neuroblasts

Fig. 1. Photoreceptors (stippled cells) initially extend their terminals through the neuroblast layer and amacrine cell layer into the IPL. Horizontal cells (diagonal lines) have already been generated and have migrated into the neuroblast layer, anticipating the site of the future OPL, yet despite their presence, the photoreceptors extend beyond them.

Their apical processes typically extend through the future outer-limiting membrane, giving rise to presumptive inner segments (Greiner and Weidman, 1981). They are detected as early as embryonic day 24 (E-24), shortly after the first cones become postmitotic, and their basally directed processes already reach into the IPL by E-30 (Johnson et al., 1999).

Signals for the nascent OPL are already present By P-1, the IPL is continuous across the entire retina, while the OPL has still yet to form (Reese et al., 1996). Horizontal cells, however, have already been

The morphology of these bipolar-shaped rod opsinand recoverin-immunoreactive cells is reminiscent of proliferating neuroepithelial cells (Hinds and Hinds, 1979; Brittis et al., 1995), raising the possibility that these cells are not postmitotic rods and cones but are precursor cells that may have already begun to express proteins characteristic of their eventual progeny. These rod opsin-positive and recoverinpositive cells, however, are situated in a narrow stratum within the neuroblast layer, while proliferating retinal cells, identified with an antibody to the cell cycle-specific nuclear antigen Ki67 (Gerdes et al., 1983; Geller et al., 1995), are distributed across the full thickness of the neuroblast layer (Reese et al., 1996), being most common in the future S- and M-phase zones (Johnson et al., 1999). In fact, the tier occupied by the rod opsin-positive and recoverinpositive cells is relatively Ki67-negative, indicating that most of the cells here are postmitotic. Further, none of the recoverin-positive cells can be double-labeled with antibodies for the Ki67 antigen (Johnson et al., 1999). Finally, injections of the thymidine analog bromodeoxyuridine on P-1 never double-labeled any of the recoverin-positive cells. This latter result rules out the possibility that the rod opsin-positive or recoverin-positive cells are a unique population of precursors that fail to express the Ki67 antigen nor show the classic pattern of interkinetic nuclear translocation associated with

8

the neuroepithelium (Robinson et al., 1985). The rod opsin-positive and recoverin-positive cells must therefore all be postmitotic, presumed to be rod or cone photoreceptors by virtue of their immunoreactivity and their positioning.

Protein-expression patterns in developing photoreceptors Most proteins associated with the visual transduction cycle and photoreceptor structure are normally detected around the time of outer segment formation (Colombaioni and Strettoi, 1993; Timmers et al., 1993), which in the ferret commences around P-15 (Greiner and Weidman, 1981). Antibodies to these proteins, including b- and g-transducin, phosducin, phosphodiesterase-g (PDEg), rhodopsin kinase, rod cGMP-gated ion channel, and peripherin, do not label photoreceptor cells in the ferret retina until the second or third postnatal week (Johnson et al., 2001b). In some cases, these proteins are compartmentally selective from the earliest stages of detection, being found exclusively within the outer segments (e.g. peripherin, the cGMP-gated cation channel and b-transducin). Others have a similar time of onset, but are found throughout the cell (e.g. gtransducin, PDEg, and phosducin), implying that distinct protein-trafficking mechanisms are at work (Fariss et al., 1997). The onset of expression of each of these proteins appears to be synchronized amongst both old and young photoreceptors, whereas the rod opsin and recoverin protein-expression patterns emerge gradually in cells in accord with their neurogenetic gradients, occurring first in a few cells in the central retina, spreading to cells at increasingly peripheral locations, and continuing to be expressed in more and more cells at all retinal loci as these cells are generated (Johnson et al., 2001b; see also Bowes et al., 1988; Treisman et al., 1988; Saha and Grainger, 1993). Because rod opsin and recoverin protein expression follow such different spatio-temporal gradients from those other outer segment-associated proteins, and since in some other species, rod opsin is reported to be expressed at the same time as these other proteins (Timmers et al., 1993; van Ginkel and Hauswirth, 1994), one might question whether the

early immunodetection of rod opsin and recoverin was spurious or artifactual. Yet independent RTPCR analyses confirm the precocious expression of these two mRNA transcripts on the day of birth, while the mRNAs for those other transductionrelated proteins mentioned above could not be detected until P-15, when outer segment assembly begins (Johnson et al., 2001b). There seems little doubt that immature rods and cones activate their rod opsin and recoverin genes and synthesize these proteins well before these cells are capable of generating a response to light. Whether these two proteins play some other precocious role in developing photoreceptors remains to be seen. Recoverin is a known calcium sensor (Ames et al., 1996; Polans et al., 1996), and given the myriad functions of calcium during development (Gu and Spitzer, 1995), it may play some other fundamental role in photoreceptor maturation. As for rod opsin, some other nonvisual function has been implicated by the fact that species of cave-dwelling crayfish, never exposed to light, should lack a functional constraint upon the frequency of mutations within the rod opsin gene, yet they show no difference from their surface-dwelling cousins (Crandall and Hillis, 1997). Regardless of whether the early expression of either of these two proteins plays a transient functional role, the fact that they are present early on and are found throughout the cell enables one to trace the complete morphology of these photoreceptors, including their projection into the IPL.

Maturational events in the OPL may trigger the elimination of this projection During the first two postnatal weeks, increasing numbers of rods become postmitotic and the number of them projecting to the IPL increases, reaching maximal density on P-15. By P-15, a cell-free OPL has formed across conspicuous stretches of the dorsal retina (Reese et al., 1996), yet the photoreceptor projections continue to extend through this region, reaching the IPL (Fig. 2). Thereafter, however, the frequency of these immunopositive processes reaching the IPL declines rapidly, falling to nearly zero by the end of the third postnatal week (Johnson et al., 1999). The gradual accumulation of these projections

9

Fig. 2. The density of photoreceptors projecting to the IPL steadily increases during the first two postnatal weeks. Horizontal cells (diagonal lines) begin elaborating their horizontally oriented dendrites toward the end of the second postnatal week, giving rise to a cell-sparse outer plexiform layer.

Fig. 3. As the horizontal and then bipolar cells (diagonal lines) continue to mature, giving rise to a continuous plexus of processes within the OPL during the third postnatal week, the photoreceptors retract their terminals from the IPL and form synapses within the OPL. Outer segment assembly is also initiated during this period.

in the IPL, between E-24 and P-15, followed by their sudden elimination during the third postnatal week, suggests that their loss is not linked to the maturational age of each cell; rather, some environmental signal has orchestrated this elimination. This loss of projections to the IPL is coincident with the further maturation of the constituents of the OPL between P-15 and P-22. A calbindin-positive plexus in the OPL gradually develops during this same period of process elimination, arising first from the differentiation of horizontal, and later, bipolar cell dendrites (Reese et al., 1996). This period is also coincident with the formation of ribbon synapses within the OPL (Greiner and Weidman, 1981;

Rapaport, 1989). Hence, maturational events in the developing OPL may trigger this elimination (Fig. 3).

Process retraction, rather than cell death or selective protein trafficking, is responsible for the elimination of these projections The decline in the number of photoreceptors projecting to the IPL does not appear to be associated with apoptosis of the parent photoreceptor cell in the ONL. Programmed cell death occurs only scarcely in the ONL, evidenced by terminal deoxytransferase dUTP nick-end labeling (TUNEL)

10

of dying cells (Johnson et al., 1999). The period of naturally occurring cell death in the ONL peaks around P-42, weeks after this period of process elimination. TUNELþ cells present within the INL during this period of process elimination serve as an internal control, confirming that their absence in the ONL during this period is not due to insufficient sensitivity of the technique. Indeed, a virtually identical time course for the relative frequency of apoptotic profiles in the ONL and INL was reported for the developing cat’s retina (Maslim et al., 1997). An alternative explanation for the transience of this projection is that it remains intact but ceases to be immunopositive for rod opsin or recoverin. This explanation, while unlikely, should not be dismissed out of hand, given that protein-trafficking mechanisms within photoreceptor cells become compartmentally selective after outer segments differentiate. Yet such an explanation can be ruled out because crystalline implants of the lipophilic carbocyanine dye, DiI, placed into the IPL of fixed specimens, readily label somata throughout both the INL and ONL on P-15, but no longer do so on P-29 or thereafter, confirming that the cells of the ONL do not maintain a process to this depth within the retina (Johnson et al., 1999). Rather, these processes must be retracted to the OPL, presumably triggered by other maturational changes at that depth within the retina (Fig. 3). A similar overextension, followed by retraction, of cone photoreceptor terminals in the rat retina expressing a glutamate transporter splice variant, and possibly also being glycine immunoreactive, has recently been reported (Pow and Hendrickson, 2000; Reye et al., 2002), and cone photoreceptors in the primate retina have also been observed to extend processes transiently to the IPL (Anita Hendrickson, personal communication). While no obvious phylogenetic correlations can be made from such a limited dataset, a parallel with a hypothesized ancestral light-sensing tissue lacking bipolar cells, in which photoreceptors directly innervate projection neurons, has been noted (Reichenbach and Robinson, 1995), akin to the pineal organ of the fish (Eckstro¨m, 1987). Yet in contrast with this example, the present photoreceptor projection to the IPL is more intimately associated with a retinal interneuron, the cholinergic amacrine cell, rather than with retinal ganglion cells.

Photoreceptors target cholinergic amacrine cells These transient projections are positioned to influence the other constituents within the IPL. Ganglion cells in the ferret retina are generated during the fourth and fifth prenatal weeks (Reese et al., 1994) and differentiate dendritic arbors shortly thereafter, forming cell class-specific morphologies during the first two postnatal weeks (Wingate and Thompson, 1994, 1995) and differentiating separate ON and OFF substrata over the first postnatal month (Bodnarenko et al., 1999; Lohmann and Wong, 2001; Wang et al., 2001). The cholinergic amacrine cells, by contrast, differentiate processes that occupy separate ON and OFF substrata much earlier, during the first postnatal week (Reese et al., 2001). Coincident with these two strata of cholinergic processes in the IPL, rod opsin-positive projections terminate at one of these same two levels during the second postnatal week (Fig. 4). Clearly, these immature photoreceptor projections recognize and respond to features defining the stratification of the developing IPL; they do not simply grow to the inner-limiting membrane (Johnson et al., 2001a).

Photoreceptor processes are immunoreactive for synaptic vesicle proteins Numerous examples of transient synaptic connectivity exist elsewhere in the CNS, including the visual system. For example, geniculo-cortical axons form transient synaptic connections with subplate cells prior to their establishing connections within the cortical plate (Chun and Shatz, 1988; Friauf and Shatz, 1991; Herrmann et al., 1994), while optic axons establish synapses within ocular domains of the lateral geniculate nucleus and superior colliculus from which they will subsequently retract (Campbell et al., 1984; Campbell and Shatz, 1992). Perhaps photoreceptors similarly form transient synapses with the cholinergic amacrine cell processes in the IPL prior to their retraction, since these cells contain synaptic proteins such as synaptophysin. Synaptophysin, an integral membrane protein of synaptic vesicles (Wiedenmann and Franke, 1985; Sudhof et al., 1987) present in both conventional and ribbon synapses (Catsicas et al., 1992; West Greenlee et al.,

11

a second synaptic vesicle protein, synaptotagmin, supporting the interpretation that these photoreceptors are preparing for, or may already be engaged in, synaptogenesis within the IPL (Johnson et al., 1999).

Early ablation of the cholinergic amacrine cells disrupts photoreceptor stratification in the IPL

Fig. 4. During the second postnatal week, photoreceptor terminals extend to one of two depths within the IPL, coincident with the stratifying processes of the cholinergic amacrine cells (dark stippled cells at the top). The photoreceptors are also immunoreactive for synaptic vesicle proteins at this stage.

1996), is generally first detectable at the onset of synaptogenesis (Knaus et al., 1986; Devoto and Barnstable, 1989; Voigt et al., 1993; Kapfhammer et al., 1994; Dhingra et al., 1997), or even preceding it (Hering and Kro¨ger, 1996). As early as the day of birth, recoverin-positive cone somata are richly synaptophysin immunoreactive, as are some rod opsin-positive somata. Conspicuously, the terminals are also richly synaptophysin immunoreactive (Johnson et al., 1999). Such synaptophysin-rich profiles extending to the IPL are increasingly frequent by P-15, despite the fact that the OPL has begun to form and to show a dense accumulation of synaptophysin itself (Reese et al., 1996). Similar results were also obtained using antibodies to

To confirm that the cholinergic amacrine cells specify the depth at which these photoreceptor projections stratify, cholinergic amacrine cells were ablated using an excitotoxic approach with L-glutamate. A single subcutaneous dose of 4 mg/g of body weight at the end of the first postnatal week was found to kill off virtually all of the cholinergic amacrine cells in the central retina, while leaving the retinal architecture relatively normal, besides a slight reduction in the thickness of the INL and IPL. This excitotoxic cell death occurs rapidly, being near-complete within one day following treatment (Reese et al., 2001). Thus, by killing off the cholinergic amacrine cells at the end of the first postnatal week, any consequence for the stratification of the photoreceptors within the IPL should then be detectable one week later, when that stratification pattern is most pronounced. In fact, one week following such cholinergic ablation, the photoreceptor projection to the IPL was no longer stratified, with terminals now found at various depths within the IPL (Fig. 5), and with a large number extending beyond the IPL into the ganglion cell layer (Johnson et al., 2001a). Unfortunately, this excitotoxicity is not selective for the cholinergic amacrine cells, as the retinal ganglion cells are also reduced by about 50%, and the alpha ganglion cells in particular are nearly completely eliminated (Reese et al., 2001), consistent with their relative glutamatergic excitability (Marc, 1999a,b); other cell types, however, are not affected or are only modestly compromised. To confirm that this change in photoreceptor stratification is not a consequence of the partial ganglion cell elimination produced by their excitotoxic ablation, the optic nerve was transected shortly after birth to kill off all of the ganglion cell population, evidenced by the loss of neurofilament immunoreactivity in the ganglion cell layer and by the loss of the optic fiber layer. While some other cell types undergo a reduction in

12

Fig. 5. Elimination of the cholinergic amacrine cells at the close of the first postnatal week disrupts the normal stratification of the photoreceptor terminals by the end of the second postnatal week.

density when the retinal ganglion cells are eliminated early on, the cholinergic amacrine cells and their strata have been shown not to be affected by this treatment (Williams et al., 2001). Under these experimental circumstances, no change to the photoreceptor stratification pattern was detected (Fig. 6), confirming that the change in photoreceptor stratification following L-glutamate exposure is not a consequence of the compromised ganglion cell population (Johnson et al., 2001a). While one should not overlook the possibility that some other cell type with processes in the IPL has been altered, and that this is the cause of the disruption of the photoreceptor stratification, none of the other stratifying processes within the IPL has been shown to be eliminated or altered following

Fig. 6. Elimination of the retinal ganglion cells during early development, by contrast, does not affect the stratification pattern of the photoreceptor terminals.

L-glutamate treatment (Reese et al., 2001). This then suggests that the alterations in photoreceptor stratification should be due to the cholinergic ablation, implying that their normal spatial coincidence indicates a causal relationship. That they do not simply grow indiscriminately beyond the IPL, but rather depend upon a cell type which itself has been shown to participate in transient retinal circuitry (Feller et al., 1996; Zhou, 1998; Wong et al., 2000), further suggests that they possess some transient functional significance.

Conclusions The early life history of the photoreceptor turns out to be far more complicated than previously considered.

13

Contrary to general opinion, these cells begin to differentiate relatively early during development, expressing photoreceptor-specific proteins well in advance of outer segment formation and the onset of phototransduction. The first cone photoreceptors in the ferret may be born as early as E-22, become immunoreactive for recoverin by E-24, and extend terminals into a differentiating IPL by E-30. Rods become postmitotic largely after the period of cone neurogenesis, through the first postnatal week, and as these cells are generated, they too extend terminals into the IPL, culminating in a maximal projection to the IPL by the end of the second postnatal week. Those photoreceptor terminals seek out the stratified processes of cholinergic amacrine cells during the second postnatal week, potentially engaging the latter in a synaptic relationship during this period. Gap junctional communication between the photoreceptors and inner retinal cells may also play a role, as outer retinal cells are connected through radial processes to differentiating inner retinal neurons during early development (Catsicas et al., 1998; Becker et al., 2002). Exactly how the photoreceptors target these cholinergic strata is unclear. Studies blocking cholinergic neurotransmission during this developmental period should clarify whether this neurotransmitter plays any role in this behavior of the photoreceptors. Alternatively, cholinergic amacrine cells and photoreceptors may express cell-surface proteins like cadherins that provide a molecular basis for this affinity (Honjo et al., 2000). Whatever its cause, this relationship is subsequently lost as the photoreceptor terminals are all retracted to the level of the OPL, apparently triggered by maturational events therein, most likely associated with the differentiation of horizontal or bipolar cell dendrites. L-type calcium channels in the photoreceptor terminal have been shown to play a role in their structural remodeling including retraction in vitro (Nachman-Clewner et al., 1999), but these effects have only been shown in mature photoreceptors, and their relevance to development is uncertain. Two other temporally related events may contribute to this retraction: the first, less-likely event, is the onset of outer segment assembly, known to be triggered by enviromental events associated with the retinal pigment epithelium (Bumsted et al., 2001).

A second possibility is the differentiation of bipolar cell terminals within the IPL, which may ‘dislodge’ the photoreceptor projection much as the normally transient retinofugal projection to the latero-posterior nucleus may be supplanted by the later invasion of other afferents to LP (Perry and Cowey, 1982). This time course of the photoreceptor retraction coincides with the transition of cholinergic-mediated spontaneous inner retinal activity to one of glutamate-mediated activity thought to reflect bipolar differentiation (Miller et al., 1999; Wong et al., 2000). Any role for the photoreceptor terminals in this activity has yet to be defined, but recent pharmacological studies reveal a significant glutamatergic component to this activity even during the earlier ‘cholinergic’ phase (Wong et al., 2000; Zhou and Zhao, 2000). Given the close temporal congruity between this transient projection and the cholinergic phase, coupled with the targeted association of the former with the cholinergic processes, the photoreceptor projection to the IPL may play a functional role in this transient retinal circuitry, providing a glutamatergic drive to the inner retina, initiating focal activity that is subsequently conveyed as waves via cholinergic and gap junctional transmission (Feller et al., 1996; Singer et al., 2001). The temporal relationship between these various anatomical and physiological events, and their relationship to other hallmark features of retinal development in the ferret, are indicated in Fig. 7. The developing retina would seem hardly the place to root out visual awareness, but if one of the wider contributions of this field is to understand visual processing as a prerequisite for the restoration of sight, then a fuller appreciation of retinal development is germane to this goal. While many of the other contributions to this volume highlight the cortical and subcortical roots of our perceptual experience, we should keep in mind the retinal processing that provides the neural blueprint for interpretation by higher visual centers. It all begins with phototransduction and signal transmission at the photoreceptor, and understanding those morphological substrates and functional characteristics of photoreceptors is only enhanced by a knowledge of their development (as is our understanding of the visual pathway in general; Reese and Cowey, 1990a,b), particularly

14

Fig. 7. Time-line depicting the major developmental milestones associated with the ferret’s retina. Two other temporal landmarks, birth and eye opening, are also indicated along the time axis. (Data are derived from the following studies: 1Reese et al., 1996; 2Reese et al., 1994; 3Johnson et al., 2001b; 4Johnson et al., 1999; 5Greiner and Weidman, 1981; 6Miller et al., 1999; 7Reese et al., 2001; 8Wong and Oakley, 1996; Bodnarenko et al., 1999; Lohmann and Wong, 2001; Wang et al., 2001; 9Cusato et al., 2001; 10Wong et al., 2000.)

if we are to develop strategies for the treatment of retinal disease.

Acknowledgments This research was supported by grants from the Santa Barbara Cottage Hospital and the National Science Foundation (IBN 9987643). I thank Pat Johnson, Mary Raven, Kathy Giannotti, Karen Cusato and Ryan Williams for their contributions to the studies described herein, and Andy Huberman, Bob Fariss and Jimmy Zhou for their comments on the manuscript.

References Ames, J.B., Tanaka, T., Stryer, L. and Ikura, M. (1996) Portrait of a myristoyl switch protein. Curr. Opin. Struct. Biol., 6: 432–438. Ary-Pires, R., Nakatani, M., Rehen, S.K. and Linden, R. (1997) Developmentally regulated release of intraretinal

neurotrophic factors in vitro. Int. J. Dev. Neurosci., 15: 239–255. Becker, D.L., Bonness, V., Catsicas, M. and Mobbs, P. (2002) Changing patterns of ganglion cell coupling and connexin expression during chick retinal development. J. Neurobiol., 52: 280–293. Blanks, J.C., Adinolfi, A.M. and Lolley, R.N. (1974) Synaptogenesis in the photoreceptor terminal of the mouse retina. J. Comp. Neurol., 156: 81–93. Blanquet, P.R. and Jonet, L. (1996) Signal-regulated proteins and fibroblast growth factor receptors: comparative immunolocalization in rat retina. Neurosci. Lett., 214: 135–138. Bodnarenko, S.R., Jeyarasasingam, G. and Chalupa, L. (1995) Development and regulation of dendritic stratification in retinal ganglion cells by glutamate-mediated afferent activity. J. Neurosci., 15: 7037–7045. Bodnarenko, S.R., Yeung, G., Thomas, L. and McCarthy, M. (1999) The development of retinal ganglion cell dendritic stratification in ferrets. NeuroReport, 10: 2955–2959. Bosco, A. and Linden, R. (1999) BDNF and NT-4 differentially modulate neurite outgrowth in developing retinal ganglion cells. J. Neurosci. Res., 57: 759–769. Bosco, A., Carri, N.G. and Linden, R. (1993) Neuritogenesis of retinal ganglion cells is differentially promoted by target extract. Brain Res., 632: 303–307.

15 Bowes, C., Van Veen, T. and Farber, D.B. (1988) Opsin, G-protein and 48 kDa protein in normal and rd mouse retinas: developmental expression of mRNAs and proteins and light/dark cycling of mRNAs. Exp. Eye Res., 47: 369–390. Brittis, P.A., Meiri, K., Dent, E. and Silver, J. (1995) The earliest patterns of neuronal differentiation and migration in the mammalian central nervous system. Exp. Neurol., 134: 1–12. Bumsted, K., Jasoni, C., Sze´l, A´. and Hendrickson, A. (1997) Spatial and temporal expression of cone opsins during monkey retinal development. J. Comp. Neurol., 378: 117–134. Bumsted, K.M., Rizzolo, L.J. and Barnstable, C.J. (2001) Defects in the MITF(mi/mi) apical surface are associated with a failure of outer segment elongation. Exp. Eye Res., 73: 383–392. Campbell, G. and Shatz, C.J. (1992) Synapses formed by identified retinogeniculate axons during the segregation of eye input. J. Neurosci., 12: 1847–1858. Campbell, G., So, K.-F. and Lieberman, A.R. (1984) Normal postnatal development of retinogeniculate axons and terminals and identification of inappropriately-located transient synapses: electron microscope studies of horseradish peroxidase-labelled retinal axons in the hamster. Neuroscience, 13: 743–759. Carwile, M.E., Culbert, R.B., Sturdivant, R.L. and Kraft, T.W. (1998) Rod outer segment maintenance is enhanced in the presence of bFGF, CNTF and GDNF. Exp. Eye Res., 66: 791–805. Catsicas, M. and Mobbs, P. (1995) Waves are swell. Curr. Biol., 5: 977–979. Catsicas, S., Catsicas, M., Keyser, K.T., Kartein, H.J., Wilson, M.C. and Milner, R.J. (1992) Differential expression of the presynaptic protein SNAP-25 in mammalian retina. J. Neurosci. Res., 33: 1–9. Catsicas, M., Bonness, V., Becker, D. and Mobbs, P. (1998) Spontaneous Ca2þ transients and their transmission in the developing chick retina. Curr. Biol., 8: 283–286. Cellerino, A. and Kohler, K. (1997) Brain-derived neurotrophic factor/neurotrophin-4 receptor trkB is localized on ganglion cells and dopaminergic amacrine cells in the vertebrate retina. J. Comp. Neurol., 386: 149–160. Cellerino, A., Pinzon-Duarte, G., Carroll, P. and Kohler, K. (1998) Brain-derived neurotrophic factor modulates the development of the dopaminergic network in the rodent retina. J. Neurosci., 18: 3351–3362. Chun, J.J.M. and Shatz, C.J. (1988) Redistribution of synaptic vesicle antigens is correlated with the disappearance of a transient synaptic zone in the developing cerebral cortex. Neuron, 1: 297–310. Cohen-Cory, S., Escandon, E. and Fraser, S.E. (1996) The cellular patterns of BDNF and trkB expression suggest

multiple roles for BDNF during Xenopus visual system development. Dev. Biol., 179: 102–115. Colombaioni, L. and Strettoi, E. (1993) Appearance of cGMPphosphodiesterase immunoreactivity parallels the morphological differentiation of photoreceptor outer segments in the rat retina. Vis. Neurosci., 10: 395–402. Cooper, A.M. and Cowey, A. (1990a) Development and retraction of a crossed retinal projection to the inferior colliculus in neonatal pigmented rats. Neuroscience, 35: 335–344. Cooper, A.M. and Cowey, A. (1990b) Retinal topography of the neonatal crossed aberrant exuberant projection to the inferior colliculus in the pigmented rat. Neuroscience, 35: 345–354. Copenhagen, D.R. (1996) On the crest of an exciting wave. Curr. Biol., 6: 1368–1370. Cramer, K.S. and Sur, M. (1997) Blockade of afferent impulse activity disrupts on/off sublamination in the ferret lateral geniculate nucleus. Dev. Brain Res., 98: 287–290. Crandall, K.A. and Hillis, D.M. (1997) Rhodopsin evolution in the dark. Nature, 387: 667–668. Cusato, K., Stagg, S.B. and Reese, B.E. (2001) Two phases of increased cell death in the inner retina following early elimination of the ganglion cell population. J. Comp. Neurol., 439: 440–449. Dann, J.F., Buhl, E.H. and Peichl, L. (1987) Dendritic maturation in cat retinal ganglion cells: a Lucifer yellow study. Neurosci. Lett., 80: 21–26. Dann, J.F., Buhl, E.H. and Peichl, L. (1988) Postnatal dendritic maturation of alpha and beta ganglion cells in cat retina. J. Neurosci., 8: 1485–1499. Devoto, S.H. and Barnstable, C.J. (1989) Expression of the growth cone specific epitope CDA 1 and the synaptic vesicle protein SVP38 in the developing mammalian cerebral cortex. J. Comp. Neurol., 190: 154–168. Dhingra, N.K., Ramamohan, Y. and Raju, T.R. (1997) Developmental expression of synaptophysin, synapsin I and syntaxin in the rat retina. Dev. Brain Res., 102: 267–273. Dizhoor, A.M., Ray, S., Kumar, S., Niemi, G., Spencer, M., Brolley, D., Walsh, K.A., Philipov, P.P., Hurley, J.B. and Stryer, L. (1991) Recoverin: a calcium sensitive activator of retinal rod guanylate cyclase. Science, 251: 915–918. Dorn, E.M., Hendrickson, L. and Hendrickson, A.E. (1995) The appearance of rod opsin during monkey retinal development. Invest. Ophthalmol. Vis. Sci., 36: 2634–2651. Eckstro¨m, P. (1987) Photoreceptors and CSF-contacting neurons in the pineal organ of a teleost fish have direct axonal connections with the brain: an HRP-electron microscopic study. J. Neurosci., 7: 987–995. Edward, D.P., Lim, K., Sawaguchi, S. and Tso, M.O.M. (1993) An immunohistochemical study of opsin in photoreceptor cells following light-induced retinal degeneration in the rat. Graefes Arch. Clin. Exp. Ophthalmol., 231: 289–294.

16 Eglen, S.J. (1999) The role of retinal waves and synaptic normalization in retinogeniculate development. Phil. Trans. R. Soc. (Lond.), B354: 497–506. Fariss, R.N., Molday, R.S., Fisher, S.K. and Matsumoto, B. (1997) Evidence from normal and degenerating photoreceptors that two outer segment integral membrane proteins have separate transport pathways. J. Comp. Neurol., 387: 148–156. Fariss, R.N., Li, Z.-Y. and Milam, A.H. (2000) Abnormalities in rod photoreceptors, amacrine cells, and horizontal cells in human retinas with retinitis pigmentosa. Am. J. Ophthalmol., 129: 215–223. Feeney, L. (1973) The interphotoreceptor space. I. Postnatal ontogeny in mice and rats. Dev. Biol., 32: 101–114. Feller, M.B. (1999) Spontaneous correlated activity in developing neural circuits. Neuron, 22: 653–656. Feller, M.B., Wellis, D.P., Stellwagen, D., Werblin, F.S. and Shatz, C.J. (1996) Requirement for cholinergic synaptic transmission in the propagation of spontaneous retinal waves. Science, 272: 1182–1187. Friauf, E. and Shatz, C.J. (1991) Changing patterns of synaptic input to subplate and cortical plate during development of visual cortex. J. Neurophysiol., 66: 2059–2071. Frost, D.O. (1984) Axonal growth and target selection during development: retinal projections to the ventrobasal complex and other ‘nonvisual’ structures in neonatal Syrian hamsters. J. Comp. Neurol., 230: 576–592. Frost, D.O., So, K.-F. and Schneider, G.E. (1979) Postnatal development of retinal projections in Syrian Hamsters: a study using autoradiographic and anterograde degeneration techniques. Neuroscience, 4: 1649–1677. Gao, H. and Hollyfield, J.G. (1996) Basic fibroblast growth factor: increased gene expression in inherited and lightinduced photoreceptor degeneration. Exp. Eye Res., 62: 181–189. Geller, S.F., Lewis, G.P., Anderson, D.H. and Fisher, S.K. (1995) Use of the MIB-1 antibody for detecting proliferating cells in the retina. Invest. Ophthalmol. Vis. Sci., 36: 737–744. Gerdes, J., Schwab, U., Lemke, H. and Stein, H. (1983) Production of a mouse monoclonal antibody reactive with a human nuclear antigen associated with cell proliferation. Int. J. Cancer, 31: 13–20. Greiner, J.V. and Weidman, T.A. (1981) Histogenesis of the ferret retina. Exp. Eye Res., 33: 315–332. Gu, X. and Spitzer, N.C. (1995) Distinct aspects of neuronal differentiation encoded by frequency of spontaneous Ca2þ transients. Nature, 375: 784–787. Hering, H. and Kro¨ger, S. (1996) Formation of synaptic specializations in the inner plexiform layer of the developing chick retina. J. Comp. Neurol., 375: 393–405. Herrmann, K., Antonini, A. and Shatz, C.J. (1994) Ultrastructural evidence for synaptic interactions between

thalamocortical axons and subplate neurons. Eur. J. Neurosci., 6: 1729–1742. Herzog, K.-H. and von Bartheld, C.S. (1998) Contributions of the optic tectum and the retina as sources of brain-derived neurotrophic factor for retinal ganglion cells in the chick embryo. J. Neurosci., 18: 2891–2906. Hicks, D. and Barnstable, C. (1987) Different rhodopsin monoclonal antibodies reveal different binding patterns on developing and adult rat retina. J. Histochem. Cytochem., 35: 1317–1328. Hicks, D. and Courtois, Y. (1992) Fibroblast growth factor stimulates photoreceptor differentiation in vitro. J. Neurosci., 12: 2022–2033. Hicks, D., Sparrow, J. and Barnstable, C.J. (1989) Immunoelectron microscopical examination of the surface distribution of opsin in rat rod photoreceptor cells. Exp. Eye Res., 49: 13–29. Hinds, J.W. and Hinds, P.L. (1979) Differentiation of photoreceptors and horizontal cells in the embryonic mouse retina: an electron microscopic, serial section analysis. J. Comp. Neurol., 187: 495–512. Hollyfield, J.G. and Witkovsky, P. (1974) Pigmented retinal epithelium involvement in photoreceptor development and function. J. Exp. Zool., 189: 357–378. Honjo, M., Tanihara, H., Suzuki, S., Tanaka, T., Honda, Y. and Takeichi, M. (2000) Differential expression of cadherin adhesion receptors in neural retina of the postnatal mouse. Invest. Ophthalmol. Vis. Sci., 41: 546–551. Jan, L.Y. and Revel, J.P. (1974) Ultrastructural localization of rhodopsin in the vertebrate retina. J. Cell Biol., 62: 257–273. Jansen, H.G., Sanyal, S., De Grip, W.J. and Schalken, J.J. (1987) Development and degeneration of retina in rds mutant mice: ultraimmunohistochemical localization of opsin. Exp. Eye Res., 44: 347–361. Jasoni, C.L. and Reh, T.A. (1996) Temporal and spatial pattern of MASH-1 expression in the developing rat retina demonstrates progenitor cell heterogeneity. J. Comp. Neurol., 369: 319–327. Jelsma, T.N., Friedman, H.H., Berkelaar, M., Bray, G.M. and Aguayo, A.J. (1993) Different forms of the neurotrophin receptor brkB mRNA predominate in rat retina and optic nerve. J. Neurobiol., 24: 1207–1214. Johnson, P.T., Williams, R.R., Cusato, K. and Reese, B.E. (1999) Rods and cones project to the inner plexiform layer during development. J. Comp. Neurol., 414: 1–12. Johnson, P.T., Raven, M.A. and Reese, B.E. (2001a) Disruption of transient photoreceptor targeting within the inner plexiform layer following early ablation of cholinergic amacrine cells in the ferret. Vis. Neurosci., 18: 741–751. Johnson, P.T., Williams, R.R. and Reese, B.E. (2001b) Developmental patterns of protein expression in photoreceptors implicate distinct environmental vs. cell-intrinsic mechanisms. Vis. Neurosci., 18: 157–168.

17 Kapfhammer, J.P., Christ, F. and Schwab, M.E. (1994) The expression of GAP-43 and synaptophysin in the developing rat retina. Dev. Brain Res., 80: 251–260. Kljavin, I.J. and Reh, T.A. (1991) Mu¨ller cells are a preferred substrate for in vitro neurite extension by rod photoreceptor cells. J. Neurosci., 11: 2985–2994. Klocker, N., Braunling, F., Isenmann, S. and Bahr, M. (1997) In vivo neurotrophic effects of GDNF on axotomized retinal ganglion cells. Neuroreport, 10: 3439–3442. Knaus, P., Betz, H. and Rehm, H. (1986) Expression of synaptophysin during postnatal development of the mouse brain. J. Neurochem., 47: 1302–1304. Langdon, R.B. and Frost, D.O. (1991) Transient retinal axon collaterals to visual and somatosensory thalamus in neonatal hamsters. J. Comp. Neurol., 310: 200–214. LaVail, M.M., Rapaport, D.H. and Rakic, P. (1991) Cytogenesis in the monkey retina. J. Comp. Neurol., 309: 86–114. LaVail, M.M., K.U., Yasumura, D., Matthes, M.T., Yancopoulos, G.D. and Steinberg, R.H. (1992) Multiple growth factors, cytokines, and neurotrophins rescue photoreceptors from the damaging effects of constant light. Proc. Natl. Acad. Sci., 89: 11249–11253. LaVail, M.M., Yasumura, D., Matthes, M.T., LauVillacorta, C., Unoki, K., Sung, C.-H. and Steinberg, R.H. (1998) Protection of mouse photoreceptors by survival factors in retinal degenerations. Invest. Ophthalmol. Vis. Sci., 39: 592–602. Lewis, G.P., Erickson, P.A., Anderson, D.H. and Fisher, S.K. (1991) Opsin distribution and protein incorporation in photoreceptors after experimental retinal detachment. Exp. Eye Res., 53: 629–640. Lohmann, C. and Wong, R.O.L. (2001) Cell-type specific dendritic contacts between retinal ganglion cells during development. J. Neurobiol., 48: 150–162. Marc, R. (1999a) Mapping glutamatergic drive in the vertebrate retina with a channel-permeant organic cation. J. Comp. Neurol., 407: 47–64. Marc, R. (1999b) Kainate activation of horizontal, bipolar, amacrine, and ganglion cells in the rabbit retina. J. Comp. Neurol., 407: 65–76. Maslim, J. and Stone, J. (1986) Synaptogenesis in the retina of the cat. Brain Res., 373: 35–48. Maslim, J., Valter, K., Egensperger, R., Hollander, H. and Stone, J. (1997) Tissue oxygen during a critical developmental period controls the death and survival of photoreceptors. Invest. Ophthalmol. Vis. Sci., 38: 1667–1677. McArdle, C.B., Dowling, J.E. and Masland, R.H. (1977) Development of outer segments and synapses in the rabbit retina. J. Comp. Neurol., 175: 253–273. Meister, M., Wong, R.O.L., Baylor, D.A. and Shatz, C.J. (1991) Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science, 252: 939–943.

Miller, E.D., Wong, W.T. and Wong, R.O.L. (1998) Developmental changes in the neurotransmitter regulation of correlated spontaneous retinal bursting activity. Soc. Neurosci. Abs., 24: 812. Miller, E.D., Tran, M.-N., Wong, G.-K., Oakley, D.M. and Wong, R.O.L. (1999) Morphological differentiation of bipolar cells in the ferret retina. Vis. Neurosci., 16: 1133–1144. Morrison, J.D. (1983) Morphogenesis of photoreceptor outer segments in the developing kitten retina. J. Anat., 136: 521–533. Muir-Robinson, G., Hwang, B.J. and Feller, M.B. (2002) Retinogeniculate axons undergo eye-specific segregation in the absence of eye-specific layers. J. Neurosci., 22: 5259–5264. Nachman-Clewner, M., St. Jules, R. and Townes-Anderson, E. (1999) L-type calcium channels in the photoreceptor ribbon synapse: localization and role in plasticity. J. Comp. Neurol., 415: 1–16. Nir, I. and Papermaster, D.S. (1986) Immunocytochemical localization of opsin in the inner segment and ciliary plasma membrane of photoreceptors in retinas of rds mutant mice. Invest. Ophthalmol. Vis. Sci., 27: 836–840. Norsat, C.A., Tomac, A., Lindqvist, E., Lindskog, S., Humpel, C., Stromberg, I., Ebendal, T., Hoffer, B.J. and Olson, L. (1996) Cellular expression of GDNF mRNA suggests multiple functions inseide and outside the nervous system. Cell Tissue Res., 286: 191–207. Okazawa, H., Damei, M., Imafuku, I. and Kanazawa, I. (1995) Gene regulation of trkB and trkC in the chicken retina by light/darkness exposure. Oncogene, 7: 1813–1818. Olney, J.W. (1968) An electron microscopic study of synapse formation, receptor outer segment development, and other aspects of developing mouse retina. Invest. Ophthalmol., 7: 250–268. Penn, A.A., Wong, R.O.L. and Shatz, C.J. (1994) Neuronal coupling in the developing mammalian retina. J. Neurosci., 14: 3805–3815. Penn, A.A., Riquelme, P.A., Feller, M.B. and Shatz, C.J. (1998) Competition in retinogeniculate patterning driven by spontaneous activity. Science, 279: 2108–2112. Perez, M.T.R. and Caminos, E. (1995) Expression of brainderived neurotrophic factor and of its functional receptor in neonatal and adult retina. Neurosci. Lett., 183: 96–99. Perry, V.H. and Cowey, A. (1979) The effects of unilateral cortical and tectal lesions on retinal ganglion cells in rats. Exp. Brain Res., 35: 85–95. Perry, V.H. and Cowey, A. (1982) A sensitive period for ganglion cell degeneration and the formation of aberrant retino-fugal connections following tectal lesions in rats. Neuroscience, 7: 583–594. Pinzo´n-Duarte, G., Kohler, K., Arango-Gonza´lez, B. and Guenther, E. (2000) Cell differentiation, synaptogenesis, and influence of the retinal pigment epithelium in a rat neonatal organotypic retina culture. Vis. Res., 40: 3455–3465.

18 Polans, A., Baehr, W. and Palczewski, K. (1996) Turned on by Ca2þ! The physiology and pathology of Ca2þ-binding proteins in the retina. TINS, 19: 547–554. Pow, D.V. and Hendrickson, A.E. (2000) Expression of glycine and the glycine transporter Glyt-1 in the developing rat retina. Vis. Neurosci., 17: 1–9. Ramoa, A.S., Campbell, G. and Shatz, C.J. (1987) Transient morphological features of identified ganglion cells in living fetal and neonatal retina. Science, 237: 522–525. Ramoa, A.S., Campbell, G. and Shatz, C.J. (1988) Dendritic growth and remodeling of cat retinal ganglion cells during fetal and postnatal development. J. Neurosci., 8: 4239–4261. Rapaport, D.H. (1989) Quantitative aspects of synaptic ribbon formation in the outer plexiform layer of the developing cat retina. Vis. Neurosci., 3: 21–32. Reese, B.E. and Cowey, A. (1990a) Fibre organization of the monkey’s optic tract: I. Segregation of functionally distinct optic axons. J. Comp. Neurol., 295: 385–400. Reese, B.E. and Cowey, A. (1990b) Fibre organization of the monkey’s optic tract: II. Noncongruent representation of the two half-retinae. J. Comp. Neurol., 295: 401–412. Reese, B.E., Thompson, W.F. and Peduzzi, J.D. (1994) Birthdates of neurons in the retinal ganglion cell layer of the ferret. J. Comp. Neurol., 341: 464–475. Reese, B.E., Johnson, P.T. and Baker, G.E. (1996) Maturational gradients in the retina of the ferret. J. Comp. Neurol., 375: 252–273. Reese, B.E., Raven, M.A., Giannotti, K.A. and Johnson, P.T. (2001) Development of cholinergic amacrine cell stratification in the ferret retina and the effects of early excitotoxic ablation. Vis. Neurosci., 18: 559–570. Reichenbach, A. and Robinson, S.R. (1995) Phylogenetic constraints on retinal organization and development. Prog. Ret. Eye Res., 15: 139–171. Reye, P., Sullivan, R. and Pow, D.V. (2002) Distribution of two splice variants of the glutamate transporter GLT-1 in the developing rat retina. J. Comp. Neurol., 447: 323–330. Rickman, D.W. and Brecha, N.C. (1995) Expression of the proto-oncogene, trk, receptors in the developing rat retina. Vis. Neurosci., 12: 215–222. Robinson, S.R. and Dreher, Z. (1990) Mu¨ller cells in adult rabbit retinae: morphology, distribution and implications for function and development. J. Comp. Neurol., 292: 178–192. Robinson, S.R., Rapaport, D.H. and Stone, J. (1985) Cell division in the developing cat retina occurs in two zones. Dev. Brain Res., 19: 101–109. Rohrer, B., Korenbrot, J.I., LaVail, M.M., Reichardt, L.F. and Xu, B. (1999) Role of neurotrophin receptor TrkB in the maturation of rod photoreceptors and establishment of synaptic transmission to the inner retina. J. Neurosci., 19: 8919–8930. Saha, M.S. and Grainger, R.M. (1993) Early opsin expression in Xenopus embryos precedes photoreceptor differentiation. Mol. Brain Res., 17: 307–318.

Sherry, D.M., St. Jules, R.S. and Townes-Anderson, E. (1996) Morphologic and neurochemical target selectivity of regenerating adult photoreceptors in vitro. J. Comp. Neurol., 376: 476–488. Singer, J.H., Mirotznik, R.R. and Feller, M.B. (2001) Potentiation of L-type calcium channels reveals nonsynaptic mechanisms that correlate spontaneous activity in the developing mammalian retina. J. Neurosci., 21: 8514–8522. Spoerri, P.E., Ulshafer, R.J., Ludwig, H.C., Allen, C.B. and Kelley, K.C. (1988) Photoreceptor cell development in vitro: influence of pigment epithelium conditioned medium on outer segment differentiation. Eur. J. Cell Biol., 46: 362–367. Stellwagen, D. and Shatz, C.J. (2002) An instructive role for retinal waves in the development of retinogeniculate connectivity. Neuron, 33: 357–367. Stiemke, M.M., Landers, R.A., Al-Ubaidi, M.R., Rayborn, M.E. and Hollyfield, J.G. (1994) Photoreceptor outer segment development in Xenopus laevis: influence of the pigment epithelium. Dev. Biol., 162: 169–180. Sudhof, T.C., Lottspeich, F., Greengard, P., Mehl, E. and Jahn, R. (1987) A synaptic vesicle protein with a novel cytoplasmic domain and four transmembrane regions. Science, 238: 1142–1144. Sze´l, A´., van Veen, T. and Ro¨hlich, P. (1994) Retinal cone differentiation. Nature, 370: 336. Timmers, A.M., Newton, B.R. and Hauswirth, W.W. (1993) Synthesis and stability of retinal photoreceptor mRNAs are coordinately regulated during bovine fetal development. Exp. Eye Res., 56: 257–265. Treisman, J.E., Morabito, M.A. and Barnstable, C.J. (1988) Opsin expression in the rat retina is developmentally regulated by transcriptional activation. Mol. Cell Biol., 8: 1570–1579. Tucker, G.S., Hamasaki, D.I., Labbie, A. and Muroff, J. (1979) Anatomic and physiologic development of the photoreceptor of the kitten. Exp. Brain Res., 37: 459–474. Ugolini, G., Cremisi, F. and Maffei, L. (1995) TrkA, trkB and p75NTR mRNA expression is developmentally regulated in rat retina. Brain Res., 704: 121–124. Unoki, K. and LaVail, M.M. (1994) Protection of the rat retina from ischemic injury by brain-derived neurotrophic factor, ciliary neurotrophic factor, and basic fibroblast growth factor. Invest. Ophthalmol. Vis. Sci., 35: 907–915. Usukura, J. and Bok, D. (1987) Changes in the localization and content of opsin during retinal development in the rds mutant mouse: immunocytochemistry and immunoassay. Exp. Eye Res., 45: 501–515. Usukura, J. and Obata, S. (1995) Morphogenesis of photoreceptor outer segments in retinal development. Prog. Ret. Eye Res., 15: 113–125. van Ginkel, P.R. and Hauswirth, W.W. (1994) Parallel regulation of fetal gene expression in different photoreceptor cell types. J. Biol. Chem., 269: 4986–4992.

19 Vogel, M. (1978) Postnatal development of the cat’s retina: a concept of maturation obtained by qualitative and quantitative examinations. Graefes Arch. Ophthal., 208: 93–107. Voigt, T., De Lima, A.D. and Beckmann, M. (1993) Synaptophysin immunohistochemistry reveals inside-out pattern of early synaptogenesis in ferret cerebral cortex. J. Comp. Neurol., 330: 48–64. Vollrath, L. and Spiwoks-Becker, I. (1996) Plasticity of retinal ribbon synapses. Microsc. Res. Tech., 35: 472–487. von Bartheld, C.S. (1998) Neurotrophins in the developing and regenerating visual system. Histol. Histopathol., 13: 437–459. Wahlin, K.J., Campochiaro, P.A., Zack, D.J. and Adler, R. (2000) Neurotrophic factors cause activation of intracellular signaling pathways in Mu¨ller cells and other cells of the inner retina, but not photoreceptors. Invest. Ophthalmol. Vis. Sci., 41: 927–936. Wahlin, K.J., Adler, R., Zack, D.J. and Campochiaro, P.A. (2001) Neurotrophic signaling in normal and degenerating rodent retinas. Exp. Eye Res., 73: 693–701. Wang, G.Y., Liets, L.C. and Chalupa, L.M. (2001) Unique functional properties of on and off pathways in the developing mammalian retina. J. Neurosci., 15: 4310–4317. Watanabe, T. and Raff, M.C. (1990) Rod photoreceptor development in vitro: intrinsic properties of proliferating neuroepithelial cells change as development proceeds in the rat retina. Neuron, 2: 461–467. Weidman, T.A. and Kuwabara, T. (1968) Postnatal development of the rat retina. An electron microscopic study. Arch. Ophthalmol., 79: 470–484. West Greenlee, M.H., Swanson, J.J., Simon, J.J., Elmquist, J.K., Jacobson, C.D. and Sakaguchi, D.S. (1996) Postnatal development and the differential expression of presynaptic terminal-associated proteins in the developing retina of the Brazilian opossum, Monodelphis domestica. Dev. Brain Res., 96: 159–172. West Greenlee, M.H., Roosevelt, C.B. and Sakaguchi, D.S. (2001) Differential localization of SNARE complex proteins SNAP-25, syntaxin, and vamp during development of the mammalian retina. J. Comp. Neurol., 430: 306–320. Wiedenmann, B. and Franke, W.W. (1985) Identification and localization of synaptophysin, an integral membrane glycoprotein of Mr 38,000 characteristic of presynaptic vesicles. Cell, 41: 1017–1028. Williams, R.R., Cusato, K., Raven, M. and Reese, B.E. (2001) Organization of the inner retina following early elimination

of the retinal ganglion cell population: effects on cell numbers and stratification patterns. Vis. Neurosci., 18: 233–244. Wingate, R.J. and Thompson, I.D. (1994) Targeting and activity-related dendritic modification in mammalian retinal ganglion cells. J. Neurosci., 11: 6621–6637. Wingate, R.J. and Thompson, I.D. (1995) Axonal target choice and dendritic development of ferret beta retinal ganglion cells. Eur. J. Neurosci., 7: 723–731. Wong, R.O.L. (1995) Cholinergic regulation of [Ca2þ]i during cell division and differentiation in the mammalian retina. J. Neurosci., 15: 2696–2706. Wong, R.O.L. (1999) Retinal waves and visual system development. Ann. Rev. Neurosci., 22: 29–47. Wong, R.O.L., Meister, M. and Shatz, C.J. (1993) Transient period of correlated bursting activity during development of the mammalian retina. Neuron, 11: 923–938. Wong, R.O.L., Chernjavsky, A., Smith, S.J. and Shatz, C.J. (1995) Early functional neural networks in the developing retina. Nature, 374: 716–718. Wong, R.O.L. and Oakley, D.M. (1996) Changing patterns of spontaneous bursting activity of on and off retinal ganglion cells during development. Neuron, 16:1087–1095. Wong, W.T., Myhr, K.L., Miller, E.D. and Wong, R.O.L. (2000) Developmental changes in the neurotransmitter regulation of correlated spontaneous retinal activity. J. Neurosci., 20: 1–10. Yan, Q., Wang, J., Matheson, C.R. and Urich, J.L. (1999) Glial cell-line derived neurotrophic factor (GDNF) promotes the survival of axotomized retinal ganglion cells in adult rats: comparison to and combination with brain-derived neurotrophic factor. J. Neurobiol., 15: 382–390. Young, R.W. (1985) Cell differentiation in the retina of the mouse. Anat. Rec., 212: 199–205. Zanellato, A., Comelli, M.C., Dal-Toso, R. and Carmignoto, G. (1993) Developing rat retinal ganglion cells express the functional NGF receptor p140trkA. Dev. Biol., 159: 105–113. Zhou, Z.J. (1998) Direct participation of starburst amacrine cells in spontaneous rhythmic activities in the developing mammalian retina. J. Neurosci., 18: 4155–4165. Zhou, Z.J. and Zhao, D. (2000) Coordinated transitions in neurotransmitter systems for the initiation and propagation of spontaneous retinal waves. J. Neurosci., 20: 6570–6577. Zimmerman, R.P., Polley, E.H. and Fortney, R.L. (1988) Cell birthdays and rate of differentiation of ganglion and horizontal cells of the developing cat’s retina. J. Comp. Neurol., 274: 77–90.

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 2

Morphology and physiology of primate M- and P-cells Luiz Carlos L. Silveira1,*, Ce´zar A. Saito1, Barry B. Lee2, Jan Kremers3, Manoel da Silva Filho1, Bjørg E. Kilavik3, Elizabeth S. Yamada1 and V. Hugh Perry4 1

Department of Physiology, Biological Science Center, Federal University of Para´, 66075-900 Bele´m, Para´, Brazil 2 SUNY Optometry, New York, NY 10036, USA and Max-Planck Institute for Biophysical Chemistry, Department of Neurobiology, D-3400, Go¨ttingen, Germany 3 Department Experimental Ophthalmology, University of Tu¨bingen, D-72076, Tu¨bingen, Germany 4 CNS Inflammation Group, University of Southampton, SO16 7PX Southampton, UK

Abstract: Catarrhines and platyrrhines, the so-called Old- and New-World anthropoids, have different cone photopigments. Postreceptoral mechanisms must have coevolved with the receptors to provide trichromatic color vision, and so it is important to compare postreceptoral processes in these two primate groups, both from anatomical and physiological perspectives. The morphology of ganglion cells has been studied in the retina of catarrhines such as the diurnal and trichromatic Macaca, as well as platyrrhines such as the diurnal, di- or trichromatic Cebus, and the nocturnal, monochromatic Aotus. Diurnal platyrrhines, both di- and trichromats, have ganglion cell classes very similar to those found in catarrhines: M (parasol), P (midget), small-field bistratified, and several classes of wide-field ganglion cells. In the fovea of all diurnal anthropoids, P-cell dendritic trees contact single midget bipolars, which contact single cones. The Aotus retina has far fewer cones than diurnal species, but M- and P-cells are similar to those in diurnal primates although of larger size. As in diurnal anthropoids, in the Aotus, the majority of midget bipolar cells, found in the central 2 mm of eccentricity, receive input from a single cone and the sizes of their axon terminals match the sizes of P-cell dendritic fields in the same region. The visual responses of retinal ganglion cells of these species have been studied using single-unit electrophysiological recordings. Recordings from retinal ganglion cells in Cebus and Aotus showed that they have very similar properties as those in the macaque, except that P-cells of mono- and dichromatic animals lack cone opponency. Whatever the original role of the M- and P-cells was, they are likely to have evolved prior to the divergence of catarrhines and platyrrhines. M- and P-cell systems thus appear to be strongly conserved in the various primate species. The reasons for this may lie in the roles of these systems for both achromatic and chromatic vision.

Vision and visual encoding

concerned with communication between individuals; a blind person is cut off from the world of things, whereas a deaf person is cut off from the world of people (Evans, 1982). This may be a partisan approach to sensory physiology, but to consider the visual system as a device built for localization and identification might be a good way to start a discussion of how the structure and function of the visual pathways serve the purpose of vision. Object reflectance modifies the spectral distribution of light generated from natural or artificial

To specify the difference between vision and hearing in humans, it is commonly said that the sense of vision is primarily devoted to object localization and identification, whereas the sense of hearing is *Corresponding author. Universidade Federal do Para´, Centro de Cieˆncias Biolo´gicas, Departamento de Fisiologia, 66075-900 Bele´m, Para´, Brazil. Tel.: þ 5591-99834133; Fax: þ 5591-2111570; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14400-2

21

22

sources to provide the stimulus configurations which reach the eye from different locations in the visual field. Thus, two stimulus properties available for use by the visual system for object localization and identification are the amplitude and frequency constituting such spectral distributions, or number of photons and photon energy. As stated in the Rushton Principle of Univariance, photoreceptors only ‘measure’ the number of photoisomerizations occurring in a certain spatiotemporal window and cannot distinguish between photons of different energy once they are absorbed. Photons of a certain energy are absorbed with the highest probability, and this peak of the absorption spectrum depends on the microelectric forces provided by the opsin protein that surrounds the retinaldehyde. A few aminoacids among those that constitute the opsin structure are critical in modifying the retinaldehyde’s environment, and have been the target for evolutionary changes leading to photopigment diversity. Thus, visual information is encoded at the photoreceptor level following simple physical principles. The electrical activity of the photoreceptor mosaic is directly related to photon catch; the photoreceptor density limits spatial resolution; and the time course of the photoreceptor potential sets the temporal resolution of this representation. What happens further down in the visual pathway? How is the visual world represented at every postreceptoral level, and what are the neural correlates of the physical stimulus properties? The ganglion cells are only two synapses away from phototransduction events and represent the link between retinal circuitry and higher-level brain processes. Thus, the retinal ganglion cell layer is a convenient place to see how postreceptoral neurons first deal with photoreceptor output to construct a neural representation of the visual world, which then is sent to the visual centers in the midbrain, thalamus, and cerebral cortex. Investigation of ganglion cell physiology has provided two ways to approach the problem. One approach, mainly performed in rabbits and lower vertebrates, has postulated the existence of trigger features, special stimulus configurations, that would drive specific ganglion cell classes (Barlow, 1961). Early feature extraction implies that a high level of information processing is already attained at the ganglion cell

level, probably involving some form of nonlinear transformation of photoreceptor signals. Another approach describes ganglion cell responses in terms of spatial and temporal properties directly related to fundamental physical parameters of the stimulus (Ku¨ffler, 1953; Enroth-Cugell and Robson, 1966). It assumes relatively simple rules for the translation of intensity parameters of the optical image into the phototransduction cascade and subsequent neuronal activity, mostly by means of linear operations. This approach has been very useful to characterize ganglion cell physiology in mammals, including primates. Primate ganglion cells have been classified and their functional roles in vision have been discussed using the abovementioned approach. In addition, a well-documented correlation between physiology and morphology has been disclosed for the most common ganglion cell classes, the M- and P-cells (Shapley and Perry, 1986; Dacey and Lee, 1994). The physiological properties of the M- and P-relay neurons of the lateral geniculate nucleus (LGN) are generally similar to those of M- and P-retinal ganglion cells, although subtle differences have been found with more detailed comparisons between neurons of these two stations of the visual pathway (e.g. Kaplan et al., 1993). In this chapter we shall focus on results obtained from ganglion cells, but the reader may find elsewhere detailed reviews on the physiology of M- and P-pathways at more central levels in the thalamus and primary visual cortex (e.g. Shapley and Hawken, 1999). This chapter presents an overview on the morphology and physiology of two major classes of primate retinal ganglion cells: the M- and P-cells. Particular emphasis will be given to results obtained from New-World primates which may be diurnal or nocturnal and have different color vision phenotypes. A comparison between the ganglion cells of OldWorld and different New-World species may shed light on M- and P-pathway function and evolution. In addition, details of M- and P-cell anatomy and physiology studied at different eccentricities are presented and differences in morphology, response characteristics in the space and time domains, and the relative contributions of rod and cone signals to their responses, are discussed and related to the diurnal activity rhythm, as well as to color vision phenotype.

23

Primate ganglion cell classes Santiago Ramo´n y Cajal’s extensive description of retinal anatomy includes little about primate ganglion cell morphology (Ramo´n y Cajal, 1904). Until the 1980s, only a few selected studies had been performed on this subject. Dogiel (1891) used the method of Ehrlich to stain human retinal flat-mounts. He described three ganglion cell classes and illustrated their morphology with drawings of stained retinal patches, showing cells that look very similar to peripheral M and P cells. The first detailed descriptions of primate ganglion cells are from the work of Polyak (1941) and Boycott and Dowling (1969) who studied retinal sections stained with the Golgi method. These authors recognized several ganglion cell classes in the retina of macaques and other nonhuman primates and showed that their morphology changes with distance from fovea. Polyak coined the terms parasol and midget that are used today for the two most frequently found cell classes. Later these cells were, respectively called Pa- and Pb-cells (Perry and Cowey, 1981) or A- and B-cells (Leventhal et al., 1981). Parasol cells are now usually termed M- (or MC-) cells and midget are now called P- (or PC-) cells (Shapley and Perry, 1986). A considerable advance in the study of primate ganglion cell morphology was achieved by combining the use of retinal flat-mount preparations with modern methods of extra and intracellular injection of neurotracers. Alan Cowey and Hugh Perry, working at the University of Oxford, were among the first to employ such techniques. They labeled macaque ganglion cells by horseradish peroxidase (HRP) retrograde transport from the optic nerve, lateral geniculate nucleus and superior colliculus, and published a series of quantitative accounts on M- and P-cell morphology (Perry and Cowey, 1981, 1984; Perry et al., 1984; Perry and Silveira, 1988). When the tracer was placed in the optic nerve, behind the eyeball, the staining quality they achieved was ‘Golgi-like’, making it easy to classify the labeled cells and to measure dendritic-field and cell-body sizes at all retinal locations accurately. Moreover, the use of flattened preparations of entire retinas, made it simple to measure the distance of labeled cells from the fovea. Using these procedures, they quantified

how the sizes of dendritic fields and cell bodies of macaque M- and P-cells vary as a function of retinal eccentricity. Finally, using retrograde labeling from retinal targets, they showed that M- and P-cells project to the magno- and parvocellular layers of the lateral geniculate nucleus, respectively, confirming the results of Leventhal and colleagues (Leventhal et al., 1981). Since 1981, different research groups have used retrograde transport of HRP or Biocytin, as well as intracellular injection of Lucifer Yellow, HRP or Neurobiotin, to label ganglion cells in retinal flat-mounts of several primate species. M- and P-cells have been identified in all primates studied so far, including human (Rodieck et al., 1985; Kolb et al., 1992; Dacey and Petersen, 1992), other diurnal catarrhines (Leventhal et al., 1981; Perry and Cowey, 1981; Watanabe and Rodieck, 1989), diurnal platyrrhines (Leventhal et al., 1989; Silveira et al., 1994; Ghosh et al., 1996; Yamada et al., 1996a,b), nocturnal platyrrhines (Silveira et al., 1994; Yamada et al., 1996b, 2001), and prosimians (Yamada et al., 1998). In all primates, M-cells have large cell bodies, thick axons, and large dendritic trees with a radial branching pattern, whereas P-cells have small cell bodies, thin axons, and small dendritic trees with a more bushy and dense branching pattern (Figs. 1–3). As in other mammalian ganglion cell classes, such as cat a- and b-cells, M- and P-cells are divided into two subclasses according to the level of dendritic branching in the inner plexiform layer. The cells of one subclass have dendrites ramifying in the outer half of the inner plexiform layer, and are called outer Mor P-cells, whereas the cells of the other subclass have dendrites in the inner half of the inner plexiform layer, being called inner M- or P-cells. The outer and inner subclasses of M- and P-cells correspond to the electrophysiologically off-center and on-center varieties (Dacey and Lee, 1994). It is of special interest to investigate whether M- and P-cell morphology is the same in primates with different life styles. The primates comprise two Suborders: Anthropoidea and Prosimii (Fleagle, 1988). The anthropoids are divided in two Infraorders, Catarrhini and Platyrrhini, inhabiting in the Old- and New-World, respectively. All the 22 genera of living Old World anthropoids are diurnal and their color vision is trichromatic with little variation among

24

Fig. 1. M- and P-cells from the central retina of Cebus and Aotus (Silveira et al., 1994; Yamada et al., 1996a, 2001). A-B. Cebus M-on cells. C. Aotus M-off cell. D-E. Cebus P-on cells. F. Aotus P-off cell. Ganglion cells were retrogradely labeled by placing Biocytin in the optic nerve, 1–3 mm behind the eyeball. After 18–48 h, the animal was euthanized with a lethal dose of barbiturate and perfused with paraformaldehyde. After perfusion, the eye was removed, the retina dissected and incubated in ABC Vectastain for 12–48 h, and then reacted for peroxidase histochemistry using diaminobenzidine as chromogen. Drawings were made using a drawing tube attached to a binocular microscope. Cebus M- and P-cells are similar to those observed in other diurnal anthropoids, M-cells being larger than P-cells at all retinal locations. Aotus M- and P-cells are larger than their Cebus counterparts at similar eccentricities, but M-cells are still larger than P-cells at all eccentricities. The figure illustrates that foveal Aotus M- and P-cells are slightly larger or about the same size as Cebus M- and P-cells located about 0.5 mm more peripherally. Both in Cebus and Aotus, the central M- and P-cells are very small and the central P-cell dendritic fields have the appropriate size to contact axon terminals of single midget bipolar cells, whose dendrites make contacts with single cones. Scale bar ¼ 50 mm.

individuals and species. In humans and other catarrhines, dichromacy or anomalous trichromacy are considered abnormal phenotypes. Thus, it is not surprising that retinal organization has been shown to be very similar in all catarrhines so far studied, with only minor differences between species. The living New-World anthropoids differ from catarrhines in that they comprise diurnal and

Fig. 2. P-cells from the peripheral retina of Cebus and Aotus (Silveira et al., 1994; Yamada et al., 1996a, 2001). A. Cebus P-on cell. B. Aotus P-off cell. In the Cebus and Aotus, similarly to other diurnal anthropoids, with increasing eccentricity, P-cells increase in size but maintain its distinctive morphology. In all anthropoids so far studied, P-cells have small to medium sized cell bodies, thin axons, and small dendritic trees bearing bushy and dense branching pattern. Note, in this figure, that the two cells have about the same dendritic-field size, but the Cebus P-cell is located 2.3 mm more peripherally than the Aotus P-cell. Scale bar ¼ 50 mm

nocturnal species and a variety of color vision phenotypes (Jacobs, 1998) (Table 1). They represent useful ‘animal models’ to test hypotheses about the organization of primate visual pathways. Amongst the platyrrhines, there are 17 diurnal genera and one nocturnal genus, Aotus. Moreover, most platyrrhine species contain a mixed population of di- and trichromatic individuals (Mollon et al., 1984). This is due to the presence of just a single gene in the X-chromosome coding for photopigments sensitive to middle or long wavelengths (MWS and LWS photopigments, respectively). As a consequence, all males are dichromats having the SWS- (short wavelength sensitive) cone, the photopigment of

25 Table 1. A summary table of the New-World anthropoids. The division in Families and genera follow the recent review by Rylands et al. (2000)

Fig. 3. M-cells from the peripheral retina of Cebus and Aotus (Silveira et al., 1994; Yamada et al., 1996a, 2001). A. Cebus M-off cell. B. Aotus M-on cell. In the Cebus and Aotus, similarly to other diurnal anthropoids, with increasing eccentricity, Mcells increase in size but maintain its distinctive morphology. In all anthropoids so far studied, M-cells have large cell bodies, thick axons, and large dendritic trees with a radial branching pattern. Aotus M- and P-cells are larger than Cebus M- and Pcells, respectively, at all retinal locations. Note, in this figure, that the Aotus M-cell is still larger than the Cebus M-cell in spite of being located 0.7 mm more centrally. Scale bar ¼ 50 mm.

which is encoded on chromosome 7, together with a single MWS/LWS-cone. Due to a polymorphism of the MWS/LWS-photopigment gene, there are three or more dichromatic phenotypes amongst males. The monozygotic females are dichromats whereas heterozygotic females are trichromats, the exact proportion of dichromatic and trichromatic females depends on the number and frequency of alleles of the MWS/LWS-photopigment genes. And again, due to gene polymorphism, there are several di- and trichromatic phenotypes amongst females.

Family Callitrichidae

Species Life style

Colour vision

M- and P-cell studies

Cebuella Mico Callithrix Saguinus Leonthopithecus Callimico

1 14 6 15 4

Diurnal Diurnal Diurnal Diurnal Diurnal

– – Polymorphic Polymorphic Polymorphic

– – 5–6, 8 – –

1

Diurnal

–

–

Family Cebidae Saimiri Cebus

5 7

Diurnal Diurnal

Polymorphic Polymorphic

1 2–4, 7–8, 11–12

Family Aotidae Aotus

8

Nocturnal Monochromatic

2–3, 8–9, 13–14

19 5 2 2

Diurnal Diurnal Diurnal Diurnal

Polymorphic – – –

– – – –

8 6 4 1 2

Diurnal Diurnal Diurnal Diurnal Diurnal

Trichromatic Polymorphic – – –

– 10 – – –

Family Pitheciidae Callicebus Pithecia Chiropotes Cacajao Family Atelidae Alouatta Ateles Lagothrix Oreonax Brachyteles

Polymorphic colour vision refers to a normal mixed population of dichromatic and trichromatic individuals (see text for explanation). Morphological studies: 1Leventhal et al. (1989), 2Lima et al. (1996), 3–4 Silveira et al. (1994, 1998), 5Ghosh et al. (1996), 6Goodchild et al. (1996), 7–9Yamada et al. (1996a, b, 2001). Electrophysiological studies: 10 Hubel and Wiesel (1960), 11–12Lee et al. (1996, 2000), 13Silveira et al. (2000), 14Saito et al. (2001).

As far as our present knowledge goes, there are at least two exceptions to this standard platyrrhine scheme of color vision (Jacobs, 1998). The Aotus is a monochromat, having a single MWS/LWSphotopigment gene on the X-chromosome and a single allele for this gene. In addition, the SWS-photopigment gene on chromosome 7 is

26

nonfunctional and there are no SWS-cones in the retina. On the other hand, Alouatta exhibits ‘routine’ trichromacy, having two different MWS/LWSphotopigment genes on the X-chromosome besides the SWS-photopigment gene on chromosome 7, all genes bearing a single allele. We have used two platyrrhine genera to investigate several comparative aspects of retinal organization: the diurnal capuchin-monkey, Cebus, which displays the standard mixture of di- and trichromats (Jacobs and Neitz, 1987); and the nocturnal and monochromatic owl-monkey, Aotus (Wikler and Rakic, 1990; Jacobs et al., 1993, 1996). These two New-World monkeys have similar eye size and retinal area, facilitating direct comparisons between retinal locations both in linear and angular metrics. Ganglion cells were retrogradely labeled by placing Biocytin in the optic nerve, behind the eyeball. Cell morphology was subsequently revealed by incubating the retina in ABC Vectastain followed by peroxidase histochemistry using diaminobenzidine as chromogen (Silveira et al., 1994). The morphology of M- and P-cells in the dichromatic male Cebus are qualitatively and quantitatively similar to those of trichromatic platyrrhines and catarrhines such as marmoset and macaque monkey (Silveira et al., 1994; Yamada et al., 1996a). Both cell classes increase in size with increasing distance from the foveal slope, conserving their distinct branching pattern at all eccentricities (Figs. 1–3). The mean M-cell dendritic-field size is about the same in the temporal, dorsal, and ventral retinal quadrants, increasing from 28 mm at 0.5 mm to 308 mm at 10–12 mm distance from the fovea, whereas in the nasal quadrant it is smaller, increasing from 25 mm at 0.5 mm to 216 mm at 10–12 mm distance from the fovea (Figs. 4 and 5). The mean P-cell dendritic-field diameter depends similarly on eccentricity, but is smaller than mean M-cell dendritic-field throughout the retina, ranging from 8 mm at 0.5 mm to 100 mm at 10–12 mm distance from the fovea in the temporal, dorsal, and ventral quadrants, and from 7 mm at 0.5 mm to 62 mm at 10–12 mm distance from the fovea in the nasal quadrant (Figs. 4 and 5). In Aotus, the fovea is absent or rudimentary, and the cone-to-rod ratio is much smaller than in Cebus and other diurnal platyrrhines and catarrhines

Fig. 4. The size of M- and P-cell dendritic fields of Cebus and Aotus as a function of retinal eccentricity (Yamada et al., 1996a,b, 2001). A. Temporal cells. B. Nasal cells. Dendritic field size was measured in drawings of selected M- and P-cells. Dendritic field was defined as the convex polygon circumscribing the tips of the distal dendrites. Cell eccentricity was corrected for shrinkage using the fovea–optic disk distance as reference. Measurements of dendritic field area were performed using a bit pad connected to a microcomputer. The results were converted to diameter of circles with equivalent area and plotted as a function of eccentricity. The Aotus M- and P-cells are larger than their Cebus counterparts at similar eccentricities. In the Cebus and Aotus, M-cell dendritic-field sizes are larger than those of P-cells at any given eccentricity along both temporal and nasal retinal regions. The difference between M and P dendritic-field sizes ranges from 2.5 to 3.5-fold and 2.3 to 2.7fold, for the Cebus and Aotus retina, respectively. In the Cebus retina, dendritic-field diameter increases less steeply along the nasal retina, so that at comparable distance from fovea, nasal M- and P-cells are smaller than temporal ones, more significantly for retinal eccentricities greater than 2 mm. In the Aotus retina, the nasotemporal asymmetry of M and P dendriticfields is less pronounced than that observed in the Cebus retina, and it attains a significant level only in the retinal periphery, at 10 mm from the fovea. Figure reproduced from Yamada et al. (2001) with the kind permission of Elsevier Science.

27

Fig. 5. Dendritic-field size of central M- and P-cells of Cebus and Aotus as a function of retinal eccentricity (Yamada et al., 1996a,b, 2001). A. Temporal cells. B. Nasal cells. Symbols as in Fig. 4: filled squares, Aotus M-cells; empty diamonds, Cebus M-cells; filled circles, Aotus P-cells; empty triangles, Cebus P-cells. As in other diurnal anthropoids, the dendritic fields of Cebus P-cells do not increase in size up to 1.75 mm and 1.25 mm from fovea in the nasal and temporal regions, respectively. On the other hand, Aotus P-cells, Cebus M-cells, and Aotus M-cells increase in size steadily with increasing distance from fovea. Aotus M-cells show the steepest change.

(Silveira et al., 1993, 2001a). However, the M- and P-cell morphology is, in most aspects, qualitatively similar to that found in diurnal anthropoids (Figs. 1–3). The main qualitative difference is that Aotus cells have thicker dendrites and lower dendritic-branching density than Cebus cells. This is better illustrated when Aotus and Cebus cells are matched for size (Figs. 1–3). Notwithstanding the qualitative similarities, there are important

quantitative differences. At similar eccentricities, Aotus M- and P-cells are larger than Cebus M- and P-cells (Figs. 4 and 5). In the central retinal region of Aotus, the mean M-cell dendritic-field diameter measures 36 mm, increasing at 10–12 mm of eccentricity to about 317 mm in the nasal and to about 381 mm in the other quadrants, respectively, whereas the mean P-cell dendritic-field diameter ranges from 14 mm in the central region to 131 mm in the nasal and 177 mm in the other quadrants at 10–12 mm of eccentricity (Yamada et al., 2001; see Figs. 4 and 5). In the central retina, the dendritic-field areas of M- and P-cells, measured in Aotus are 3.9 and 5.9 times larger respectively than those of Cebus. This ratio decreases towards the retinal periphery to 1.9 and 3.5 for M- and P-cells, respectively. The size difference between Aotus and Cebus M- and P-cells is related to the relative density of cones and rods in the retina of these two platyrrhine species (Silveira et al., 1994; Yamada et al., 2001). Figure 6 shows the cone and rod convergence to M- and P-cells as a function of retinal eccentricity. Although there are large differences in the dendritic field area between Aotus and Cebus, the cone convergence to M- and P-cells is of the same magnitude. Similar cone convergences were found for other platyrrhines, such as Callithrix, and for catarrhines, such as humans and macaques (Goodchild et al., 1996). This finding indicates that during development, ganglion cells from different primate species adjust their dendritic size to collect signals from similar numbers of cones. Consistent with this hypothesis is the fact that ganglion cells and cones are generated during the first phase of retinal neurogenesis, while rods appear at a later stage (La Vail et al., 1991; Yamada et al., 2001). An important characteristic of the P-pathway is the existence of very small P-cells, that are connected to single-cone midget bipolar cells in the central 2 mm around the fovea (Polyak, 1941; Boycott and Dowling, 1969; Kolb and DeKorver, 1991). This one-to-one neuronal circuitry is thought to form the basis of red–green color opponency. A single MWS- or LWS-cone provides the signal for the P-cell receptive-field center, whereas the surround is either driven by signals selectively coming from the other cone class (Lee et al., 1998), or by a mixture of

28

Fig. 6. Photoreceptor convergence to M- and P-cells as a function of retinal eccentricity in the Cebus and Aotus retina (Yamada et al., 2001). A–B. Cone convergence to temporal and nasal cells, respectively. C–D. Rod convergence to temporal and nasal cells, respectively. The number of cone and rod per ganglion cell was obtained by multiplying the ganglion-cell dendritic-field area by photoreceptor density (Yamada et al., 2001). Cone convergence to M- and P-cells is similar for the Cebus and Aotus. Cebus M-cells: 20 cones/cell at 1 mm from fovea, 500 cones/cell at the retinal periphery. Aotus M-cells: 17 cones/cell in the fovea, 40–50 cones/cell at 1 mm from fovea, 340–420 cones/cell at the retinal periphery. Cebus P-cells: 1–2 cones/cell at 1 mm from fovea, 60 cones/cell at the retinal periphery. Aotus P-cells: 2.5 cones/cell in the fovea, 2.5–3 cones/cell at 1 mm from fovea, 60–110 cones/cell at the retinal periphery. Rod convergence to M- and P-cells is higher in Aotus. Aotus M-cells: 410 rods/cell in the fovea, 1,600–2,100 rods/cell at 1 mm from fovea, 13,900–16,200 rods/cell at the retinal periphery. Cebus M-cells: 70–80 rods/cell at 1 mm from fovea, 4,700–6,500 rods/cell at the retinal periphery. Aotus P-cells: 60 rods/cell in the fovea, 110–130 rods/cell at 1 mm from fovea to 2,400–4,400 rods/cell at the retinal periphery. Cebus P-cells: 4–7 rods/cell at 1 mm from the fovea to 570–880 rods/cells at the retinal periphery. Figure reproduced from Yamada et al. (2001) with the kind permission of Elsevier Science.

MWS- and LWS-cones. However, recent physiological evidence suggests a more complex physiological reality than this simple anatomical picture would suggest (McMahon et al., 2000). To test whether similar pathways are present in dichromats and monochromats, Cebus and Aotus

bipolar cells were stained by placing the lipophilic carbocianine dye DiI in fixed retinas (Silveira et al., 1998, 2001b). As in trichromatic catarrhines, in the dichromatic male Cebus and the monochromatic Aotus, the majority of midget bipolar cells, found in the central 2 mm of eccentricity, receive input from

29

a single cone and the sizes of their axon terminals match the sizes of P-cell dendritic-fields in the same region. These findings support the view that, similar to catarrhines (all being diurnals and trichromats), central P-cells of platyrrhines receive input from single midget bipolar cells, which in turn, receive input from single MWS/LWS-cones irrespective of whether they are diurnal, nocturnal, mono-, di-, or trichromatic. The results are consistent with the idea that a P-pathway with one-to-one connectivity was present in the anthropoid ancestor before the divergence between catarrhines and platyrrhines (Mollon and Jordan, 1988). It will be interesting to ascertain the presence of M- and P-cells in prosimians. Recently, a genetic investigation on 20 species, representing the major prosimian lineages, indicated that several forms of color vision might be found among them (Tan and Li, 1999). Some species are monochromatic, others dichromatic, and some others are potentially trichromatic, similar to the polymorphic color vision of platyrrhines. Little is known about retinal ganglion cell morphology and physiology of prosimians. The greater bushbaby Otolemur, a nocturnal and monochromatic prosimian, which has an MWS/LWS-cone and no SWS-cones, also has M- and P-ganglion cells (Yamada et al., 1998). The central P-cells receive the signals from about five cones, resulting in a cone to P-cell convergence that is higher than that found in diurnal and nocturnal anthropoids but still lower than in central cat b-cells, which receive 30 cones per cell. While little is known of the M- and P-pathways in prosimians, an analysis of their axon diameters across the depth of the optic tract has shown that Otolemur contains an organization identical to that found in humans and other anthropoids. Specifically, the deeper parts of the optic tract contain purely medium-caliber axons, while the more superficial parts of the tract contain both fine and coarse caliber axons (Reese, 1996). While this sheds little light on the morphology of their ganglion cells, it would appear strongly supportive of the view that the fundamental categories of ganglion cell classes (defined anatomically and embryologically, rather than physiologically) are conserved across primates.

The correlation between morphology and physiology Physiological studies had demonstrated that the primate retina had a variety of functional classes of ganglion cells (Hubel and Wiesel, 1960; Gouras, 1968; de Monasterio and Gouras, 1975; de Monasterio et al., 1975a,b; de Monasterio, 1978a,b,c; Kaplan and Shapley, 1986; Lee et al., 1988, 1989a,b,c, 1990, 1994; Purpura et al., 1988, 1990; Kremers et al., 1993; Croner and Kaplan, 1995). The correlation between morphology and physiology remained inferential until the development of an in vitro retino-choroidal preparation, which allows simultaneous light stimulation and intracellular recording and labeling of ganglion cells. Using this procedure, Dacey and Lee (1994) were able to confirm that the phasic, electrophysiological broad-band ganglion cells were the M-cells and that the tonic, red–green color-opponent ganglion cells were the P-cells. In addition, they also confirmed that dendrites of off- and on-center varieties of both cell classes branch in the outer and inner halves of the inner plexiform layer, respectively. Other ganglion cell classes in the primate retina have been described using morphological criteria (Perry and Cowey, 1984; Rodieck and Watanabe, 1993; Kolb et al., 1992). One of them is a small-field bistratified cell and others comprise a heterogeneous group of wide-field cells. Electrophysiologically, there are also several other classes, including blueyellow opponent cells, very phasic cells, and cells responsive only to moving stimuli (de Monasterio and Gouras, 1975; de Monasterio et al., 1975a; de Monasterio, 1978a). By intracellular labeling of physiologically identified blue-on/yellow-off coloropponent cells, Dacey and Lee (1994) were able to show that they were the small-field bistratified cells, previously described by morphological techniques. Presently, using similar techniques, other ganglion cell classes are being studied in order that the correspondence between physiology and morphology can be established. Using tritan stimuli, blue-off/ yellow-on color-opponent cells were identified in the macaque LGN (Valberg et al., 1986a). There is recent evidence that such cells correspond to a class of wide-field ganglion cells in the retina (Dacey et al., 2002).

30

Spatial properties of M- and P-cells How M- and P-cells share the duties of visual information encoding may be understood through a thorough characterization of their physiological properties. In addition, it is relevant to know in which way the physiological properties of M- and P-cells from dichromatic and monochromatic, diurnal and nocturnal primates differ from the well-studied M- and P-cells from trichromatic diurnal anthropoids. Physiological characterization is accomplished by recording ganglion-cell action potentials from axons in the optic nerve (e.g. Hubel and Wiesel, 1960) or from cell soma directly in the retina (e.g. Gouras, 1968). In addition, ganglion cell activity can be indirectly monitored by recording the presynaptic potential from LGN cells (e.g. Kaplan and Shapley, 1986). M- and P-cells are devices that code for a visual intensity dimension in both space and time. What is coded is the absolute level of light intensity and its change in time and space, expressed by its contrast. The responses of M- and P-cells are influenced by the spatial and temporal contrast enabling the identification of edges and temporal changes. However, P-cells have a sustained component in their response, which enables them to respond in some degree to the level of light intensity. This sustained component may not only be important for color and brightness vision but also for certain aspects of form perception (Kremers, 1999). Do these two cell classes differ in the size of the spatial window which they analyze at a given visual field location? Receptive-field shape and size are usually quantified by using different techniques such as thresholds to small spots of light across the receptive field (de Monasterio and Gouras, 1975), measurement of area-threshold curves (Crook et al., 1988), and stimulation of cell receptive field with bipartite fields sinusoidally modulate in counterphase (Kremers and Weiss, 1997; Lee et al., 1998). In addition, assuming linearity, it is also possible to measure cell contrast sensitivity as a function of spatial frequency and then calculate receptive field profiles by Fourier transformation of the frequency response (Derrington and Lennie, 1984; Crook et al., 1988; Croner and Kaplan, 1995). The assumption that M- and P-cells respond to visual stimuli as

linear filters is only valid within certain limits, but it is possible to obtain useful values for M- and P-cell properties in the spatial domain from measurements in the spatial frequency domain (see Kremers et al., 2001, for a discussion of some aspects of this problem). As mentioned above, M- and P-cells differ from each other in dendritic field size, M-cells being larger than P-cells. For a given ganglion cell class, the receptive-field center sizes are generally proportional to the sizes of the dendritic fields at corresponding eccentricity (Peichl and Wa¨ssle, 1979). In macaques, it has been shown that M- and P cells have approximately circular receptive fields with a centersurround organization (e.g. de Monasterio and Gouras, 1975; Passaglia et al., 2002). In this regard, they are similar to a variety of ganglion cells found in other mammals such as cat a- and b-cells. In addition, it has also been shown that the receptivefield center sizes of M- and P-cells increase with eccentricity, except for P-cells in the first 10
of visual field (de Monasterio and Gouras, 1975; Derrington and Lennie, 1984). In this region, P-cells show a large degree of scatter and little change with eccentricity. Despite the anatomical difference in dendritic field diameter, physiological studies have shown a large degree of overlap between the receptive-field center sizes of M- and P-cells, especially in the central retina (Derrington and Lennie, 1984; Lee et al., 1998; Kremers and Weiss, 1997; Kremers et al., 2001; Kilavik et al., 2003). In the first 10
of visual field, the P-cell center sizes are larger than expected if they are solely or largely driven by a single cone. This is partly due to image blur imposed by the eye optics (Lee et al., 1998), but results obtained by measuring P-cell receptive-fields bypassing the eye’s optics, using interference fringes generated directly in the retina, are only partially consistent with this explanation (McMahon et al., 2000). Very little is known about ganglion cell receptivefield size in platyrrhines, limited to the very first observations in Ateles (Hubel and Wiesel, 1960). Some measurements have been made on the magnoand parvocellular relay neurons of Saimiri, Callithrix, and Aotus LGN (Usrey and Reid, 2000; Xu et al., 2001; Kremers and Weiss, 1997; Kremers et al., 2001; Kilavik et al., 2003). The results are qualitatively

31

similar to those obtained in Macaca, but M- and P-cells from Aotus have consistently larger receptive fields than their diurnal counterparts (Usrey and Reid, 2000; Xu et al., 2001).

transformation (Purpura et al., 1990; Lee et al., 1994; Bernadete and Kaplan, 1999a). At a given eccentricity, M-cells have a shorter temporal response than P-cells (Solomon et al., 2002), and thus M-cells should signal a time event with better precision than P-cells (Silveira, 1996; Silveira and de Mello Jr., 1998).

Temporal properties of M- and P-cells Spatial properties of M- and P-cells change dramatically with eccentricity and it has been suggested that there should be some change also in their temporal properties (Silveira, 1996; Silveira and de Mello Jr., 1998). To show this, the temporal response should be measured for both cell classes at all locations of the visual field. However, until recently, only results for M- or P-cells studied within a restricted range of eccentricities have been reported (Lee et al., 1990; Purpura et al., 1990; Lee et al., 1994; Bernadete and Kaplan, 1997a,b, 1999a,b). More recent results indeed strongly indicate that the temporal contrast gain and the critical flicker fusion frequency of M- and P-cells in the retina and the LGN increase with increasing cell size and increasing retinal eccentricity (Solomon et al., 1999, 2002; Kremers et al., 2001; Kilavik et al., 2003). One of the first achievements of electrophysiological recordings performed in the primate retina was the establishment of the phasic/tonic dichotomy for M- and P-cells by probing them with temporal step functions (Gouras, 1968). M-cells discharge transiently whereas P-cells have a sustained component in their discharges to maintained stimuli (Gouras, 1968; de Monasterio and Gouras, 1975). The difference between them can be estimated by calculating the tonic/phasic index (Purpura et al., 1990). Trichromatic and dichromatic Cebus, as well as monocromatic Aotus, have M- and P-cells that can be distinguished by the tonic/phasic index (Lee et al., 2000; Silveira et al., 2000). Figure 7 illustrates examples taken from a dichromatic Cebus. The M- and P-cell impulse response functions can be estimated with temporal pulses of different durations and intensities (Lee et al., 1994); or spots, annuli, or gratings the contrast of which is modulated according to m-sequences (Bernadete and Kaplan, 1997a, 1999b). In addition, the temporal frequency response of M- and P-cells can be measured and then converted to the time domain by Fourier

Contrast sensitivity of M- and P-cells Contrast is a critical intensity parameter for the visual system. It is a comparative measurement of luminance values of adjacent spatial regions or successive instants of time. To measure contrast, the visual system at least partially adapts to environmental mean luminance through mechanisms in the photoreceptors (Yau, 1994), as well as at the postreceptoral level. These postreceptoral mechanisms include compressive nonlinearities of contrast response functions (in the spatial and temporal domains), and high-pass spatial filtering by lateral inhibition (Shapley and Enroth-Cugell, 1984; Barlow and Levick, 1976; Laughlin, 1981). However, light adaptation in P-cells is incomplete, which permits them to signal also steady levels of luminance and chromaticity (Lee et al., 1990). The M- and P-cells differ considerably in their luminance contrast sensitivity (Kaplan and Shapley, 1986; Purpura et al., 1988; Lee et al., 1990, 1994; Kremers et al., 1993). This finding was first demonstrated in LGN neurons and later extended to their retinal afferents. M- and P-cell contrast sensitivity can be evaluated by stimulating the cell’s receptive field with drifting sine-wave gratings (Kaplan and Shapley, 1986; Purpura et al., 1988), temporal sine-wave modulation (Lee et al., 1990; Kremers et al., 1993), and light pulses of different durations (Lee et al., 1994). The cell response amplitude, expressed in impulses per second, is measured as a function of contrast. M-cells are about eight to ten times more sensitive than P-cells, but their responses saturate at a relatively low contrast level, whereas P-cells are insensitive to contrast but their responses show little saturation when contrast is increased. These differences in response amplitude as a function of contrast between M- and P-cells are observed at different retinal eccentricities, retinal illuminance levels, sizes of spatial targets, duration of

32

light pulses, and at most of spatial and temporal frequencies at which both cells are sensitive. The differential sensitivity of M- and P-cells to luminance contrast is also observed across species and in different color vision phenotypes of a given species (Lee et al., 1996, 2000; Silveira et al., 2000). M- and P-cells of dichromatic Cebus are very similar in contrast sensitivity to their counterparts of macaques and trichromatic Cebus (Fig. 8); the main difference among them is that P-cells from dichromatic Cebus are color blind (Lee et al., 1996, 2000). M- and P-cells of monochromatic Aotus are less sensitive to contrast than M- and P-cells of diurnal anthropoids at high temporal frequencies, but they still differ from each other, M-cells being more contrast sensitive than P-cells (Silveira et al., 2000). The reflectance of objects in natural scenes creates spatiotemporal patterns ranging over a wide band of contrast values (Laughlin, 1983). Thus it is reasonable to suppose that the M- and P-channels in all primates are tuned for different ranges of contrasts and therefore their presence will be of value for the analysis of visual information (Yamada et al., 1996a; van Hateren et al., 2002).

The role of M- and P-cells in achromatic and chromatic vision In 1878, Ewald Hering pointed out that human color vision is not characterized by three fundamental hue sensations, but four—blue, green, yellow, and red. To explain perceptual opponency, he postulated that blue and yellow cause antagonistic effects in one color mechanism, whereas green and red cause opposing effects in another mechanism. Human visual experience results from the interaction of three opponent channels: black–white, blue–yellow, and red–green. Although the long standing inconsistency between the Young–Helmholtz trichromatic theory (Young, 1802; von Helmholtz, 1867) and the Hering coloropponent theory (Hering, 1878) can be partly resolved if color-opponency is a sign of postreceptoral neuronal processing of cone signals, it remains puzzling why the Hering unique hues do not precisely correspond to the physiological cone-opponent mechanisms (Mollon and Jordan, 1997).

In contrast to nonmammalian vertebrates, the outer retina of primates and other mammals is not the site of color-opponent processes. Primate horizontal cells exhibit to some extent cone selectivity, but they respond to light of all wavelengths with the same polarity (Dacey et al., 1996). Thus, the search for the initial site of postreceptoral mechanisms of color-opponency has to focus on the selective connections between cones and bipolar cells in conjunction with interactions between bipolar cells, amacrine cells, and ganglion cells in the inner plexiform layer. First studies on retinal ganglion cells (Gouras, 1968; de Monasterio and Gouras, 1975; de Monasterio et al., 1975a,b; de Monasterio, 1978a,b) and their LGN relay neurons (De Valois et al., 1958; Wiesel and Hubel, 1966) were able to demonstrate that trichromatic primates, such as macaques, have color-opponent neurons, which can be divided into two broad categories. One group is numerous, comprising cells that respond with one polarity in the red, with opposite polarity in the green, and have a null point in the yellow region of the spectrum. The other group is less numerous, comprising cells that respond with one polarity in the blue, with opposite polarity in the yellow, and have a null point in the green region of the spectrum. The chromatic specificity of retinal ganglion cells can be tested in various ways, for example with circular or annular stimuli of which size, intensity and wavelength are varied. It was possible to relate the characteristics of receptive-field spatial organization to the spectral response of retinal ganglion cells and to their tonic/phasic temporal properties. Much was already known of the response of LGN neurons to the same kind of stimulus (De Valois et al., 1958; Wiesel and Hubel, 1966), making it possible to hypothesize about the projection of the main classes of retinal ganglion cells to the different LGN layers. The most numerous ganglion cell class are the tonic, red/green color-opponent cells (Fig. 9). Their receptive fields have a center–surround antagonistic organization, the center giving an on- or off-response and the surround responding with opposite polarity. Moreover, the receptive-field centers and surrounds receive signals from different cone classes, either MWS or LWS, resulting in red–green color-opponency. Combining the chromatic and spatial characteristics

33

of their receptive fields, they can be grouped in four subclasses: center red-on/surround green-off; center green-on/surround red-off; center red-off/surround green-on; and center green-off/surround red-on. Based on the similar physiological properties of tonic, red/green color-opponent ganglion cells and the parvocellular relay-neurons of the LGN and the labeling of Polyak’s midget ganglion cells by HRP injections in the parvocellular LGN layers, it was proposed that the midget cells, the P-cells, correspond to the red/green color-opponent cells (Leventhal et al., 1981; Perry et al., 1984). As mentioned above, this correspondence was later directly demonstrated by in vitro recording of ganglion cells followed by intracellular injection of Neurobiotin (Dacey and Lee, 1994). A comparison between psychophysical detection thresholds and ganglion cell responses to combined chromatic and achromatic modulation plotted in an MWS/LWS-cone space, has shown that the properties of P-cells were sufficiently linear and homogeneous to support a linear, red–green opponent, chromatic mechanism (Kremers et al., 1992; Lee et al., 1993a). What is the retinal circuitry that makes P-cells red–green color-opponent and how did they evolve from some cell class in a primitive dichromatic ancestor? P-cells receive input from midget bipolar cells which, over most of the retina of macaques and other catarrhines, have single dendritic clusters connected to single MWS- or LWS-cones. In the central 2 mm of the macaque retina (about 10
of visual field), P-cell dendritic-fields are very small and connect with single midget-bipolar axon-terminals, thence to single MWS- or LWS-cones. Thus, in this region, P-cell receptive-field centers necessarily receive cone-specific inputs (but see McMahon et al., 2000). P-cells could have cone-specific or cone-mixed surrounds (Lennie et al., 1991). Although there is evidence that the P-cell receptivefield surrounds are specific (Reid and Shapley, 1992; Lee et al., 1998), the issue is not fully resolved and the neuronal circuitry that makes this possible is not known. Human red–green color vision becomes quickly degraded with visual field eccentricity (Mullen and Kingdom, 1996). Between about 10–50
of visual field, most of the midget bipolars are connected to single MWS- or LWS-cones. However at these eccentricities, P-cell dendritic-fields are

connected to an increasing number of midgetbipolar axon-terminals. Nevertheless, in this region, about 64% of P-cells show overt red–green coloropponency and a degree of red–green coloropponency can be demonstrated in as many as 80% (Martin et al., 2001). These results suggest that during development, there may be a mechanism which results in specific connections to midget bipolars that are themselves connected to either MWS- or LWS-cones, thus obtaining select conespecific inputs (Martin et al., 2001). Finally, in the far peripheral retina, at 50
eccentricity or more, the majority of midget bipolars are connected to two or more cones and there is no indication that they select either MWS- or LWS-cones. At this eccentricity, in vitro recordings show that P-cells are not color-opponent (Dacey, 1999). The P-cells can support only the red–green color-opponent channel of primate chromatic vision, since these cells have no SWS-cone input. Another, less numerous, ganglion cell class of the macaque retina comprises blue-on/yellow-off cells without clear center–surround spatial organization of the receptive fields (de Monasterio and Gouras, 1975; de Monasterio et al., 1975a; de Monasterio, 1978c; Lee et al., 1988, 1989a,b). They correspond morphologically to the small-field bistratified cells (Dacey and Lee, 1994) and project to the koniocellular layers of the LGN (Martin et al., 1997). The sizes of their receptive fields are similar to those of M-cells (de Monasterio and Gouras, 1975) and correlate with their dendritic field sizes (Dacey, 1993). Ganglion cells that have an inhibitory SWS-cone input are required to fully support the blue–yellow color-opponent channel of primate chromatic vision. The presence of such cells have been reported (de Monasterio and Gouras, 1975; de Monasterio et al., 1975a; Valberg et al., 1986a; Lee et al., 1989a,b) but their morphology has been elusive until recently, when they were associated to a class of wide-field ganglion cells (Dacey et al., 2002). The achromatic channel of primate vision is supported at least partially by cells that are insensitive to color, since there are many conditions, both physiological and pathological, in which most of achromatic stimulus detection persists in the absence of color sensation (Merigan, 1989; Lynch et al., 1992).

34

The primary studies on macaques described a class of retinal ganglion cells and LGN neurons with receptive fields with a center–surround spatial organization, and which received additive signals from MWS- and LWScones in center and surround (Wiesel and Hubel, 1966; Gouras, 1968; de Monasterio and Gouras, 1975; de Monasterio, 1978a,b). These cells give responses of the same polarity, either on or off, to all light wavelengths. Based on the similarity of physiological properties between broad-band ganglion cells and the magnocellular relay neurons of the LGN, as well as on the results of HRP injections in the magnocellular LGN layers, which labeled Polyak’s parasol ganglion cells, it was proposed that parasol cells correspond to broadband cells (Leventhal et al., 1981; Perry et al., 1984). This was later confirmed by ganglion cell intracellular recording and labeling (Dacey and Lee, 1994). Shapley and Perry (1986) proposed the broadband, magnocelullar-projecting ganglion cells are called M-cells. M-cells have been found to support psychophysical tasks such as heterochromatic flicker photometry (Lee et al., 1988), minimally distinct border (Kaiser et al., 1990; Valberg et al., 1992), and spatial position in vernier acuity (Lee et al., 1993b, 1995). This is consistent with their important role in achromatic vision. The positional signal from single M-cells is more precise than that from P-cells, especially at lower contrasts, making M-cells the likely candidates to support hyperacuity. Besides M-cells, other ganglion cell classes may also be important to the primate achromatic vision. In other highly visual mammals, two or more ganglion cell classes, such as the cat a- and b-cells, appear to share the duties of solving complex achromatic tasks needed for animal behavior. The P-cells are the most numerous ganglion cell class, corresponding to 80% of all ganglion cells in the macaque retina. These cells respond to achromatic stimulus at medium and high contrast levels. In addition, their response does not saturate, or exhibits only a small degree of saturation when contrast is increased. M-cells, on the contrary, are very sensitive to contrast, but their response saturates at a relatively low contrast level. Thus, it is possible that M- and P-cells code low and high contrast levels, respectively, and work synergistically to support achromatic vision at intermediate levels of contrast. In particular, P-cells may contribute to achromatic

lightness perception (Valberg et al., 1986b) or discrimination of suprathreshold targets (Pokorny and Smith, 1997). Finally, although P-cells are much more sensitive to chromatic than achromatic contrast, in the natural environment the range of achromatic contrasts is much greater than of chromatic contrasts, and most information in P-cell spike trains is associated with achromatic content (van Hateren et al., 2002). Platyrrhines provide an opportunity to study the contribution of M- and P-cells to the chromatic and achromatic aspects of vision, since in a single species, there are trichromatic and dichromatic individuals bearing a variety of color vision phenotypes. Anatomical studies have shown that the retina of dichromatic Cebus has similar ganglion cell densities and identical proportions of M- and P-cells as those observed in trichromatic anthropoids (Silveira et al., 1989; Lima et al., 1996; Yamada et al., 1996a). To investigate dichromatic P-cells, ganglion cell responses were recorded in trichromatic and dichromatic Cebus and compared with data obtained from macaques (Lee et al., 1996, 2000). Color vision phenotype of each animal was determined by electrophysiological procedures (see the next section) and confirmed by DNA genetic analysis of blood and liver samples. Despite some differences in quantitative details, results from M- and P-cells of trichromatic Cebus strongly resemble those from trichromatic Macaca. In particular, P-cells respond to chromatic stimuli with vigorous red–green opponency (Fig. 9). More importantly, P-cells from dichromatic Cebus appear to be blind versions of those from trichromatic Cebus and Macaca (Figs. 7 and 8). The presence of similar numbers of P-cells in dichromats and trichromats, and the notion that they respond in a very similar manner to stimuli, except for the absence of color-opponency in the dichromatic P-cells, might suggest that P-cells have a role to play in achromatic vision. Furthermore, the presence of P-cells in primates with different life styles, different cone-to-rod ratios, and different numbers of cone classes (Silveira et al., 1994; Yamada et al., 1996a,b, 1998, 2001) may indicate that the original P-cells evolved for the needs of spatiotemporal achromatic vision and became colorcoded when evolution provided two MWS/LWS

35

Fig. 7. Temporal properties of dichromatic Cebus M- and P-cells. Cells were stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated in phase to generate 400 ms square pulses with different Weber contrasts (top row in each set). Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. The bottom rows in each set depict the responses of a M-on cell and a P-off cell as perstimulus time histograms (PSTH) (total recording ¼ 6 s; sweep duration ¼ 800 ms; bin size ¼ 4 ms; vertical bar ¼ 100 impulses/s). The tonic/phasic index (TPI) was calculated according to Purpura et al. (1990) and the values are given above each cell response histogram. M-cell response is phasic and more sensitive to contrast whereas P-cell response is tonic and less sensitive to contrast.

Fig. 8. Temporal contrast sensitivity of dichromatic Cebus M- and P-cells. Cells were stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated in phase to generate sine-wave temporal modulation of 9.76 Hz of frequency with different Michelson contrasts. Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. For each stimulus condition a PSTH was recorded, a Fourier analysis was performed, and the first harmonic amplitude was extracted to be plotted as a function of stimulus contrast. NakaRushton functions were fitted to the data point. As contrast increases, both M- and P-cell responses increase, however displaying a very different behavior. M-cell response is very sensitive to contrast but saturates at intermediate and high contrast levels. P-cell response is relatively insensitive to contrast but exhibits little saturation when contrast is increased.

Fig. 9. Chromatic and achromatic responses of a trichromatic Cebus P-cell. The cell was stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated to generate 400 ms square pulses with different chromatic and achromatic Weber contrasts. Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. The cell response is shown as PSTH (total recording ¼ 6 s; sweep duration ¼ 800 ms; bin size ¼ 4 ms; vertical bar ¼ 100 impulses/s). The top three rows illustrate the P-cell chromatic response to greenward and redward increments. The response reveals that this P-cell is a center green-on/surround red-off. The bottom two rows illustrate the achromatic response to luminance pulses. P cells from trichromatic female Cebus are generally less sensitive to achromatic in comparison to chromatic contrast. P-cells from dichromatic male or female Cebus are colour blind.

36

cones (Mollon and Jordan, 1988; Kremers, 1999; Kremers et al., 1999). This is in agreement with the idea that the appearance of trichromacy in mammals was a relatively recent phylogenetic event, and probably occurred in a primate ancestor (Tan and Li, 1999) which probably already possessed a P-pathway. The P-cell cone selectivity might be simply due to the ‘hit and miss’ mechanism proposed by Shapley and Perry (1986), but, as mentioned above, it is probable that some form of reinforcement of synaptic connections, operating in trichromatic animals, can strengthen P-cell color-opponency during development (Martin et al., 2001).

Photoreceptor signals to M- and P-cells The primate retina, as do the retina of other mammals, possesses several parallel pathways which convey signals from cones to ganglion cells. This is reflected in the existence of multiple classes of cone bipolar cells, including those that connect cones to M- and P-cells (Boycott and Wa¨ssle, 1991). On the other hand, rods are connected to the inner retina by means of a single class of rod bipolar cells. A specific amacrine cell class, the AII amacrine cell, transfers rod signals from rod to cone bipolars, determining that from this point onwards, rod and cone driven signals share the same pathways (Kolb and Famiglietti, 1974). This rod pathway was initially dissected in nonprimate mammals, but more recently several studies have shown that the rod pathway is similarly organized in primates (Gru¨nert and Martin, 1991; Wa¨ssle et al., 1995; Dacey, 1999). For primates and other mammals, there is at least one alternative route, which uses gap junctions to feed rod signals directly into cones (Nelson, 1977; Schneeweis and Schnapf, 1995; Sharpe and Stockman, 1999; Verweij et al., 1999). A third possibility has been demonstrated only in rodents and consists of rod contacts with off cone-bipolar cells (Soucy et al., 1998). M- and P-cells convey cone and rod signals to the LGN with different strengths. Although anatomical studies performed in the macaque retina suggest that both cell classes receive significant rod input (Gru¨nert, 1997), ganglion cell recordings in

the retina of the same species have shown that while M-cells receive a strong rod signal, P-cells receive a weak or negligible rod input (Purpura et al., 1988, 1990; Lee et al., 1997). Human psychophysical studies also suggest that this might be the case (Sun et al., 2001). A survey of the literature on photoreceptor density distribution in primates, including recent comparative studies of several platyrrhines (Franco et al., 2000), reveals the presence of three basic patterns. Diurnal primates with small eyes, such as Callithrix and Saguinus exhibit a high cone-to-rod ratio, whereas those with medium to large eyes, such as Cebus and Macaca, exhibit a lower cone-to-rod ratio. All diurnal primates, independently of eye dimensions, have a well-developed fovea the size of which is constant in different species (Franco et al., 2000). Finally, nocturnal primates, such as Aotus and Otolemur, exhibit a very low cone to rod ratio and lack a well-developed fovea. Thus, it is interesting to see how M- and P-cells from primates, that have very different cone-to-rod ratios, handle photoreceptor signals. Among platyrrhines, there are examples of all three patterns of cone-to-rod ratios and so they represent a good animal model to investigate this question. As previously mentioned, M- and P-cells of Aotus (nocturnal, very low cone-to-rod ratio), Cebus (diurnal, low cone-to-rod ratio), and Callithrix (diurnal, high cone-to-rod ratio), adjust their dendritic field size to keep approximately the same cone convergence to either M- or P-cells for a given eccentricity. Consequently, the rod convergence to M- and P-cells are very different in these three platyrrhines and a physiological difference among them is to be expected. This question has been investigated by recording from ganglion cells in the retina of Aotus and dichromatic Cebus (Saito et al., 2001). As the retinas of both animals have a single MWS/LWS-cone class, their M- and P-cells show no additive or subtractive interactions between M and L cones, making it easier to study the presence of cone and rod signals in their responses. There are no data available for Callithrix ganglion cells, but cone–rod interactions have been studied in this platyrrhine by recording from LGN relay neurons of the M- and P-pathway (Yeh et al., 1995; Weiss et al., 1998).

37

Two protocols were used to study the contribution of cone and rod signals for the Cebus and Aotus ganglion cell responses. One of them uses a modified heterochromatic flicker photometry (HFP) paradigm: the lights of a 554 nm green LED and a 638 nm red LED were sinusoidally modulated in counterphase and the modulation amplitudes of the two LEDs were varied in relation to each other, while keeping the mean luminance and average chromaticity

constant. The amplitudes and phases of cell responses changed as a function of the ratio between the modulation amplitude of the red and green light present in the stimulus. The response amplitudes were minimal for a certain red/green ratio, depending on which photoreceptor was driving the cell responses. In addition, there was a large phase shift in the cell responses when the stimulus condition crosses the null point (not illustrated).

Fig. 10. The weighting of cone and rod signals present in Cebus and Aotus M-cell responses, measured at 2000 Trolands by heterochromatic flicker photometry (HFP). The cells were recorded in a dichromatic Cebus, having a 563 nm LWS-cone, and a monochromatic Aotus, having a 543 nm MWS-cone. Cells were stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated in counterphase to generate sine-wave temporal modulation at different frequencies. The amplitude modulation of the two LEDs was varied to found the null point for cell response. Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. For each stimulus condition a PSTH was recorded, a Fourier analysis was performed, and the first harmonic amplitude and phase were extracted. A. Photopigment templates for Cebus monkeys. B. Photopigment templates for Aotus monkeys. C. Cebus M-cell response is strongly cone-dominated in all temporal frequencies. D. Aotus M-cell response is strongly rod-dominated in all temporal frequencies.

38

Figure 10A–B shows HFP templates predicting how the response amplitudes change with red–green ratio for each rod or MWS/LWS cone photopigment that may be present in the Cebus and Aotus retina (Fig. 10A and B, respectively). The templates were obtained by convolution of photopigment absorption spectrum with LED light-emission spectra. The retina of a dichromatic Cebus has one of three MWS/LWS photopigments, with absorption spectra peaking at 535 nm, 548 nm or 563 nm. Figure 10A illustrates the response amplitude templates for these photopigments and also for the rod photopigment, which peaks at 500 nm. The Cebus retina also has an S photopigment, peaking at 440 nm, but M- and P-cells do not receive input from SWS-cones. On the other hand, the Aotus possesses a single cone photopigment peaking at 543 nm, the template of which is illustrated together the template for the rod photopigment in Fig. 10B. The results of HFP for Cebus and Aotus M-cells are illustrated in the bottom panels of Fig. 10. The results for three temporal frequencies (4.88, 9.76, and 19.53 Hz) and 2000 Trolands of retinal illuminance are shown. Cell responses were recorded and the first harmonic amplitudes and phases were extracted by Fourier analysis. The response amplitudes are plotted in Fig. 10C and D as a function of red–green ratio for Cebus and Aotus M-cells, respectively. Assuming that cell responses reflect a linear addition of cone and rod inputs, a linear vector addition model was used to fit response amplitude and phase data in the complex plane (Weiss et al., 1998). A least-square method was used to find four free parameters: cone amplitude and phase, and rod amplitude and phase. The predicted amplitudes obtained from the fits of the model to the data are displayed in Fig. 10C–D together with the measured response amplitudes. Overall, the model fitted the data well. The null-point found for the Cebus M-cell is close to the template prediction for an animal having a 563 nm cone photopigment, and this result was confirmed by genetic analysis of blood and liver samples (Lee et al., 2000). The fraction of the cone driven signal, R, present in the cell response, was calculated by dividing the cone amplitude by the sum of cone and rod amplitudes. The larger the value of R, the more the cell response is driven by cone signals. The

Cebus M-cell response is heavily cone-dominated, while the Aotus M-cell response is heavily roddominated at all temporal frequencies. The cone and rod signal contribution to Cebus and Aotus ganglion cell responses was also studied using the Smith et al. (1992) phase paradigm. In this protocol, the modulation amplitudes of the red and green lights are kept constant. The relative phase of the green to the red LED was varied in 22.5
steps. As with HFP, it is possible to predict the ganglion cell response amplitude and phase for animals of different phenotypes. Figure 11A–B shows, the amplitude and phase templates for dichromatic Cebus having different MWS/LWS-photopigments, while Fig. 12A–B shows the templates for the Aotus. In this paradigm, both the response amplitudes and phases change as a function of the relative phase between the LEDs according to the photopigment that is driving the cell responses, but the response phase is a more reliable photopigment signature due to its relative insensitivity to signal-to-noise ratio (Smith et al., 1992). The results for a Cebus M-cell are illustrated in the middle and bottom panels of Fig. 11. The results for four temporal frequencies (2.44, 9.76, 19.53, and 39.06 Hz) and two illuminance levels (20 and 2000 Trolands) are shown. The model fitted to the data points had four free parameters: cone and rod phases, cone signal fraction (R), and a scalar factor used to displace vertically the fitting curve (Smith et al., 1992; Lee et al., 1997). The results for the amplitudes and phases, at 2000 Trolands, are displayed in Fig. 11C and D, respectively, whereas those at 20 Trolands are shown in Fig. 11E and F. The model fitted the data well and the phase plots can be used to predict which photopigment is dominating the cell response. At high illuminance, the cell response is heavily dominated at all temporal frequencies by signals from a 563 nm cone photopigment. At low illuminance levels, there are signs of rod intrusion especially at low temporal frequencies. The presence of a 563 nm photopigment was confirmed by genetic analysis (Lee et al., 2000). The templates and results for an Aotus M-cell are illustrated in Fig. 12. The response phase indicates that cell responses are heavily rod-dominated even at this high illuminance level of 2000 Trolands.

39

Fig. 11. The weighting of cone and rod signals present in Cebus M-cell responses measured by the Smith et al. (1992) phase paradigm. This cell was recorded in a dichromatic Cebus, having a 563 nm LWS-cone. The cell was stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated in counterphase to generate sine-wave temporal modulation at different frequencies. The phase of the green LED was varied to evoke cell response change. Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. For each stimulus condition a PSTH was recorded, a Fourier analysis was performed, and the first harmonic amplitude and phase were extracted. A–B. Amplitude and phase of the photopigment templates for Cebus monkeys. C–D. Response amplitude and phase of Cebus M-cell response under 2000 Trolands retinal illuminance is strongly cone-dominated in all temporal frequencies. E–F. Response amplitude and phase of Cebus M-cell response under 20 Trolands retinal illuminance shows some rod intrusion, mostly in the low temporal frequencies.

40

Fig. 12. The weighting of cone and rod signals present in Aotus M-cell responses, measured by the Smith et al. (1992) phase paradigm. This cell was recorded in the nocturnal Aotus, which is monochromat and has a single 543 nm MWS-cone. The cell was stimulated using two LEDs of 554 nm and 636 nm peak emission, which were modulated in counterphase to generate sine-wave temporal modulation at different frequencies. The phase of the green LED was varied to evoke cell response change. Maxwellian view was used. Cell response was extracellularly recorded with tungsten-in-glass microelectrode inserted into the retinal tissue. For each stimulus condition a PSTH was recorded, a Fourier analysis was performed, and the first harmonic amplitude and phase were extracted. A–B. Amplitude and phase of the photopigment templates for Aotus monkeys. C–D. Response amplitude and phase of Aotus M-cell response under 2000 Trolands retinal illuminance is strongly rod-dominated in all temporal frequencies. As temporal frequency increases there is some sign of cone intrusion.

Figure 13 shows the cone signal fraction for a sample of M- and P-cells recorded from the retinas of dichromatic Cebus and monochromatic Aotus at 2000 Trolands. At this high illuminance level, the Aotus cells are still heavily rod-dominated, whereas Cebus cell responses are mainly cone dominated. This is consistent with the larger rod convergence to M- and P-cells in this nocturnal monkey found in anatomical experiments (Yamada et al., 2001).

Conclusions In this chapter, we have compared the anatomy and physiology of two main classes of primate retinal ganglion cells, the M- and P-cells. There is compelling evidence that the anatomical and physiological properties of M-cells are very similar in all anthropoids so far studied, both from the Old- and NewWorld. P-cells also have similar properties in these animals with the exception that they are color blind in

41

grant R01-13112. LCLS and ESY are CNPq research fellows. CAS has a CAPES studentship for graduates. JK is supported by a Heisenberg fellowship of German Research Council. The authors thank Dr. Jose´ Augusto P. C. Muniz, Head of the Centro Nacional de Primatas (Ananindeua, State of Para´, Brazil) for his support of primate research. The authors have been supported on different occasions by international travel grants held by CNPq, CAPES, Max Planck Gesellschaft, Deutscher Akademischer Austauschdientst, The Royal Society, and The British Council. Fig. 13. Distribution of cone/rod ratio in samples of Cebus and Aotus ganglion cells measured by HFP and phase paradigms. The results obtained with the two paradigms were shown separately. The Smith et al. (1992) phase paradigm was used to analyze, respectively, the responses of 17 Cebus ganglion cells and 28 Aotus ganglion cells. The HFP paradigm was used for 17 Cebus ganglion cells and 19 Aotus ganglion cells. At 9.76 Hz most of Aotus cells were rod-dominated even at 2000 Trolands.

the monochromatic and dichromatic platyrrhines. Moreover, there are consistent differences between closely related diurnal and nocturnal anthropoids concerning the contribution of cone and rod driven signals to ganglion cell responses: rods have a much stronger influence on the ganglion cell responses in the nocturnal Aotus than in the diurnal Cebus. Whatever the original role of the M- and P-cells, they are likely to have evolved prior to the divergence of catarrhines and platyrrhines. This suggests that they should also be present in prosimians. Very little is known about retinal ganglion cells of prosimians, but the few studies that have been done in these primates indicate that indeed they follow a general primate scheme, having M- and P-cells similar to those of anthropoids. Apart from the differences mentioned above, M- and P-cell systems thus appear to be strongly conserved in the various primate species. The reasons for this may lie in the roles of these systems for both achromatic and chromatic vision.

Acknowledgments The authors have been supported by FINEP, CNPq, and CAPES. BBL is currently supported by the NEI

References Barlow, H.B. (1961) Possible principles underlying the transformation of sensory messages. In: Rosenblith, W. (Ed.), Sensory Communication. MIT Press, Cambridge, MA, pp. 217–234. Barlow, H.B. and Levick, W.R. (1976) Threshold setting by the surround of cat retinal ganglion cells. J. Physiol. (Lond.), 259: 737–757. Bernadete, E.A. and Kaplan, E. (1997a) The receptive field of the primate P retinal ganglion cell. I: Linear dynamics. Vis. Neurosci., 14: 169–185. Bernadete, E.A. and Kaplan, E. (1997b) The receptive field of the primate P retinal ganglion cell. II: Nonlinear dynamics. Vis. Neurosci., 14: 187–205. Bernadete, E.A. and Kaplan, E. (1999a) The dynamics of primate M retinal ganglion cells. Vis. Neurosci., 16: 355–368. Bernadete, E.A. and Kaplan, E. (1999b) Dynamics of primate P retinal ganglion cells: responses to chromatic and achromatic stimuli. J. Physiol. (Lond.), 519.3: 775–790. Boycott, B.B. and Dowling, J.E. (1969) Organization of the primate retina: light microscopy. Phil. Trans. R. Soc. Lond. B, 255: 109–184. Boycott, B.B. and Wa¨ssle, H. (1991) Morphological classification of bipolar cells of the primate retina. Eur. J. Neurosci., 3: 1069–1088. Crook, J.M., Lange-Malecki, B., Lee, B.B. and Valberg, A. (1988) Visual resolution of macaque retinal ganglion cells. J. Physiol. (Lond.), 396: 205–224. Croner, L.J. and Kaplan, E. (1995) Receptive fields of P and M ganglion cells across the primate retina. Vision Res., 35: 7–24. Dacey, D.M. (1993) Morphology of a small-field bistratified ganglion cell type in the macaque and human retina. Vis. Neurosci., 10: 1081–1098. Dacey, D.M. (1999) Primate retina: cell types, circuits and colour opponency. Prog. Retin. Eye Res., 18: 737–763. Dacey, D.M. and Lee, B.B. (1994) The ‘blue-on’ opponent pathway in primate retina originates from a distinct bistratified ganglion cell type. Nature, 367: 731–735.

42 Dacey, D.M. and Petersen, M.R. (1992) Dendritic field size and morphology of midget and parasol ganglion cells of the human retina. Proc. Natl. Acad. Sci. USA, 89: 9666–9670. Dacey, D.M., Lee, B.B., Stafford, D.K., Pokorny, J. and Smith, V.C. (1996) Horizontal cells of the primate retina: cone specificity without spectral opponency. Science, 271: 656–659. Dacey, D.M., Peterson, B.B. and Robinson, F.R. (2002) Identification of an S-cone opponent OFF pathway in the macaque monkey retina: morphology, physiology and possible circuitry. ARVO Annual Meeting Abstract Search and Program Planner, Program No. 3796. de Monasterio, F.M. (1978a) Properties of concentrically organized X and Y ganglion cells of macaque retina. J. Neurophysiol., 41: 1394–1417. de Monasterio, F.M. (1978b) Center and surround mechanisms of opponent-colour X and Y ganglion cells of retina of macaques. J. Neurophysiol., 41: 1418–1434. de Monasterio, F.M. (1978c) Properties of ganglion cells with atypical receptive-field organization in retina of macaques. J. Neurophysiol., 41: 1394–1417. de Monasterio, F.M. and Gouras, P. (1975) Functional properties of ganglion cells of the rhesus monkey retina. J. Physiol. (Lond.), 251: 167–195. de Monasterio, F.M., Gouras, P. and Tolhurst, D.J. (1975a) Trichromatic colour opponency in ganglion cells of the rhesus monkey retina. J. Physiol. (Lond.), 251: 197–216. de Monasterio, F.M., Gouras, P. and Tolhurst, D.J. (1975b) Concealed colour opponency in ganglion cells of the rhesus monkey retina. J. Physiol. (Lond.), 251: 217–229. Derrington, A.M. and Lennie, P. (1984) Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. J. Physiol. (Lond.), 357: 219–240. De Valois, R.L., Smith, C.J., Kitai, S.T. and Karoly, A.J. (1958) Responses of single cells in different layers of the primate lateral geniculate nucleus to monochromatic light. Science, 127: 238–239. Dogiel, A.S. (1891) Ueber die nervo¨sen Elemente in der Retina des Menschen. Archiv fu¨r Mikroskopische Anatomie und Entwicklungsmechanik, 38: 317–344. Enroth-Cugell, C. and Robson, J.G. (1966) The contrast sensitivity of retinal ganglion cells of the cat. J. Physiol. (Lond.), 187: 517–552. Evans, E.F. (1982) Basic physics and psychophysics of sound. In: Barlow, H.B. and Mollon, J.D. (Eds.), The Senses. Cambridge University Press, Cambridge, pp. 239–250. Fleagle, J.G. (1988) Primate Adaptation and Evolution. Academic Press, San Diego. Franco, E.C.S., Finlay, B.L., Silveira, L.C.L., Yamada, E.S. and Crowley, J.C. (2000) Conservation of absolute foveal area in New World primates: a constraint on eye size and conformation. Brain Behav. Evol., 56: 276–286. Ghosh, K.K., Goodchild, A.K., Sefton, A.E. and Martin, P.R. (1996) The morphology of retinal ganglion cells in the New

World marmoset monkey Callithrix jacchus. J. Comp. Neurol., 366: 76–92. Goodchild, A.K., Ghosh, K.K. and Martin, P. (1996) A comparison of photoreceptor spatial density and ganglion cell morphology in the retina of human, macaque monkey, cat, and the marmoset Callithrix jacchus. J. Comp. Neurol., 366: 55–75. Gouras, P. (1968) Identification of cone mechanisms in monkey ganglion cells. J. Physiol. (Lond.), 199: 533–547. Gru¨nert, U. (1997) Anatomical evidence for rod input to the parvocellular pathway in the visual system of the primate. Eur. J. Neurosci., 9: 617–621. Gru¨nert, U. and Martin, P.R. (1991) Rod bipolar cells in the macaque monkey retina: immunoreactivity and connectivity. J. Neurosci., 11: 2742–2758. Hering, E. (1878) Zur Lehre vom Lichtsinne. Carl Gerold’s Sohn, Wien. Hubel, D.H. and Wiesel, T.N. (1960) Receptive fields of optic nerve fibres in the spider monkey. J. Physiol. (Lond.), 154: 572–580. Jacobs, G.H. (1998) Photopigments and seeing—lessons from natural experiments. Invest. Opthalmol. Vis. Sci., 39: 2205–2216. Jacobs, G.H. and Neitz, J. (1987) Polymorphism of the middle wavelength cone in two species of South American monkey: Cebus apella and Callicebus molloch. Vision Res., 27: 1263–1268. Jacobs, G.H., Deegan II, J.F., Neitz, J., Crognale, M.A. and Neitz, M. (1993) Photopigments and colour vision in the nocturnal monkey. Aotus. Vision Res., 33: 1773–1783. Jacobs, G.H., Neitz, M. and Neitz, J. (1996) Mutations in S-cone pigment genes and the absence of colour vision in two species of nocturnal primate. Proc. R. Soc. Lond. B, 263: 705–710. Kaiser, P.K., Lee, B.B., Martin, P.R. and Valberg, A. (1990) The physiological basis of the minimally distinct border demonstrated in the ganglion cells of the macaque retina. J. Physiol. (Lond.), 422: 153–183. Kaplan, E. and Shapley, R.M. (1986) The primate retina contains two types of ganglion cells, with high and low contrast sensitivity. Proc. Natl. Acad. Sci. USA, 83: 2755–2757. Kaplan, E., Mukherjee, P. and Shapley, R.M. (1993) Information filtering in the lateral geniculate nucleus. In: Shapley, R.M. and Lam, D.M.K. (Eds.), Contrast Sensitivity. MIT Press, Cambridge, MA, pp. 183–200. Kilavik, B.E., Silveira, L.C.L. and Kremers, J. (2003) Centre and surround responses of marmoset lateral geniculate neurones at different temporal frequencies. J. Physiol. (Lond.), 546.3: 903–919. Kolb, H. and DeKorver, L. (1991) Midget ganglion cells of parafovea of the human retina: a study by electron microscopy and serial-section reconstruction. J. Comp. Neurol., 303: 617–636.

43 Kolb, H. and Famiglietti, E.V., Jr. (1974) Rod and cone pathways in the inner plexiform layer of cat retina. Science, 186: 47–49. Kolb, H., Linberg, K. and Fisher, S.K. (1992) Neurons of the human retina: a Golgi study. J. Comp. Neurol., 318: 147–187. Kremers, J. (1999) Spatial and temporal response properties of the major retino-geniculate pathways of Old and New World monkeys. Doc. Ophthalmol., 95: 229–245. Kremers, J. and Weiss, S. (1997) Receptive field dimensions of lateral geniculate cells in the common marmoset (Callithrix jacchus). Vision Res., 37: 2171–2181. Kremers, J., Lee, B.B. and Kaiser, P.K. (1992) Sensitivity of macaque retinal ganglion cells and human observers to combined luminance and chromatic temporal modulation. J. Opt. Soc. Am. A, 9: 1477–1485. Kremers, J., Lee, B.B., Pokorny, J. and Smith, V.C. (1993) Responses of macaque ganglion cells and human observers to compound periodic waveforms. Vision Res., 33: 1997–2011. Kremers, J., Silveira, L.C.L., Yamada, E.S. and Lee, B.B. (1999) The ecology and evolution of primate colour vision. In: Gegenfurtner, K.R. and Sharpe, L.T. (Eds.), Color Vision: From Molecular Genetics to Perception. Cambridge University Press, Cambridge, pp. 123–142. Kremers, J., Silveira, L.C.L. and Kilavik, B.E. (2001) Influence of contrast on the responses of marmoset lateral geniculate cells to drifting gratings. J. Neurophysiol., 85: 235–246. Ku¨ffler, S.W. (1953) Discharge patterns and functional organization of mammalian retina. J. Neurophysiol., 16: 37–68. Laughlin, S.B. (1981) Neural principles in the peripheral visual systems of invertebrates. In: H. Autrum (Ed.), Handbook of Sensory Physiology, Vol. VII, Part 6 Comparative Physiology and Evolution of Vision in Invertebrates, B: Invertebrate Visual Centers and Behavior I. Springer-Verlag, Berlin, pp. 133–280. Laughlin, S.B. (1983) Matching coding to scenes to enhance efficiency. In: Braddick, O.J. and Sleigh, A.C. (Eds.), Physical and Biological Processing of Images. Springer, Berlin, pp. 42–52. La Vail, M.M., Rapaport, D.H. and Rakic, P. (1991) Cytogenesis in the monkey retina. J. Comp. Neurol., 309: 86–114. Lee, B.B., Martin, P.R. and Valberg, A. (1988) The physiological basis of heterochromatic flicker photometry demonstrated in the ganglion cells of the macaque retina. J. Physiol. (Lond.), 404: 323–347. Lee, B.B., Martin, P.R. and Valberg, A. (1989a) Sensitivity of macaque retinal ganglion cells to chromatic and luminance flicker. J. Physiol. (Lond.), 414: 223–243. Lee, B.B., Martin, P.R. and Valberg, A. (1989b) Amplitude and phase of responses of macaque retinal ganglion cells to flickering stimuli. J. Physiol. (Lond.), 414: 245–263.

Lee, B.B., Martin, P.R. and Valberg, A. (1989c) Nonlinear summation of M- an L-cone inputs to phasic retinal ganglion cells of the macaque. J. Neurosci., 9: 1433–1442. Lee, B.B., Pokorny, J., Smith, V.C., Martin, P.R. and Valberg, A. (1990) Luminance and chromatic modulation sensitivity of macaque ganglion cells and human observers. J. Opt. Soc. Am. A, 7: 2223–2236. Lee, B.B., Martin, P.R., Valberg, A. and Kremers, J. (1993a) Physiological mechanisms underlying psychophysical sensitivity to combined luminance and chromatic modulation. J. Opt. Soc. Am. A, 10: 1403–1412. Lee, B.B., Wehrhahn, C., Westheimer, G. and Kremers, J. (1993b) Macaque ganglion cell responses to stimuli that elicit hyperacuity in man: detection of small displacements. J. Neurosci., 13: 1001–1009. Lee, B.B., Pokorny, J., Smith, V.C. and Kremers, J. (1994) Responses to pulses and sinusoids in macaque ganglion cells. Vision Res., 34: 3081–3096. Lee, B.B., Wehrhahn, C., Westheimer, G. and Kremers, J. (1995) The spatial precision of macaque ganglion cell responses in relation to vernier acuity of human observers. Vision Res., 35: 2743–2758. Lee, B.B., Silveira, L.C.L., Yamada, E.S. and Kremers, J. (1996) Parallel pathways in the retina of Old and New World primates. Braz. J. Biol., 56(1): 323–338. Lee, B.B., Smith, V.C., Pokorny, J. and Kremers, J. (1997) Rod inputs to macaque ganglion cells. Vision Res., 37: 2813–2828. Lee, B.B., Kremers, J. and Yeh, T. (1998) Receptive fields of primate retinal ganglion cells studied with a novel technique. Visual Neurosci., 15: 161–175. Lee, B.B., Silveira, L.C.L., Yamada, E.S., Hunt, D.M., Kremers, J., Martin, P.R., Troy, J.B. and da Silva Filho, M. (2000) Visual responses of ganglion cells of a New World primate, the capuchin monkey, Cebus apella. J. Physiol. (Lond.), 528.3: 573–590. Lennie, P., Haake, P.W. and Williams, D.R. (1991) The design of chromatically opponent receptive fields. In: Landy, M.S. and Movshon, J.A. (Eds.), Computational Models of Visual Processing. The MIT Press, Cambridge, MA, pp. 71–82. Leventhal, A.G., Rodieck, R.W. and Dreher, B. (1981) Retinal ganglion cell classes in the Old World monkey: morphology and central projections. Science, 213: 1139–1142. Leventhal, A.G., Ault, S.J., Vitek, D.J. and Shou, T. (1989) Extrinsic determinants of retinal ganglion cell development in primates. J. Comp. Neurol., 286: 170–189. Lima, S.M.A., Silveira, L.C.L. and Perry, V.H. (1996) Distribution of M retinal ganglion cells in diurnal and nocturnal New-World monkeys. J. Comp. Neurol., 368: 538–552. Lynch III, J.J., Silveira, L.C.L., Perry, V.H. and Merigan, W.H. (1992) Visual effects of damage to P ganglion cells in macaques. Vis. Neurosci., 8: 575–583.

44 Martin, P.R., White, A.J., Goodchild, A.K., Wilder, H.D. and Sefton, A.E. (1997) Evidence that blue-on cells are part of the third geniculocortical pathway in primates. Eur. J. Neurosci., 9: 1536–1541. Martin, P.R., Lee, B.B., White, A.J.R., Solomon, S.G. and Ru¨ttiger, L. (2001) Chromatic sensitivity of ganglion cells in the peripheral primate retina. Nature, 410: 933–936. McMahon, M.J., Lankheet, M.J.M., Lennie, P. and Williams, D.R. (2000) Fine structure of parvocellular receptive fields in the primate fovea revealed by laser interferometry. J. Neurosci., 20: 2043–2053. Merigan, W.H. (1989) Chromatic and achromatic vision of macaques: role of the P pathway. J. Neurosci., 9: 776–783. Mollon, J.D. and Jordan, G. (1988) Eine evolutiona¨re Interpretation des menschlichen Farbensehens. Die Farbe, 35: 139–170. Mollon, J.D. and Jordan, G. (1997) On the nature of unique hues. In: Dickinson, C., Murray, I. and Carden, D. (Eds.), John Dalton’s Colour Vision Legacy. Taylor & Francis, London, pp. 381–392. Mollon, J.D., Bowmaker, J.K. and Jacobs, G.H. (1984) Variations of colour vision in a New World primate can be explained by polymorphism of retinal photopigments. Proc. R. Soc. Lond. B, 222: 373–399. Mullen, K.T. and Kingdom, F.A.A. (1996) Losses in peripheral colour sensitivity predicted from ‘hit and miss’ postreceptoral cone connection. Vision Res., 36: 1995–2000. Nelson, R. (1977) Cat cones have rod input: a comparison of the response properties of cones and horizontal cell bodies in the retina of the cat. J. Comp. Neurol., 172: 109–135. Passaglia, C.L., Troy, J.B., Ru¨ttiger, L. and Lee, B.B. (2002) Orientation sensitivity of ganglion cells in primate retina. Vision Res., 42: 683–694. Peichl, L. and Wa¨ssle, H. (1979) Size, scatter and coverage of ganglion cell receptive field centres in the cat retina. J. Physiol. (Lond.), 291: 117–141. Perry, V.H. and Cowey, A. (1981) The morphological correlates of X- and Y-like retinal ganglion cells in the retina of monkeys. Exp. Brain Res., 43: 226–228. Perry, V.H. and Cowey, A. (1984) Retinal ganglion cells that project to the superior colliculus and pretectum in the macaque monkey. Neuroscience, 12: 1125–1137. Perry, V.H. and Silveira, L.C.L. (1988) Functional lamination in the ganglion cell layer of the macaque’s retina. Neuroscience, 25: 217–223. Perry, V.H., Oehler, R. and Cowey, A. (1984) Retinal ganglion cells that project to the dorsal lateral geniculate nucleus in the macaque monkey. Neuroscience, 12: 1101–1123. Pokorny, J. and Smith, V. (1997) Psychophysical signatures associated with magnocellular and parvocellular pathway contrast gain. J. Opt. Soc. Am. A, 14: 2477–2486. Polyak, S.L. (1941) The Vertebrate Retina. University of Chicago Press, Chicago.

Purpura, K., Kaplan, E. and Shapley, R.M. (1988) Background light and the contrast gain of primate P and M retinal ganglion cells. Proc. Natl. Acad. Sci. USA, 85: 4534–4537. Purpura, K., Kaplan, E., Tranchina, D. and Shapley, R.M. (1990) Light adaptation in the primate retina: analysis of changes in gain and dynamics of monkey retinal ganglion cells. Vis. Neurosci., 4: 75–93. Ramo´n y Cajal, S. (1904) Textura del Sistema nervioso del hombre y de los vertebrados. English translation (1995): N. Swanson and L.W. Swanson (Translators) Histology of the Nervous System of Man and Vertebrates. Oxford University Press, New York. Reese, B.E. (1996) The chronotopic reordering of optic axons. Perspect. Dev. Neurobiol., 3: 233–242. Reid, R.C. and Shapley, R.M. (1992) Spatial structure of cone inputs to receptive fields in primate lateral geniculate nucleus. Nature, 356: 716–718. Rodieck, R.W. and Watanabe, M. (1993) Survey of the morphology of macaque retinal ganglion cells that project to the pretectum, superior colliculus, and parvicellular laminae of the lateral geniculate nucleus. J. Comp. Neurol., 338: 289–303. Rodieck, R.W., Binmoeller, K.F. and Dineen, J. (1985) Parasol and midget ganglion cells of the human retina. J. Comp. Neurol., 233: 115–132. Rylands, A.B., Schneider, H., Langguth, A., Mittermeier, R.A., Groves, C.P. and Rodrı´ guez-Luna, E. (2000) An assessment of the diversity of New World Primates. Neotropical Primates, 8: 61–93. Saito, C.A., Lee, B.B., Kremers, J., Silveira, L.C.L., da Silva Filho, M. and Kilavik, B.E. (2001) Rod–cone interactions in the ganglion cell response: studies using the diurnal capuchinmonkey and the nocturnal owl-monkey. Invest. Ophthalmol. Vis. Sci., 42: S676. Schneeweis, D.M. and Schnapf, J.L. (1995) Photovoltage of rods and cones in the macaque retina. Science, 268: 1053–1056. Shapley, R.M. and Enroth-Cugell, C. (1984) Visual adaptation and retinal gain controls. Prog. Retinal Res., 3: 263–346. Shapley, R.M. and Hawken, M.J. (1999) Parallel retinocortical channels and luminance. In: Gegenfurtner, K.R. and Sharpe, L.T. (Eds.), Colour Vision—From Genes to Perception. Cambridge University Press, Cambridge, pp. 221–234. Shapley, R.M. and Perry, V.H. (1986) Cat and monkey ganglion cells and their visual functions roles. Trends Neurosci., 9: 229–235. Sharpe, L.T. and Stockman, A. (1999) Rod pathways: the importance of seeing nothing. Trends Neurosci., 22: 497–504. Silveira, L.C.L. (1996) Joint entropy loci of M and P cells: a hypothesis for parallel processing in the primate visual system. Braz. J. Biol., 56(1): 345–367. Silveira, L.C.L. and de Mello, Jr. H.D. (1998) Parallel pathways of the primate vision: sampling of the information in the Fourier space by M and P cells. In: Chalupa, L.M. and

45 Finlay, B.L. (Eds.), Development and Organization of the Retina: From Molecules to Function. Plenum Press, New York, pp. 173–199. Silveira, L.C.L., Picanc¸o-Diniz, C.W., Sampaio, L.F.S. and Oswaldo-Cruz, E. (1989) Retinal ganglion cell distribution in the Cebus monkey: a comparison with the cortical magnification factors. Vision Res., 29: 1471–1483. Silveira, L.C.L., Perry, V.H. and Yamada, E.S. (1993) The retinal ganglion cell distribution and the representation of the visual field in area 17 of the owl-monkey Aotus trivirgatus. Vis. Neurosci., 10: 887–897. Silveira, L.C.L., Yamada, E.S., Perry, V.H. and PicancoDiniz, C.W. (1994) M and P retinal ganglion cells of diurnal and nocturnal New World monkeys. NeuroReport, 5: 2077–2081. Silveira, L.C.L., Lee, B.B., Yamada, E.S., Kremers, J. and Hunt, D.M. (1998) Post-receptoral mechanisms of colour vision in new world primates. Vision Res., 38: 3329–3337. Silveira, L.C.L., Lee, B.B., Kremers, J., da Silva Filho, M., Saito, C.A. and Kilavik, B.E. (2000) Receptor inputs and temporal dynamics of owl monkey ganglion cells. Invest. Ophthalmol. Vis. Sci., 41: S937. Silveira, L.C.L., Yamada, E.S., Franco, E.C.S. and Finlay, B.L. (2001a) The specialisation of the owl monkey retina for night vision. Colour Res. Appl., 26: S118–S122. Silveira, L.C.L., dos Santos, S.N. and Kremers, J. (2001b) The presence of single-cone midget bipolar cells in the retina of the owl-monkey. Invest. Ophthalmol. Vis. Sci., 42: S375. Smith, V.C., Lee, B.B., Pokorny, J., Martin, P.R. and Valberg, A. (1992) Responses of macaque ganglion cells to the relative phase of heterochromatically modulated lights. J. Physiol. (Lond.), 458: 191–221. Solomon, S.G., White, A.J. and Martin, P.R. (1999) Temporal contrast sensitivity in the lateral geniculate nucleus of a New World monkey, the marmoset Callithrix jacchus. J. Physiol. (Lond.), 3, 517: 907–917. Solomon, S.G., Martin, P.R., White, A.J.R., Ru¨ttiger, L. and Lee, B.B. (2002) Modulation sensitivity of ganglion cells in peripheral retina of macaque. Vision Res., 42: 2893–2898. Soucy, E., Wang, Y., Nirenberg, S., Nathans, J. and Meister, M. (1998) A novel signaling pathway from rod photoreceptors to ganglion cells in mammalian retina. Neuron, 21: 481–493. Sun, H., Pokorny, J. and Smith, V.C. (2001) Rod–cone interactions assessed in inferred magnocellular and parvocellular postreceptoral pathways. J. Vision, 1: 42–54. Tan, Y. and Li, W.H. (1999) Trichromatic vision in prosimians. Nature, 402: 36. Usrey, W.M. and Reid, R.C. (2000) Visual physiology of the lateral geniculate nucleus in two species of New World monkey: Saimiri sciureus and Aotus trivirgatus. J. Physiol. (Lond.), 3, 523: 755–769.

Valberg, A., Lee, B.B. and Tigwell, D.A. (1986a) Neurones with strong inhibitory S-cone inputs in the macaque lateral geniculate nucleus. Vision Res., 26: 1061–1064. Valberg, A., Seim, T., Lee, B.B. and Tryti, J. (1986b) Reconstruction of equidistant colour space from responses of visual neurones of macaques. J. Opt. Soc. Am. A, 3: 1726–1734. Valberg, A., Lee, B.B., Kaiser, P.K. and Kremers, J. (1992) Responses of macaque ganglion cells to movement of chromatic borders. J. Physiol. (Lond.), 458: 579–602. van Hateren, J.H., Ru¨ttiger, L., Sun, H. and Lee, B.B. (2002) Processing of natural temporal stimuli by macaque retinal ganglion cells. J. Neurosci., 22: 9945–9960. Verweij, J., Dacey, D.M., Peterson, B.B. and Buck, S.L. (1999) Sensitivity and dynamics of rod signals in H1 horizontal cells of the macaque monkey retina. Vision Res., 39: 3662–3672. von Helmholtz, H. (1867) Handbuch der Physiologischen Optik. Voss, Hamburg and Leipzig. Wa¨ssle, H., Gru¨nert, U., Chun, M.H. and Boycott, B.B. (1995) The rod pathway of the macaque monkey retina: identification of AII-amacrine cells with antibodies against calretinin. J. Comp. Neurol., 361: 537–551. Watanabe, M. and Rodieck, R.W. (1989) Parasol and midget ganglion cells. J. Comp. Neurol., 289: 434–454. Weiss, S., Kremers, J. and Maurer, J. (1998) Interaction between rod and cone signals in responses of lateral geniculate neurons in dichromatic marmosets (Callithrix jacchus). Vis. Neurosci., 15: 931–943. Wikler, K.C. and Rakic, P. (1990) Distribution of photoreceptor subtypes in the retina of diurnal and nocturnal primates. J. Neurosci., 10: 3390–3401. Wiesel, T.N. and Hubel, D.H. (1966) Spatial and chromatic interactions in the lateral geniculate body of the rhesus monkey. J. Neurophysiol., 29: 1115–1156. Xu, X., Ichida, J.M., Allison, J.D., Boyd, J.D., Bonds, A.B. and Casagrande, V.A. (2001) A comparison of koniocellular, magnocellular and parvocellular receptive field properties in the lateral geniculate nucleus of the owl monkey (Aotus trivirgatus). J. Physiol. (Lond.), 531(1): 203–218. Yamada, E.S., Silveira, L.C.L. and Perry, V.H. (1996a) Morphology, dendritic field size, somal size, density and coverage of M and P retinal ganglion cells of dichromatic Cebus monkeys. Vis. Neurosci., 13: 1011–1029. Yamada, E.S., Silveira, L.C.L., Gomes, F.L. and Lee, B.B. (1996b) The retinal ganglion cell classes of New World primates. Braz. J. Biol., 56(1): 381–396. Yamada, E.S., Marshak, D.W., Silveira, L.C.L. and Casagrande, V.A. (1998) Morphology of P and M retinal ganglion cells of the bush baby. Vision Res., 38: 3345–3352. Yamada, E.S., Silveira, L.C.L., Perry, V.H. and Franco, E.C.S. (2001) Morphology and dendritic field size of M and P retinal ganglion cells of the owl monkey. Vision Res., 41: 119–131.

46 Yau, K.W. (1994) Phototransduction mechanisms in retinal rods and cones. Invest. Ophthalmol. Vis. Sci., 35: 9–32. Yeh, T., Lee, B.B., Kremers, J., Cowing, J.A., Hunt, D.M., Martin, P.R. and Troy, J.B. (1995) Visual responses in the

lateral geniculate nucleus of dichromatic and trichromatic marmosets (Callithrix jacchus). J. Neurosci., 15: 7892–7904. Young, T. (1802) The Bakerian lecture: On the theory of light and colours. Phil. Trans. R. Soc. Lond., 92: 12–48.

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 3

Identifying corollary discharges for movement in the primate brain Robert H. Wurtz* and Marc A. Sommer Laboratory of Sensorimotor Research, National Eye Institute, National Institutes of Health, Bethesda, MD 20892-4435, USA

Abstract: The brain keeps track of the movements it makes so as to process sensory input accurately and coordinate complex movements gracefully. In this chapter we review the brain’s strategies for keeping track of fast, saccadic eye movements. One way it does this is by monitoring copies of saccadic motor commands, or corollary discharges. It has been difficult to identify corollary discharge signals in the primate brain, although in some studies the influence of corollary discharge, for example on visual processing, has been found. We propose four criteria for identifying corollary discharge signals in primate brain based on our experiences studying a pathway from superior colliculus, in the brainstem, through mediodorsal thalamus to frontal eye field, in the prefrontal cortex. First, the signals must originate from a brain structure involved in generating movements. Second, they must begin just prior to movements and represent spatial attributes of the movements. Third, eliminating the signals should not impair movements in simple tasks not requiring corollary discharge. Fourth, eliminating the signals should, however, disrupt movements in tasks that require corollary discharge, such as a double-step task in which the monkey must keep track of one saccade in order to correctly generate another. Applying these criteria to the pathway from superior colliculus to frontal eye field, we concluded that it does indeed convey corollary discharge signals. The extent to which cerebral cortex actually uses these signals, particularly in the realm of sensory perception, remains unknown pending further studies. Moreover, many other ascending pathways from brainstem to cortex remain to be explored in behaving monkeys, and some of these, too, may carry corollary discharge signals.

Introduction

to keep track of movements as they are generated and predict the sensations that will result from them. The second challenge is in the motor domain. As behaviors become more elaborate, the need for internal information about movements becomes more critical. During quick, complex motor sequences such as those produced while fighting a competitor, information about prior actions helps to generate appropriate future ones. For both sensory perception and motor production, therefore, nervous systems need to keep track of the movements they generate. In this chapter, we consider how the brain might monitor movement information in the primate visual-oculomotor system. We review studies exploring how visual input from the world is distinguished from visual input caused by eye movements, and how primates keep track of

Generating movements is a key to survival for animals. Food gathering, escape from predators, and reproduction all involve coordinated movements. Generating movements, however, presents two major challenges to the nervous system. The first is in the sensory domain. Many movements cause sensory input identical to that elicited by external events, and consequently animals must be able to distinguish whether they, or another entity, caused the sensory input. A valuable aid in making this distinction is *Corresponding author. Building 49, Room 2A50, MSC 4435, NEI, NIH, 9000 Rockville Pike, Bethesda, MD 20892-4435, USA. Tel.: þ1-301-496-7170; Fax: þ1-301-402-0511; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14400-3

47

48

their eye movements while they look around rapidly. Based on experience from our own laboratory we also propose criteria for identifying internal records of movement within the primate brain.

Sources of knowledge about self-movement Among the most common movements made by primates are eye movements, and how these movements are internally monitored has been the focus of speculation for centuries and quantitative study for decades (Bridgeman, 1995a; Gru¨sser, 1995; Colby and Goldberg, 1999). As a primate makes rapid or saccadic eye movements to explore the visual scene, the apparent motion of objects in the scene is an artefact of the saccadic eye movement and is not due to actual object movements. How does the brain distinguish this self-induced, illusory object motion from real motion? As might be expected from a biological system that undoubtedly resulted from eons of evolution, there are multiple mechanisms for making this potentially life and death distinction. One useful clue is contained in the visual signal from the retinas: when the eyes move the whole visual field moves, whereas when a visual object moves it moves alone. Full-field motion, often referred to as optic flow (Fig. 1A), is a reasonable indicator of eye movement as long as the head and body remain stationary. Optic flow so frequently indicates selfmotion that it provides critical information about the heading taken by an animal as it proceeds through its environment (Warren and Hannon, 1990; Wurtz and Duffy, 1997; Duffy, 2000). This clue to self-motion, however, requires a lighted, contoured environment that of course is not always present. A second clue comes from proprioceptors in the eye muscles (Fig. 1A). As the eyes move, proprioceptive input may report eye-muscle contraction to the brain, providing information that apparent visual motion is due to eye movements. The role of the proprioceptors has been investigated for many years (Ruskell, 1999; Donaldson, 2000) and yet their exact contribution remains to be determined. There is growing evidence, however, that the major contribution of proprioception is in long-term calibration of the eye-movement system rather than in monitoring

Fig. 1. The three major sources of information about one’s own eye movements. (A) At left, a source of retinal information is indicated: optic flow, or full-field visual motion caused by a saccade. At right, two sources of extraretinal information are diagrammed. Proprioception, or input to the brain from receptors in the eye muscles, and corollary discharge, a signal within the brain representing the movement command, both accompany a saccade. (B) Time course of the three sources of information. Corollary discharge signals can occur before, during, and after a saccade. Proprioception and optic-flow signals, however, are available only after a saccade, following afferent delays from periphery to the brain.

movements on a saccade-by-saccade basis (Keller and Robinson, 1971; Guthrie et al., 1983; Lewis et al., 2001). These two sources of information are sensory in nature, arising peripherally in either the retinas or the proprioceptors. They provide clues about eye movements through afferent inputs to the brain. A third source of information is from within the brain itself (Fig. 1A), and we refer to it as a corollary discharge. This is also known as an efference copy; for a discussion of the nomenclature, see Bell (1984). A corollary discharge for movement is just that: it is a corollary signal sent to other regions of the brain at the same time that the signal is sent on the pathway

49

to activate the muscles to generate the movement. The corollary logically could be from any level of the circuit within the brain generating the movement, including the final common path to the eye muscles. The advantages of corollary discharges are that they are generated within the brain itself, making them impervious to disruptions of the peripheral receptors, and that they are available even before the movement begins, whereas sensory information is available only afterward (Fig. 1B). The specific idea of a corollary discharge evolved from the 18th century onward (McCloskey, 1981; Bridgeman, 1995b; Gru¨sser, 1995), culminating in Hermann von Helmholtz’s 19th century reference to an ‘effort of will’ as the mechanism compensating for the spurious visual motion caused by one’s own eye movements. The most influential papers of the 20th century were published by Sperry (1950) and by von Holst and Mittelstaedt (1950), who examined the behaviors of fish and flies, respectively, after ocular rotation/inversion. In both preparations the animals’ abnormal behaviors could be explained most easily by postulating that internal copies of motor commands were monitored by the nervous system. Since then the concept of corollary discharge has been invoked to help explain a wide range of animal behaviors, such as electrolocation in fish (Bell, 1984), song learning in birds (Troyer and Doupe, 2000), and chirping in crickets (Poulet and Hedwig, 2002). In all these behaviors the animals must distinguish the sensory consequences of their own actions from environmentally produced sensations. Psychophysical and lesion studies have demonstrated that corollary discharge signals exist in humans (McCloskey, 1981; Skavenski, 1990; Haarmeier et al., 1997; Thier et al., 2001; Pierrot-Deseilligny et al., 2002). Much current work on human motor control is focusing on how the generation of limb movements, especially during motor learning, relies on corollary discharge signals (or ‘forward internal models’; Jordan and Rumelhart, 1992; Frith et al., 2000; Wolpert and Ghahramani, 2000). In principle, neurophysiologists can take at least two approaches to demonstrating the existence of corollary discharge signals in neurons of any sensorimotor system (Fig. 2). The first approach is to identify the effect of the corollary discharge on a neuron’s sensory responses. The second is to identify

Fig. 2. Two ways of detecting corollary discharge in the visualoculomotor system. Experimenters usually detect corollary discharges indirectly by demonstrating otherwise inexplicable changes in sensory processing (right). For example, a modified visual signal, such as a visual response that changes just prior to saccade initiation, may suggest that a corollary discharge is present. The more direct approach is to identify the corollary discharge itself (left). To do this, one must establish criteria for determining whether movement-related neuronal activity (as in the example shown with rasters and a spike density function) is a corollary discharge or a movement command. The corollary discharge would interact at a later stage with visual input to produce a modified visual signal. Many types of interactions are possible (MacKay, 1966; Bell, 1984).

the corollary itself, but this raises the question of how to distinguish a corollary discharge signal from a movement command. We consider both of these approaches in turn as they have been applied in the monkey visual-oculomotor system.

Searching for the influence of corollary discharge on visual processing The classical approach to studying corollary discharge in the primate visual-oculomotor system has been to search not for the corollary itself but instead for the impact of the corollary on visual processing. The logical first place to look for the effect of a corollary discharge was in primary visual cortex, which receives input from the retinas via the lateral geniculate nuclei. The principle was to compare neuronal activity evoked by motion of an object (with the eyes still) with activity evoked by movement of the eyes (with the object still). If the neuron responded differently to the nearly identical object

50

motion on the retina in the two conditions, then the neuron had to be receiving information that an eye movement was occurring. This meant the neuron’s activity was influenced by corollary discharge signals. This experiment was performed in the awake, trained monkey (Wurtz, 1968), and in fact was the very first recording of visual neurons achieved in an awake, trained monkey. No clear difference was detected in the two conditions, indicating that corollary discharge signals probably have little influence on processing in primary visual cortex. There was, however, evidence that the presence or absence of a visual background influenced the neuronal responses, emphasizing that other lines of information such as optic flow (Fig. 1A) can provide clues as to the cause of visual motion. A corollary discharge associated with pursuit movements also has been sought in primary visual cortex, but none has been found (Ilg and Thier, 1996). Subsequent studies on saccades have reported slight effects of corollary discharge on primary visual-cortex neurons (Bridgeman, 1973; Galletti et al., 1984). These latter results might indicate true corollary discharge influences, but they also may be due instead to significant differences in the motion produced by the saccade versus the stimulus movement generated by the experimenter. Primary visual cortex is not the only recipient of visual signals from the retina in primates; the retina also projects directly to the superficial layers of the superior colliculus (SC), a structure on the roof of the midbrain. Neurons in the SC superficial layers respond to visual stimuli and do not increase their activity before eye movements (in contrast to neurons just below them in the SC intermediate layers that discharge in tight correlation with saccades; Schiller and Koerner, 1971; Wurtz and Goldberg, 1971; Sparks and Hartwich-Young, 1989). The same test for the presence of a corollary discharge was done on these SC superficial-layer neurons as on the primary visual-cortical neurons, but the outcome was substantially different. In contrast to the results in primary visual cortex, many SC superficial-layer neurons showed strong differences in their responses to moving visual stimuli (Robinson and Wurtz, 1976a) depending upon whether the motion was caused by visual stimulus motion with the eyes stationary (Fig. 3A, left) or by a saccade with the visual stimulus stationary (Fig. 3A,

Fig. 3. Identifying the effects of corollary discharge on visual neurons of the SC superficial layers. (A) Example of an SC neuron that may have been influenced by corollary discharge signals. The neuron showed a clear visual response when a spot of light moved across its receptive field while the eye was stationary (see the rasters and histogram of neuronal activity, left panel), but it did not respond when a saccade moved the receptive field across a stationary stimulus at about equal speed (right panel). In fact, background activity was actually suppressed when the eye moved. This was evidence that there was an extraretinal input (corollary discharge or proprioception) to these neurons. H, V, horizontal and vertical components of the eye position; Sp/s, spikes per second. From Robinson and Wurtz (1976b). (B) Demonstration that the effect is due to corollary discharge. The saccade-related suppression of background activity of an SC superficial-layer neuron (left panel) continued when the monkey attempted to move its eyes even though a retrobulbar block prevented movement (middle panel). Because the eye muscles did not contract, there were no proprioceptive signals. The attempted eye movement was indicated by an increase in activity from integrated multiple neuron activity recorded from the oculomotor nucleus (Oc. Nuc.). Activity after the block recovered is shown in the right panel. E.O.G., electrooculogram. From Richmond and Wurtz (1980). Only a few of the rasters contributing to the histograms are shown. The rasters were retouched to compensate for faint dots resulting from digitization.

right). The motion resulting from saccades frequently did not produce the usual increase of activity at all, but instead produced suppression in the background activity.

51

This difference in visual responses to eye- versus stimulus-generated motion was necessary, but not sufficient, to demonstrate the influence of corollary discharges. There was still the possibility that the effects were due entirely to proprioceptive input. Therefore, a further test was done to determine whether the suppression of activity accompanying the saccade persisted in the absence of proprioception (Richmond and Wurtz, 1980). Proprioception was eliminated by stopping the movement of the eye by numbing eye muscles with xylocaine. During the block the monkey attempted in vain to move its eyes, as indicated by bursts of activity recorded from the oculomotor nucleus, and corollary discharge signals still should have been generated accordingly. The suppression persisted (Fig. 3B, middle), so it must have been dependent upon corollary discharge. This experiment probably provides the best evidence in the primate visual-oculomotor system for the action of a corollary discharge on early visual processing. The inverse experiment was not done (eliminate the corollary discharge and keep the proprioception), so the possibility remains that proprioception may contribute to some extent; however, corollary discharge alone was sufficient to explain the effect. We noted above that corollary discharge has little, if any, influence on activity in primary visual cortex. However, it does seem to exert an effect later in the visual stream. For example, visual receptive fields of many cerebral cortical neurons suddenly shift to new locations just prior to a saccade; the new locations are those where the receptive field would be just after the saccade (Duhamel et al., 1992a; Colby and Goldberg, 1999). This predictive remapping must use corollary discharge information because it occurs before the eye actually moves. This effect has been seen in the frontal eye field (FEF) of prefrontal cortex (Umeno and Goldberg, 1997) and seems to diminish gradually in extrastriate cortex as one approaches primary visual cortex (Nakamura and Colby, 2002). Recently, a difference between stimulus- and eyeproduced motion was found for neurons in extrastriate cortical area MT of the monkey (Thiele et al., 2002). This demonstrates the presence of an extraretinal input that may be a corollary discharge, although influences of proprioception were not explicitly ruled out.

Identifying the corollary discharge itself Demonstration of an effect of corollary discharge has been accomplished many times, not only in the monkey with respect to the modification of visual processing, but also in a large number of vertebrate and invertebrate species. In contrast, the identification of the corollary discharge signal itself has been attempted in only a few studies, among them investigations of the corollary discharge of weak electric signals (the generation of which involve a muscle-like organ) in mormyrid fish (Bell, 1984) and of the corollary discharge of leg movements in cockroach and cricket (Delcomyn, 1977; Poulet and Hedwig, 2002). A critical issue in such experiments is to differentiate the signal that is the corollary from that which is the movement command (Fig. 2, left). For example, in monkeys the saccade-related discharges of SC intermediate-layer neurons could logically be either movement commands or corollaries of the commands. Certainly many are movement commands because low-threshold electrical stimulation or reversible inactivation of the SC intermediate layers elicits or impairs saccade generation, respectively (Robinson, 1972; Hikosaka and Wurtz, 1985). Whether some of the saccade-related discharges in SC are actually corollary discharge signals, however, has been unknown. In our own attempts to investigate corollary discharge signals we developed a list of criteria for identifying them within the complex circuits of the primate brain (Table 1). First, putative corollary discharges should originate from a brain structure known to be involved in the generation of the movement as indicated by changes in activity preceding the movement and alterations in the movement resulting from activating or inactivating the structure. Second, the signals should occur just prior to the movement and represent spatial

Table 1. Criteria for identifying corollary discharges 1. The signals originate in a motor area 2. The signals precede and spatially represent the movement 3. Eliminating the signals does not impair movements in tasks not requiring corollary discharge 4. Eliminating the signals does impair movements in tasks requiring corollary discharge

52

parameters of the movement. Third, eliminating the signals should not impair movements in simple tasks not requiring corollary discharge. Fourth, eliminating the signals should, however, disrupt the performance of tasks that require corollary discharge. While we think these criteria should apply to the identification of corollary discharge in systems other than the visual oculomotor and in animals other than the monkey, we make no pretense that these are the only criteria that could be used. Using these criteria, we considered whether neurons in a pathway from SC up to frontal cortex could be regarded as conveying corollary discharges for saccades, as will be discussed next.

Criterion 1: The signals originate from a motor area We investigated a pathway suspected on anatomical grounds to run from a clearly established brainstemoculomotor region up to the cerebral cortex. It was thought to originate from SC intermediate-layer neurons that project to relay neurons in the mediodorsal nucleus of the thalamus (MD) that in turn project to the FEF (Fig. 4A). Evidence for the existence of this pathway came from retrogradelabeling and anterograde-degeneration studies (Benevento and Fallon, 1975; Goldman-Rakic and Porrino, 1985) taken together with a transynaptic retrograde-labeling study using herpes simplex virus (Lynch et al., 1994). To confirm that this pathway existed and was functional, we first attempted to identify and record from MD relay neurons. The activity of thalamic neurons in and around MD during visuosaccadic behavior had been studied only once before in the monkey (Schlag and Schlag-Rey, 1984; Schlag-Rey and Schlag, 1984). While finding MD neurons in the awake monkey is itself an experimental challenge, identifying the small subset of MD neurons that relay signals from SC to FEF would seem even harder. There are, however, electrophysiological methods for identifying MD neurons that project to FEF and receive SC input, namely, antidromic and orthodromic stimulation techniques (Fig. 4B) that we described in detail previously (Sommer and Wurtz, 1998, 2002). Using these techniques, we identified 51 neurons in two monkeys that were clearly MD relay

Fig. 4. Technique for satisfying Criterion 1, ensuring that the signals under study originate in a motor area. (A) Anatomical studies indicated that some neurons in the SC intermediate layers project to mediodorsal thalamus (MD), onto relay neurons that in turn project to the frontal eye field (FEF). The SC intermediate layers also send commands that ultimately cause saccade generation down to the brainstem saccade-generating circuits. Arrows indicate direction of signal flow. (B) Method used to identify the neurons in MD that both receive input from SC and project to FEF. Every MD relay neuron was double-identified: it was both antidromically activated from the FEF (showing that it projected to FEF) and orthodromically activated from the SC (showing that it received input from the SC). Arrows show direction of action potential propagation from the stimulating electrodes.

neurons, in that each one was both antidromically activated from the FEF and orthodromically activated from the SC (Sommer and Wurtz, 2002). They may project additionally to frontal cortical areas other than FEF and they may receive other inputs besides that from the SC, but all of them

53

at least were positively identified as relay neurons between SC and FEF. After studying the MD relay neurons we then examined the SC neurons that projected up to them (Wurtz and Sommer, 2000). This was done by looking for SC neurons antidromically activated from the locations of previously recorded MD relay neurons. We also identified FEF neurons that seemed to receive the signals flowing in this ascending pathway (Sommer and Wurtz, 1998). This was done by searching for FEF neurons orthodromically activated from the SC. In sum, we recorded from identified neurons all along a pathway originating in the SC, a structure crucial for generating saccades.

Criterion 2: The signals precede and spatially represent the movements

Fig. 5. Evidence satisfying Criterion 2, showing that signals in the pathway precede and spatially represent the movements. (A) Presaccadic bursts of activity recorded from MD relay neurons. Once an MD relay neuron was isolated it was studied by having the monkey perform a delayed saccade task. The monkey looked at a fixation spot, then a target (Visual Stim.) appeared in the periphery, and after a delay period of 500–1000 ms the fixation spot disappeared (Cue to Move), which was the cue to start the eye movement (Saccade) and look at the target. Shown are examples of two major types of MD relay neurons, Visuomovement and Movement Neurons. Neurons of both types had bursts of activity beginning just prior to the saccade. The pie chart shows the percentage of each neuron type in our sample of MD relay neurons (VM, Visuomovement Neurons; M, Movement Neurons; ‘Others’ include neurons with only visual responses and those with neither visual or saccadic activity). Presaccadic bursts of activity were present in 74% of

For brevity we will focus on the MD relay neurons, which represents the crucial node in the pathway. We studied their activity while monkeys made delayed saccades to visual targets (Sommer and Wurtz, 2002) and found that most of them increased their activity just before the saccade (see Fig. 5A). Of 46 neurons tested, 57% were visuomovement neurons (having both a presaccadic burst and a visual response) and 17% were movement neurons (having a presaccadic burst but no visual response). In net, 74% of the neurons increased their activity before the saccade, on average starting their saccade-related burst 66 ms prior to the onset of movement. Note that this presaccadic initiation meant that the activity could not have resulted from proprioceptive input from eyemuscle contraction. We examined the relationship

the neurons (MþVM neurons), as indicated by the bold outline. (B) Representation of saccadic vectors by MD relay neurons. The movement field (gray oval) of an example neuron is shown at left. The neuron exhibited presaccadic bursts of activity only for saccadic vectors made from the origin into this field. The saccadic vector encoded by the peak firing of the neuron (bold arrow) was directed 27
up from horizontal and was 16
in amplitude. This vector was determined by having the monkey make various directions and amplitudes of saccades (right) and fitting the presaccadic firing rate data with Gaussians and spline curves (solid curves), respectively. Dashed lines show mean baseline activity, and dotted lines show 2 SDs above that, which was the criterion level for significance. Ipsi, ipsilateral space; Contra, contralateral space.

54

between the saccadic activity and the saccadic vector for 29 of the neurons, and 23 of them (79%) had distinct peaks in their movement fields, firing strongest for saccades of a certain amplitude and direction (Fig. 5B). For all tuned neurons the best direction was into the contralateral visual field. Many MD relay neurons, therefore, have activity preceding the saccade and representing the spatial aspects of the saccade. Incidentally, nearly identical results were found for saccade-related bursts of SC neurons projecting up to the MD, consistent with our assumption that the MD relay neurons were driven in large part, if not completely, by SC neurons. These ascending saccadic bursts are excellent candidates to be corollaries of motor commands, because they are qualitatively similar to saccadic bursts exhibited by the general population of SC neurons (Sparks and Hartwich-Young, 1989) and in particular by those SC neurons identified as projecting downstream to saccadic-generating circuits (Guitton and Munoz, 1991; Munoz and Guitton, 1991; Munoz et al., 1991).

Criterion 3: Eliminating the signals does not impair movements in a simple task not requiring corollary discharge At this point we know that signals related to impending saccades are sent from SC up to FEF. But might these signals actually be causing saccade generation through some loop involving cerebral cortex and brainstem? To answer this question we capitalized on the presence of the MD relay neurons in the ascending pathway — an experimental gift to the physiologist. By inactivating them we could specifically interrupt transmission from SC to FEF. [Directly inactivating the SC or FEF, instead, would have caused extensive unwanted effects due to perturbing the myriad other networks involving these structures, including the descending, motordedicated pathways to the pons; we already know that inactivating either SC or FEF impairs saccade generation itself (Hikosaka and Wurtz, 1985; Sommer and Tehovnik, 1997; Dias and Segraves, 1999).] We inactivated the MD relay neurons using muscimol, a GABAA agonist. Muscimol inhibits neuron cell bodies, not axons (Lomber, 1999), so it

should suppress MD relay neurons without affecting transthalamic fibers passing nearby. While MD relay neurons were inactivated, we had monkeys make single saccades to visual or remembered targets at several eccentricities and directions. Making a single saccade does not require corollary discharge information. Thus if the ascending pathway’s saccade-related signals are corollary discharges, single saccades should not be affected by MD inactivation; however, if the signals instead are needed for making saccades, then single saccades should be impaired by MD inactivation. Figure 6A (left) shows the average trajectories of saccades made to targets at 10
eccentricity and eight directions, before versus during inactivation of MD neurons in one experiment. The monkey still made saccades, and quantification showed that the accuracy and latency of these saccades was not altered by inactivation (Sommer and Wurtz, 2002). Throughout a series of like experiments, significant changes in the accuracy and latency of single saccades were infrequent and small. To examine saccadic dynamics we plotted peak speed as a function of amplitude (referred to as the main sequence, Fig. 6A, right). There were no clear impairments during inactivation; the logarithmic fits of the values before and during the injection were not significantly different. The significance of this lack of effect during MD inactivation is brought into sharper perspective by considering previous experiments in which the SC was inactivated with muscimol. Figure 6B (left) shows that during an example of SC inactivation, saccades made to the upper right quadrant were shortened and their trajectories altered. In addition, SC inactivation markedly slowed saccades (Fig. 6B, right). Similar effects have been reported for FEF inactivation (not shown; Sommer and Tehovnik, 1997; Dias and Segraves, 1999). Thus, eliminating the saccade-related signals coursing through MD does not eliminate, or even significantly affect, the generation of single saccades in simple tasks. This supports the idea that these signals provide information about saccades but are not critical for generating them. This is in contrast to inactivation of SC or FEF, which can severely impair saccade generation presumably by shutting off descending efferents to brainstem saccade-generating circuits.

55

Fig. 6. Evidence satisfying Criterion 3, showing that eliminating the signals does not impair saccades made in a simple task not requiring corollary discharge. (A) Results of inactivating the MD relay neurons while monkeys made single saccades to visual targets. Left, average trajectories of saccades made in one experiment, before versus during the inactivation. Saccades traveled from the center of the screen to each of eight targets at 10
eccentricity. Inactivation did not significantly impair saccades in any direction. Right, graphs summarizing the dynamics of contraversive single saccades. The curves show logarithmic fits. (From data presented in Sommer and Wurtz, 2002.) (B) Analogous saccade data from an experiment in which the SC was inactivated (Hikosaka and Wurtz, 1985; Aizawa and Wurtz, 1998).

Criterion 4: Eliminating the signals disrupts movements in a task requiring corollary discharge Many tasks can be imagined that require corollary discharge for their execution, for example tasks that require distinguishing sensations caused by selfmovement as opposed to external forces or tasks that require generation of fast, complex motor acts. The task we used was the double-step task, in which the monkey had to make successive saccades to two flashed targets (Fig. 7A, left). We selected this task because it is widely used as an assay for the presence

of corollary discharge, particularly in patients with cortical lesions (Duhamel et al., 1992b). Correct execution of the second saccade (the upward saccade) requires knowledge of where the eye lands after the first (horizontal) saccade. Visual feedback indicating where the eye is after the first saccade is not available because the saccades begin after the targets disappear; additionally, the experimental room is usually in total darkness. Proprioception probably does not contribute to successful performance, because it likely has little influence in the online control of saccades (Lewis et al., 2001) and has been

56

Fig. 7. Evidence satisfying Criterion 4, showing that eliminating the signals does impair saccades made in a task requiring corollary discharge. (A) We used the double-step task, which requires corollary discharge for correct performance. Left, the monkey first looked at a fixation spot (shown in gray, center of screen), which then disappeared as two targets were flashed sequentially (shown in white, T1 and T2). The monkey then made two saccades (arrows) to the target locations. Due to the reaction time of the saccades, all stimuli were gone before the saccades started. With corollary discharge intact, the first saccade would go rightward and the second saccade would go straight up from there. Right, without corollary discharge, the first saccade would go rightward but there would be no internal record of this. Hence the monkey would not know that its eyes are at a new position, and to complete the trial it would be expected to make the second saccade as if it were still looking at the center of the screen, i.e. the second saccade should travel diagonally (dashed arrow). Since the first saccade was in fact made correctly, however, the pattern of saccades should be as shown with the solid arrows. (B) Left, individual saccadic sequences from an example MD inactivation, before (top) and during (bottom) inactivation. Right, means (and SDs) of the initial fixation locations, first-saccade endpoints, and second-saccade endpoints for the same example. The only significant change was that predicted by loss of corollary discharge: there was a shift of second-saccade endpoints in the contraversive (rightward) direction. From Sommer and Wurtz (2002). S1, first saccade; S2, second saccade; n.s.d., not significantly different.

shown to be unnecessary for double-step task (Guthrie et couraged memorization or sequences by randomizing a

performing a similar al., 1983). We dispreplanning of the variety of sequences

across trials and changing the sequences between experiments. The indicator of a loss of corollary discharge in this task is specific and quantifiable. If inactivation

57

totally eliminates corollary discharge (Fig. 7A, right), the monkey should make a contraversive first saccade correctly but should not have internal information that it did so. Therefore, if the monkey tries to complete the trial by looking at the second target, it should make a second saccade as if it never made the first, i.e. as if the eyes were still looking at the fixation point; hence the second saccade should be made diagonally (Fig. 7A, right, dashed arrow). But since the first saccade actually was correct (the monkey just did not know this), the second saccade will begin at the endpoint of the first target and will land to the right of the second target location (Fig. 7A, right, diagonal arrow). The indicator of lost corollary discharge therefore would be a shift of secondsaccade endpoints contraversively (in this example, rightward) during inactivation. No vertical shifts should occur, however, nor any changes in the initial fixation locations or first-saccade endpoints. Figure 7B shows the results from one injection of muscimol into MD (Sommer and Wurtz, 2002). Before inactivation the monkey made saccadic sequences correctly. Because the saccades were made in total darkness, first-saccades were shifted upward slightly (Gnadt et al., 1991). Second saccades went nearly straight up, indicating that the corollary discharge was intact. Following inactivation of MD, the second-saccade endpoints shifted contraversively (to the right) as expected if the corollary discharge was impaired. Quantitatively the second-saccade endpoints were shifted 2.5
to the right ( P1 Hz) frequencies. Thus, the basic gain of the VOR was not stored in the flocculus itself but in the brainstem (Luebke and Robinson, 1994; McElligott et al., 1998; Rambold et al., 2002). The second component was a leaky integrator with time constant 0.5 s, to be consistent with the observation that after cerebellar inactivation the time constant of postsaccadic drift is longer than that obtained for the plant alone (Carpenter, 1972; Robinson, 1974; Zee et al., 1981; Godaux and Vanderkelen, 1984). The performance of the brainstem controller is shown in Fig. 5. The retinal slip found in response to the training stimulus (head-velocity signals with a mixture of frequencies) shows good compensation at high frequencies (Fig. 5A), and indeed the gain of the system above about 1 Hz is close to one (Fig. 5B). After a velocity-pulse input, eye position relaxes back to the primary position with a time constant of about 1 s (Fig. 5C). Finally, because the brainstem controller is insufficient on its own to produce accurate motor commands, there are indeed correlations between components of the motor command and the subsequent sensory error, namely retinal slip (Fig. 5D).

Results of decorrelation control The effects of training the system just described with the decorrelation-control algorithm are shown in Fig. 6. Retinal slip declined rapidly at first, then more slowly (Fig. 6A), and was still continuing to decline at the end of 1000 trials of training (each trial ¼ 5 s of colored noise head-velocity input). At this point the remaining slip was very slight (Fig. 6B), and the ability of the system to hold eccentric gaze after a velocity pulse was almost perfect (Fig. 6C). Finally, the correlations between motor-command components and sensory error had almost completely disappeared (Fig. 6D). These findings demonstrate that the decorrelationcontrol algorithm is capable of learning accurate velocity commands, and thus compensating for the oculomotor plant, with the particular modeling assumptions outlined in the section on model

68

Fig. 5. Performance of the model before training, with a first-order plant P (time constant ¼ 0.2 s). The brainstem controller B was a leaky integrator with time constant 0.5 s and accurate high-frequency gain. (A) Head velocity and retinal slip. The colored-noise headvelocity signal (root-mean-square amplitude 1 /s) produced a relatively smooth retinal slip signal. (B) The reason for the smoothing is evident from the Bode plot of VOR gain against frequency of head velocity. For frequencies above about 1 Hz the VOR gain is close to 1.0, because of the properties of the brainstem controller. (C) Eye-position response of system to a head-velocity pulse (equivalent to head-position step, and similar to a saccadic eye-movement command). The eye position returns to its initial value with a time course determined by the characteristics of both the plant and the brainstem controller. (D) The correlations present between delayed versions of the eye-movement command and retinal slip, measured over a period of 500 s (modified from Dean et al., 2002).

structure. The next test for the algorithm is whether it is robust, that is to say whether it can still cope when those assumptions are relaxed. The following assumptions were investigated. (i) There are still uncertainties about the precise characteristics of the brainstem controller B (De Zeeuw et al., 1995). We tested the extreme case of having no brainstem controller at all (i.e. B set to a gain of 1) Although learning was slow, eventual convergence was good and the asymptotic performance for both retinal slip and eccentric gaze resembled that shown in Fig. 6. Thus, the success of the decorrelationcontrol algorithm does not depend on the precise characteristics of the brainstem controller. (ii) The first-order plant used above is the simplest dynamical system possible. What happens

when decorrelation control is confronted with a more realistic model plant? We approached this question in two ways. First, we replaced the single-element plant of Fig. 3 with a two-element model (details in Appendix), of the kind suggested by behavioral and electrophysiological data (Optican and Miles, 1985; Optican et al., 1986; Fuchs et al., 1988; Stahl, 1992; Goldstein and Reinecke, 1994; Goldstein et al., 2000). This plant shows substantially more complex behavior and requires more sophisticated control, including a ‘slide’ of innervation after a velocity pulse (Optican and Miles, 1985; Goldstein and Reinecke, 1994; Goldstein et al., 2000). Nonetheless, the decorrelation-control algorithm was able to learn to compensate a two-element plant (Fig. 7, details in legend). Secondly, the learning

69

Fig. 6. Performance of model during and after training, with a first-order plant P (time constant ¼ 0.2 s) and a brainstem controller B with a leaky integrator (time constant 0.5 s) and accurate high-frequency gain. (A) Typical decline in retinal-slip amplitude with training. Root-mean-square retinal-slip amplitude, measured over a 5-s training trial as shown in Fig. 4A, plotted on a log scale against number of training trials. (B) Posttraining reduction in retinal slip (note change in scale from Fig. 4A). (C) Eye-position response of system to a head-velocity pulse. The resultant eccentric eye position is maintained. (D) The pretraining correlations between delayed versions of the eye-movement command and retinal slip have almost disappeared (modified from Dean et al., 2002).

properties of the configuration shown in Fig. 4 were analyzed mathematically (Porrill et al., 2003). The analysis revealed that the synaptic weights become more accurate as long as output errors are being made. Thus, the algorithm is guaranteed to learn to compensate for any plant (subject to certain technical limitations). The crucial point is that the system operates in ‘feedback’ mode, i.e. a copy of the motor command is fed back to the cerebellum. This general result is important, not least for the specific case of oculomotor plant compensation where a variety of data suggest that the oculomotor plant may contain at least three viscoelastic elements (Robinson, 1965; Sklavos et al., 2002). The mathematical analysis indicates that the decorrelation-control algorithm is capable of compensating for these more complex plants.

(iii) Concerns have been expressed about the capacity of the climbing-fiber pathway to convey detailed information because the maximum firing rate of an individual fiber is rather low, that is about 10 Hz. However, when the decorrelation-control algorithm was tested with a climbing-fiber signal that conveyed only the direction of retinal-slip (not its magnitude) learning was still similar to that illustrated in Fig. 6. The main difference was that final performance needed to be improved slightly by reducing the learning rate ( in Eq. 1) near to convergence. (iv) A further problem with the climbing-fiber pathway is that the retinal-slip signal it delivers to the flocculus is delayed by about 100 ms (Miles, 1991). Such a delay introduces instabilities into the learning process if the training data contain frequencies higher than about 2.5 Hz (see Appendix). These instabilities can be

70

Fig. 7. The decorrelation-control algorithm used with a second-order plant P and a leaky-integrator brainstem controller B. (A) Learning as measured by reduction in root-mean-square retinal-slip amplitude. Note log scale on both axes. The two curves are for decorrelators with either the ‘delay’ or the ‘spectral’ set of basis functions. The latter were an orthogonal set derived from the principal components of compensated motor commands. The final performance of the trained filter was little affected by the basis functions used. (B) Pre- and posttraining retinal slip in response to a colored-noise head-velocity input. (C) Pre- and post-training Bode gains for the VOR. (D) Pre- and posttraining eye-position response to a head-velocity pulse (from Dean et al., 2002).

avoided by what has been termed an ‘eligibility trace’, which acts as a delay and smoothing filter to remove high frequencies from the motorcommand components (details in Appendix). A variety of behavioral and electrophysiological evidence points to the existence of an eligibility trace (Raymond and Lisberger, 1998; Wang et al., 2000; Kehoe and White, 2002). (v) Finally, very little is known about the way mossy-fiber signals are decomposed into parallel-fiber components. Our use of different delays in the simulation described above is essentially an educated guess. However, by trying different schemes for decomposing signals in the adaptive linear filter, we were able to show that their main influence was on the speed with which the decorrelationcontrol algorithm learns, rather than its final convergence. Suitable choice of decomposition method could in fact speed learning very

considerably (Fig. 7). Suggestions that the method of decomposition can itself be influenced by learning (implemented for example by synaptic plasticity between mossy fiber– granule cell complex) have been made elsewhere (Schweighofer et al., 2001). To summarize, the above results indicate that in the context of the flocculus and (linearized) oculomotor plant compensation, the decorrelation-control algorithm is an effective and robust method of ensuring that a simple velocity command into the system generates the corresponding velocity output.

Decorrelation control and visual awareness One of the roles suggested for the cerebellum in relation to awareness is that it carries out the ‘elaboration’ of simple motor commands issued by the forebrain, thereby freeing the forebrain’s computational

71

resources. But it seemed that in order to learn such elaboration, cerebellar models — at least those based on the ideas of Marr and Albus — required a signal that in principle could not be available, namely motor error. However, the decorrelation control algorithm is a possible solution to this problem, since it requires an available signal of the sensory consequences of motor error, not motor error itself. The results described above indicate that for eye movements decorrelation control used by a simplified Marr–Albus model was effective in learning to compensate for a linearized oculomotor plant, thus enabling higher centers to send only simple velocity commands downstream with consequent easing of their computational load. The second role mentioned above for the cerebellum in visual awareness concerned the provision of sensory information uncontaminated by the organism’s own activity. In the case of oculomotor plant compensation the sensory signal is whole-field retinal image movement (retinal slip), potentially contaminated by inaccurate eye-movement commands. Inasmuch as decorrelation control successfully removes this contamination, any retinal slip

remaining is a genuine external signal. This can be seen in a redrawing of the VOR circuitry (Fig. 4) to emphasize its sensory-processing aspect (Fig. 8). In the redrawn version the retinal slip that would occur if the retina did not move can be considered as a sensory ‘target variable’. This has two components: an external signal of interest u, combined with selfproduced interference n. What the system is trying to do is move the sensor surface (i.e. the eye) so as to cancel n, leaving behind the ‘real’ signal u. The eye movement can thus be regarded as an estimate of that ˆ and the resultant retinal slip an interference n, estimate of the real signal u^ . The more accurate the eye movement, the better the estimate u^ (so that if u were zero, for example, there would be no retinal slip at all). Thus, the decorrelation-control algorithm that learns to produce accurate eye movements necessarily produces a good estimate of the signal of interest. Consequently, decorrelation control is a candidate algorithm for securing both of the proposed functions of the cerebellum in visual awareness. Of course, many questions remain. One of the most important concerns movements of parts of the

Fig. 8. Redrawing of the vestibulo-ocular circuitry shown in Fig. 4 to emphasize its sensory-processing aspects. Inputs to the system are: (i) the retinal slip that would occur if the eyes remain stationary is treated as a target variable. As such it consists of an external signal of interest u(t) corrupted by additive interference n(t); and (ii) predictor variables p(t). The task of the system is to extract an ˆ of the estimate of the signal of interest uˆ(t) from the target variable. It does so by subtracting from the target variable an estimate n(t) interference, in this case by physically moving the eye. Sensor output is no longer the target variable u(t) þ n(t) but the estimate uˆ(t) of the signal of interest u(t). The decorrelator must therefore learn the motor command m(t) which will act on the plant to produce the appropriate interference estimate (from Dean et al., 2002).

72

body other than the eyes. Unfortunately, control of multijoint movements is more complex than eyemovement control, and less is known about the anatomical details of the projections of cerebellar microzones to and from the relevant premotor circuitry in cortex, brainstem, and spinal cord. However, the mathematical analysis of decorrelation control indicated that it was in principle capable of compensating for very complex plants provided a copy of the motor command was made available to the relevant region of the cerebellum. It is therefore interesting that Eccles (1973) supposed this to be the case for motor cortex itself (the basis of his ‘dynamic loop’ hypothesis). More recently anatomical investigations using transneuronal transport methods have indicated that a given area of cerebral cortex which projects to cerebellar cortex via the pons receives a projection back from that selfsame region of cerebellar cortex via the thalamus. These ‘‘closedloop circuits may be a fundamental feature of cerebellar interactions with the cerebellar cortex’’ (Middleton and Strick, 2000, p. 240). It is possible therefore that the closed-loop arrangements required by decorrelation control are characteristic not just of eye movements but of movements in general. Further investigation of cerebro-cerebellar connectivity is but one example of the extensive work required to establish decorrelation control (or any other candidate) as the generic cerebellar method. It is of course a form of detective work, the kind of work of which, as this volume attests, Alan Cowey is a master.

Appendix The model architecture of Fig. 4 was programmed in MATLABTM. P, V, B, and C were treated as linear processes, allowing use of functions in the control system toolbox. The characteristics of the linear processes in initial training were: (i) V was a unit gain. (ii) P was a first-order plant, with the transfer function Hp(s) between eye-in-head velocity eh and motor command y given by Eq. (A1). Hp ðsÞ ¼

eh ðsÞ s ¼ yðsÞ s þ 1=Tp

ðA1Þ

where s denotes the Laplace complex frequency variable and Tp the time constant of the plant ( ¼ 0.2 s). (In subsequent equations with transfer functions, the argument (s) of transfer functions is omitted for simplicity.) (iii) The brainstem B had the transfer function Hb given by: Hb ¼ Gd þ

Gi s þ 1=Ti

ðA2Þ

corresponding to a brainstem controller with two paths: (a) a direct path which passed the head-velocity signal to the plant with the correct gain (Gd ¼ 1); and (b) an indirect path in which the head-velocity signal was integrated and passed to the plant also with the correct gain (Gi ¼ 1/Tp ¼ 5). The brainstem integrator was leaky with time constant Ti ¼ 0.5 s. (iv) The input to the adaptive filter C was split into 100 components with delays between components of 0.02 s (2 s total). C was thus effectively a finite impulse-response filter of length 100, with output c(t) given by:

cðtÞ ¼

100 X

wi yi ðt
0:02iÞ

ðA3Þ

i¼1

where wi was the weight of component yi. The rule for adjusting the weights was equivalent to that given in Eq. (1) in the text. The value of the learning-rate constant  in that equation was adjusted to give rapid learning without instability. The training input to the system was a headvelocity signal modeled as colored noise with unit power. The power had its peak value at 0.2 Hz, then varied with increasing frequency f as 1/f (as would occur if white-noise head acceleration were integrated to head velocity). For efficiency weight update was implemented in batch mode using 5 s batches of head-velocity data. After training with the basic system described above, a number of variants were investigated. (i) Variants of B: The integrator pathway was removed (Eq. A2, with Gi ¼ 0).

73

(ii) Variants of P: A second-order version of P was used with transfer function Hp given by: Hp ¼

sðs þ 1=Tz Þ ðs þ 1=T1 Þðs þ 1=T2 Þ

ðA4Þ

where T1 ¼ 0.37 s, T2 ¼ 0.057 s, Tz ¼ 0.2 s, taken from Stahl’s estimate (Stahl, 1992, p. 361) of the best-fit two-pole one-zero transfer function (for eye position from eye-movement command) to the data of Fuchs et al. (1988). This plant was combined with a leaky undergained integrator (Eq. A2, with Gi ¼ 5.05, Ti ¼ 0.5). (iii) Learning rule: The learning rule was changed from that shown in Eq. (1) to:
wi ¼
 sign½eðtÞyi ðtÞ

Acknowledgments Support for this work was Biotechnology and Biological Council (BBSRC). J.V.S. was Wellcome mathematical biology

provided by the Sciences Research the recipient of a fellowship.

ðA6Þ

and used to train an adaptive filter C with a first-order plant (Eq. A1) and a leaky undergained brainstem controller (Eq. A2, Gi ¼ 2.5, Ti ¼ 0.5). (iv) Delay: The retinal-slip signal arriving at C was delayed by d ¼ 100 ms. The system was trained with a first-order plant (Eq. A1) and a leaky undergained brainstem controller (Eq. A2, with Gi ¼ 2.5, Ti ¼ 0.5). It was found that the delay caused unstable learning if the input to C contained frequencies above 1/4d (at these frequencies the input becomes >90 out of phase with the retinal-slip signal). The components yi(t) were therefore convolved with an ‘eligibility trace’ r(t). The equation for the eligibility trace was taken from Eqs. (11) and (12) of Kettner et al. (1997): rðtÞ / t e
ðt=tpeak Þ

motor outputs for a perfectly compensated firstorder plant were subjected to principal component analysis. The 100 eigenvectors derived from the analysis were then used as basis functions. Learning was examined for the second-order plant with leaky undergained brainstem controller (variant 2 above).

ðA5Þ

where tpeak was set to 0.1 s. (v) Basis functions: The different delays used as basis functions for the mossy-fiber input y(t) were subsequently replaced by alternative functions. These included sine waves of different frequencies and decaying exponentials of different time constants, as well as basis functions that were orthogonalized with respect to the motor commands themselves. One method of achieving this was by spectral decomposition, in which the

References Albus, J.S. (1971) A theory of cerebellar function. Math. Biosci., 10: 25–61. Blakemore, S.J., Wolpert, D. and Frith, C. (2000) Why can’t you tickle yourself ? Neuroreport, 11: R11–R16. Blakemore, S.J., Frith, C.D. and Wolpert, D.M. (2001) The cerebellum is involved in predicting the sensory consequences of action. Neuroreport, 12: 1879–1884. Bower, J.M. (1997) Control of sensory data acquisition. Int. Rev. Neurobiol., 41: 489–513. Brindley, G.S. (1964) The use made by the cerebellum of the information that it receives from sense organs. IBRO Bull., 3: 80. Bu¨ttner-Ennever, J.A. and Horn, A.K.E. (1996) Pathways from cell groups of the paramedian tracts to the floccular region. In: Highstein S.M., Cohen B. and Bu¨ttner-Ennever J.A. (Eds.), New Directions in Vestibular Research. New York Academy of Sciences, New York, pp. 532–540. Carpenter, R.H.S. (1972) Cerebellectomy and the transfer function of the vestibulo-ocular reflex in the decerebrate cat. Proc. R. Soc. Ser. B, 181: 353–374. Carpenter, R.H.S. (1988) Movements of the Eyes. Pion, London. Cowey, A., Parkinson, A.M. and Warnick, L. (1975) Global stereopsis in rhesus monkeys. Q. J. Exp. Psychol., 27: 93–109. Dean, P., Porrill, J. and Stone, J.V. (2002) Decorrelation control by the cerebellum achieves oculomotor plant compensation in simulated vestibulo-ocular reflex. Proc. R. Soc. Ser. B, 269: 1895–1904. De Zeeuw, C.I., Wylie, D.R., Stahl, J.S. and Simpson, J.I. (1995) Phase relations of Purkinje cells in the rabbit flocculus during compensatory eye movements. J. Neurophys., 74: 2051–2064.

74 Eccles, J.C. (1973) The Understanding of the Brain. McGrawHill, New York. Eccles, J.C., Ito, M. and Szenta´gothai, J. (1967) The Cerebellum as a Neuronal Machine. Springer-Verlag, Berlin. Fuchs, A.F., Scudder, C.A. and Kaneko, C.R.S. (1988) Discharge patterns and recruitment order of identified motoneurons and internuclear neurons in the monkey abducens nucleus. J. Neurophys., 60: 1874–1895. Fujita, M. (1982) Adaptive filter model of the cerebellum. Biol. Cybern., 45: 195–206. Gilbert, P.F.C. (1974) A theory of memory that explains the function and structure of the cerebellum. Brain Res., 70: 1–8. Godaux, E. and Vanderkelen, B. (1984) Vestibulo-ocular reflex, optokinetic response and their interactions in the cerebellectomized cat. J. Physiol., 346: 155–170. Goldstein, H. and Reinecke, R. (1994) Clinical applications of oculomotor plant models. In: Fuchs A.F., Brandt T., Bu¨ttner U. and Zee D. (Eds.), Contemporary Ocular Motor and Vestibular Research: A Tribute to David A. Robinson; International Meeting, Eibsee 1993. Georg Thieme Verlag, Stuttgart, pp. 10–17. Goldstein, H.P., Bockisch, C.J. and Miller, J.M. (2000) Muscle forces underlying saccades. Invest. Ophthal. Vis. Sci., 41: S315. Ito, M. (2001) Cerebellar long-term depression: characterization, signal transduction, and functional roles. Physiol. Rev., 81: 1143–1195. Kandel, E.R., Schwartz, J.H. and Jessell, T.M. (2000) Principles of Neural Science. McGraw-Hill, New York. Kehoe, E.J. and White, N.E. (2002) Extinction revisited: similarities between extinction and reductions in US intensity in classical conditioning of the rabbit’s nictitating membrane response. Anim. Learn. Behav., 30: 96–111. Kettner, R.E., Mahamud, S., Leung, H.C., Sitkoff, N., Houk, J.C., Peterson, B.W. and Barto, A.G. (1997) Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement. J. Neurophys., 77: 2115–2130. Lisberger, S.G. and Fuchs, A.F. (1978) Role of primate flocculus during rapid behavioral modification of vestibuloocular reflex. II. Mossy fiber firing patterns during horizontal head rotation and eye movement. J. Neurophys., 41: 764–777. Llina´s, R. and Welsh, J.P. (1993) On the cerebellum and motor learning. Curr. Opin. Neurobiol., 3: 958–965. Luebke, A.E. and Robinson, D.A. (1994) Gain changes of the cat’s vestibulo-ocular reflex after flocculus deactivation. Exp. Brain Res., 98: 379–390. Marr, D. (1969) A theory of cerebellar cortex. J. Physiol., 202: 437–470. Marr, D. (1982) Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. W.H. Freeman, San Francisco.

McElligott, J.G., Beeton, P. and Polk, J. (1998) Effect of cerebellar inactivation by lidocaine microdialysis on the vestibuloocular reflex in goldfish. J. Neurophys., 79: 1286–1294. Middleton, F.A. and Strick, P.L. (2000) Basal ganglia and cerebellar loops: motor and cognitive circuits. Brain Res. Rev., 31: 236–250. Miles, F.A. (1991) The cerebellum. In: Carpenter R.H.S. (Ed.), Eye Movements. MacMillan Press, Basingstoke, pp. 224–243. Miles, F.A., Fuller, J.H., Braitman, D.J. and Dow, B.M. (1980) Long-term adaptive changes in primate vestibuloocular reflex. III. Electrophysiological observations in flocculus of normal monkeys. J. Neurophys., 43: 1437–1476. Nixon, P.D. and Passingham, R.E. (2001) Predicting sensory events: the role of the cerebellum in motor learning. Exp. Brain Res., 138: 251–257. Optican, L.M. and Miles, F.A. (1985) Visually induced adaptive changes in primate saccadic oculomotor control signals. J. Neurophys., 54: 940–958. Optican, L.M., Zee, D.S. and Miles, F.A. (1986) Floccular lesions abolish adaptive control of post-saccadic ocular drift in primates. Exp. Brain Res., 64: 596–598. Paulin, M.G. (1993) The role of the cerebellum in motor control and perception. Brain Behav. Evol., 41: 39–50. Porrill, J., Dean, P. and Stone, J.V. (2003) Recurrent cerebellar architecture solves the motor error problem: application to 3-D VOR. Program No. 882.12. 2003 Abstracts Viewer/ Itinerary Planner. Society for Neuroscience, Washington, DC. Rambold, H., Churchland, A., Selig, Y., Jasmin, L. and Lisberger, S.G. (2002) Partial ablations of the flocculus and ventral paraflocculus in monkeys cause linked deficits in smooth pursuit eye movements and adaptive modification of the VOR. J. Neurophys., 87: 912–924. Raymond, J.L. and Lisberger, S.G. (1998) Neural learning rules for the vestibulo-ocular reflex. J. Neurosci., 18: 9112–9129. Robinson, D.A. (1965) The mechanics of human smooth pursuit eye movement. J. Physiol., 180: 569–591. Robinson, D.A. (1974) The effect of cerebellectomy on the cat’s vestibulo-ocular integrator. Brain Res., 71: 195–207. Robinson, D.A. (1981) Models of the mechanics of eye movements. In: Zuber B.L. (Ed.), Models of Oculomotor Behaviour. CRC Press, Boca Raton, FL, pp. 21–41. Schweighofer, N., Doya, K. and Lay, F. (2001) Unsupervised learning of granule cell sparse codes enhances cerebellar adaptive control. Neuroscience, 103: 35–50. Sejnowski, T.J. (1977) Storing covariance with nonlinearly interacting neurons. J. Math. Biol., 4: 303–321. Simpson, J.I., Wylie, D.R. and De Zeeuw, C.I. (1996) On climbing fiber signals and their consequence(s). Behav. Brain Sci., 19: 384–398. Sklavos, S., Gandhi, N.J., Sparkls, D.L., Porrill, J. and Dean, P. (2002) Mechanics of oculomotor plant estimated from effects of abducens microstimulation. In 2002 Abstract

75 Viewer, Society for Neuroscience, Washington, DC, Program No. 463.5. Stahl, J.S. (1992) Signal Processing in the Vestibulo-ocular Reflex of the Rabbit. PhD Thesis, New York University. Stone, L.S. and Lisberger, S.G. (1990) Visual responses of Purkinje cells in the cerebellar flocculus during smoothpursuit eye movements in monkeys. II. Complex spikes. J. Neurophys., 63: 1262–1275. Voogd, J., Gerrits, N.M. and Ruigrok, J.H. (1996) Organization of the vestibulocerebellum. In: Highstein S.M., Cohen B. and Bu¨ttner-Ennever J.A. (Eds.), New Directions in Vestibular Research. New York Academy of Sciences, New York, pp. 553–579.

Wang, S.S.-H., Denk, W. and Ha¨usser, M. (2000) Coincidence detection in single dendritic spines mediated by calcium release. Nat. Neurosci., 3: 1266–1273. Weiskrantz, L., Elliott, J. and Darlington, C. (1971) Preliminary observations of tickling oneself. Nature, 230: 598–599. Widrow, B. and Stearns, S.D. (1985) Adaptive Signal Processing. Prentice-Hall Inc., Engelwood Cliffs, NJ. Zee, D.S., Yamazaki, A., Butler, P.H. and Gu¨cer, G. (1981) Effects of ablation of flocculus and paraflocculus on eye movements in primate. J. Neurophys., 46: 878–899.

SECTION II

Cortical Visual Systems

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 5

Some effects of cortical and callosal damage on conscious and unconscious processing of visual information and other sensory inputs Giovanni Berlucchi* Dipartimento di Scienze Neurologiche e della Visione, Universita` di Verona, I-37134, Verona, Italy

Abstract: Although new methods of investigation from the molecular level to cognition are promoting major advances in the study of the functions of the human brain, the analysis of behavioral and psychological deficits following brain damage is still a major tool for the understanding of cerebral organization. The present paper reviews some aspects of work on functional losses and residual abilities following cortical damage that have allowed to distinguish conscious and unconscious levels of visual input processing. Attention is given to the possible contribution of residual conscious vision of color to unconscious form analysis in visual agnosia. The paper also reviews findings on temporary and permanent deficits that occur after selective lesions of a prominent input–output system of the cerebral cortex, the corpus callosum, with the aim of assessing the possibility of establishing a functional callosal topography.

The work of Alan Cowey is noted for its achievements on the path to a thorough understanding of the relations between specific neural structures and specific cognitive functions. In the author’s contribution to this Festschrift honoring him, he summarizes the results of studies that his colleagues and he have carried out in the past 3 or 4 years in an attempt to link particular forms of brain damage with loss or preservation of given higher-order neural functions. The two main topics the author will deal with belong to areas of neuroscience to which Alan has made lasting contributions: blindsight and callosal hemispheric interactions. The conceptual link between these two approaches is that they both aim at understanding cognitive and behavioral functions of the cortex as inferred from the functional changes following direct lesions of specific cortical regions or the interruption of specific corticofugal and corticopetal

pathways in the corpus callosum. In the former case one can assess functional losses and functional sparings or recoveries after circumscribed or diffuse damage to the cortex. In the latter case it is possible to observe the behavioral and cognitive consequences of the loss of interhemispheric interactions between given cortical areas, as well as the differential effects exerted on cognition and behavior by cortical centers in each hemisphere that have become functionally autonomous because of their reciprocal disconnection.

Interactions between blindsight and conscious visual awareness The term blindsight was originally coined to denote the ability of patients with primary visual cortex lesions to emit adequate behavioral reactions to visual inputs from the supposedly blind contralesional part of their visual field, in the face of their proclaimed unawareness of those inputs (Weiskrantz, 1986; Stoering and Cowey, 1997). The term is now used to refer to a range of behaviors guided by visual

*Corresponding author. Dipartimento di Scienze Neurologiche e della Visione, Sezione Fisiologia Umana, Universita` di Verona, Strada Le Grazie, 8, I-37134, Verona, Italy. Tel.: þ 39-045-8027141; Fax: þ 39-045-580881 E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14400-5

79

80

cues that do not give rise to conscious visual perceptions, as can be observed in patients with various neurological disorders, and even in normal subjects submitted to specific kinds of visual stimulation (Milner, 1995; Milner and Goodale, 1995; Kolb and Braun, 1995; Driver and Mattingley, 1998; Marcel, 1998; Savazzi and Marzi, 2002). Although the phenomena subsumed under the term blindsight may occur in apparently identical fashion in these various cases, the underlying neural mechanisms may be different in different conditions, and our understanding of them is still largely incomplete.

Blindsight may influence conscious vision Most reports of blindsight have come from studies of patients with unilateral brain damage whose behavioral responses to visual stimuli from a contralesional field affected by hemianopia, extinction or neglect can be compared and contrasted with their responses to visual signals from the normal ipsilesional field. Evidence for blindsight is based not only on overt behavioral responses to otherwise unperceived visual stimuli, but also on the influence that such unperceived stimuli can exert on the processing of other visual stimuli that have access to consciousness. One search for the latter influence has involved the analysis of modulatory effects of light stimuli from the impaired field on speed of reaction to light stimuli from the good field (Marzi et al., 1986; Corbetta et al., 1990). As an example, the speed of detection of a simple flash stimulus in the intact visual field ipsilateral to a complete hemispherectomy can be significantly decreased by the simultaneous presentation of an identical flash stimulus in the opposite hemianopic field, even though the latter stimulus is incapable of eliciting any overt response, whether associated with awareness or unaccompanied by it (Tomaiuolo et al., 1997). In another approach, the discrimination of patterned stimuli in the intact visual field of patients with severe contralateral neglect has been shown to be aided by the previous presentation in the neglected field of stimuli identical to or belonging in the same category as the target, notwithstanding the patients’ consistent denial of the the occurrence

of the facilitating stimuli. In contrast, no facilitation was obtained with stimuli physically and categorically unrelated to the targets (Berti and Rizzolatti, 1992). Yet another approach was employed by Danckert et al. (1998) with a patient with a hemianopia contralateral to a one-sided occipital lesion who consistently denied seeing letter and color stimuli presented in his hemianopic field. Nevertheless, the patient’s reaction time for verbal identification of letter or color stimuli presented at the fixation point was influenced by flanker letter or color stimuli simultaneously presented in the intact or the hemianopic field. Reaction time was longer when the letter or color flankers differed from the corresponding letter or color targets, as compared to when flankers and targets were the same, or when there were no flankers. The effect was obtained with flankers presented in either visual field, implying that both color and letter information was unconsciously processed in the hemianopic field up to a degree that could interfere with the processing of concurrent central targets. Finally, discrimination of the emotional expression of a half face in the intact visual field was found to be facilitated by the simultaneous presentation of a half face with a congruent expression to the blind field, whereas discrimination of the expression of whole faces in the intact field was interfered with by incongruent facial expressions presented in the blind field (De Gelder et al., 2001). All these results speak strongly in favor of the existence of a variety of interhemispheric effects of unseen inputs from the blind field on the processing of inputs from the intact field that can enter consciousness.

Conscious vision may influence blindsight A few other studies have been aimed at exploring possible converse effects of conscious visual processes on blindsight phenomena. There is evidence to suggest that blindsight can be enhanced, or even result in conscious experiences, under the influence of visual information processed by intact brain systems. As a counterpart to facilitation of reactivity to stimuli in an intact visual hemifield by stimuli in an impaired hemifield, unconscious detection of light stimuli in an impaired hemifield can be facilitated by concurrent

81

stimuli in the intact hemifield (Ward and Jackson, 2002). After being presented with unilateral or bilateral visual stimuli, a patient with unilateral damage to the primary visual cortex indicated his detection of right, left or bilateral stimuli by making unspeeded choice key-pressing responses, and also reported his awareness of the stimuli. Although he never reported awareness of stimuli in the blind field, his manual responses evinced an imperfect, but clearly above-chance detection of such stimuli, i.e. blindsight, when such stimuli were presented alone. The finding relevant here was that his detection rate of stimuli in the blind field almost doubled when they were presented along with stimuli in the intact field. The latter stimuli were consistently detected, regardless of the presence or absence of a contralateral stimulus. The same patient was studied by Kentridge et al. (1999) in a spatial two-alternative forced choice discrimination in the contralesional and ipsilesional visual fields. Above-chance performance in the contralesional field could be dissociated from stimulus awareness, and could be significantly improved by cues that signaled the time of occurrence of the stimuli for discrimination. By being presented at the fixation point, such temporal cues had direct access to the intact hemisphere, and thus presumably to conscious processes of attentional control that could influence the decision underlying the blindsight performance.

Conscious vision in a hemianopic field Torjussen (1978) found in three hemianopic patients that unperceived stimuli flashed in the hemianopic field did not produce after-images; yet the patients experienced veridical bilateral after-images when exposed to bilateral complementary stimuli. Conscious experience of the half after-image in the hemianopic field could not be attributed to a completion effect, because no such completion was reported with stimuli restricted to the good field, which gave rise to after-images also strictly localized to that field. These findings were confirmed and extended by Marcel (1998) in two patients with unilateral hemianopia, who not only had conscious experiences of bilateral after-images generated by good Gestalten crossing the midline, but also

consciously saw complete figures with illusory contours partly lying in the hemianopic field. Conscious vision of the Gestalt’s part lying in the hemianopic field is probably made possible by complementary visual inputs to the damaged hemisphere from the intact hemisphere. That the intact hemisphere can relay visual inputs to the damaged hemisphere is suggested by the finding that extrastriate visual areas in the latter hemisphere are activated by appropriate stimuli presented in either hemifield (Goebel et al., 2001). Instances of conscious vision of stimuli in a hemianopic field, especially moving stimuli, have also been reported in a patient who lost the primary visual cortex at an age that may have allowed a substantial reorganization of his visual system (Stoerig and Cowey, 1997; Sahraie et al., 1997; Zeki and fftyche, 1998; Stoerig and Barth, 2001).

Blindsight and visual agnosia Patients with severe visual agnosia caused by diffuse cortical damage can exhibit visually guided behaviors that fit the definition of blindsight. Analyses of such cases have emphasized the dissociations, rather than the possible interactions, between a severely impaired conscious vision and the unconscious guidance of action towards visual targets, thus offering a starting point for general hypotheses about an at least partial separation between the neural substrates for perception and those for action (Milner, 1995; Milner and Goodale, 1995). The hallmark of cortical visual agnosia is usually a profound inability to identify and discriminate visual objects, whereas perception of color and visual motion, as well as general visual imagery, can often be preserved to at least some degree (Milner and Goodale, 1995; Servos and Goodale, 1995; Zeki et al., 1999). In one of these cases, Aglioti et al. (1999) have investigated whether preserved color vision, in addition to providing cues that help the patient to arrive at conscious inferences about the nature of visual objects, can by itself bring out blindsight responses to visual shapes. The study was performed on a patient suffering from a dense apperceptive agnosia for visual shapes and objects associated with a bilateral parieto-occipital atrophy. The brain damage and the resulting

82

visual agnosia were the consequences of a prolonged cardio-respiratory arrest sustained in the course of an endoscopic extraction of a foreign body fom the trachea. The patient’s most conspicuous deficit consisted in a complete incapacity to identify and discriminate even simple visual shapes and objects, such as single large black letters presented against a white background. When repeatedly tested in the latter task, his performance never deviated from chance, and eventually he asked to be spared such a futile exercise. In contrast, his color perception was nearly normal and he consistently reported a distinct awareness of the color stimuli that he was asked to point to or name. Similarly intact was his visual imagery, as assessed by his good ability to write and draw from memory, in striking contrast with his failure to copy drawings and verbal material, and to read what he had written minutes beforehand. The purpose of our study was to use a modified version of the Stroop test in order to assess whether the patient’s fully conscious chromatic vision could bear out some latent, implicit or explicit capacity for the processing of visual shape (Aglioti et al., 1999).

The Stroop test with single letters In the standard version of the Stroop test, normally seeing subjects are comparatively fast in naming the color in which a word is written if the word matches the name of the color, and comparatively slow if the word denotes a competing color. The effect is best accounted for by a parallel distributed processing model whereby the two pathways, one for color processing and the other for word processing, converge onto a shared response mechanism. In the case of incongruency between the information in the color pathway and the word pathway, the well learned and presumably automatic tendency to read the word is bound to interfere with the production of the competing color-naming response (MacLeod, 1991). Recently it has been reported that the Stroop effect is diminished if only one letter selected at random in the presented color name word is colored (Monahan, 2001), but work by Regan (1978) had previously shown that a robust Stroop effect could be obtained with the presentation of appropriately colored single-letter stimuli corresponding to the

initials of color names. We have used a simplified version of Regan’s (1978) task for testing first normal observers, and then the agnosic patient. The test involved repeated discriminations between two colors, red and green, and two letters, a capital R and a capital V. These two letters are the initials of the Italian words ‘rosso’ for ‘red’ and ‘verde’ for ‘green’ (all normal controls as well as the patient were Italian by birth and upbringing). The R letters were 5.2 cm high and 3.5 cm wide; the V letters were 5.5 cm high, and their width was 4 cm at the top and 0.6 cm at the bottom. Four stimuli, a red R, a green V, a green R and a red V, were presented one at a time in random sequences on the screen of a computer. The observer, positioned at a distance of 40 cm from the screen, was instructed to fixate the screen center at the beginning of each trial, but was allowed to move head and eyes when the stimulus was present. The stimulus remained on until the observer’s response. The letter discrimination task required speeded choice key-pressing responses, with one key to the letter R and another key to the letter V, regardless of their color. The color discrimination task similarly required speeded choice key-pressing responses to the red color and the green color, regardless of whether the colors were carried by an R or a V. A forced-response paradigm was used throughout, and no feedback about accuracy and speed of performance was provided. A computer controlled the sequences of stimulus presentation and recorded response accuracy and speed. There were three sessions of 40 trials each for both the color discrimination and the letter discrimination, and the order of the two types of discrimination was counterbalanced across sessions. The succession of stimuli was random with the constraint that in each session there were 20 congruent stimuli (10 red Rs and 10 green Vs) and 20 incongruent stimuli (10 green Rs and 10 red Vs). Eight normal observers matched for age with the patient showed no differences in accuracy between tasks and between congruent and incongruent stimuli. In the color task, the accuracy was 98.2% for congruent stimuli, and 96.1% for incongruent stimuli. In the letter task the accuracy was 97.2% for congruent stimuli and 96.8% for incongruent stimuli. In both tasks, t-tests for matched pairs indicated that the difference in accuracy between congruent and incongruent stimuli

83

fell far from statistical significance. However, in typical Stroop-effect fashion, the color task yielded RTs that were significantly longer for incongruent stimuli (mean RT 415.3 ms) than congruent stimuli (mean RT 393.5 ms), a significant difference by a t-test for matched pairs. No significant difference was found in the letter task, where mean RT was 424.3 ms for congruent stimuli and 429.2 ms for incongruent stimuli. In confirming a Stroop-like effect based on the initials of color names (Regan, 1978), the findings suggest that the initials can induce an automatic activation of the representations of the corresponding color names, with the result that responses to namecongruent colors are expedited and responses to name-incongruent colors are retarded.

The simplified Stroop test with the visually agnosic patient When performing the two tasks, the agnosic patient consistently reported that he could perceive the colors but not the letters, and that he felt that his forced responses to the letter stimuli were the result of mere guessing. In the color task there were no statistical differences in either accuracy or RT between congruent and incongruent stimuli. Accuracy was 95.8% with congruent stimuli and 97.5% with incongruent stimuli, and RT was 1.5 s with congruent stimuli and 1.4 s with incongruent stimuli. In contrast, in the letter task accuracy was at chance with the incongruent stimuli (50.8%), as it had been in many previous tests of letter recognition, but it was clearly and significantly above chance with the congruent stimuli. The accuracy advantage for congruent over incongruent stimuli was 23.4% with the letter R and 20% with the letter V. Overall RT of correct responses was significantly longer for incongruent stimuli (mean 8.6 s) than for congruent stimuli (mean 6.8 s). The RT advantage for congruent over incongruent stimuli was 0.9 s for the letter R and 2.7 s for the letter V. RTs for incorrect responses were not significantly different for congruent and incongruent pairings (10.4 s vs. 9.2 s). These results with the agnosic patient differed markedly from those of normal subjects on at least three counts. First, unlike the normal controls, the patient did not exhibit a standard Stroop effect insofar as his performance

on the color discrimination was unaffected by the letter stimuli. This result was to be expected on the basis of the patient’s preserved color perception and his general inability to recognize letter stimuli in many previous clinical and experimental tests. Even assuming a latent implicit potential for processing the letter stimuli, such potential would be preempted from influencing color discrimination by the fast processing of the color stimuli. Second, the normal controls did not show any interference of the color stimuli on accuracy and speed of the letter discrimination, supposedly because reading of a single letter can easily take precedence over any automatic color processing. In contrast, the accuracy of the patient’s responses to letter stimuli showed a clear effect of the congruency or incongruency of such stimuli with the letters to be discriminated. As could be expected from many previous tests, the patient’s performance did not show any evidence of letter discrimination with color-incongruent letter stimuli, but successful discrimination clearly emerged with color-congruent letter stimuli. That such performance reflected a real potential for letter discrimination was supported by the faster response speed to color-congruent than to color-incongruent letter stimuli. Third, throughout the testing the patient consistently reported that he had no conscious awareness of the letter presented on any given trial, so that his better-than-chance ability to discriminate such stimuli and his greater speed of response to color-congruent than color-incongruent letters could legitimately be defined as blindsight.

The role of visual imagery To account for this finding, we proposed that the patient’s preserved visual imagery would allow perceived colors to activate an orthographic representation of the corresponding word name. According to McClelland and Rumelhart’s (1981) computational model of interactive letter and word perception, this word name representation would in turn activate the orthographic representation of the word’s component letters. Viewed as a top-down influence, this activation would act on any existing ability for visual processing of letters by giving an advantage to inputs consistent with the activated letter representation over inputs inconsistent with it. It thus seems possible

84

that in the case of an input consistent with the activated letter representation, the advantage afforded to that input could bear out a partial residual ability of the patient for the implicit processing of letters. Inconsistency between the input and the activated letter representation, and the resulting absence of a top-down support would preclude the emergence of a successful letter discrimination even with the protracted processing attested by the patient’s very long response times. In the letter discrimination the patient’s reaction time was indeed longer with incongruent pairings, but even in the case of congruent pairings reaction time was much slower than it was in the color discrimination task. This difference strongly suggests that the proposed activation of the color name representation and its initial letter by the color stimuli lagged behind the patient’s color perception that was fully available to his awareness. If so, the patient’s discrimination of letters of which he was consistently unaware qualifies as a form of blindsight that was contingent on normal vision, i.e. vision which is ordinarily conscious, and was thus very different from reported facilitations of object or shape recognition that rely on the direct generation of shape from color or wavelength.

Speculations about the possible neural substrates of the interactions between ‘normal’ vision and blindsight In commenting on the results of Aglioti et al. (1999), Danckert and Goodale (2000) suggested that intact visual functions may aid the processing of visual information transmitted by a damaged system in two ways. Intact visual processes may either enhance the weak signals conveyed by the system, as proposed by Aglioti et al. (1999), or facilitate the access to such signals by another processing system, resulting in blindsight or even conscious experiences. The available evidence is insufficient to decide between these two possibilities, but they may be examined and discussed in the light of a few recent studies that bear on various forms of interaction between blindsight and normal vision. Suzuki and Yamadori (2000) have reported a surprising dissociation between letter reading and awareness of the form of letter stimuli. A Japanese woman with a lesion of the lower bank of

the calcarine fissure in the left hemisphere could read aloud kana and kanji characters and Arabic numerals in the scotomatous part of her right visual field, although she claimed that she perceived the stimuli as simple light flashes and denied any visual awareness of their form. According to the authors, the damage to the visual system reduced signal processing to a degree that was compatible with a reading vocal response, but too low for accessing consciousness. This suggestion bears on Danckert and Goodale’s (2000) concept of the reduced strength of the signal within a damaged visual system, insofar as one can envisage different degrees of information processing dysfunctions, ranging from intact signal processing accompanied by awareness in the case of minimal lesions, to signal processing dissociated from awareness, or even complete functional loss in the case of more severe lesions. If it is possible for top-down controls to modulate signal strength in a damaged visual system, it would have been interesting to explore whether Suzuki and Yamadori’s (2000) patient could become aware of the stimuli as a result of appropriate signal strengthening effects by attentional or arousing mechanisms. Potential top-down controls that can strengthen the signal in a damaged visual system are the so called feedback cortico-cortical connections from higher-order visual cortical areas to lower-order ones, or even cortical projections to subcortical visual centers such as the superior colliculus. According to Lamme (2001), unconscious visually guided behaviors can be executed on the basis of entirely feedforward input–output transformations, whereas conscious vision would also require the action of feedback inputs from higher order cortical areas to the primary visual cortex. Whereas the role of feedback corticocortical connections in conscious vision is compatible with data from normal and brain damaged observers, the primary visual cortex may not be the sole recipient of these connections in the mediation of conscious vision, as suggested by the results of a recent study by Weiskrantz et al. (2002). In a patient with unilateral visual cortex damage, visual stimuli such as gratings, colors, shapes, etc. that could elicit blindsight responses (i.e. correct discriminations associated with denial of stimulus awareness) in the contralesional field could also give rise to consciously experienced negative after-images after being turnedoff. These delayed subjective experiences might be

85

accounted for by the time necessary for feedback cortico-cortical connections to act on residual substrates for conscious vision in the damaged hemisphere. In other words, feedback signals from intact higher-order cortical areas in the damaged hemisphere, or perhaps even in the intact hemisphere, might induce a delayed enhancement of information processing in cortical or subcortical centers directly targeted by the original signal. The study by Goebel et al. (2001) did not find activation of the intact hemisphere by visual inputs to the damaged hemisphere, but it is possible that signals thus generated in the intact hemisphere are too weak to be detected by neuroimaging. If a cross-talk between the hemispheres is possible, interhemispheric feedback signals from the intact to the damaged hemisphere (possibly traveling via the corpus callosum or other interhemispheric connections) were involved in the after-images experienced in the hemianopic field by the patient described by Weiskrantz et al. (2002), it appears that such signals are apt to preserve veridical spatial relations, because the after-images were localized to the original stimulation site. Incidentally, the above-chance discrimination of color-congruent letters by the patient of Aglioti et al. (1999) cannot have depended on negative after-images, because red stimuli produce green negative after-images, and green stimuli produce red negative after-images, hence a Strooplike effect based on negative after-images would have paradoxically consisted in a facilitated discrimination of color-incongruent letters.

Channels in the corpus callosum and the multifunctional splenium For centuries, the corpus callosum has posed problems to the neurosciences as a most conspicuous feature of the brain of placental mammals, and especially of the human brain, which did not possess an obvious function. The problem of its functional significance was considered largely solved when Sperry and his collaborators demonstrated a major role for the corpus callosum in the interhemispheric transfer of information and in the unification of the independent cognitive domains of the two cerebral hemispheres (Sperry, 1982). Yet other issues about

callosal functions in man remain open. One question of interest to all involved in the search for anatomofunctional correlations is: can the human corpus callosum be seen as an ensemble of ‘channels’, each of which is used for the interhemispheric transmission of specific kinds of signals, from simple sensory messages and motor commands to highly digested information underlying learning, memory, thinking, emotion and so on? The corpus callosum is a cortical commissure, and to the extent that different functions can be attributed to different cortical areas, it seems logical that the callosal connections of an area or a set of areas with a specific function should subserve that same function for the purposes of interhemispheric communication.

Maps in the corpus callosum Anatomical investigations in animals have provided evidence that certain contingents of fibers belonging to different cortical areas are compartmentalized within the corpus callosum, but such compartmentalization seems far from strict and precise. Studies in cats indicate that fibers from discrete parts of the cortex disperse through large portions of the corpus callosum, where they intermix with fibers with different cortical relations and functions (Matsunami et al., 1994). In macaque monkeys, the majority of commissural fibers from a given cortical region tend to occupy a distinct location in the corpus callosum, but overlaps of callosal fibers from different cortical areas have also been noted in the body of the corpus callosum, suggesting that the anatomic segregation of functionally diversified contingents of callosal fibers is by no means complete (Pandya and Seltzer, 1986; Lamantia and Rakic, 1990). Different deficits in interhemispheric communication resulting from surgical sections of different callosal portions in various mammalian species have generally conformed with the expectations based on anatomical knowledge, but the bulk of evidence is largely restricted to impairments in visual interhemispheric transfer and splenial lesions (Berlucchi, 1990). In contrast with animal studies, anatomical evidence about the topographical organization of human corpus callosum is severely limited. Analyses of partial callosal degenerations or atrophies after

86

Fig. 1. The anatomo-functional map of the corpus callosum according to Habib (reproduced with permission from Neurochirurgie, Vol. 44, Suppl. 1, 1998, Masson Editeur).

(2) some notable discrepancies between different reports about the location of the callosal fibers supposedly involved in the performance of verbal dichotic listening tasks.

Interhemispheric communication after surgical sections of the corpus callosum that spare the splenium

Fig. 2. The anatomo-functional of the corpus callosum according to Funnell et al. (2000a).

neuronal losses in select neocortical areas of the human brain do not clearly indicate that fibers with a specific function, or a specific cortical origin and destination, cross the midline within a circumscribed portion of the corpus callosum. Figs. 1 and 2 show tentative anatomo-functional maps of the human corpus callosum based on observed associations between discrete callosal lesions on one hand, and specific behavioral deficits or scanty direct anatomical knowledge on the other. The systematicity of such proposed associations is at least partly questionable on the basis of various considerations. Here the author will deal with (1) the considerable sparing of interhemispheric communication that is known to obtain following extensive callosal sections that leave the splenium intact, and

Several years ago Gordon et al. (1971) reported the surprising finding that two patients submitted to section of the anterior two-thirds of the corpus callosum for relief of epilepsy did not show any of the interhemispheric disconnection deficits exhibited by epileptic patients with complete callosal sections (Bogen, 1993), or by patients with spontaneous anterior callosal lesions of vascular or tumoral origin. More specifically, the patients with surgical callosal sections sparing the splenium did not present with any of the signs of alexia in the left visual field and anomia for objects felt with the left hand that are so evident in patient with complete callosotomies. One of the callosotomy patient with preserved splenium could even name olfactory stimuli presented to either nostril, whereas complete callosotomy appears to limit this ability to the left nostril, projecting to the speaking left hemisphere (Sperry, 1982). In emphasizing the remarkable degree to which the small intact posterior sector of the corpus callosum could help attain a near-normal interhemispheric

87

communication, Gordon et al. (1971) maintained that signs of interhemispheric disconnection observed after vascular or tumoral lesions of the anterior callosum may be due to the association of callosal and extracallosal lesions, an association lacking in cases with clean surgical callosal sections. According to these results, still unchallenged to the author’s knowledge, either the posterior callosum may by itself be able to sustain normal interhemispheric interactions in all or most sectors of brain activity, or it may constitute a major site of a compensatory reordering of commissural mechanisms that prevents the occurrence of interhemispheric disconnection symptoms following anterior callosotomy. While this alternative is still undecided, work from the author’s laboratory has pointed to a similar role of the posterior corpus callosum in ensuring normal interhemispheric interactions in the taste modality.

Taste and the corpus callosum As shown by conflicting statements by anatomy and physiology textbooks, the lateral organization of the gustatory pathway in man is incompletely understood. A majority of studies support an uncrossed projection from each side of the tongue to the cortex (Norgren, 1990), but reports of an opposite crossed organization continue to appear in the neurological literature (Sa´nchez-Juan and Combarros, 2001). We studied the lateral organization of the gustatory pathway in eight normal controls, a man with a complete callosal agenesis, three men with a complete section of the corpus callosum, one man with a callosal section sparing the genu and the rostrum, and a man with a callosal section sparing the posterior callosum including the splenium. (Aglioti et al., 2000, 2001). Sapid solutions containing one of three basic taste stimuli (sour, bitter, salty) were applied to one or the other side of the tongue, and subjects reported the taste of the stimulus either verbally or by manually pointing to the name of the taste. Since it was known that in the subjects with complete callosotomies and in the posterior callosotomy subject verbal responses to tactile and visual stimuli were possible only with inputs to the left hemisphere, it was felt that verbalization of lateralized taste stimuli could provide information on the lateral organization of the taste pathway. There

were no differences in accuracy and reaction time between the right and left hemitongues of the normal controls, in accord with more precise psychophysical tests (Kroeze, 1979; McMahon et al., 2001). Similar results were obtained with the genetically acallosal observer, in accord with the notion that an inborn lack of the corpus callosum is generally compatible with an effective cross-integration in most functions (Jeeves, 1990). By contrast, the three complete callosotomy subjects and the subject with a sparing of the rostrum and the genu (Fig. 3) showed a significant advantage of the left hemitongue over the right hemitongue for response accuracy, and in one case for speed of response as well, although performance with right stimuli was clearly above chance in all four cases. Finally, and quite surprisingly, the callosotomy subject with an intact splenium (Fig. 3) showed no differences between the two hemitongues. Aglioti et al. (2000, 2001) concluded from these results that: (1) gustatory pathways from the tongue to the cortex are bilaterally distributed, so that taste information from either side of the tongue can reach the left hemisphere in the absence of the corpus callosum; (2) the uncrossed input from the left hemitongue to the left hemisphere is actually more potent functionally than the contralateral input; and (3) in the normal brain, the corpus callosum appears to equalize the effects of the ipsilateral and contralateral gustatory inputs on the left hemisphere, a callosal function also suggested by electrophysiological findings from animal experiments (Kadohisa et al., 2000). These conclusions are in agreement with some old and recent evidence about lateralized taste deficits following unilateral cortical lesions (Motta, 1958; Pritchard et al., 1999; Small et al., 2001). For present purposes, they specifically suggest that it is the posterior part of the corpus callosum, including the splenium, that ensures the functional equivalence between the two sides of the tongue in normal observers. The remarkable capacity of the posterior callosum alone to maintain effective interactions between the hemispheres in vision, touch and olfaction, as shown by Gordon et al. (1971), can therefore be regarded to extend to taste as well. As in the case of olfaction, the evidence for an involvement of the splenium in interhemispheric integration of taste information is puzzling in the light of the known

88

Fig. 3. Magnetic resonance imaging of the brains of two callosotomy patients studied by Aglioti et al. (2001). Evidence for asymmetries in taste discrimination between the two hemitongues, attributable to callosal disconnection, was found in the patient whose brain appears on the left, but not in the other patient with an intact splenium, shown on the right.

location of the cortical areas for taste in the frontal lobe and the insula (Frey and Petrides, 1999; Small et al., 1999). Since callosal connections of these areas run in the anterior corpus callosum, one would have expected effects on taste perception from anterior rather than posterior lesions.

Left ear suppression and the corpus callosum The suppression of left ear signals in such tasks is a typical symptom of interhemispheric disconnection in patients treated with complete callosal sections for drug-refractory forms of epilepsy (Milner et al., 1968). Left ear suppression is generally attributed to the interruption of callosal fibers that convey verbal auditory information from the left ear/right hemisphere to the left hemisphere for report. Some findings in patients with surgical or spontaneous partial callosal lesions have suggested that the critical interruption affects fibers running in a specific portion of the posterior trunk of the corpus callosum in front of the splenium, but not in the splenium itself. Different conclusions have been drawn from studies of other patients, also with partial surgical or spontaneous callosal sections, where a permanent left ear suppression in verbal dichotic tests has been found to result from lesions involving the splenium in addition to the posterior callosal trunk.

The human corpus callosum is frequently damaged by closed head traumas (Vuilleumier and Assal, 1995), and the specific position of the lesion within the corpus callosum can be precisely visualized in vivo by noninvasive brain imaging methods, so that clinical and laboratory observations can be carried out on single cases with identified callosal lesions. The results from a single case study in our laboratory (Peru et al., 2003) has recently afforded evidence that has a direct bearing on the location of the callosal lesion giving rise to left ear suppression in verbal dichotic listening task. A young male patient who had sustained a severe closed head trauma followed by coma underwent a nearly complete functional recovery in the course of several months. Magnetic resonance imaging examinations showed a complete interruption of the posterior third of the body of the corpus callosum with a minimal involvement of the splenium, which appeared substantially preserved (Fig. 4) as also demonstrated by tests of visual interhemispheric transfer. Given the site of the callosal lesion, some previous studies would have predicted a left ear suppression in verbal dichotic listening tasks (Springer and Gazzaniga, 1975; Alexander and Warren, 1988), whereas other studies would have predicted a substantial sparing of left ear signals due to the intactness of the splenium (Sugishita et al., 1995; Pollmann et al., 2002). We employed a dichotic listening test in which each trial consisted in the

89

Fig. 4. Magnetic resonance imaging of the traumatic callosal lesion in a patient who showed a left ear suppression in a dichotic listening task shortly after the lesion (a) but not in a follow-up retest a few months later (b) (from Peru et al., Neuropsychologia, 41: 634–643, 2003).

simultaneous presentation through headphones of 40 tape-recorded series of four digits, spoken by a male voice. One series was presented to the right ear and the other was presented to the left ear, and the digits occurring simultaneously were always different for the two ears. Immediately after the presentation the patient was to report, in a free order, all the digits he remembered to have heard. When tested 4 months after the trauma, the patient was very accurate in reporting series of four digits presented monoaurally to either ear, but showed an almost complete left-ear suppression in the dichotic listening paradigm. He reported 39 digits out of 40 presented to the right ear, but only one digit out of 40 presented to the left ear, and the difference between the two ears remained virtually the same when he was explicitly required to ignore digits presented to the right ear, and to report only digits presented to the left ear. However, this striking left ear suppression was no longer observable 3 months later, when the patient reported 25 out of 40 digits presented to the left ear, and 29 out of 40 digits presented to the right ear, a performance which is hardly different from that of normal subjects who divide their attention between the ears. We believe that the performance in the later dichotic listening test is compatible with the presence of fibers carrying auditory information in the splenium as well as in presplenial callosal portions. Anatomical findings in

the cat demonstrate a spread of fibers from cortical auditory areas over the entire posterior half of the corpus callosum, where they are interspersed with interhemispheric connections of other cortical areas (Matsunami et al., 1994; Clarke et al., 1995). If that anatomical pattern obtains in the human brain as well, the auditory callosal fibers running in the intact splenium of the patient may have compensated for the initial deficit caused by the injury to the auditory presplenial fibers. Alternatively, the splenial auditory fibers might have been nonfunctional soon after the trauma, and might have recovered their function with the elapsing of time and the waning of the causal factor, for example edema. In addition to suggesting a rather widespread distribution of auditory fibers in splenial and presplenial portions of the corpus callosum, the results from this case argue for the need to analyze effects of partial callosal disconnections over time in order to distinguish temporary from permanent interhemispheric transfer deficits.

Suggestions from partial callosal agenesis Total callosal agenesis is not associated with major symptoms of functional interhemispheric disconnection, due to compensatory processes that are still largely unknown. By comparison, partial callosal

90

agenesis may manifest itself in more conspicuous signs of functional interhemispheric disconnection, possibly because the extant callosal connections do not allow a full activation of the mechanisms for functional compensation (Dennis, 1976). In support of this hypothesis, in collaboration with Aglioti et al. (1998), the author reported clear-cut deficits of interhemispheric communication, including left hand anomia, partial left field alexia and poor tactile cross-localization in a subject with a congenital absence of the posterior part of the corpus callosum including the splenium. Such deficits were similar to those exhibited by a subject with a complete surgical section of the corpus callosum, but were lacking in a subject with a total callosal agenesis (Aglioti et al., 1998), as in many other genetically acallosal cases (Jeeves, 1990). It would be interesting to study whether the rare cases with anterior callosal agenesis and apparent preservation of the splenium (e.g. Sener, 1995) are completely free of interhemispheric disconnection symptoms, including those minor ones that are as a rule observable in total callosal agenesis (Jeeves, 1990).

In conclusion, as originally argued by Gordon et al. (1971), the recognized importance of the splenium for interhemispheric communication leaves unsolved the question of what functions are mediated by the large anterior sectors of the corpus callosum. Some of these putative anterior callosal functions have been discussed by Gazzaniga and his coworkers (Gazzaniga, 2000; Funnell et al., 2000a,b), who have also claimed that there is a remarkable functional specificity in callosal information transmission. However, where such remarkable specificity has been demonstrated, the data indicate that it obtains within the splenium rather than within the entire corpus callosum (Funnell et al., 2000b). The author likes to call attention to the scanty available evidence on the anatomical origin of the contingents of fibers that cross in the splenium of the human corpus callosum (De Lacoste et al., 1985). As shown in Fig. 5, large expanses of the posterior cortex in the occipital, parietal and temporal lobes, including and extending beyond areas with recognized visual functions, contribute fibers to the splenium. If these anatomical relations are confirmed with more precise methods, it

Fig. 5. Connections between posterior cortical areas and the splenium according to De Lacoste et al. (1985). Dense degenerating fibers (black) were found in callosal sector V (the splenium) after lesions affecting all cortical sectors indicated by arrows. (A) Lesion in parieto-temporal cortex (tentative degeneration also indicated in stippled sector IV). (B) Lesion in superior parietal cortex. (C) Lesion in occipital cortex. Reproduced with permission from the Journal of Neuropathology and Experimental Neurology.

91

follows that the splenium is connected with cortical regions that handle not only visual information, but also information from all other sensory modalities. In this vein, the possibility of the splenium to transmit integrated multisensory information between the hemispheres would be consistent with a special role of this part of the corpus callosum for human cognition and general behavioral control.

Conclusion The present era of neuroscience is witnessing triumphal technical and theoretical achievements. A wide ranging cooperation between molecular and cellular biology, genetics, embriology, morphology, physiology and pharmacology has resulted in an unprecedented in-depth exploration of the organization of the central and peripheral nervous system. Modern in vivo neuroimaging technologies allow the visualization of the human brain, whether normal or damaged, during complex cognitive and motor tasks. In such an era the time-honored exploration of brain functions based on the study of the effects of brain lesions may appear largely outdated or even totally obsolete. Yet the search for precise relations between cognition and behavior on one hand and nervous structures and mechanisms on the other has still a long way to go, and there continue to be ample opportunities and justifications for studying the effects of experimental brain lesions in animals and of neuropathological damage in man, as attested by the work of Alan Cowey during four decades. Studies of the effects of brain damage are still producing results that not rarely provide ultimate tests of hypotheses generated by more modern approaches to the knowledge of cerebral organization. To the extent that brain diseases will continue to afflict humankind, a systematic investigation of the related deficits will remain indispensable for understanding the underlying physiopathological mechanisms and for planning rational pharmacological treatments and rehabilitation procedures.

Acknowledgments Work by the author reported here has been supported by grants from the Ministero della Istruzione,

della Universita` e della Ricerca Scientifica e Tecnologica, and by the Consiglio Nazionale delle Ricerche. The author is grateful to Marco Veronese for assistance with the illustrations.

References Aglioti, S., Beltramello, A., Tassinari, G. and Berlucchi, G. (1998) Paradoxically greater interhemispheric transfer deficits in partial than complete callosal agenesis. Neuropsychologia, 36: 1015–1024. Aglioti, S., Bricolo, E., Cantagallo, A. and Berlucchi, G. (1999) Unconscious letter discrimination is enhanced by association with conscious color perception in visual form agnosia. Curr. Biol., 9: 1419–1422. Aglioti, S., Tassinari, G., Corballis, M.C. and Berlucchi, G. (2000) Incomplete gustatory lateralization as shown by analysis of taste discrimination after callosotomy. J. Cogn. Neurosci., 12: 238–245. Aglioti, S.M., Tassinari, G., Fabri, M., Del Pesce, M., Quattrini, A., Manzoni, T. and Berlucchi, G. (2001) Taste laterality in the split brain. Eur. J. Neurosci., 13: 195–200. Alexander, M.P. and Warren, R.L. (1988) Localization of callosal auditory pathways: a CT case study. Neurology, 38: 802–804. Berlucchi, G. (1990) Commissurotomy studies in animals. In: Boller F. and Grafman J. (Eds.), Handbook of Neuropsychology. Vol. 4. Elsevier, Amsterdam, pp. 9–47. Berti, A. and Rizzolatti, G. (1992) Visual processing without awareness: evidence from unilateral neglect. J. Cogn. Neurosci., 4: 347–351. Bogen, J.E. (1993) The callosal syndromes. In: Heilman K.H. and Valenstein E. (Eds.), Clinical Neuropsychology. Oxford University Press, Oxford, pp. 337–407. Clarke, S., de Ribaupierre, F., Bajo, V.M., Rouiller, E.M. and Krafsik, R. (1995) The auditory pathway in cat corpus callosum. Exp. Brain Res., 104: 534–540. Corbetta, M., Marzi, C.A., Tassinari, G. and Aglioti, S. (1990) Effectiveness of different task paradigms in revealing blindsight. Brain, 113: 603–616. Danckert, J. and Goodale, M.A. (2000) Blindsight: A conscious route to unconscious vision. Curr. Biol., 10: R64–R67. Danckert, J., Maruff, P., Kinsella, G., de Graaf, S. and Currie, J. (1998) Investigating form and colour perception in blindsight using an interference task. NeuroReport, 9: 2919–2925. De Gelder, B., Pourtois, G., van Raamsdonk, M., Vroomen, J. and Weiskrantz, L. (2001) Unseen stimuli modulate conscious visual experience: evidence from inter-hemispheric summation. NeuroReport, 12: 385–391. De Lacoste, M.C., Kirkpatrick, J.B. and Ross, E.D. (1985) Topography of the human corpus callosum. J. Neuropathol. Exp. Neurol., 44: 578–591.

92 Dennis, M. (1976) Impaired sensory and motor differentiation with corpus callosum agenesis: a lack of callosal inhibition during ontogeny? Neuropsychologia, 14: 455–469. Driver, J. and Mattingley, J.B. (1998) Parietal neglect and visual awareness. Nature Neurosci., 1: 17–22. Frey, S. and Petrides, M. (1999) Re-examination of the human taste region: a positron emission study. Eur. J. Neurosci., 11: 2985–2988. Funnell, M.G., Corballis, P.M. and Gazzaniga, M.S. (2000a) Cortical and subcortical interhemispheric interactions following partial and complete callosotomy. Arch. Neurol., 57: 185–189. Funnell, M.G., Corballis, P.M. and Gazzaniga, M.S. (2000b) Insights into the functional specificity of the human corpus callosum. Brain, 123: 920–926. Gazzaniga, M.S. (2000) Cerebral specialization and interhemispheric communication. Does the corpus callosum enable the human condition? Brain, 123: 1293–1336. Goebel, R., Muckli, L., Zanella, F.E., Singer, W. and Stoerig, P. (2001) Sustained extrastriate cortical activation without visual awareness revealed by fMRI studies of hemianopic patients. Vision Res., 41: 1459–1474. Gordon, H.W., Bogen, J.E. and Sperry, R.W. (1971) Absence of deconnexion syndrome in two patients with partial section of the neocommissures. Brain, 94: 327–336. Habib, M. (1998) Syndromes de de´connexion calleuse et organisation fonctionelle du corps calleux chez l’adulte. Neurochirurgie, 44: 102–109. Jeeves, M.A. (1990) Agenesis of the corpus callosum. In: Boller F. and Grafman J. (Eds.), Handbook of Neuropsychology. Vol. 4. Elsevier, Amsterdam, pp. 99–114. Kadohisa, M., Shinohara, M. and Ogawa, H. (2000) Effects of reversible block of callosal afferent inputs on the response characteristics of single neurons in the cortical taste area in rats. Exp. Brain Res., 135: 311–318. Kentridge, R.W., Heywood, C.A. and Weiskrantz, L. (1999) Effects of temporal cueing on residual visual discrimination in blindsight. Neuropsychologia, 37: 479–483. Kolb, F.C. and Braun, J. (1995) Blindsight in normal observers. Nature, 377: 336–338. Kroeze, J.H. (1979) Functional equivalence of the two sides of the human tongue. Percept Psychophys., 25: 115–118. Lamantia, A.S. and Rakic, P. (1990) Cytological and quantitative characteristics of four cerebral commissures in the rhesus monkey. J. Comp. Neurol., 291: 520–537. Lamme, V.A.F. (2001) Blindsight: the role of feedforward and feedback corticocortical connections. Acta Psychol., 107: 209–228. MacLeod, C.M. (1991) Half a century of research on the Stroop effect: an integrative review. Psychol. Rev., 109: 163–203. Marcel, A.J. (1998) Blindsight and shape perception: deficit of visual consciousness or of visual function? Brain, 121: 1565–1588.

Marzi, C.A., Tassinari, G., Aglioti, S. and Lutzemberger, L. (1986) Spatial sumation across the vertical meridian in hemianopsics: A test of blindsight. Neuropsychologia, 24: 749–758. Matsunami, K., Kawashima, T., Ueki, S., Fujita, M. and Konishi, T. (1994) Topography of commissural fibers in the corpus callosum of the cat: a study using WGA-HRP method. Neurosci. Res., 20: 137–148. McClelland, J.L and Rumelhart, D.E. (1981) An interactive activation model of context effects in letter perception. P.1. An account of basic findings. Psychol. Rev., 88: 375–407. McMahon, D.B., Shikata, H. and Breslin, P.A. (2001) Are human taste thresholds similar on the right and left sides of the tongue? Chem. Senses, 26: 875–883. Milner, A.D. (1995) Cerebral correlates of visual awareness. Neuropsychologia, 33: 1117–1130. Milner, A.D. and Goodale, M.A. (1995) The Visual Brain in Action. Oxford University Press, Oxford. Milner, B., Taylor, L. and Sperry, R.W. (1968) Lateralized suppression of dichotically presented digits after commissural section in man. Science, 161: 184–186. Monahan, J.S. (2001) Coloring single Stroop elements: reducing automaticity or slowing color processing? J. Gen. Psychol., 128: 98–112. Motta, G. (1958) I Fattori Centrali delle Disgeusie. Tipografia Luigi Parma, Bologna, pp. 291–513. Norgren, R. (1990) Gustatory system. In: Paxinos G. (Ed.), The Human Nervous System. Academic Press, San Diego, pp. 845–861. Pandya, D.P. and Seltzer, B. (1986) The topography of commissural fibres. In: Lepore, F., Ptito, M. and Jasper, H.H. (Eds.), Two Hemispheres – One Brain: Functions of the Corpus Callosum. New York: Alan R. Liss, pp. 47–73. Peru, A., Beltramello, A., Moro, V., Sattibaldi, L. and Berlucchi, G. (2003) Temporary and permanent signs of interhemispheric disconnection after traumatic brain injury. Neuropsychologia, 41: 634–643. Pollmann, S., Maertens, M., von Cramon D.Y., Lepsien, J. and Hugdahl, K. (2002) Dichotic listening in patients with splenial and nonsplenial callosal lesions. Neuropsychology, 16: 56–64. Pritchard, T.C., Macaluso, D.A. and Eslinger, P.J. (1999) Taste perception in patients with insular cortex lesions. Behav. Neurosci., 113: 663–671. Regan, J. (1978) Involuntary automatic processing in colornaming tasks. Perc. Psychophys., 24: 130–136. Sahraie, A., Weiskrantz, L., Barbur, J.L., Simmons, A., Williams, S.C. and Brammer, M.J. (1997) Pattern of neuronal activity associated with conscious and unconscious processing of visual signals. Proc. Natl. Acad. Sci. USA, 94: 9406–9411.

93 Sa´nchez-Juan, P. and Combarros, O. (2001) Sı´ ndromes lesionales de las vı´ as nerviosas gustativas. Neurologia, 16: 262–271. Savazzi, S. and Marzi, C.A. (2002) Speeding up reaction time with invisible stimuli. Curr. Biol., 12: 403–407. Sener, R.N. (1995) Anterior callosal agenesis in mild, lobar holoprosencephaly. Pediatr. Radiol., 25: 385–386. Servos, P. and Goodale, M.A. (1995) Preserved visual imagery in visual form agnosia. Neuropsychologia, 33: 1383–1394. Small, D.M., Zald, D.H., Jones-Gotman, M., Zatorre, R.J., Pardo, J.V., Frey, S. and Petrides, M. (1999) Human cortical gustatory areas: a review of functional neuroimaging data. NeuroReport, 10: 7–14. Small, D.M., Zatorre, R.J. and Jones-Gotman, M. (2001) Increased intensity perception of aversive taste following right anteromedial temporal lobe removal in humans. Brain, 124: 1566–1575. Sperry, R.W. (1982) Some effects of disconnecting the cerebral hemispheres. Science, 217: 1223–1226. Springer, S.P. and Gazzaniga, M.S. (1975) Dichotic testing of partial and complete split-brain subjects. Neuropsychologia, 13: 341–346. Stoerig, P. and Barth, E. (2001) Low-level phenomenal vision despite unilateral destruction of primary visual cortex. Conscious. Cogn., 10: 574–587. Stoerig, P. and Cowey, A. (1997) Blindsight in man and monkey. Brain, 120: 535–559. Sugishita, M., Otomo, K., Yamazaki, K., Shimizu, H., Yoshioka, M. and Shinohara, A. (1995) Dichotic listening

in patients with partial section of the corpus callosum. Brain, 118: 417–427. Suzuki, K. and Yamadori, A. (2000) Intact verbal description of letters with diminished awareness of their forms. J. Neurol. Neurosurg. Psychiat., 68: 782–786. Tomaiuolo, F., Ptito, M., Marzi, C.A., Paus, T. and Ptito, A. (1997) Blindsight in hemispherectomized patients as revealed by spatial summation across the vertical meridian. Brain, 120: 795–803. Torjussen, T. (1978) Visual processing in cortically blind hemifields. Neuropsychologia, 16: 15–21. Vuilleumier, P. and Assal, G. (1995) Le´sions du corps calleux et syndromes de de´connexion interhe´misphe´rique d’origine traumatique. Neurochirurgie, 41: 98–107. Ward, R. and Jackson, S.R. (2002) Visual attention in blindsight: sensitivity in the blind field increased by targets in the sighted field. NeuroReport, 13: 301–304. Weiskrantz, L. (1986) Blindisght: A Case Study and Implications. Oxford University Press, Oxford. Weiskrantz, L., Cowey, A. and Hodinott-Hill, I. (2002) Prime-sight in a blindsight subject. Nature Neurosci., 5: 101–102. Zeki, S. and fftyche, D.H. (1998) The Riddoch syndrome: insights into the neurobiology of conscious vision. Brain, 121: 25–45. Zeki, S., Aglioti, S., McKeefry, D. and Berlucchi, G. (1999) The neurological basis of conscious color perception in a blind patient. Proc. Natl. Acad. Sci. USA, 96: 14124–14129.

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 6

Consciousness absent and present: a neurophysiological exploration Edmund T. Rolls* University of Oxford, Department of Experimental Psychology, Oxford OX1 3UD, UK

Abstract: Backward masking was used to investigate the amount of neuronal activity that occurs in the macaque inferior temporal visual cortex when faces can just be identified. It is shown that the effect of the pattern mask is to interrupt neuronal activity in the inferior temporal visual cortex. This reduces the number of action potentials that occur to a given stimulus, and decreases even more the information that is available about which stimulus was shown because the variance of the spike counts is increased. When the onset of the mask follows the onset of the test stimulus by 20 ms, each neuron fires for approximately 30 ms, provides on average 0.06 bits of information, and human observers perform at approximately 50% better than chance in forced choice psychophysics, yet say that they are guessing, and frequently report that they are unable to consciously see the face and identify which face it is. At a longer Stimulus Onset Asynchrony of 40 ms, the neurons fire for approximately 50 ms, the amount of information carried by a single neuron is 0.14 bits, and human observers are much more likely to report conscious identification of which face was shown. The results quantify the amount of neuronal firing and information that is present when stimuli can be discriminated but not reported on consciously, and the additional amount of neuronal firing and information that is required for humans observers to consciously identify the faces. It is suggested that the threshold for conscious visual perception may be set to be higher than the level at which small but significant information is present in neuronal firing, so that the systems in the brain that implement the type of information processing involved in conscious thoughts are not interrupted by small signals that could be noise in sensory pathways.

Introduction

visual perception of the test visual stimulus, and this paradigm has been widely used in psychophysics (Humphreys and Bruce, 1989). In this chapter the author considers how much information is present in neuronal firing in the part of the visual system that represents faces and objects, the inferior temporal visual cortex (Rolls and Deco, 2002), when human subjects can discriminate in forced choice, but cannot consciously perceive, face identity. The author also considers the implications that the neurophysiological findings have for consciousness. The representation of faces and objects is in the inferior temporal visual cortex as shown by evidence that position, size and even for some neurons view, invariant representations of objects and faces are provided by neurons in the inferior temporal visual cortex (Rolls, 2000a; Rolls and

Damage to the primary (striate) visual cortex can result in blindsight, in which patients report that they do not see stimuli consciously, yet when making forced choices can discriminate some properties of the stimuli such as motion, position, some aspects of form, and even face expression (Weiskrantz et al., 1974; Stoerig and Cowey, 1997; Weiskrantz, 1997, 1998; De Gelder et al., 1999). In normal human subjects, backward masking of visual stimuli, in which another visual stimulus closely follows the short presentation of a test stimulus, reduces the *Corresponding author. Tel.: þ 44-1865-271348; Fax: þ 44-1865-310447; Web: www.cns.ox.ac.uk; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14400-6

95

96

Deco, 2002); that this is the last stage of unimodal visual processing in primates; and that lesions of what may be a homologous region in humans, the fusiform gyrus face and object areas (Ishai et al., 1999; Kanwisher et al., 1997), produce face and object identification deficits in the absence of low-level impairments of visual processing such as visual acuity (Rolls and Deco, 2002; Farah, 1990; Farah et al., 1995a,b). The inferior temporal visual cortex is, therefore, an appropriate stage of processing at which to relate quantitative aspects of neuronal processing to the visual perception of faces and objects. We have, therefore, studied the quantitative relationship between neuronal activity in the macaque inferior temporal visual cortex and visual perception (Rolls and Deco, 2002), and in this article the focus is on the relation between inferior temporal visual cortex and conscious visual perception, using the results from combined neurophysiological studies on the inferior temporal

visual cortex and perceptual studies in humans with the paradigm of backward masking of visual stimuli (Rolls et al., 1994, 1999; Rolls and Tove´e, 1994). A subsequent study by Kovacs et al. (1995) using a similar backward masking paradigm combined with primate electrophysiology confirmed the results.

Neurophysiology of the backward masking of visual stimuli Rolls and Tove´e (1994) and Rolls et al. (1994) measured the responses of single neurons in the macaque inferior temporal visual cortex during backward visual masking. Neurons that were selective for faces, using distributed encoding (Rolls and Tove´e, 1995; Rolls et al., 1997; Treves et al., 1999; Rolls and Deco, 2002), were tested in a visual fixation task run as shown in Fig. 1. The visual

Tone

Fixation spot

Visual stimulus S.O.A. Mask

Firing rate measurement

-500

0

500

1000

1500

2000

Time (ms)

Fig. 1. The timing used in the backward masking visual fixation blink task. The Stimulus Onset Asynchrony is the time between the onset of the visual test stimulus and the onset of the pattern mask stimulus. The test stimulus duration was 16 ms. (After Rolls et al., 1994.)

97

fixation task was used to ensure that the monkey looked at the visual stimuli. The methods used are described by Rolls and Tove´e (1994) and Rolls et al. (1994), and a few salient points follow. As shown in Fig. 1, at
100 ms, the fixation spot was blinked off so that there was no stimulus on the screen in the 100 ms period immediately preceding the test image. The screen in this period, and at all other times including the interstimulus interval and the interval between the test image and the mask, was set at the mean luminance of the test images and the mask, so that pattern discrimination with equally intense test and mask stimuli was investigated (see Bruce and Green, 1989). At 0 ms, the 500 ms warning cure tone was switched off and the test visual image was switched on for one 16 ms frame of a raster display image. The monitor had a persistence of less than 3 ms, so that no part of

the test image was present at the start of the next frame. Stimulus Onset Asynchrony (S.O.A.) values of 20, 40, 60, 100 or 1000 ms (chosen in a random sequence by the computer) were used. (The Stimulus Onset Asynchrony is the time between the onset of the test stimulus and the onset of the mask.) The duration of the masking stimulus was 300 ms. The stimuli were static visual stimuli subtending 8 degrees in the visual field presented on a video monitor at a distance of 1.0 m. The faces used as test stimuli are illustrated in Fig. 2. The usual masking stimulus (to which the neuron being analysed did not respond) was made up of letters of the alphabet {N,O}, as shown in Fig. 2. The masking pattern consisted of overlapping letters, and this masking pattern was used because it is similar to the mask used in the previous psychophysical experiments (see Rolls et al., 1994). (In some cases the masking stimulus was a face

Fig. 2. Examples of the test images used. The mask is also shown. (After Rolls et al., 1994.)

98

stimulus that was ineffective for the neuron being recorded.) Figure 3 shows examples of the effects of backward masking on the responses of a single inferior temporal cortex neuron in peristimulus rastergram and time histogram form. The top rastergram/spike density histogram pair shows the responses of the neuron to a single frame of the test stimulus (an effective face stimulus for that neuron). Relative to the prestimulus rate, there was an increase in the firing produced with a latency of approxi-

mately 75 ms, and this firing lasted for 200–300 ms, that is for much longer than the 16 ms presentation of the target stimulus. In the next pairs down, the effects of introducing a non-effective face as the masking stimulus with different S.O.A.s are shown. It is shown that the effect of the mask is to limit the duration of the firing produced by the target stimulus. Very similar masking was obtained with the standard N–O pattern mask. Similar experiments were repeated on 42 different cells (Rolls et al., 1994; Rolls and Tove´e, 1995), and in all cases the

Fig. 3. Peristimulus rastergrams and smoothed peristimulus spike density histograms based on responses in 8–16 trials to the test face alone (top raster-histogram pair), and to the test face followed by a masking stimulus (which was a face that was ineffective in activating the cell) with different S.O.A. values. (S.O.A. ¼ Stimulus Onset Asynchrony) The mask alone did not produce firing in the cell. The target stimulus was shown for 16 ms starting at time 0. (The top trace shows the response to the target stimulus alone, in that with this 1000 ms S.O.A., the mask stimulus was delayed until well after the end of the recording period shown.) (After Rolls and Tove´e, 1994.)

99

temporal aspects of the masking were similar to those shown in Fig. 3. One important conclusion from these results is that the effect of a backward masking stimulus on cortical visual information processing is to limit the duration of neuronal responses, by interrupting neuronal firing. The neuronal firing of inferior temporal cortex neurons often persisted for 200–300 ms after a 16 ms presentation of a stimulus. With a 20 ms Stimulus Onset Asynchrony, the neuronal firing was typically limited to 30 ms. With a 40 ms Stimulus Onset Asynchrony, the neuronal firing was typically limited to 50 ms. This persistence of cortical neuronal firing when a masking stimulus is not present is probably related to cortical recurrent collateral connections which could implement an autoassociative network with attractor and short-term memory properties (see Rolls and Treves, 1998; Rolls and Deco, 2002), because such continuing post-stimulus neuronal firing is not observed in the lateral geniculate nucleus (K. Martin, personal communication).

responses, averaged across the population of 15 neurons for which a sufficient number of trials was available, is shown in Fig. 4. The responses for the most (max) and the least (min) effective stimuli are shown for the period 0–200 ms with respect to stimulus onset. There was little effect (not significant) of the mask on the responses to the least effective stimulus in the set, for which the number of spikes was close to the spontaneous activity. The transmitted information carried by neuronal firing rates about the stimuli was computed with the use of techniques that have been described previously (e.g., Rolls et al., 1997; Rolls and Treves, 1998; Rolls and Deco, 2002), and have been used previously to analyse the responses of inferior temporal cortex neurons (Optican and Richmond, 1987; Gawne and Richmond, 1993; Tove´e et al., 1993; Tove´e and Rolls, 1995; Rolls et al., 1997). In brief, the general procedure was as follows (Rolls et al., 1999). The response r of a neuron to the presentation of a particular stimulus s was computed by measuring the firing rate of the neuron in a fixed time window after

Information available in inferior temporal cortex visual neurons during backward masking

Max. Min.

11

10

9 Number of spikes in 0 - 200 ms

To fully understand quantitatively the responses of inferior temporal cortex neurons at the threshold for visual perception, Rolls et al. (1997) applied information theoretic methods (see Shannon, 1948; Rolls and Treves, 1998; Rolls and Deco, 2002) to the analysis of the neurophysiological data with backward masking obtained by Rolls et al. (1994) and Rolls and Tove´e (1994). One advantage of this analysis is that it shows how well the neurons discriminate between the stimuli under different conditions, by taking into account not only the number of spikes, but also the variability from trial to trial in the number of spikes. Another advantage of this analysis is that it evaluates the extent to which the neurons discriminate between stimuli in bits, which can then be directly compared with evidence about discriminability obtained using different measures, such as human psychophysical performance. The analysis quantifies what can be determined about which of the set of faces was presented from a single trial of neuronal firing. As a preliminary to the information theoretic analysis, the effect of the S.O.A. on the neuronal

8

7

6

5

4

20 40

60

100

no mask

Stimulus Onset Asynchrony (ms)

Fig. 4. The mean (sem) across cells of the number of spikes produced by the most effective stimulus (max) and the least effective stimulus (min) as a function of Stimulus Onset Asynchrony (S.O.A.). (After Rolls et al., 1999.)

100

the stimulus presentation. The firing rates were then quantized into a smaller number of bins d than there were trials for each stimulus. After this response quantization, the experimental joint stimulusresponse probability table P(s, r) was computed from the data (where P(r) and P(s) are the experimental probability of occurrence of responses and of stimuli respectively), and the information I(S, R) transmitted by the neurons averaged across the stimuli was calculated by using the Shannon formula (Shannon, 1948; Rolls and Deco, 2002): IðS, RÞ ¼

X s, r

Pðs, rÞ log2

Pðs, rÞ PðsÞPðrÞ

and then subtracting the finite sampling correction of Panzeri and Treves (1996), to obtain estimates unbiased for the limited sampling. This leads to the information available in the firing rates about the stimulus. Information in 0 - 200 ms

Figure 5 shows the average across the cells of the cumulated information available in a 200 ms period from stimulus onset from the responses of the 15 neurons as a function of the S.O.A. This emphasizes how as the S.O.A. is reduced towards 20 ms the information does reduce rapidly, but that nevertheless at an S.O.A. of 20 ms there is still considerable information about which stimulus was shown. The reduction of the information at different S.O.A.s was highly significant (one way ANOVA) at PV

Hyb>HT

Hyb>HD

Hyb>VT

Hyb>VD

64.2 (7.6)

52.6 (25.1)

65.1 (8.1)

45.4 (21.7)

44.8 (21.6)

23.3 (16.5)

31.2 (23.1)

28.6 (20.4)

The first symbol in each column heading refers to the stimulus; the second symbol refers to the response. T, search target; D, search distracter.

of the hybrids as Vs line when they were search target versus distracter (t ¼ 0.380, df ¼ 9, P ¼ 0.7128, Bcorr). RTs to probes on sessions when they were targets versus distracters did not differ according to paired t-tests. Significant differences (or differences approaching significance) were not present when RTs were averaged over the two characters (mean RT to targets ¼ 0.68 s, S.D. ¼ 0.15; mean RT to distracters ¼ 0.70 s, S.D. ¼ 0.20). Similarly, negative findings obtained when H and V lines were considered separately (mean RT to Hs as targets ¼ 0.60 s, S.D. ¼ 0.18; mean RT to Hs as distracters ¼ 0.70 s, S.D. ¼ 0.20; mean RT to Vs as targets ¼ 0.64 s, S.D. ¼ 0.24; mean RT to Vs as distracters ¼ 0.73 s, S.D. ¼ 0.197). Paired t-tests provided no evidence than RTs to hybrids misidentified as targets (mean RT ¼ 0.68, S.D. ¼ 0.16) differed from hybrids misidentified as distracters (mean RT ¼ 0.72, S.D. ¼ 0.17). Negative outcomes also were found when misidentifications of hybrids as Hs were analyzed. When H was the target, mean RT of misidentifications as Hs ¼ 0.64, S.D. ¼ 0.16; when H was distracter, mean RT of misidentifications as Hs ¼ 0.68, S.D. ¼ 0.20). Similar results were found when misidentifications of hybrids as Vs were analyzed (when V was target, mean RT of misidentifications as V ¼ 0.62 s, S.D. ¼ 0.22; when V was distracter, mean RT of misidentifications as V ¼ 0.70, S.D. ¼ 0.23) (Table 2). The effects of search on probe identification in this study were weak and less consistent than those found in two prior studies in which targets and distracters differed in feature configuration (Butter and Goodale, 2000). Apparently, the search task used here was not sufficiently difficult to exert effects like those found in earlier studies. Indeed, the increase of 15.5% in search times on target-absent relative to target-present trials was not significantly different (t ¼ 1.736, df ¼ 9, P ¼ 0.117) from the increase (7.5%) found in Experiment 4 (Z vs. O), where no effects of search on probe identification were found.

General discussion Summarizing the findings presented here, search increased identification of Ls when they were targets and decreased identification when Ls were distracters in concurrent search (Exp. 1). Subjects in Experiment 2 showed increases in sensitivity, but not in response bias, to Ls. Search had a weaker effect on d 0 for identification of Ts, which was not reliably altered by search. In Experiment 3, enhancing the difficulty of searching for Ts or Ls by increasing the number of distracters augmented identification of targets versus distracters compared to the effect of search found previously (Exps. 1 and 2). Two studies investigated the effects on target identification of search tasks that were easier than those employed in prior studies here and elsewhere (Butter and Goodale, 2000). When targets were distinguished from distracters by straight versus curved lines (Z vs. O), there were no reliable effects of search on probe identification (Exp. 4) When horizontal and vertical lines were targets and distracters, subjects showed weak and inconsistent effect on probe identification (Exp. 5). The conclusion that conjunction search (T vs. L) affects target identification is tempered by lack of significant effects of search on identification of Ts in Experiments 1 and 2. However, predicted trends for this effect appeared in these studies as well as in Experiment 2 in Butter and Goodale (2000). Moreover, when search for Ts was made more difficult by employing 32 distracters (Exp. 3), a significant effect of search on identification of Ts appeared. Also, the d 0 values for identification of Ts as targets and distracters were marginally different. These findings suggest that identification of Ts is altered by search, but this effect is less reliable and more dependent on search difficulty than identification of Ls. RTs to probes were not significantly affected by search tasks in this or subsequent experiments, as found in the earlier study (Butter and Goodale,

194

2000), although in several studies there were trends in the direction of faster RTs to targets than to distracters. These findings suggest that search may affect speed of RTs to probes when statistical power is increased. The prolongation of search times on target-absent trials in feature-conjunction search (Exps. 1 and 2) was similar in degree to earlier findings (Butter and Goodale, 2000). This finding, plus the greater increase in target-absent search times in Experiment 3, suggests that these search tasks were difficult. In contrast, search times on target-absent trials were not prolonged (or much less prolonged) when features distinguished targets and distracters. The present studies did not examine slopes of functions relating set size to search times, a method commonly used for distinguishing parallel from serial search. This method requires a large number of search trials, a procedure that in pilot studies reduced or abolished the effect of search on target identification. There may be a general problem with the use of search times to distinguish different stages of processing. As Wolfe (1998) has pointed out, search times may not be the ideal measure to distinguish parallel from serial search, which he asserted may be an artificial distinction. An alternative interpretation of the findings presented here attributes enhancement of target identification to the assumption that the target stimulus was likely the last stimulus seen (on target-present trials). Direct test of this possibility would require analysis of responses to probes following targetpresent versus target absent trials. Unfortunately, these data are not available. However, this interpretation does not account for the finding that subjects identified distracters less frequently than neutral stimuli (Exp. 1), nor for the greater effect of search on target identification when the number of distracters was increased (Exp. 3). By this interpretation, one might expect increasing distracters would reduce, and not enhance, target identification, because on many target-present trials one or more distracters was close to the target and thus was identifiable. The findings that search affects responses to probe targets are consistent with those derived from studies of single-unit recordings in monkeys engaged in visual search (Chelazzi et al., 1993; Luck et al., 1997;

Reynolds et al., 1999) and fMRI studies in humans (Kastner and Ungerleider, 2000). These findings and the results presented here converge on the view that executive mechanisms augment the activity of goal representations, thus increasing the likelihood of target identification, as shown here. (See Introduction for a more detailed description of this effect.) This view implies that executive modulation of representations held in working memory enhances the efficiency of search. This would be especially useful when searching a cluttered field of irrelevant objects for an object, a common situation in everyday search. Executive activation of representations in working memory would provide the selective bias required for the object sought to compete successfully with other objects for the searcher’s attention (Desimone and Duncan, 1995). Support for this view derives from findings that visual working memory influences visual selective attention (Downing, 2000). The findings that targets are recognized more frequently than neutral stimuli (Exp. 1) and that their sensitivity is enhanced by search (Exp. 2) are consistent with the activating effects of executive modulation on neurons coding for search stimuli. Enhancement of sensitivity would be a consequence of selectively increasing the activity level of neurons encoding targets. Search may also decrease sensitivity to distracters, a conclusion suggested by subjects’ less-frequent identification of distracters compared to neutral stimuli in Experiment 1. (For neurophysiological evidence for such inhibitory effects, see Moran and Desimone, 1985 and Luck et al., 1997.) However, this inhibitory effect may be limited to search tasks where irrelevant items are homogeneous and appear repeatedly. Sensitivity for irrelevant items is less likely to be reduced when searching for an object in a cluttered setting of irrelevant and different objects. In this common situation, inhibitory control over large numbers of visual coding neurons would be an inefficient way of guiding search to a particular object. The studies reported here show that search difficulty determines to what degree search enhances identification of targets. A four-fold increase in distracters (distinguished from targets by feature conjunctions) made search more difficult, and improved target identification (Exp. 3). This finding

195

implies that when input processing is more difficult, executive mechanisms increases the activity level of the target’s representation, thus making activation by an appropriate stimulus to its threshold for identification more likely. The same interpretation applies to search in which difficulty is varied by size differences in targets and distracters. When subjects in a pilot study searched for a target differing in size from distracters by 6%, identification of probe targets was significantly better than it was when the size difference between target and distracters was 80%. Conversely, when search was relatively easy, its effects on probe identification were weak (Exp. 5) or not apparent (Exp. 4). It is possible that a more sensitive measure of probe identification would have disclosed effects of easy search. This kind of search task (involving feature differences) may benefit from top-down enhancement of target identification, as well as from stimulus salience (see Mack and Rock, 1998; Yantis and Egeth, 1999). These findings imply that bottom-up processing and top-down modulation of visual input work together in a coordinated manner. When processing becomes difficult (because of many distracters or small differences between target and distracters), top-down modulation increases; consequently, targets are more easily distinguished from distracters. When input processing is less demanding, executive controlled modulation is reduced. Studies of visual search inform us that our awareness of things is controlled in part by the modulating effects of executive mechanisms on internal representations of future goals. These modulating effects regulate awareness of searched-for objects and, in doing so, take into account the demands of input processing. Further studies of ways in which awareness is modulated by selective attention may bring us closer to understanding visual awareness, a phenomenon to which Alan Cowey has made outstanding contributions and to whom this chapter is dedicated.

References Baddeley, A. (1986) Working Memory. Oxford University Press, New York. Bundeson, C. (1990) A theory of visual attention. Psychol. Rev., 97: 523–547.

Butter, C.M. and Goodale, M.A. (2000) Visual search selectively enhances recognition of the search target. Vis. Cogn., 7: 7669–7782. Chelazzi, L., Miller, E.K., Duncan, J. and Desimone, R. (1993) A neural basis of visual research in inferior temporal cortex. Nature, 363: 345–347. Chelazzi, L., Duncan, J., Miller, E.K. and Desimone, R. (1998) Responses of neurons in inferior temporal cortex during memory-guided visual search. J. Neurophysiol., 80: 2918–2940. Cohen, A. and Ivry, R.B. (1991) Density effects in conjunction search: evidence for coarse location mechanism of feature integration. J. Exp. Psychol. Hum. Percept. Perform., 17: 891–901. Cowey, A. (1997) Current awareness: spotlight on consciousness. Dev. Med. Child Neurol., 39: 54–62. Crick, F. and Koch, C. (1998) Consciousness and neuroscience. Cereb. Cortex, 8: 97–107. Desimone R. and Duncan J. (1995) Neural mechanisms of selective visual attention. In: W. Maxwell Cowan, (Ed.), Annual Review of Neuroscience, Vol. 18, Annual Reviews. Palo Alto, California, pp. 193–222. Downing, P.E. (2000) Interactions between visual working memory and selective attention. Psychol. Sci., 11: 467–473. Duncan, J. (1984) Selective attention and the organization of visual information. J. Exp. Psychol. Gen., 113: 501–517. Duncan, J. and Humphreys, G.W. (1989). Visual search and stimulus similarity. 96: 433–458. James, W. (1890) The Principles of Psychology. Holt, New York. Kahneman, D. and Treisman, A. (1984) Changing views of attention and automaticity. In: Parasuraman R. and Davies D.R. (Eds.), Varieties of Attention. Academic Press, New York. Kastner, S. and Ungerleider, L.G. (2000) Mechanisms of visual attention in human cortex. In: W.M. Cowan (Ed.), Annual Review of Neuroscience, Vol. 23, Annual Reviews. Palo Alto, California, pp. 315–341. LaBerge, D. (1995) Attentional Processing. Harvard University Press, Cambridge, MA. Luck, S.J., Chelazzi, L., Hillyard, S.A. and Desimone, R. (1997) Neural mechanisms of spatial selective attention in areas V1, V2 and V4 of ht macaque visual cortex. J. Neurophysiol., 77: 24–42. Mack, I. and Rock, I. (1998) Inattentional Blindness. MIT Press, Cambridge, MA. Macmillan, N.A. and Creelman, C.D. (1992) Detection Theory: A User’s Guide. Cambridge University Press, New York. McLeod, P., Driver, J. and Crisp, J. (1988) Visual search for conjunctions of movement and form is parallel. Nature, 332: 154–155. Merikle, P.M., Smilek, D. and Eastwood, J.D. (2001) Perception without awareness: perspectives from cognitive psychology. Cognition, 79: 115–124.

196 Moran, J. and Desimone, R. (1985) Selective attention gates visual processing in the extrastriate cortex. Science, 229: 782–784. Nakayama, K. and Silverman, G.H. (1986) Serial and parallel processing of visual feature conjunctions. Nature, 320: 264–265. Norman, D.A. and Shallice, T. (1980) Attention to Action: Willed and Automatic Control of Behavior. University of California, Center for Human Information Processing, San Diego, CA, Report No. 8006. Rensink, R.A. (2000) Seeing, sensing and scrutinizing. Vis. Res., 40: 1469–1487. Reynolds, J.H., Chelazzi, L. and Desimone, R. (1999) Competitive mechanisms subserve attention in macaque areas V2 and V4. J. Neurosci., 19: 1736–1753. Shiffrin, R.M. and Schneider, W. (1977) Controlled and automatic human information processing. II. Perceptual learning, automatic attending and a general theory. Psychol. Rev., 84: 127–158. Treisman, A. (1993) The perception of features and objects. In: Baddeley A. and Weiskrantz L. (Eds.), Attention: Selection.

Awareness and Control A tribute to Donald Broadbent. Oxford University Press, Oxford. Treisman, A. and Gelade, G. (1980) A feature-integration theory of attention. Cogn. Psychol., 12: 97–136. Treisman, A. and Sato, S. (1990) Conjunction search revisited. J. Exp. Psychol., Hum. Percept. Perform., 16: 459–478. Wolfe, J.M. (1992) ‘Effortless’ texture segmentation and ‘parallel’ visual search are not the same thing. Vis. Res., 32: 757–763. Wolfe, J.M. (1998) What can 1 million trials tell us about visual search? Psychol. Sci., 9: 33–39. Wolfe, J.M., Cave, K.R. and Franzel, S.L. (1989) Guided search: an alternative to the feature integration model for visual search. J. Exp. Psychol. Hum. Percept. Perform., 15: 419–433. Yantis, S. and Egeth, H.E. (1999) On the distinction between visual salience and stimulus-driven attentional capture. J. Exp. Psychol. Hum. Percept. Perform., 25: 661–676.

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 14

First-order and second-order motion: neurological evidence for neuroanatomically distinct systems Lucia M. Vaina1,2,* and Sergei Soloviev1 1

Department of Biomedical Engineering, Brain and Vision Research Laboratory, Boston University, BME, 44 Cummington Str. 2, Boston, MA 02215, USA 2 Department of Neurology, Harvard Medical School, 75 Francis Str., Boston, MA 02215, USA

Abstract: An unresolved issue in visual motion perception is how distinct are the processes underlying ‘first-order’ and ‘second-order’ motion. The former is defined by spatio-temporal variations of luminance and the latter by spatiotemporal variations in other image attributes such as contrast or depth, for example. Using neuroimaging and psychophysics we present data from four neurological patients with unilateral and mostly cortical infarcts, which strongly suggest that first- and second-order motion have a different neural substrate. We found that from the early stages of processing, these two types of motions are mediated by two distinct pathways: first-order motion is carried out by mechanisms along the dorsal pathway in the occipital lobe, while the second-order motion by mechanisms mostly along the ventral pathway. The data reported here also suggest that different cortical regions may be in charge of processing direction-discrimination in second-order motion defined by different second-order attributes.

Introduction

approaches to study visual motion perception converge towards firmly elucidating the underlying neural substrate of motion mechanisms (for reviews Nakayama, 1985; Snowden, 1994; Sekuler et al., 2002). However, it has proved surprisingly difficult to achieve a consensus among motion-perception researchers on some basic yet fundamental questions, particularly on how many distinct motion systems human vision embodies. Some research groups maintain that a single mechanism is sufficient to account for human motion sensing regardless of the image cue (Johnston et al., 1992; Johnston and Clifford, 1995; Taub et al., 1997); for a summary review see Table 1 in Clifford and Vaina, 1999). Others have put forth the hypothesis that human vision may embody at least two motion sensing systems: a first-order system consisting of luminanceor color-defined attributes and at least another, second-order, system consisting of moving patterns whose motion attributes are not first-order, but most frequently are defined by texture flicker,

Visual motion perception is one of the most fundamental abilities of our visual system. Visual motion can be sensed from spatio-temporal variations in numerous and very different cues in the image, such as luminance, color, local contrast, texture, flicker or disparity. Over the past few decades neurophysiological and neuroanatomical studies have described motion selectivity in many cortical areas, and there is a consensus that motion processing occurs at different stages (for reviews see Albright and Stoner, 1995; Andersen, 1997) in the visual system. Psychophysical and computational research has defined and characterized a large number of motion processes and their relationships. The close link between the neurophysiological, psychophysical and computational *Corresponding author. Tel.: þ 1-617-353-2455; Fax: þ 1-617-353-6766; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14401-4

197

198

texture-contrast, texture spatial frequency (Chubb and Sperling, 1988; Cavanagh and Mather, 1990; Derrington and Badcock, 1992; Derrington et al., 1993; Fleet and Langley, 1994; Nishida et al., 1997; Baker Jr., 1999; Clifford and Vaina, 1999). The firstorder motion mechanisms are blind to second-order motion because the latter contains no consistent difference in luminance. Yet psychophysics has convincingly demonstrated that normal human observers have no trouble in perceiving purely second-order motion (Chubb and Sperling, 1989). Psychophysical evidence suggests that, at least initially, the visual system analyzes first- and second-order motion by distinct visual pathways and different mechanisms (i.e. a quasi-linear process detects first-order motion, and a nonlinear mechanism detects second-order motion (for a review, see Baker, 1999). In a recent study of first- and secondorder motion, Schofield (2000) argues convincingly that the visual world contains both first-order (luminance) and second-order (contrast) defined information. While first-order motion mechanisms are blind to second-order motion, physiological studies in both awake and anesthesized animals provide clear evidence for the existence of neurons sensitive to second-order motion. Directionally selective neurons in primate cortical areas that are strongly responsive to first-order visual motion stimuli are found in the middle temporal area (MT or V5) (Albright, 1992; O’Keefe and Movshon, 1996, 1998; Churan and Ilg, 2001), the middle superior temporal area (both MSTd, Geesaman and Andersen, 1996, and MSTl, Churan and Ilg, 2001) and superior temporal polysensory area (STP) (O’Keefe and Movshon, 1998), and cat areas 17 and 18 (Zhou and Baker, 1993), have been found to respond to a variety of second-order motion stimuli. The study of O’Keefe and Movshon (1998) compared responses to first and second-order motion in both MT/V5 and V1 and concluded that while most MT/V5 neurons responded poorly and nonselectively to second-order motion, none of the V1 cells did. A fundamental question common to these studies is whether any of these known motion-responsive cortical areas provide the substrate for the secondorder motion mechanism in the same way it has been described for specific first-order mechanisms. There

was no strong evidence that a subpopulation of neurons in any of the cortical areas studied thus far are selective for the different kinds of second-order stimuli employed. Thus, the question still confronting us is whether there are visually responsive areas specifically devoted to processing second-order motion, or at least significantly involved in this processing. Several recent fMRI studies in normal human observers point to areas outside of human hMT þ (MT and MST) as being strongly responsive to several types of second-order motion stimuli (Smith et al., 1998; Somers et al., 1999; Wenderoth et al., 1999). In particular, Smith et al. (1998) report that the cortical area V3 (lower hemifield) and its ventral counterpart, VP (upper hemifield) have stronger responses to second-order than to first-order motion. Based on these results, they speculated that V3 and VP may be the first visually responsive areas in which second-order motion is explicitly represented. They also reported that hMT þ was activated by both first and second-order motion. However, the experimental design and stimuli employed in this study were not conducive to clarifying whether second-order motion is detected in V1 and V2. In a recent carefully designed fMRI study, Dumoulin and collaborators (Dumoulin et al., 2002, 2003) used first- and second-order stimuli with identical spatial and temporal properties (Boulton and Baker, 1993a,b; Clifford et al., 1998) to further investigate the possibility of cortical specialization for these motions in normal human observers. While many cortical areas had similar responses to both types of motion, a region-of-interest analysis indicated that V1 and V2 were more involved in the processing of first-order motion, while a region posterior to hMT þ was preferentially activated by secondorder motion. The neuroanatomical substrate of first- and second-order motion was also investigated behaviorally in a few neurological studies. For example, Plant and collaborators (Plant and Nakayama, 1993; Plant et al., 1993) studied perceptual abilities for both types of motion stimuli in patients before and after unilateral occipital–temporal resection. For firstorder motion they used speed and direction discrimination tasks, while for second-order motion subjects were required to discriminate direction in

199

a ‘beat’ pattern, resulting from the sum of two oppositely drifting sine-wave gratings. After the surgery, but not before, contrast threshold for direction discrimination was more impaired for second-order than for first-order motion patterns for stimuli presented in the contralesional visual field. However, the threshold of contrast discrimination of static orientation gratings was normal. Plant and colleagues interpreted their results to indicate that the mechanisms underlying secondorder motion are not as widely distributed in the extrastriate cortex as the first-order motion mechanisms, and therefore they are less likely to survive insults to the posterior part of the brain. The lesions in these patients were quite large and involved significant white matter, so these studies tell us little about a potential neuroanatomical substrate for second-order motion mechanisms. Moreover, one must be cautious in interpreting the results since, as pointed out by Clifford and Vaina (Clifford and Vaina, 1999), the ‘beat stimuli’ of the form used in these studies are not well suited to isolating secondorder motion processing. Chubb and Sperling (Chubb and Sperling, 1988) showed that stimuli for studying second-order motion should be ‘driftbalanced’, so as not to contain first-order motion components. Beat stimuli are not drift-balanced because they are formed by the additive superposition of two sinusoidal components (Barron et al., 1994). For a static carrier, as in Plant and Nakayama’s stimulus, the two components have equal and opposite temporal frequencies, but differ in spatial frequency, and therefore are not drift balanced. (Clifford and Vaina, 1999, provide a detailed theoretical discussion of the problem.) Greenlee and Smith (Greenlee and Smith, 1997) compared detection and discrimination of first- and second-order motion in twenty-one neurosurgery patients with unilateral lesions in the posterior cerebral cortex and normal control subjects. In the first set of experiments, thresholds for orientation and direction of moving patterns (first-order) or contrastmodulation depth (second-order) were measured using the method of constant stimuli. In another experiment speed discrimination thresholds were determined for both first and second-order gratings. The first interesting outcome of this study is that direction thresholds were slightly elevated especially

for second-order stimuli in patients with lesions in the lateral intraparietal (LIP) and superior temporal (ST) areas, as compared with lesions in the inferotemporal region (IT). The second outcome was that speed discrimination thresholds for first-order motion stimuli were just slightly more elevated than for second-order motion in patients with damage to ST and LIP areas. These results suggest a significant overlap in the neural substrate of first- and secondorder speed discrimination. Nawrot and colleagues (Nawrot et al., 2000) studied a patient with a bilateral resection of the occipital–temporal areas to treat epilepsy. This patient presented with transient deficits of both first- and second-order motion perception which, however, recovered within a few weeks. Similarly, recovery of second-order motion perception was found 20 months after the surgical lesion in the patients studied by Braun et al. (Braun et al., 1998). Azzopardi and Cowey (2001) studied motion perception of a patient, G.Y., with a striate cortex lesion and cortically blind in the contralesional field. They investigated G.Y.’s ability to discriminate direction of motion in several first- and secondorder stimuli. While G.Y. reliably discriminated the direction of isolated first-order moving bars even when presented in the scotoma, he failed on discriminating motion direction in random dot kinematograms (RDK) even at 100% coherence, despite correctly detecting the presence of movement in the stimulus. The same was true for stimuli embodying gratings, plaids, or RDK of motion in depth. For second-order motion stimuli, he could easily detect and discriminate the direction of motion of second-order bars which had no luminance cues associated with them. However, in his scotoma G.Y. could not even detect RDK stimuli defined by dynamic texture contrast, at any speed or contrast. Of relevance for this article is that G.Y. failed to discriminate direction in RDK stimuli, no matter whether they were first- or second-order. Overall G.Y.’s performance on a series of carefully designed psychophysical tests demonstrates that motion discrimination is severely impaired in the scotoma, suggesting that the relevant motion information does not arrive to the intact motion-sensitive extrastriate cortical areas (i.e. MT and MST) by a pathway bypassing the damaged area V1. The other

200

neurological studies discussed above suggest that various extrastriate lesions may affect the perception of second-order motion, often together with that of first-order motion. However, the lesions in these were sufficiently large to include quite a few putative extrastriate areas, and they involved significant amounts of white matter which complicate the precision of the functional-anatomical localization. Moreover, because of the diversity of the stimuli used, comparison across studies is difficult. We are therefore still left with the questions: which type of motion is processed, where and how? Over the past 10 years, as part of our research on the effects of lesions on visual motion perception in humans, we compared the ability of discriminating direction of first- and second-order motion in several stroke patients with unilateral lesions. In a few patients (F.D., R.A., T.F. and J.V.) with small, circumscribed, single and cortically centered unilateral infarctions, we carried out a detailed neuroanatomical analysis of their structural MRI data. In this article I will relate their performance on firstand second- psychophysical tests and the locus of their lesions on the basis of detailed analysis of MRI data of their brains. To allow a direct comparison among these patients, the anatomical data from all four patients was reprocessed such that their lesion localization can be shown on the 3-D surface of an inflated brain accompanied by coronal or transverse brain slices that show the lesion in relation to various landmarks. For the first two patients, F.D. and R.A., a direct comparison with recent fMRI data on first- and second-order motion has also been made (Dumoulin et al., 2002, 2003). During these patients’ weekly visits to the laboratory for a period of over a year, we were interested to determine the integrity of their visual motion perceptual abilities and whether deficits remained stable or recovered over time. Our data provide more specific evidence than the previously reported neurological cases for the hypothesis that first and second-order motion are separate mechanisms and that they can be selectively damaged by lesions. The specificity of these lesions and the double dissociation of deficits suggest that at least to some extent first- and second-order motion are mediated by different pathways in the visually responsive cortex.

Patients F.D. and R.A. Patients F.D. and R.A. will be discussed first as both their lesions have been analyzed in more detail and the psychophysical study was more detailed. Here we will focus only on the comparable first- and secondorder motion tasks and on a task of long range motion (Green, 1986). Detailed longitudinal studies of these patients on a large battery of motion tasks have been published previously (Vaina and Cowey, 1996; Vaina et al., 1996, 1998, 1999b, 2000). Second, patients T.F. and J.V. will be briefly discussed, with the intent to demonstrate a finer dissociation of the anatomical pathways proposed to mediate first- and second-order motion analysis (Vaina, Soloviev and Dumoulin, in preparation). Patient F.D. is a right-handed male, collegeeducated social worker, who suffered a left hemisphere infarct at the age of 41. Neurological examination at the time of the cerebrovascular accident (CVA) revealed slight right-sided weakness, lasting a few days, and a mild anomia lasting a few weeks. For a few weeks after the infarct he complained of feeling disturbed by visually cluttered moving scenes and by (auditorily) noisy surroundings. Neuroophthalmological examination, including visual fields, was normal. Contrast sensitivity for detection of static or moving gratings and for discrimination of direction and speed of motion were normal as was temporal frequency discrimination. The anatomical locus of lesion is shown first on the inflated probabilistic brain (Fig. 1, top left), in relation to anatomical landmarks, and in Fig. 1 (bottom right) in comparison with areas of activity elicited by a recent fMRI study of firstand second-order motion study in normal subjects (Dumoulin et al., 2002, 2003). The 2-D MR slices shown illustrate in the coronal plane F.D.’s lesion on the dorso-lateral surface in the left hemisphere (shown on the right). In this figure, the lesion has been localized using the Cardview software package (Rademacher et al., 1992, 1993). The segmented cortical surface of each hemisphere is subdivided by topographic criteria into parcellation units (PU.) The parcellation system is relatively fine grained and retains the principal topographic landmarks. The spatial extent of the lesion was outlined on each slice where it was identified. The lesion was

201

Fig. 1. Lesion location in Patient F.D. and fMRI data is schematically drawn on the average unfolded surfaces of the left hemisphere: (top left): T1-weighted image of F.D.’s brain was co-registered with the Talairach coordinate system using automatic 3D inter-subject registration tools of MR volumetric data in standardized Talairach space from the Montreal Neurological Institute, which is an average brain volume derived from roughly 300 brains. On the surface F.D.’s lesion location affecting second-order motion perception is plotted on the inflated brain. Coronal slices: Using the Cardview parcellation and lesion localization software (Rademacher et al., 1993), F.D.’s lesion is shown in coronal slices in relation to the relevant parcellation units, which are familiar gross anatomical structures. The lesion is located dorsolaterally in the left hemisphere involving the superior (OLs) and inferior (OLi) occipital lateral cortex. It extends anteriorly into the angular gyrus (AG) and middle temporo-occipital cortex (TO2) and terminates in the supramarginal gyrus (Sgp). (bottom right): The fMRI clusters preferentially activated by second-order motion (Worsley et al., 1996, 2002) are outlined in green (Dumoulin et al., 2000). The circles indicate the peaks of the second-order activation in the parietal (P ¼ 0.001) and occipital (P ¼ 0.001) lobe. The occipital activation and lesion sites are mainly dorsal to hMTþ (depicted in black). F.D.’s lesion is portrayed in green (filled area). It overlaps with the most posterior area of cortical activity elicited by second-order motion stimuli. (S. Dumoulin helped in generating the inflated brain.)

small enough not to disrupt the identification of the requisite sulcal trajectories and landmarks. The Cardview system allows illustration of the lesion in relation to the cortical parcellation units. The lesion involves both the superior (Ols) and inferior (Oli) lateral occipital cortex, it extends anteriorly and involves a portion of the angular gyrus (AG) and middle temporo-occipital cortex (TO2) and terminates in the inferior portion of the posterior supramarginal gyrus (Vaina et al., 1999b). To compare the location of the lesion with the cortical areas identified by fMRI studies, we registered F.D.’s T2 weighted structural MRI volume in the Talaraich space (Talairach and Tournoux, 1988), and subsequently we applied specially devel-

oped scripts which identify the retinotopic areas and the hMT þ based on published fMRI studies. The coordinates of the region of interest (ROI) representing F.D.’s center of the lesion were {
48(6),
55(8), 11(4)} suggesting that the lesion is dorsal to hMT þ reported by several studies to roughly correspond to {
46(6),
73(10), 4(4)} (i.e. Van Oostende et al., 1997; Mendola et al., 1999; Sunaert et al., 1999; Vaina et al., 2001; Zeki et al., 2003). Patient R.A. is a right-handed retired computer manager who suffered a sudden right hemisphere embolic stroke at the age of 66. Visual fields obtained by both Goldmann and Humphrey perimetry revealed a left inferior quadrantopsia which resolved over a period of 16 months, when the behavioral data

202

Fig. 2. Lesion location in Patient R.A. and fMRI data schematically drawn on the average unfolded surfaces of the left hemisphere, viewed medially: (Top left): On the surface R.A.’s lesion location affecting first-order motion perception (Vaina et al., 1998) is plotted on the inflated brain using the same method as for F.D. Coronal slices: Using the Cardview parcellation and lesion localization software (Rademacher et al., 1993), R.A.’s lesion is shown in coronal slices in relation to the relevant parcellation units, which are familiar gross anatomical structures. The outline of the occipital cortex is shown in red and the outline of the lesion is shown in green. The lesion is located medially in the right hemisphere. It begins in the occipital pole, extending into the cuneus (CN) and the supracalcarine cortex (SCLC) and the lingual gyrus (LG). (Bottom right): The fMRI activation for first-order motion task falls partially in visual areas V1 and V2 (average V1/V2 border is shown with black lines). Though the first-order activation did not reach statistical significance in a stereotaxic analysis it did in a volume-of-interest analysis on the early visual areas identified in a separate scanning session (Dumoulin et al., 2003). The ROI analysis provides a signal-to-noise improvement due to intra- and inter-subject averaging, i.e. averaging of voxels within a functional area and averaging of the same area across subjects. Using a ROI analysis, area V1 was found to preferentially respond to first-order motion (P ¼ 0.01), a trend that decreased and eventually reversed in later visual areas. (S. Dumoulin helped generating the inflated brain.)

presented here were obtained. Contrast sensitivity was normal. Like in F.D., R.A.’s lesion is shown first on both the inflated probabilistic brain (Fig. 2, top left) and on coronal MR images of his own brain using the Cardview parcellation system. The lesion is predominantly dorsal to the striate cortex of the calcarine sulcus, involves the cuneus (CN) and the supracalcarine cortex (SCLC) and then descends to slightly include portions of the calcarine cortex and the lingual gyrus. Figure 2 (bottom right) shows the lesion again on the probabilistic brain in relation to the fMRI activation to first-order motion stimuli (Dumoulin et al., 2002, 2003) on which R.A. was selectively impaired (Vaina et al., 1999b). It is worth noting that among the tasks that compared R.A.’s ability to perform direction discrimination in firstand second-order motion, were the exact stimuli used by Dumoulin et al. (2002) and Dumoulin et al. (2003) to localize the neuroanatomical substrate of these two motion mechanisms.

We registered R.A.’s lesion in the Talaraich space and obtained the x, y, z, coordinates of the region of interest (ROI) delineating his right hemisphere cortical lesion. Consistent with the methods of lesion localization reported above, R.A.’s lesion was confined to the occipital lobe, with the centre in {14,
92,
2}. The result of processing R.A.’s structural MRI data with our anatomical templates based on reports from the fMRI literature of coordinates for the retintotopic areas and hMT þ , suggest that the lesion overlaps with cortical areas V1v and V2v, sparing area VP (is posterior to VP). For example, the fMRI studies of Mendola et al. (1999) and Sunaert et al. (1999), identified the following Talaraich coordinates for the areas V1v in the right hemisphere {8(2),
81(4),
5(5)}, and for V2v the coordinates reported are {9 (8),
78 (7),
6(5)}. The coordinates reported in these studies for the area VP are {16 (10),
79(7),
11(5)}.

203

Screening for visual motion perception: direction discrimination Among other visual motion tests, F.D. and R.A. were evaluated on their ability to discriminate direction in two first-order motion tests. In the first, Direction discrimination, the stimulus was a random dot kinematogram (RDK) where all the dots moved with a variable angle to the right or left of an imaginary vertical line, and subjects were required to report whether the RDK moved left or right. Figure 3B shows that R.A. was very impaired on this task for stimuli shown in his contralesional visual field, yet his performance for stimuli shown in the ipsilesional field and that of F.D. in both visual fields were normal. In the second test, Motion coherence (Fig. 3C), the stimulus was a RDK in which the variable parameter was the proportion of coherently moving dots necessary for reliable global direction discrimination of the dynamic cloud of dots

Fig. 3. Screening battery for motion direction discrimination: (A) Schematic view of the direction discrimination task. The stimulus presented a field of evenly distributed dots all moving in the same direction, either slightly to the right or to the left of an imaginary vertical line (two small lines placed outside the stimulus indicated true vertical). (B) Results on this task from normal controls and patients F.D. and R.A. for stimuli presented in the right and left visual field, at 2 eccentricity. The y axis indicates the smallest angular difference (in degrees of visual angle) from vertical needed for reliable discrimination of direction of motion. (C) A schematic view of the motion coherence stimulus. The filled circles denote signal, that is dots that translate in the same direction — up, down, left, or right. Open circles denote masking motion noise, dots that are replotted at random spatial locations from one frame to another within the stimulus aperture. The arrows refer to the magnitude of the dot jumps. (D) Results on this task from normal controls and patients F.D. and R.A. for stimuli presented in the right and left visual field, at 2 eccentricity. The y axis represents the proportion of the coherently moving dots for reliable direction discrimination. (E) A schematic view of the flickering bar test. The stimulus consists of dense static random dots and a flickering bar (square-wave grating) moving up and down over the static random dot pattern background. The percentage of flickering dots was varied in a staircase procedure. Flickering is

obtained by random changing polarity of the dots within the bar. (F) Results from F.D. and R.A. on this task. The y axis represents proportion of the flickering dots necessary to determine the direction of motion. (G) A schematic view of the display in the long range motion task The display consisted of two pairs of vertically oriented Gabor patches positioned orthogonally to each other and arranged at the four corners of an imaginary square centered on a cross hair fixation mark. The stimulus consisted of four consecutive frames, displayed twice in succession to give a total of eight frames in one trial. The eight frames, each visible for 75 ms, interleaved with seven 45 ms stimulus intervals, are displayed in one of two sequences, corresponding to clockwise or counterclockwise rotation. The Gabors of each pair have the same spatial frequency and during a ‘rotation’ only the position of the Gabors changes, not their orientation. One pair of Gabors, the reference, was held constant at 5 cycles/deg. The other pair could have one of five different central spatial frequencies: 1, 1.7, 3, 5 and 10 cycles/deg. The separation between centers of like Gabors was 3.6 and for a 45 rotation each Gabor travelled 1.4 . The viewer discerns movement from the change in position of the textures. (H) The results of the long range motion tasks from control subjects (the gray area indicates mean and standard deviation) and F.D. (open squares) and R.A. (filled circles) show that F.D. was impaired while R.A. showed normal performance. In all the tests, the data illustrate threshold and error bars representing  1 SD. In all the tests, except Longrange motion, subjects fixated 2 off the lateral edge of the stimulus at midline level. In Long-range motion, fixation was in the middle. Using a staircase procedure the subject was shown all these stimuli (except the motion coherence test which was a 4AFC) in a 2AFC paradigm.

204

(Newsome et al., 1990; Newsome and Pare´, 1988). Figure 3D shows that R.A. was impaired on this task, while F.D.’s performance was normal. (It is interesting to note that F.D.’s performance on this test was initially in the impaired range for stimuli presented in the visual field contralateral to the lesion but, within 2 months, had recovered to normal (Vaina and Cowey, 1996)). In both these tests the speed of the dots was 3 /s. Two screening tests of direction discrimination in second-order motion were also used: motion defined by flicker, the Flickering Bar test (Fig. 3E) adapted from Albright (1992) and Long Range Motion (Fig. 3G) in which direction of motion was defined by differences in spatial frequency matching and position changes (adapted from Green, 1986). The two patients had very different performance on these tests. On the Flickering Bar test (presented at 6 Hz) R.A. performed normally while F.D. was severely impaired for stimuli presented in the right visual field contralateral to his lesion (Fig. 3F). Their performance on this test was in stark contrast with their performance on the first-order direction test. On the Long Range Motion test, F.D. was impaired, while R.A.’s performance was normal on this task. (Details on this task are provided in the legend for Fig. 3G.)

Comparison of direction discrimination in firstand second-order motion perception We used a single kind of stimulus to compare F.D. and R.A.’s ability to discriminate direction in firstand second-order motion. Each pair of first- and second-order stimuli had identical spatial and temporal properties. The reason for carrying out this specific comparison was that these patients’ performance suggested a double dissociation on direction-discrimination between first- and secondorder motion tests on the standard screening tests.

Direction discrimination in first- and second-order motion: motion coherence The pair of global motion-coherence tasks is illustrated schematically in Fig. 4A (first-order) and Fig. 4C (second-order). The task is conceptually

similar to the motion coherence test described above (Fig. 3C). The background consisted of flickering random dots, and subjects had to perform a direction discrimination (left or right) in stochastic first-order or second-order global motion displays. A variable proportion of the signal ‘tokens’ (i.e. small binary black and white texture patches) move coherently, left or right, while the other ‘tokens’ are presented from frame to frame at random location within the aperture. In the first-order version of the stimulus, there is a difference between the mean luminance of the tokens and the background while the contrast is identical. In the second-order version of the stimulus, the tokens differ in mean contrast from the background but not in mean luminance. The stimulus field subtended 10  10 arc, and was presented against a uniform gray background (9.5 cd/ m2) at 2 left or right of a small fixation mark placed at eye level. The stimulus area was divided into a notional grid of 38  38 blocks, each subtending 16 min  16 min. Each block consisted of a dense random dot microtexture made of pixels whose luminance was one of 256 possible gray levels. The dots defining the microtexture could have one of two states: on or off, represented by different gray levels. The number of ‘on’ and ‘off ’ dots within a block was evenly distributed. The mean luminance of a block was the average of its ‘on’ and ‘off ’ dots. Its contrast was the ratio of the difference of the ‘on’ and ‘off ’ dots luminance divided by twice the mean luminance. A block can differ from the background either in mean luminance but not contrast (first-order motion), or in contrast but not mean luminance (second-order motion), and in both cases is called a token. Whether a block was ‘token’ or ‘background’, was randomly assigned at the beginning of every trial and token-block density remained constant at 42% throughout the test. The mean luminance of firstorder motion token blocks was 12.3 cd/m2 and contrast within the block was 0.2, while the mean luminance of second-order blocks was 9.5 cd/m2 and internal contrast was 0.6. The mean luminance of the background was in both cases 9.5 cd/m2 with internal contrast of 0.2. The strength of the motion signal in the stimulus was varied by changing the proportion of the tokenblock micropatterns in a given trial that carried the same undirectional motion signal. The remainder of

205

Direction discrimination in first- and second-order motion: D-max

Fig. 4. Direction discrimination in first- and second-order motion: Motion Coherence. On the left are shown schematic views of the first-order motion coherence task (A) and the second-order motion coherence task (C). On the right, B and D show the threshold coherence necessary to reliably perform these tasks for control subjects and F.D. and R.A. Note that compared to the normal controls, F.D.’s performance on the second order task was impaired for stimulus presentation in his right visual field, while R.A.’s performance was impaired on the first order motion task for stimuli shown in the left visual field contralateral to his lesion.

the token-block micropatterns appeared from frame to frame at random locations, creating the impression of flickering noise. When all the micropatterns reappeared with the same spatial and temporal offset, the display appeared as a cluster of micropatterns all moving to the left or to the right across the flickering background. Token density was 2 tokens per degree, and speed was 3 /s. Using an adaptive staircase procedure the stimulus was presented for 12 frames, with each frame shown for 45 ms, with zero interframe interval. Observers were asked whether the direction of the global motion was rightward or leftward. Threshold of proportion of signal tokens (percent coherent motion) necessary for reliable discrimination of stimulus direction was computed as the mean of the last six reversals in the staircase. Figures 4B and D show that in the visual hemifield contralateral to the lesion R.A. was impaired on the first-order but not on the second-order motion stimulus, whereas F.D. showed the opposite dissociation.

The second-order global motion stimulus contained a high proportion of both temporal (the flickering background) and motion (the ‘noise’ tokens) noise, and, initially F.D. was significantly impaired on global motion direction discrimination (motion coherence task) when the signal was embedded in masking motion noise (Vaina and Cowey, 1996b). R.A. was impaired on first-order global motion (the motion coherence task in the screening test battery). Was the deficit restricted to global motion, or was it a more general direction-discrimination deficit? For these reasons we assessed R.A. and F.D.’s ability to discriminate first and second-order motion in a task which addresses local motion measurements. The concept of this task is similar to the classic D-Max tests (Braddick, 1974, 1980), and at least for the first-order displays, the motion measurements are mediated by motion mechanisms that are spatially local. The characteristics of the display are identical to those in the global stimulus. Motion stimuli consist of two successively presented frames (frame duration 45 ms, and zero inter-frame interval) (Fig. 5). From one frame to the next the ‘token’ blocks are shifted coherently either to the left or to the right, with the remaining background acting as a static viewing window. The specific spatial pattern of texture defining the tokens is independent from frame-to-frame, thus from one frame to the next, the component pixels of the ‘token’ blocks and of the background flickered by randomly changing their state (from ‘on’ to ‘off ’ and vice-versa), yet keeping their mean luminance and contrast identical throughout the trial. Subjects were instructed to keep their gaze on a fixation mark 2 to the left or right of the lateral edge of the display. In a 2AFC task they reported whether the direction of motion was to the left or right. Stimuli were varied by an adaptive staircase procedure and the threshold for the maximum displacement for which observers correctly perceived the direction of motion, was averaged over the last six reversals. The results reveal that R.A. was selectively impaired on the first-order motion for stimuli presented in his left visual field. F.D.’s performance on this task and R.A.’s performance for stimuli

206

Fig. 5. First- and second-order D-max motion task. A and C show schematically the display used to measure direction discrimination in textured random dot kinematograms. In A, the motion is first-order: a group of micropatterns differing from the background in luminance is shifted coherently to the left or to the right. In C, the motion is second-order: the shifting micropatterns differ from the background in contrast but not in mean luminance. B and D show the performance of F.D. and R.A. and control subjects on the two tasks for stimuli presented in the inferior quadrants. F.D. was impaired in his left visual field for the second order stimulus but not for the first order stimulus. R.A. was impaired on first order motion but not on second, for stimuli shown in his contralesional field.

shown in the right visual hemifield were normal. The results on the second-order motion task were exactly the opposite; R.A.’s performance was normal, while F.D. was impaired for stimuli shown in the contralesional visual field.

Discussion of patients R.A. and F.D. R.A.’s lesion is probably centered on the putative functionally defined areas V2 in the occipital lobe, as illustrated in Fig. 2 (also slightly involving V1). In this figure (bottom right) we see the peak of fMRI activation (from Dumoulin et al., 2003) for the firstorder stimulus in normal subjects (on a task not discussed in this paper, but which R.A. actually performed and was very impaired on in the left visual field, contralateral to his right hemisphere lesion (Vaina et al., 1998). This task was not available in our laboratory when F.D. participated in the study). F.D.’s lesion appears to be dorsal to hMT þ , and is

consistent with the findings of the fMRI study of Dumoulin et al. (2002) and Dumoulin et al., 2003) that the anatomical substrate of second-order motion involves this region (Fig. 1, bottom right). Furthermore, F.D.’s psychophysical performance is consistent with the hypothesis that this area might provide at least in part an underlying neural substrate for second-order motion processing. Thus, our psychophysical results in these two patients and the significantly different locations of their lesions support the idea of relative cortical specialization for first- and second-order motion in the early visual areas of the occipital lobe, and in a cortical region posterior to hMT þ , respectively. The first is more specific for processing first-order motion, the latter for the processing of second-order motion. While the anatomical location of the lesion and the psychophysical data from F.D. and R.A. is consistent with that of Dumoulin and colleagues (2002), it is at odds with the functional imaging studies of Smith et al. (1998) and Wenderoth et al. (1999) which suggest the cortical areas VP, or ventral V3, as the putative substrate for second-order motion processing. This area was not directly, and almost certainly even not indirectly, involved in R.A.’s or F.D.’s lesions.

Two patients with dorsal and ventral V3 unilateral lesions I will briefly discuss similarities and differences in behavioral data on first- and second-order motion tasks of two patients with small unilateral infarcts centered in the ventral occipital region, V2/VP (Patient J.V.) and in the dorsal V2/V3 (Patient T.F.; Vaina et al., 2000). Patient J.V., was a 60-year-old right-handed college educated woman who suffered an infarct in the left occipital lobe, as shown in Fig. 6. For 6 weeks after the stroke, she reported that the world looked fragmented, ‘like a Picasso painting’, and she felt very uncomfortable coping with the visual world surrounding her. Patient T.F., was a 60-year-old right handed college educated man who suffered a mild almost infarct in the occipital lobe. Formal Humphrey and Goldmann perimetry visual field testing showed a well defined upper right

207

Fig. 6. Lesion localization in Patients J.V. and T.F. The lesion location in Patient J.V. is shown in green (the first row) and in Patient T.F. is shown in red (the second row). In the first column, lesion locations are shown on inflated images of J.V. and T.F.’s brains. The inflation was obtained using T-1 weighted images of the patients’ brains and was processed with the FreeSurfer software package (Dale et al., 1999; Fischl et al., 1999). For each patient the lesion was identified based on T2-weighted high resolution MRI images and their contours were overlaid on the inflated brain surface. The adjacent columns show again the lesions of J.V. and T.F. registered on T-1 weighted axial brain images in the Talairach space (Talairach and Tournoux, 1988). The axial slices also show the overlap of the lesions with Brodmann areas 17 (light blue), 18 (dark blue), and 19 (yellow) in the occipital lobe.

quadratopsia in J.V., but T.F.’s visual fields were normal. However, he spontaneously reported that he had difficulties seeing motion in the lower left visual quadrant. In both patients full vision was restored within 6 months when the data reported here were obtained. Figure 6 (top), shows J.V.’s infarct localized on the inflated 3D brain obtained from her MRI study together with axial slices registered into the Talairach space (Talairach and Tournoux, 1988), illustrating the lesion encroaching Brodmann areas 18 and 19 ventrally in the left hemisphere. Figure 6 (bottom), shows T.F.’s lesion localized on the inflated 3D brain obtained from the MRI study and relevant axial slices in the Talairach space, illustrating a small right hemisphere infarct in Brodmann area 18, just above the calcarine fissure. Using anatomical templates generated on the basis of the Talaraich coordinates defined in the literature for the retinotopic areas and for hMT þ , the ROI corresponding to J.V.’s lesion showed that it encroached BA 18 with the centre in {
14,
78,
10} and BA 19 with the centre in {
20,
80,
12}.

The lesion closely overlapped with the ventral areas V2v {
10 (7),
79 (8),
10(6); in BA18} and VP {
19(8),
79(8)
13 (6); in BA19} as defined by Mendola et al. (1999) and Sunaert et al. (1999). The center of the ROI corresponding to T.F.’s lesion, overlapped with the dorsal retinotopic areas, V2d {11(9),
93(6), 11(10)} and V3 {18(11),
93(6), 13(10)}. Figure 7 shows the results of J.V. and T.F. on the same tests for screening for visual motion perception that were reported above for F.D. and R.A. (Fig. 3). T.F.’s performance on both the direction discrimination and motion coherence tasks, was impaired for stimuli shown in the left visual field contralateral to his right hemisphere lesion (Fig. 7B and D). J.V.’s performance on these tasks was normal. On the other hand, T.F.’s performance was entirely normal on the flickering bar test, which is a second-order motion task. J.V. was impaired for stimuli shown in her contralesional visual field. It is of note that both patients performed within the normal range on the long range motion test (Fig. 7H).

208

Fig. 7. Screening battery for motion direction discrimination. As in Fig. 3, a schematic view of the stimuli and performance of normal controls and two patients is illustrated. A and B: Direction discrimination; C and D: Motion Coherence; E and F: Flickering bar; G and H: Long-range motion. Specific to the comparison of J.V. and T.F. on these tasks, is that in the first three they show a double dissociation of deficits (T.F. is normal on second order but impaired in the first order in the visual field contralateral to the lesion, and J.V. has the opposite performance). However, on the Long-range motion task both patients had normal performance.

The same two pairs of direction discrimination tasks in first- and second-order motion, Motion Coherence and D-max were also administered to these patients (Fig. 8). On these tests T.F. and J.V. presented a double dissociation of deficits. Similar to patient R.A., T.F. was normal on the secondorder motion tasks (Fig. 8D and H), but selectively impaired on the first-order tasks (Fig. 8B and F) for stimuli presented in the contralesional field. J.V., similar to patient F.D., had the opposite pattern of performance. She had normal performance

Fig. 8. Top two rows: First- and second-order D-max motion tasks. A and C show schematically the display used to measure direction discrimination in textured random dot kinematograms. In A, the motion is first-order: a group of micropatterns differing from the background in luminance is shifted coherently to the left or to the right. In C, the motion is second-order: the shifting micropatterns differ from the background in contrast but not in mean luminance. B and D show the performance of J.V. and T.F. and control subjects on the two tasks for stimuli presented in each visual field separately. J.V. was impaired in her right visual field for the second order stimulus but not for the first- order stimulus. T.F. was impaired on first order motion but not on second, for stimuli shown in the contralesional field. Error bars represent  1 SD. Bottom two rows: First- and second order motion coherence tasks. E and G show schematic views of the first- and second-order global motion coherence task, respectively. F and H show the threshold coherence necessary to reliably perform these tasks from control subjects and patients J.V. and T.F. Error bars represent  1 SD. Compared to the normal control subjects, T.F.’s performance on the first order task was impaired for stimulus presentation in his contralesional field. His performance on the second-order motion stimuli was normal for presentation in both visual fields. J.V.’s performance was the opposite. She had normal performance on the first-order task, but was selectively impaired on the second-order task for stimuli presented in the contralesional field.

209

on the first-order motion stimuli (Fig. 8B and F) but scored at a very impaired level on both second-order motion tasks when the stimuli were presented in the contralesional field (Fig. 8D and H).

Conclusion In this chapter we discussed selective deficits on direction discrimination selective to first- or secondorder motion in four neurological patients who were also studied neuroanatomically on the basis of the MRI’s of their brains. The MRI’s of the patients’ brains were resliced and registered in the Talairach space (Talairach and Tournoux, 1988) in order to provide a uniform coordinate system for comparison and reference to the fMRI localization of retinotopic areas and of hMTþ . However, as a point of caution, while a fine structural localization of the lesion can be obtained, it is unlikely that just based on structural MRI information one can determine precisely what cortical areas have a reduced input or output. Since for all four patients the MR images were registered in stereotaxic space (Talairach and Tournoux, 1988), we applied scripts developed in our laboratory to describe the location of the lesions using as reference the spatial coordinates (x, y, z) of visual areas published in the literature. Human fMRI studies have implicated the early visual areas (V1 and V2) in processing first-order motion (Dumoulin et al., 2002, 2003). Smith et al. (1998) found that areas V3 and VP are more responsive to second-order than to first-order motion, while Dumoulin et al. (2002) found instead that a region posterior to hMT þ was preferentially activated by second-order motion. How can we relate the psychophysical data from the four patients presented here to the fMRI result? Where does this leave us in explaining the performance of the four patients discussed here on direction discrimination in first- and second-order motion tasks? Area VP was clearly not involved in R.A.’s lesion and so he might have used it in processing the secondorder motion stimuli. Area VP is connected to other motion areas that are rich in direction-selective neurons, for example area MT (Felleman and Van Essen, 1991) and physiological studies have shown that neurons in MT respond to direction of motion

whether motion is first- or second-order. This could explain R.A.’s normal performance on a broad range of direction discrimination tasks of second-order motion. Moreover, the involvement in R.A.’s lesion of areas V2, known to contribute to the analysis of first-order stimuli and to project to MT and other higher motion areas, may explain his severe impaired performance on first-order motion direction discrimination tasks. T.F.’s lesion encroached with areas V2d and V3, which may explain his normal performance on the second-order motion and his impaired performance on the first-order motion tasks. However, we should note that Smith et al. (1998), found that V3 also responds to second-order motion. This is at odds with our findings, and should be further investigated in patients with small cortical lesions and by fMRI. J.V.’s lesion, located more ventrally directly involves area VP and her impairment in the contralesional visual field on local or global secondorder motion defined by flicker or contrast supports a role of this area in second-order motion. However, her normal performance on the long-range motion task suggests that perhaps the different second-order motion mechanisms have different neuroanatomical substrates. F.D., whose lesion was mainly posterior to hMT þ , was selectively impaired on direction discrimination in all the second-order motion tasks described here. It is relevant to recall that F.D.’s lesion intersected with the ROI found by Dumoulin et al. (2002, 2003) to strongly respond to second-order motion stimuli. It is also interesting to note that, different from J.V., F.D. was impaired on all secondorder motion tasks, including long-range motion which is higher level. Performance on this task depends on the difference in the carrier spatial frequency between the pairs of Gabor patches forming the stimulus, which is consistent with the operations of a secondorder long-range mechanism (Werkhoven et al., 1993). Different from the other tasks described here, it is possible that long-range motion is actually a higher level mechanism based on attention. The idea of several types of second-order motion mechanisms, and even a third-order motion system (Lu and Sperling, 2001) and that they may have different neuronal substrates is an important one and worth pursuing. Studies of neurological patients can provide double dissociations of deficits (as shown in

210

this paper) and the patients’ self-report of what they do and do not perceive in the everyday world will provide invaluable information about these motion mechanisms’ relevance to perception beyond laboratory setting where there are carefully controlled stimuli. The tentative suggestions made here on the basis of psychophysical and indirectly on fMRI studies would have been undoubtedly stronger if we had been able to also obtain a functional anatomical map of these patients, and thus register their functionally defined retinotopic areas and hMT þ with their brain activity specific to the first- or second-order motion tasks described here. Unfortunately this was not possible for these patients. We believe that carefully designed fMRI studies of neurological patients with small cortically centered lesions will refine our understanding of the functional architecture of the human visual motion system.

Acknowledgments This research has been supported in part by the NIH grant 2 RO1 EY-07861 to LMV. We are grateful to Serge Dumoulin for providing parts of Figures 1 and 2.

References Albright, T. and Stoner, G. (1995) Visual motion perception. Proc. Natl. Acad. Sci. USA, 92: 2433–2440. Albright, T.D. (1992) Form-cue invariant motion processing in primate visual cortex. Science, 255: 1141–1143. Andersen, R.A. (1997) Neural mechanisms of visual motion perception in primates. Neuron, 18: 865–872. Azzopardi, P. and Cowey, A. (2001) Motion discrimination in cortically blind patients. Brain, 124: 30–46. Baker, C.L. (1999) Central neural mechanisms for detecting second-order motion. Curr. Opin. Neurobiol., 9: 461–466. Barron, J.L., Fleet, D.J. and Beauchemin, S.S. (1994) Performance of optical flow techniques. Int. J. Comp. Vis.: 43–77. Boulton, J. and Baker, C. (1993a) Different parameters control motion perception above and below a critical density. Vision Res., 33: 1803–1811. Boulton, J.C. and Baker, C.L. (1993b) Dependence on stimulus onset asynchrony in apparent motion: evidence for two mechanisms. Vision Res., 33: 2013–2019.

Braddick, O. (1974) A short-range process in apparent motion. Vision Res., 14: 519–527. Braddick, O.J. (1980) Low-level and high-level processes in apparent motion. Phil. Trans. R. Soc. B., 290: 137–151. Braun, D., Petersen, D., Scho¨nle, P. and Fahle, M. (1998) Deficits and recovery of first- and second-order motion perception in patients with unilateral cortical lesions. Europ. J. Neurosci., 10: 2117–2128. Cavanagh, P. and Mather, G. (1990) Motion: the long and short of it. Spatial Vision, 4: 103–129. Chubb, C. and Sperling, G. (1988) Drift-balanced random stimuli: a general basis for studying non Fourier motion perception. J. Opt. Soc. Am. A, 5: 1986–2007. Chubb, C. and Sperling, G. (1989) Two motion perception mechanisms revealed through distance driven reversal of apparent motion. Proc. Natl. Acad. Sci. USA, 86: 2985–2989. Churan, I. and Ilg, U.J. (2001) Processing of second-order motion stimuli in primate middle temporal area and medial superior temporal area. J. Opt. Soc. Am. A, 18: 2297–2306. Clifford, C. and Vaina, L. (1999) A computational model of selective deficits in first and second-order motion processing. Vision Res., 39: 113–130. Clifford, C.W., Freedman, J.N. and Vaina, L.M. (1998) Firstand second-order motion perception in Gabor micropattern stimuli: psychophysics and computational modelling. Cogn. Brain Res., 6: 263–271. Dale, A.M., Fischl, B. and Sereno, M.I. (1999) Cortical surfacebased analysis: I. Segmentation and surface reconstruction. NeuroImage, 9: 179–194. Derrington, A.M. and Badcock, D.R. (1992) Two-stage analysis of the motion of 2-dimensional patterns, what is the first stage? Vision Res., 32: 691–698. Derrington, A.M., Badcock, D.R. and Henning, G.B. (1993) Discriminating the direction of second-order motion at short stimulus durations. Vision Res, 33: 1785–1794. Dumoulin, S., Baker, C.L. and Hess, R.F. (2002) Cortical specialization for processing first- and second-order motion in parietal and occipital lobe: an fMRI study. Soc. Neurosci. Abstr., 219.1. Dumoulin, S., Bitta, R., Kabani, N., Baker, C., Goualher, G., Pike, G. and Evans, A. (2000) A new anatomical landmark for reliable identification of human area V5/Mt: a quantitative analysis of sulcal patterning. Cereb. Cortex, 10: 454–463. Dumoulin, S., Baker, C.L., Hess, R. and Evans, A. (2003) Cortical specialization for first and second-order motion. Cereb. Cortex, in press. Felleman, D.J. and Van Essen, D.C. (1991) Distributed hierarchical processing in the primate cerebral cortex. Cereb. Cortex, 1: 1–47. Fischl, B., Sereno, M.I. and Dale, A.M. (1999) Cortical surfacebased analysis II: inflation, flattening, and a surface-based coordinate system. NeuroImage, 9: 195–207.

211 Fleet, D.J. and Langley, K. (1994) Computational analysis of non-Fourier motion. Vision Res., 34: 3057–3079. Geesaman, B.J. and Andersen, R.A. (1996) The analysis of complex motion patterns by form/cue invariant MSTd neurons. J. Neurosci., 16: 4716–4732. Green, M. (1986) What determines correspondence strength in apparent motion? Vision Res., 26: 599–607. Greenlee, M.W. and Smith, A.T. (1997) Detection and discrimination of first- and second-order motion in patients with unilateral brain damagem. J. Neurosci., 17: 804–818. Johnston, A., McOwan, P.W. and Buxton, H. (1992) A computational model of the analysis of some first-order and second-order motion patterns by simple and complex cells. Proc. R. Soc. Lond. B., 250: 297–306. Johnston, A.J. and Clifford, C.W.G. (1995) Perceived motion of contrast modulated gratings: predictions of the multichannel gradient model and the role of full-wave rectification. Vision Res., 35: 1771–1783. Lu, Z.-L. and Sperling, G. (2001) Three-systems theory of human visual motion perception: review and update. J. Opt. Soc. Am. A, 18: 2331–2370. MacDonald, D., Kabani, N., Avis, D. and Evans, A. (2000) Automated 3-D extraction of inner and outer surfaces of cerebral cortex from MRI. Neuroimage, 12: 340–356. Mendola, JD, Dale, AM, Fischl, B., Liu, AK. and Tootell, RBH (1999) The representation of illusory and real contours in human cortical visual areas revealed by functional magnetic resonance imaging. J. Neurosci., 19(19): 8560–8852. Nakayama, K. (1985) Biological image motion processing: A review. Vision Res., 25: 625–660. Nawrot, M., Rizzo, M., Rockland, K.S. and Howard, M. (2000) A transient deficit of motion perception in human. Vision Res., 40(24): 3435–3446. Newsome, W.T., Britten, K.H., Salzman, C.D. and Movshon, J.A. (1990) Neuronal mechanisms of motion perception. Cold Spring Harb. Symp. Quant. Biol., 55: 697–705. Newsome, W.T. and Pare´, E.B. (1988) Relation of cortical areas MT and MST to pursuit eye movements. II. Differentiation of retinal from extraretinal inputs. J. Neurosci., 8: 2201–2211. Nishida, S.Y., Ledgeway, T. and Edwards, M. (1997) Dual multiple-scale processing for motion in the human visual system. Vision Res., 37: 2685–2698. O’Keefe, L.P. and Movshon, J.A. (1996) First- and secondorder motion processing in the superior temporal sulcus of the alert macaque. Soc. Neurosci. Abstr., 22: 716. O’Keefe, L.P. and Movshon, J.A. (1998) Processing of first- and second-order motion signals by neurons in area MT of the macaque monkey. Vis. Neurosci., 15: 305–317. Plant, G.T., Laxer, K.D., Barbaro, N.M., Schiffman, J.S. and Nakayama, K. (1993) Impaired visual motion perception in the contralateral hemifield following unilateral posterior cerebral lesions in humans. Brain, 116: 1303–1335.

Plant, G.T. and Nakayama, K. (1993) The characteristics of residual motion perception in the hemifield contralateral to lateral occipital lesions in humans. Brain, 116: 1337–1353. Rademacher, J., Caviness, V.S., Steinmetz, H. and Galaburda, A.M. (1993) Topographical variation of the human primary cortices: implications for neuroimaging, brain mapping, and neurobiology. Cereb. Cortex, 3: 313–329. Rademacher, J., Galaburda, A.M., Kennedy, D.N., Filipek, P.A. and Caviness, V.S., Jr. (1992) Human cerebral cortex: localization, parcellation, and morphometry with magnetic resonance imaging. J. Cogn. Neurosci., 4: 352–374. Schofield, A.J. (2000) What does second-order vision see in an image? Perception, 29: 1071–1086. Sekuler, S, Watamaniuk, S. and Blake, R. (2002). Stevens’ handbook of experimental psychology. In: Yantis, H.S. (Ed.), Sensation and Perception, Vol. 1. Wiley, New York, pp. 121–153. Smith, A.T., Greenlee, M.W., Singh, K.D., Kraemer, F.M. and Hennig, J. (1998) The processing of first- and second-order motion in human visual cortex assessed by functional magnetic resonance imaging (fMRI). J. Neurosci., 18: 3816–3830. Snowden, R.J. (1994). Motion processing in the primate cerebral cortex. In: Smith, A.T. and Snowden, R.J. (Eds.), Visual Detection of Motion. Academic Press, London. Somers, D.C., Dale, A.M., Seiffert, A.E. and Tootell, R.B. (1999) Functional MRI reveals spatially specific attentional modulation in human primary visual cortex. Proc. Natl. Acad. Sci. USA, 96: 1663–1668. Sunaert, S., van Hecke, P., Marchal, G. and Orban, G.A. (1999) Motion-responsive regions in the human brain. Exp. Brain Research, 127: 355–370. Talairach, J. and Tournoux, P. (1988). Co-Planar Stereotaxic Atlas of the Human Brain. Thieme Medical Publishers, New York. Taub, E., Victor, J.D. and Conte, M.M. (1997) Nonlinear processing in short-range motion. Vision Res., 37: 1459–1477. Vaina, L. and Cowey, A. (1996) Impairment of the perception of second-order motion but not first-order motion in a patient with unilateral focal brain damage. Proc. R. Soc. Lond. B., 263: 1225–1232. Vaina, L.M., Makris, N. and Cowey, A. (1996) The neuroanatomical damage producing selective deficits of first or second-order motion in stroke patients provides further evidence for separate mechanisms. NeuroImage, 3: 360. Vaina, L.M., Makris, N., Kennedy, D. and Cowey, A. (1998) The selective impairment of the perception of first-order motion by unilateral cortical brain damage. Vis. Neurosci., 15: 333–348. Vaina, L., Cowey, A. and Kennedy, D. (1999b) The neuroanatomical damage producing selective deficits to first or

212 second-order motion in stroke patients. Human Brain Mapping, 7: 67–77. Vaina, L.M., Soloviev, S., Bienfang, D.C. and Cowey, A. (2000) A lesion of cortical area V2 selectively impairs the perception of the direction of first-order visual motion. NeuroReport, 11: 1039–1044. Vaina, L.M., Solomon, J., Chowdhury, S., Sinha, P. and Belliveau, J.W. (2001) Functional neuroanatomy of biological motion perception in humans. Proc. Nat. Acad. Sci., 98(20): 11,656–11,661. Van Oostende, S., Sunaert, S., Van Hecke, P., Marchal, G. and Orban, G.A. (1997) The kinetic occipital (KO) region in man: an fMRI study. Cerebral Cortex, 7: 690–701. Wenderoth, P., Watson, J.D.G., Egan, G.F., TochonDanguy, H.J. and O’Keefe, G.J. (1999) Second order components of moving plaids activate extrastriate cortex:

a positron emission tomography study. NeuroImage, 15: 227–234. Zeki, S., Perry, R.J. and Barteis, A. (2003) The processing of kinetic contours in the brain. Cerebral Cortex, 13: 189–202. Werkhoven, P., Sperling, G. and Chubb, C. (1993) The dimensionality of texture-defined motion: a single channel theory. Vision Res., 33: 463–486. Worsley, K., Marrett, S., Neelin, P., Vandal, A., Friston, K. and Evans, A. (1996) A unified statistical approach for determining significant signals in images of cerebral activation. Human Brain Mapping, 4: 58–73. Worsley, K.J., Liao, C.H., Aston, J., Petre, V., Duncan, G.H., Morales, F. and Evans, A.C. (2002) A general statistical analysis for fMRI data. NeuroImage, 15: 1–15. Zhou, Y.-X. and Baker, C.L. (1993) A processing stream in mammalian visual cortex neurons for non-Fourier responses. Science, 261: 98–101.

Progress in Brain Research, Vol. 144 ISSN 0079-6123 Copyright ß 2004 Elsevier BV. All rights reserved

CHAPTER 15

Reaching between obstacles in spatial neglect and visual extinction A. David Milner* and Robert D. McIntosh Cognitive Neuroscience Research Unit, Wolfson Research Institute, University of Durham, Queen’s Campus, University Boulevard, Stockton-on-Tees TS17 6BH, UK

Abstract: The aim of the present studies was to investigate whether ‘perception’ and ‘visually guided action’ could be dissociated with regard to two different aspects of the neglect syndrome. In the first study we tested a group of patients with neglect in two tasks, both within the same experimental setting. One task was to bisect a space between two objects, while the other required subjects to reach between the same pair of objects en route to a target area, so that the objects became potential obstacles to the reach. In the second study we tested a patient with visual extinction to double simultaneous stimulation, using a similar reaching task. Our aim was to determine whether visual awareness of obstacles in the workspace was necessary for successful navigation. In both studies we found evidence that reaching responses took normal account of the presence and location of obstacles on the left side, despite the tendency to neglect such left-sided information in more explicit perceptual tasks. We interpret both sets of results within a theoretical framework that identifies on-line visuomotor control with the occipito-parietal ‘dorsal stream’ (along with associated premotor and subcortical structures), and visual perception with the occipito-temporal ‘ventral stream’, plus associated temporo-parietal areas.

Introduction

Extinction can be characterized as a lateral bias of spatial attention occurring in the context of a reduced attentional capacity (e.g. Driver et al., 1997), whereby a stimulus on the ipsilesional side briefly attracts attention to the exclusion of simultaneous (or near-simultaneous) stimuli located in more contralesional locations. Other symptoms of neglect, such as rightward errors in line bisection, and left-sided deficits in visual search tasks, are generally regarded as more central to the essence of spatial neglect, though theories as to their causation range widely. The present chapter is concerned with exploring the phenomena of visual neglect and extinction from an unconventional starting point, by asking whether these conditions affect a person’s ability to guide their hand to a desired location while avoiding intervening obstacles lying on the right and left of the workspace. The logic of this approach derives from the visual processing model set out

The neglect syndrome is a complex and multi-faceted group of symptoms (Heilman et al., 2002), some of which hang together better than others. The brain lesions that cause neglect symptoms vary widely, though there is a region around the parieto-temporal junction, particularly in the right hemisphere in cases of left-sided neglect, that is included in the majority of cases (Vallar, 1993; Vallar et al., 1994; Karnath et al., 2001). Perceptual extinction to double simultaneous stimulation is traditionally included as one of the cardinal symptoms of the neglect syndrome (Heilman et al., 2002), though it frequently occurs in the absence of other signs of neglect, and several authors have argued that it may be causally independent (e.g. Milner, 1987, 1997). *Corresponding author. Tel.: þ 44-1642-333850; Fax: þ 441642-385866; E-mail: [email protected] DOI: 10.1016/S0079-6123(03)14401-5

213

214

by Milner and Goodale (1995). They argued that a distinction should be drawn between visual processing for perception and visual processing for action. According to the model, the former is embodied in the occipito-temporal ‘ventral stream’ of cortical processing, while the latter is embodied in the occipito-parietal ‘dorsal stream’ (Glickstein and May, 1982; Ungerleider and Mishkin, 1982). Although all visual processing can ultimately be expressed in the form of overt behavior (and indeed that is why it exists) the dorsal stream is thought to interact directly with premotor cortex and brainstem structures to transform visual information into motor coordinates, while the ventral stream can only influence action indirectly. What the latter route loses in immediacy, of course, it gains in flexibility. We have suggested (Milner and Goodale, 1995; Milner, 1997; Milner and McIntosh, 2002) that the temporo-parietal region of the right hemisphere that is most heavily implicated in the causation of neglect probably functions in good part as a high-level representational system that is fed principally by visual inputs arising from the ventral stream. The region can perhaps be regarded as the endpoint of the perceptual processing pathway, where the everchanging contents of our visual consciousness are represented, operating under the flexible control of attentional and executive systems located in superior parietal and prefrontal regions (Driver and Mattingley, 1998; Driver and Vuilleumier, 2001). Of course there is no doubt that overt behavior is affected by neglect, not just the patient’s inner experience. Nonetheless, the hypothesis is that much of this abnormal behavior is driven by the indirect ‘perceptual’ route, rather than by the direct ‘visuomotor’ route. In other words, it is an expression of the distorted and disrupted perceptual experience of the patient rather than a direct result of a damaged visuomotor control system in the dorsal stream. Indeed, even some apparently pure motor manifestations of neglect, such as biased ocular exploration in darkness (e.g. Hornak, 1992; Karnath et al., 1998), might derive from a distorted internal representation of external space. A strong prediction of this hypothesis is that, in many cases of neglect, visual processing for direct action should be free from the pronounced perceptual biases that characterize the

syndrome. It is this proposal that we aimed to test in the present studies. It is necessary to distinguish this proposal of dissociated perceptual and visuomotor processing in neglect from the more familiar concept of a division between perceptual and motor contributions to neglect. Traditionally, there has been an assumption that the symptoms of neglect arise from spatial biases either in the processing of sensory input or in the programming of motor responses. A number of attempts have been made to distinguish between input- and output-related biases in neglect, though the validity of many of these studies has been questioned (see Mattingley and Driver, 1997 for a review). In contrast to this serial input–output distinction, however, the model of Milner and Goodale (1995) is concerned with a distinction between parallel visual processing systems, respectively underlying conscious perceptual awareness and automatic goal-directed actions. In relation to neglect symptoms, this model proposes that the expression of spatial biases in visual processing should depend upon the behavioral context in which the visual processing takes place. Specifically, the symptoms of neglect should be more often linked to the visual processing subserving perceptual awareness than to the processing underlying the guidance of automatic goal-directed actions. Whether on not the visual guidance of simple direct actions in neglect patients is subject to lateral spatial biases has been a matter of some debate. Goodale et al. (1990) and Jackson et al. (2000) reported rightwardly curved trajectories in the pointing movements of recovered left neglect patients, and Harvey et al. (1994) observed a similar effect in right hemisphere damaged patients without neglect. Such effects, however, are not ubiquitous. For instance, Chieffi et al. (1993) observed no abnormal curvature in the pointing movements of a recovered left neglect patient to isolated visual targets. More importantly, these phenomena have not been substantiated by direct tests of patients with full left neglect. Perenin (1997) failed to observe any directional skewing of visually guided (open-loop) pointing movements among four neglect patients. Similarly, Karnath and colleagues (1997) reported accurate open- and closed-loop visual guidance in five chronic neglect patients, both in terms of terminal

215

accuracy and hand trajectories, a result replicated recently by Harvey et al. (2002) in four neglect patients performing a simple grasping task. On the basis of such findings, Karnath et al. (1997) have argued that abnormally curved movement trajectories are not characteristic of patients with neglect, although they may be characteristic of optic ataxia (Perenin, 1997). The evidence for a systematic spatial bias in the visually guided actions of neglect patients is therefore not strong. However, at least two published studies have claimed to observe specifically visuomotor manifestations of neglect. Behrmann and Meegan (1998) required six patients with left visual neglect to reach for a target LED, presented alone or simultaneous with a distractor LED, with target and distractor distinguished by color. Like normal subjects, neglect patients were slower to initiate a response to any target that was accompanied by a distractor but, compared to controls, they had an increased RT cost for a distractor on the right and a reduced cost for a distractor on the left. Behrmann and Meegan concluded that information from the neglected side must be processed minimally, if at all, in the visuomotor domain. However, we suggest that it is misleading to portray this effect as visuomotor, since conscious perceptual discrimination of the target from any distractor would be a necessary prerequisite for selecting which stimulus to respond to in the task. Thus, the asymmetrical influence of left and right-sided distractors is most likely to reflect a difficulty in choosing the left stimulus in the presence of the irrelevant stimulus on the right, irrespective of what response has to be made to the target. An accentuated influence of right-sided distractors has also been reported for a recovered left neglect patient performing a grasping task (Chieffi et al., 1993). Again, however, it is unclear whether this reflects unbalanced visuomotor processing, or interference with the perceptual discrimination of the target from the distractor object. A more direct strategy to assess the normality of visuomotor processing in neglect is to compare performance between tasks where the stimuli are matched as closely as possible, but the mode of response differs. Several studies have already employed this general strategy to investigate reaching and grasping in neglect. Robertson et al. (1995, 1997)

found that neglect patients showed significantly less bisection error when asked to pick up a rod at its midpoint than when asked only to point to the rod’s midpoint. They argued that the pointing task reflected disordered perception in their patients, but that reaching to grasp was an action driven more directly by the dorsal than the ventral stream, thus enabling an improvement in bisection accuracy. In a similar vein, Pritchard et al. (1997) found that a neglect patient (E.C.) was able to calibrate her finger-thumb grip aperture accurately when reaching to grasp different sized cylinders, with no asymmetry in grip size between target locations on the two sides of space. Yet when asked to indicate her perceived size of the cylinders, she consistently underestimated them when they were located on her left as compared with her right side. Later studies with groups of neglect patients have replicated this symmetrical grasping behavior (Harvey et al., 2002; McIntosh et al., 2002), though there was no direct demonstration in those papers that the cylinders were perceived as having different sizes on the two sides of space. In the present paper we report a series of studies aimed at assessing the visuomotor processing underlying simple reaching movements in neglect. In the first study, we asked neglect patients to perform two tasks, both involving the presentation of two upright cylindrical stimuli, whose locations varied from trial to trial. In the ‘bisection’ task, the patient was asked to judge the midpoint between the two cylinders and to place their finger at that location. In the ‘reaching’ task, the patient was asked to move the hand rapidly from a start point to touch a ‘target zone’ located beyond the two objects. This second task was designed to be a simple act of reaching under direct visual control, with an implicit requirement that the reach needs to be executed so as to minimize the risk of collision with either cylinder. The conceptual similarity between these two tasks is that they both require the subject to take account of the location of the cylinders on the left and right simultaneously. In one situation, this demand is part of the explicit spatial analysis underlying a bisection response. In the other, it arises implicitly in computing the optimal spatial path for a visually guided reaching movement. Preliminary data on these tasks have been presented already (Milner and

216

McIntosh, 2002). We found good evidence in this experiment for a dissociation between the two tasks: most of the patients behaved like healthy controls in the reaching task, though they showed asymmetrical behavior when attempting to make explicit bisection responses. In other words, we found that most neglect patients were able to take a potential obstacle on the left as fully into account as one located on the right when reaching between them. However these results do not of course demonstrate that such obstacle avoidance can take place even when there is no conscious perception of the left obstacle. After all, there was no restriction of viewing time, and no independent evidence that the patients were unaware of the presence of the left-side object. We therefore conducted a single-case study in which we attempted to address this question. We tested a patient (V.E.) with persistent visual extinction who, under appropriate conditions, would often report seeing only the right stimulus when in fact a left stimulus was present as well. Again using a reaching task, we assessed whether his awareness or unawareness of the left object, when two objects were present, affected the trajectories taken by his hand. Normal subjects vary their reaches systematically according to the presence of a left-alone, right-alone, or bilateral pair of objects. Our question was whether V.E. would show these same phenomena, even when he did not ‘see’ the obstacle on the left — or would his reaches on such ‘unaware’ trials be more similar to those he made when there literally was only one object, located on the right? Our data indicate that a failure to ‘see’ an obstacle, due to visual extinction, does not necessarily compromise the ability to process the location of that obstacle in order to avoid it when reaching.

Experiment 1: Reaching and bisection in neglect Twelve patients with left visual neglect following unilateral right hemisphere stroke, and ten agematched controls, took part in this experiment. All patients displayed neglect on three or more of five standard diagnostic tests. Full details of the patients can be found in McIntosh et al. (submitted). Subjects were seated in front of a 60 cm square white stimulus board depicted in Fig. 1, with the right index finger

placed at the start position. Two dark gray cylinders (24.5 cm tall and 3.5 cm in diameter) could be fixed into the board, one to either side of the midline, at a depth of 25 cm with respect to the start position, and 20 cm in front of a 5 cm deep gray strip that spanned the far edge of the board. Each cylinder could occupy one of two possible locations, with its inside edge either 8 cm or 12 cm away from the midline. The factorial combination of these four locations thus created four stimulus configurations. Each patient performed two different tasks on this stimulus board, with the order of tasks alternating between subjects within each group. All responses were recorded by sampling the position of a magnetic marker attached to the nail of the right index finger, at a frequency of 86.1 Hz (Minibird, Ascension Technology Ltd.). Full

Fig. 1. Plan view of the apparatus used in Experiment 1. Cylinders were always presented in pairs, one on the left and one on the right of the midline. The open circles show their possible locations, with inside edges 8 and 12 cm from the midline. Each trial began with the subject’s right index finger resting on the start position (black dot). In the bisection task, the subject was requested to place their finger midway between the two cylinders, and the dependent measure was the lateral position of the response with respect to middle of the stimulus board. In the reaching task, the subject was instructed to reach out and touch the gray strip as fast as possible. The dependent measure was the lateral position of the right index finger as it crossed the virtual line joining the two cylinder locations (dotted line).

217

methodological details can be found elsewhere (McIntosh et al., submitted). In the bisection task, the subject was asked to place their right index finger midway between the two cylinders. The subject was told that this was a test of ‘accuracy of judgment’, and that an unlimited time was available for each response. On each trial, the subject was allowed to adjust their finger position until satisfied that it was exactly midway between the cylinders. The position of the marker attached to the index finger was then sampled for 1 s. The dependent measure was the average lateral position (P) of the finger marker, with respect to the midline of the stimulus board, during this 1-s period. Each subject made 48 bisection responses, with each of the four stimulus configurations presented 12 times in a fixed pseudo-random order. In the reaching task, the subject received the instruction ‘On the go signal, reach out and touch the gray strip as quickly as you can’, and was told that this was a test of ‘speed of movement’. Prior to the task, they were informed that, whenever a cylinder was present, there would be one on the left and one on the right, and that they should pass their hand between the two cylinders, rather than around the outside edge of the board. The presence of the cylinders was not otherwise mentioned. Each reaching response was recorded in full, and the dependent measure was the lateral position (P) of the index finger marker as it crossed the virtual line joining the two cylinder locations (the exact value of P was

estimated by linear interpolation). Each subject made 60 reaches, 12 for each of the four cylinder configurations, and 12 in which no cylinder was present on the board, in a fixed pseudo-random order. The 12 trials with no cylinder in place were included to check for any systematic reaching biases when the response was not constrained by potential obstacles. An independent t-test (corrected for unequal variances) performed on these responses alone found no difference between the groups [t (13) ¼ 0.59, P ¼ 0.57], with both groups passing on average slightly to the left of the board midline. This constant bias is unsurprising, since the marker from which responses were recorded was attached to the right index finger and was thus on the left side of the hand in its palm-down reaching posture. These no-cylinder trials were excluded from all further analyses. Figure 2 shows the group mean responses for each stimulus configuration in each task. A repeated measures ANOVA was performed with the withinsubjects factors of task (bisection, reaching), left cylinder location (near, far) and right cylinder location (near, far), and the between-subjects factor of group (control, neglect). Overall, the neglect group responded slightly further rightwards than the controls in the bisection task, and slightly further leftwards than controls in the reaching task, as reflected in a significant interaction of task by group [F(1, 20) ¼ 6.67, P ¼ 0.02]. Due to the presence of this interaction, and a significant three-way interaction of task by group by left cylinder location [F(1, 20) ¼

Fig. 2. Experiment 1: Mean responses in the bisection task (left) and the reaching task (right). The large gray circles depict the stimulus cylinders.

218

20.03, P