Brain Theory - Biological Basis and Computational Principles

Preface This book is the fifth in a series of volumes, trying to define a theory of the brain by bringing together formal reasoning and experimental facts. This endeavour was started some ten years ago by a group of researchers, mostly of theoretical inclination, who were guided by the uneasy feeling that more facts were being produced than they had been able to understand. At the same time, there was an awareness that theoretical reasoning about the brain was perhaps not duly constrained by factual considerations. Thus, the first volumes were dedicated mainly to the task of making neural theories and neural facts more palatable to each other. This epistemological problem is not anymore in the foreground now. A new breed of brain scientists has learned to appreciate and to competently use both the methods of sophisticated experimentation and those of model building. Rather than striving to forge a marriage, we can now happily draw on a new generation, the offspring of a connubium already consumed. The present collection of papers focuses on the subject of vision, and brings together new insights and facts from various branches of experimental and theoretical neuroscience. The experimental facts presented in this book stem from disparate fields, such as neuroanatomy, electrophysiology, optical imaging, and psychophysics. The theoretical models in part are home-spun, but no less inspiring for that, while others judiciously apply sophisticated mathematical reasoning to results of experimental measurements. We trust that the reader will feel, like we do, that these various attempts may well present the prelude to a new kind of brain science, where facts and theory begin to blend in a manner reminiscent of the development of physics in the last centuries. Starting point of this enterprise were the presentations and discussions at the Fifth International Meeting on Brain Theory, held at the Istituto per la Ricerca Scientifica e Tecnologica (IRST) in Trento (Italy) on April 5-7, 1994. This meeting, organized by Moshe Abeles, Ad Aertsen, Valentino Braitenberg and Luigi Stringa, was the fifth in a series, starting in 1984 at the International Center for Theoretical Physics in Trieste [1], and continuing in 1986 in Bad Homburg [2], in 1990 at Schloss Ringberg [3] and in 1992 at the IRST in Trento [4]. The meeting lasted three full days, providing a natural segmentation of the presentations and discussions according to three main headings: (1) Visual perception: psychophysics and physiology (2) Cortical implementation: physiology and anatomy (3) Cortical models and computational principles. We adopted the same tripartition to organize the material in the present book. The Fifth International Meeting on Brain Theory was jointly sponsored by the European Commission (DC XII), the Fritz Thyssen Stiftung, the Istituto per la Ricerca

VI

Scientifica e Tecnologica (Trento), as well as by contributions from the various institutions who financed the participation of their delegates. Splendid hospitality, together with most efiicient organization (thank you, Nadia Oss!) was provided by IRST, Trento. The generous support by all these institutions is most gratefully acknowledged. Ad Aertsen Valentino Braitenberg REFERENCES 1. G. Palm and A. Aertsen (eds.), Brain Theory. Springer Verlag, Berlin, 1986. 2. W. von Seelen, U. Leinhos and G. Shaw (eds.). Organization of Neural Networks: Structures and Models, VCH Verlag, Weinheim, 1987. 3. A. Aertsen and V. Braitenberg (eds.). Information Processing in the Cortex: Experiments and Theory, Springer Verlag, Berlin, 1992. 4. A. Aertsen (ed.). Brain Theory - Spatio-Temporal Aspects of Brain Function, Elsevier, Amsterdam, 1993.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) © 1996 Elsevier Science B.V. All rights reserved.

Early vision: Images, context and memory Dov Sagi* Department of Neurobiology, Brain Research The Weizmann Institute of Science Rehovot 76100, Israel Visual perception involves image transformations, producing internal images with some useful features enhanced, possibly linked. Enhancement is a result of local transformations being context dependent, affected either by remote image regions or by visual memory. Psychophysical experiments show that early vision generates an image that, although dominated by the retinal output, contains a context dependent component. Context affects response of oriented filters through lateral interactions. Efficacy of lateral connections may change with experience (perceptual learning) and can be modulated by higher levels of processing (visual attention and imagery). Psychophysical results suggest that long range interactions are achieved by activation spread through multiple connections. Some properties of this model of early vision are examined and applied to a variety of problems, as lateral masking, texture discrimination and visual grouping. 1. I n t r o d u c t i o n Our visual system transforms the apparently meaningless reality inflicted on our retinas into meaningful and behaviorally relevant objects. As the retinas are specialized in detecting picture points (pixels), we are faced with the problem of attributing these point data to objects. The embedment of point measurements within the proper context of a perceivable object, requires integration between remote image parts as well as with memory. This problem is best exemplified by the hidden figure puzzle [1] shown in Figure 1. The figure seems to be composed of black and white patches devoid of any semantic significance, though a face is present. As this puzzle is difficult to solve by trial and error, it is suggested to look at Figure 5 briefly and then to return to the hidden figure, if only to be faced with the deep look of a face decorated with Christ styled hair and beard. Perception is changed dramatically, with some 'illusory' contours added (as at the top of the right shoulder), yet, these contours are supported by some minimal physical cues. It seems that the solution reached by our brains is based on integration * Supported by the Israel Science Foundation administered by The Israel Acfidemy of Sciences and Hiunanities - The Charles H. Revson Foimdation.

Figure 1. A hidden figure: the puzzle. From Porter P. [1], copyright by the Board of Trustees of the University of Illinois.

of the physical context with some memory cues (previous experience). At what level of processing does this integration take place? Recent findings [2-6] suggest that both spatial and memory dependent context affect visual perception at a very early stage of processing. It is assumed here that a stage of processing is characterized by integration of information from different sources (lower, as well as higher stages) followed by a decision (threshold). Accordingly, the first stage of visual processing is described as spatial (linear) integration of the retinal input followed by a threshold. This stage, encapsulated within the preattentive bottom-up frame, is a principal component of theories of early vision [7-9], though formulation may differ between theories. Here the emphasis is being shifted to the context dependency of early vision, suggesting that already the first stage of processing is affected by 'late' processes and is involved in the creation of a unified perception (Gestalt). It is suggested that first stage filters are arranged topographically with excitatory connections between neighbors. Activation may spread in this network, and remote filters may interact, depending on efficacy of excitatory connections. Efficacy is determined by experience and is modulated by visual attention. 2. T h e primary visual network 2.1. Spatial filters One of the basic findings in visual perception is that there exist many parallel pathways within the visual system, each of them is specialized to carry information about a different stimulus aspect. These findings were demonstrated in physiology with the introduction of the 'receptive field' concept [10,11] and in visual psychophysics with the introduction of parallel processing of color by Young [12] and Helmholtz [13]. Later work

by Hubel and Wiesel [14] and Campbell and Robson[15] opened a period of extensive research in determining the properties of visual channels analyzing form. Although the term 'channel' is not well defined, there is a general agreement on the existence of some mechanisms operating in parallel across the visual field and responding significantly to some stimuli but not others (selectivity). Some theories assume that these channel can be described as linear filters [15-18] while others elaborate nonlinear feature detectors [7-9,19]. Here we assume that the basic mechanisms are linear filters, selective for orientation, location and spatial frequency. It is possible to describe these filters as some function of retinal coordinates, orientation and spatial frequency, F{x^y \ uj^O^Xc^yc)^ with parameters representing filter location (a:c,2/c), its prefered orientation (^), and its prefered spatial frequency (a;). Popular functions used for spatial filters are Difference Of Gaussians (DOG) [18] and Gabor functions [20]. Both filter types were used to model receptive fields of cortical simple cells [8,20]. Filter response is obtained by convolution with an input image L(x,y), followed by a nonlinear transducer function (trf[]): Rfi{xc, Vc \^,0) = trflY,F{x,2/

I a;, 20%). Following Polat and Sagi [4] we assume here that these excitatory connections are limited to between close neighbors, but signal transmission through multiple connections is possible. Due to transmission constrains (synaptic) signal propagation through lateral connections involves attenuation (by a factor 0 < /c < 1) and requires additional time steps (with a temporal attenuation by a factor 0 < i/ < 1), resulting the equation: Q-^Roire,t) = -i^Reire.t)

+ K[Re{re - Sr,,t) + Re{re + Sr,,t)] -\- Q^^re^t)

(2)

with re as the axis of propagation in direction 6. Re is the response of a spatial filter with an orientation 0, and some spatial frequency parameter a;, and $(r^, t) is the sensory input to this filter. (Only filters with the same spatial frequency parameter are assumed to be laterally connected.)

And if sampling density is high enough: |i? = ( 2 K - . ) i l + < ^ i ? + | $ .

(3)

It is further assumed that excitatory weights are modulated by visual attention (see Section 4). This assumption can allow for a selective increase (or decrease) in connectivity at selected image regions within a time frame defined by network integration time. On a slower time scale, excitatory weights are subjected to perceptual learning [4]. Network response level is also controlled by a normalization process, that is by divisive inhibition. 2.3. Inhibitory interactions Response normalization is supported by data from masking experiments, where contrast thresholds increase with increasing background contrast. Psychophysical [28,31] and physiological [32] models of filter contrast response assume divisive inhibition, where filter response is divided by a 'local energy' measure reflecting the total activation at some small neighborhood around the normalized filter. These inhibitory filters seem to have an isotropic spatial weighting function[30] and probably operate on a slower time scale [33]. Thus, the normalization factor is considered here to be filter response averaged across time. The orientation selectivity of the detection suppression (see Figure 2) observed in lateral masking experiments [30] indicates short range (of about twice the filter size) orientation selective lateral inhibition. Thus the response of a given filter is inhibited by activity of filters with different orientations at the same location, and by spatially adjacent filters with the same orientation. Such a connectivity pattern may serve, in addition to local response normalization, detection of texture boundaries by reducing activity at regions with uniform orientation. 3. Application t o s o m e perceptual p h e n o m e n a 3 . 1 . Lateral masking: a probe on spatial interactions The range of spatial interactions (i.e the space constant) can be estimated by measuring detection thresholds for a target with high contrast signals (masks) positioned at some distance away from it [3,30]. Data from such an experiment is depicted in Figure 2. These experiments show detection suppression when masks are presented in close neighborhood to the target (less than twice the target wavelength, cis predicted by the divisive inhibition). At larger distances an enhancement is observed, reflecting an excitatory input from a distance up to ten times signal wavelength, much beyond the first stage receptive field size. In order to derive model predictions for this case, we cissume a narrow input at the origin with a space/time spread of one unit, $ ( r ^ , t ) = e"^*'^''^^ (in Equation 3). Taking a non-oscillating solution of Equation 3 one finds (for some u,^ with V = U-K{2-{-

8l^C) > 0),

R = i?y[e-("*+^'-) -h Ce-(*+^^)].

(4)

Model predictions for this case, depicted in Figure 2, were calculated assuming temporal integration and using a Gaussian weighting function for the inhibitory filter with

+

X X X X

+

X X +

• V X + < C C T ^

+

+

'V

y . A - A ' + A ' ^ x X

xA'

+

Jr-VY.+

A + J r + x x

+

" l ^ ^ - 7 C 4 > N / " \

L ^ r A ' T ' . ^ . r C r

V \ / r > \ / < L > v

x +

^ y ^ ^ ' y ^ ^ < ' < ^

xA-Tt

B ^ L - i " i r > ' " v / s

Figure 3. Effortless texture segmentation in (a) is somewhat easier than in (b), though boundaries aie the same. Orientation selective spatial filters predict larger response variability for textures composed of L's, as L's are more elongated than X's (when convolved with larger receptive fields). As a consequence, filter based models generate more boundaries in (b), making foreground localization difEcult.

a standard deviation equal to the first stage filter wavelength. Excitation was calculated as a spatial convolution of filter responses with a sum of exponential weighting function, as derived in Equation 4. The best data fitting parameters for Equation 4 yield an input dependent weighting function which is 16 times stronger than the weighting function accounting for lateral range interactions (i.e C = 16). The same ratio was also used by Zenger and Sagi [28]. The space constant of the long range excitation depends on experience. The experimental data depicted in Figure 2 show an increase range of interactions with practice (perceptual learning), implying an increase of the corresponding space constant from 6 to 12 filter wavelengths(^ = 0.16 to ^ = 0.08). 3.2. Texture segmentation In texture segmentation tasks one has to detect or locate a texture region (foreground) embedded within another texture (background). A typical example is shown in Figure 3. As texture segmentation is sometimes 'effortless' [7] or attentiveless [47], it is believed to be carried out, and thus to be limited, by early visual mechanisms. Human performance on texture segmentation tasks can be accounted for by a two stage filtering model [41-44] with a similar design to the one described above. The key component in texture segmentation is the first stage filter, being local, orientation selective and spatial-frequency selective. Detection of texture boundaries can be modeled as detection of local activity differences (edge detection) within a filtered image. As a typical texture creates local activity differences across all image regions (e.g. the textures depicted in Figure 3 generate highly variable activity patterns when convolved with oriented filters, due to the random local orientation), the problem is one of detecting an activity edge in a noisy image [44]. Here, spurious activity variations are smoothed by excitatory lateral interactions.

"A V

-^

Tt

-^

y.

4

J A T^

r

' L

>

+

^

+

+

>y

T^

n

V.

Ar

n

+

J

+

A -A

^

^ >

+ -V

n L

+ Ar

+ + x - v y . x

+

x

JrX"J«.

B+

+

+

+-v

+

+

x

T^^'

+

x - V x A - T ^

i f c ^ ^ v - ^ A - n + j x J C -^^

r n

L +

^

C

X

V T t J

>

y +

+

J

x

^ - A

Figure 4. Visuai grouping by similaxity (a.), proximity (b) and both (c).

and activity at regions of uniform response is reduced by (divisive) inhibition. A similar model was shown to successfully predict human performance on texture segmentation tasks [44]. To account for preattentive texture segmentation, where all image regions are processed in parallel and texture boundaries are to be detected, we assume excitatory weights to be low and constant across the processed image, allowing for good localization performance. However, performance is experience dependent. Observers practicing texture segmentation improve with practice [2]. Improvement occurs through a period of a few days and lasts for years [45]. Learning is specific for texture orientation, location and eye (practicing with one eye does not produce improvement when testing with the other eye), implying plasticity at an early stage of visual processing. These results imply that the network weights used here should be modifiable with experience. It is possible that increase in efficacy of excitatory weights produces a more efficient smoothing of spurious local variations in filters activity. It is also possible that an increase of inhibitory gain further reduces activity at regions with low response variability, thus enhancing texture boundaries.

3.3. P e r c e p t u a l grouping Visual stimuli, as Figure 4, give rise to spontaneous perception of some global organization. Thus it seems that our visual system implements specific rules for the creation of structure in seemingly ambiguous situations. While texture segmentation is marking object boundaries, grouping is a process involved in linking image regions into unified objects. The Gestalt laws of perceptual organization [48] assume that image parts group together when being similar in shape or when being in close proximity, or when creating a

10 'good form'. The Gestalt laws, though applicable in many situations, lack a quantitative formulation. It is not clear how to define shape and similarity or what entities to use for proximity measurements. Furthermore, often images contain multiple organization cues (e.g. proximity and similarity in Figure 4c) thus calling for some quantitative method to combine the different rules into a unified framework. The primary visual network described above provides a natural framework for handling internal image relations, as required in perceptual organization. Spatial filters can be used to construct a similarity metric (as in texture segmentation), and anisotropic excitatory lateral connections can be used to implement spatial relations. A recent theory of grouping accounts for human performance on proximity and similarity based grouping tasks by assuming an autocorrelation type of process [49]. Though the autocorrelation computations can be applied to filter responses, a simpler model is possible. In this model [49], the input image is represented as an intensity map, /(a:,T/), thus skipping the filtering stage. The basic operation performed is an estimate of total weighted directional correlation at each image point:

Re{x, y) = E H^^ y)' ^(^ + ^^^^^' y + ^^^^^) • ^^~^^^°^

(5)

Here, Re{x^ y) represents the total weighted correlation of the image point /(a:, y) with all image points along a line with orientation 0 passing through the point {x,y). Long range correlations are given a smaller weight (exponentially decaying), thus introducing proximity effects. This equation was successfully applied to psychophysical experiments using stimuli (Figure 4) where global orientation was defined either by elements proximity and/or shape similarity and/or luminance similarity. As the psychophysical task in the experiments was a two alternative forced choice between vertical and horizontal global organizations, a model decision parameters was constructed (A), A

2^x,y ^0=vertical\X-,y) 2^x,y •^0=horizontal\X^

^^x yj

SO that A > 1 indicates vertical organization, A < 1 indicates horizontal organization, and an unbiased guess is forced with A = 1. Performance on the grouping tasks was found to be time dependent [49]. In the experiments, stimuli (like in Figure 4) were briefly presented (20 msec) followed by noisy unorganized pattern to mask the grouping perception. For smaller stimulus to mask temporal separations {t < 100 msc), the perceived stimulus organization was dominated by proximity relations, while additional processing time allowed for similarity based grouping to take over. Equation 4 can account for this behavior if the space constant ^o (the model free parameter) is allowed to increase with time. Thus, the model suggests an increasing integration range with time. Our network can account for human behavior by assuming that the weighted spatial integration is performed by the excitatory lateral connections. The exponential distance

11 weight applied to the correlation equation is predicted by signal propagation through lateral connections (section 2.2). Dynamic control of connectivity can result in an increase of excitatory spread within network integration time (< 200 msc), predicting increasing space constants with time and thus accounting for the psychophysical findings [49]. This fast dynamic of excitatory transmission is assumed here to be controlled by visual attention. It is also possible that fast modulations of synaptic efficacies are generated by response correlations [50,51](in filters or some other responding units), however, such a mechanism would not allow transmission of activity through unstimulated image regions. Perceptual organization involves, not only enhancement of activity correlations, but also filling in gaps where information is missing (as in Figure 1). Psychophysical experiments indicate that perceptual grouping depends on the availability of visual attention [39,40], supporting the assumption of attentional control over excitatory efficacy.

4. Visual a t t e n t i o n Visual attention is assumed to link image features into objects [7,9], to integrate filter responses [34,35], or to select a region in the visual field for enhanced processing [36]. As image features (e.g. orientation, size, color) are not represented uniquely within the filter representation used here (assuming filters perform weighted linear broadband integration within a multidimensional space), attention is assumed here to derive image features from filter representations by integrating filter responses. On this account, attention drives visual process that reconfigure filters by modulating their (excitatory) interactions to create ensembles that are better tuned to specific image features. Though these processes can be implemented in a feed-forward fashion by creating higher level filters using dynamic receptive fields [37], it is attractive to assume a single layer feedback network capable of handling global process (as needed for shape analysis). Selective attention seems to be controlled by grouping process [35,38], as the number of image items that can be processed in parallel within the 'spot-light' of attention depends on stimulus parameters affecting perceptual grouping. Two image items having the same spatial frequency, or location (e.g Gabor signals [35]) require the same processing time as a single item to be identified, implying some similarity and proximity based grouping processes preceding attention allocation. Excitatory in^teractions can underlie these grouping process, with a similarity metric defined by filter parameters (orientation, spatial frequency and location). It is interesting to note that grouping processes (see section 3.3) are critically dependent on the availability of attention [39,40], in agreement with the assumption of attentive control over the efficacy of excitatory interactions. Thus attention and grouping seem to be mutually dependent. Increasing efficacy of excitatory interactions enables long range correlations to affect grouping and thus an optimization of attentive resources allocation. Duncan and Humphreys [38] suggested that hierarchical segmentation and grouping is a basic capacity-free stage in visual processing. This visual selection theory has three components: a parallel stage of perceptual description, a selection process and the entry

12 of selected information into visual-short-term-memory (VSTM) which allows control of access to awareness. According to this theory, the perceptual description is made by a process of hierarchical segmentation of the image into linked groups and subgroups (structural units). Each structural unit is described by its elementary sensory properties (relative position, color, size, motion etc) and categorical properties (based on meaning). The above described process of segmentation and description is said to be parallel and resource free, selection starts when all parts of the display compete for access to the VSTM. Linked structural units tend to gain or lose resources (activity strength) together. Increased assignment of resources to any structural unit increases its speed and probability for access to VSTM. Once a structural-unit emerges the 'winner' in the visual selection process it accesses VSTM. Duncan and Humphreys [38] propose that "Structural units act as wholes, competing for and gaining access to VSTM with all their associated descriptions" . It is possible that structural units are created by linking (grouping) processes within the early filter representation. If this is the case, attention allocation should be constrained by early vision architecture, that is, by excitatory connectivity. Filters that are connected by excitatory interactions can be linked to create a 'structural unit'. The linking process would involve 'attentive' fast synaptic modifications to strengthen the desired links, or to weaken the links between other filters. However, on this account, the linking processes is not 'resource free' as the system is limited by connectivity, and to some extent by the flexibility of the attentive synaptic gating mechanism.

5. From a t t e n t i o n t o m e m o r y Lateral interactions are probably modulated by attention on a fast time scale (msecs) and can go under longer term (years) modifications as in perceptual learning. These two processes operate on different time scales but are probably linked through some intermediate memory structures operating on time scales of several minutes to a few hours. Experiments involving lateral masking learning show an increase of interaction range with practice [4]. In order for practice to be successful, observers had to practice all target to mask distances within each practice session. Learning was not observed when observers were practicing on long distances alone, without performing on intermediate distances. Increasing the difference between the distance samples used in the experiments also prevented learning [4]. It seems that different distances have to be experienced within a single session, probably with neighboring filters activated within a time window of a few minutes, enabling generation of associations between stimuli that are close in space and time. Further evidence for a low level memory operating on a time scale of a few minutes come from visual imagery experiments. In these recent experiments [55] observers were asked to imagine the high contrast masks while detecting the low contrast target in the lateral masking experiment (see Figure 2). Results from these imagery experiments show enhanced detection following the perception effects, though at about half the magnitude. However, enhancement by imagery can be obtained only within a time window of a few

13 minutes after performing the 'real' perception task, and with the same targeted eye. Passive inspection of the mask prior to the imagery action, or running only as few as ten trials before the imagery task are not sufficient to obtain the imagery based enhancement. These results support the existence of a low level (monocular) memory, probably iconic in nature, that stores filters activity for a few minutes. This storage is activated by higher level processes (e.g. attention, as task is necessary) and is accessible by higher level processes (as imagery). Thus this memory seems to match the one that is used for establishing spatio-temporal associations. [Though spatio-temporal associations are established during a time period of a few minutes, a few hours may be required for their consolidation [45,54].]

6. Conclusion Non-isotropic excitatory interactions between oriented spatial filters provide activity enhancement from neighboring filters, thus enabling some useful global image characteristics to spread within the network of early vision and to be detected (e.g. enhancement of long lines [3,53], closure [48]), enhancement of closed contours [52]). Furthermore, it is assumed here that excitatory weights are modulated by a top-down process (i.e. visual attention), introducing memory dependent context. It is suggested that excitatory transmission efficacy is set to some low default value under nonattentive conditions and it can increase with increasing levels of attention. Though a global control of excitatory efficacy is assumed here, it is also possible that some specific weights, or pattern of weights, can be modulated by selective attention providing a fast space invariant object filter. Spatial selectivity is inherent in the concept of visual attention as visual detection is enhanced within attended regions of the visual field [36,46]. Attention was also suggested to be involved in binding image features to objects [7,9], thus serving as the bridge between featural image representation and the perceived Gestalt. Within the context of the present framework, binding can be achieved by selective modulation of weights, probably determined also by memory stored patterns, with objects parts linked by increased connectivity between their corresponding filters. Note that we do not allow for memory dependent context to be effective without any input dependent context available, as memory affects only connectivity. (Transmission efficacy may also be controlled by weak top-down signals providing subthreshold input to filters and thus improving their responsiveness to other inputs [56], sensory as well as lateral.) Visual inputs should assume some spatial correlations in order for connectivity modulations to be efficient. However, a rich enough (or noisy) input can generate patterns of spurious correlations with sufficiently organized filter activity to enable modulation by memory. The effect of these spurious meaningful images on perception will depend then on the weighting factors, which may vary with time, or between brains.

14 REFERENCES 1. Porter, P. (1954). Another picture-puzzle. American Journal of Psychology 67, 550551. 2. Kami, A. and D. Sagi (1991). Where practice makes perfect in texture discrimination: Evidence for primary visual cortex plasticity. Proceedings of the National Academy of Science, USA 88, 4966-4970. 3. Polat, U. and D. Sagi (1993). Lateral interactions between spatial channels: Suppression and facilitation revealed by lateral masking experiments. Vision Research 33, 993-999. 4. Polat, U. and D. Sagi (1994). Spatial interactions in human vision: from near to far via experience dependent cascades of connections. Proceedings of the National Academy of Science, USA 91, 1206-1209. 5. Polat, U. and A. M. Norcia (1996). Neurophysiological evidence for contrast dependent long range facilitation in the human visual cortex. Vision Research, in press. 6. Sagi, D. and D. Tanne (1994). Perceptual learning: learning to see. Current Opinion in Neurobiology 4-, 195-199. 7. Julesz B. (1981). Textons, the elements of texture perception and their interactions. Nature 290, 91-97. 8. Marr, D. (1982). Vision. Freeman, New-York. 9. Treisman, A. (1985). Preattentive processing in vision. Computer Vision Graphics and Image Processing 31, 156-177. 10. Hartline H.K. (1938). he response of single optic nerve fibers of the vertebrate eye to illumination of the retina. American Journal of Physiology 121, 400-415. 11. Barlow, H. B. (1953). Summation and inhibition in the frog's retina. Journal of Physiology, London 119, 69-88. 12. Young, T. (1802). On the theory of light and colors. Philosophycal Transactions of the Royal Society 92, 12. 13. von Helmholtz, H. (1909). Physiological Optics. Optical Society of America, 1925. Republished by Dover, New York. 14. Hubel, D. and T. Wiesel (1962). Receptive fields, binocular interaction and functional architecture in the cat's striate cortex. Journal of Physiology, London 166, 106-154. 15. Campbell, F. and J. Robson (1968). Application of fourier analysis to the visibility of gratings. Journal of Physiology, London 197, 551-566. 16. Graham N. (1977). Visual detection of aperiodic spatial stimuli by probability summation among narrowband channels. Vision Research 17, 637-652. 17. Watson A.B. (1983). Detection and recognition of simple spatial forms. In Physical and Biological Processing of Images. Eds. Braddick O.J. &; Sleigh A.C., Berlin: Springer-Verlag. 18. Wilson H.R. (1983). Psychophysical evidence for spatial channels. In Physical and Biological Processing of Images. Eds. Braddick O.J. k Sleigh A.C., Berlin: SpringerVerlag. 19. Barlow, H. B. (1972). Single units and sensation: a neuron doctrine for perceptual

15 psychology? Perception i, 371-394. 20. Marcelja, S. (1980). Mathematical description of the responses of simple cortical cells. Journal of the Optical Society of America 70, 1297-1300. 21. Campbell, F. and J. Kulikowski (1966). Orientationonal selectivity of the human visual system. Journal of Physiology, London 187, 437-445. 22. Phillips, G. C. and H. R. Wilson (1984). Orientation bandwidths of spatial mechanisms measured by masking. Journal of the Optical Society of America A 1, 226-232. 23. Blakemore, C. and F. Campbell (1969). On the existence of neurones in the human visual system selectively sensitive to the orientation and size of retinal images. Journal of Physiology, London 203, 237-260. 24. Stromeyer C.F. and Julesz B. (1972). Spatial-frequency masking in vision: Critical bands and spread of masking. Journal of the Optical Society of America, 62, 12211232. 25. Legge G. E. (1979). Spatial frequency masking in human vision: binocular interactions. Journal of the Optical Society of America, 69, 838-847. 26. Sagi, D. and S. Hochstein (1983). Discriminability of suprathreshold compound spatial frequency gratings. Vision Research, 23, 1595-1606. 27. Watson A.B., Barlow H.B. and Robson J.G. (1983) What does the eye see best? Nature, 302, 419-422. 28. Zenger, B and D. Sagi (1996). Isolating excitatory and inhibitory non-linear spatial interactions involved in contrast detection. Vision Research, in press. 29. Gilbert, C D . (1993) Circutry, architecture and functional dynamics of visual cortex. Cerebral Cortex 3, 373-386. 30. Polat, U. and D. Sagi (1994). The architecture of perceptual spatial interactions. Vision Research 34, 73-78. 31. Foley, J. M. (1994). Human luminance pattern-vision mechanisms: mctsking experiments require a new model. Journal of the Optical Society of America All, 1710-1719. 32. Heeger, D. (1992). Normalization of cell responses in cat striate cortex. Visual Neuroscience 9, 181-197. 33. Georgeson, M. A. and J. M. Georgeson (1987). Facilitation and masking of briefly presented gratings: Time-course and contrast dependence. Vision Research 27, 369379. 34. Sagi, D. (1990). Detection of an orientation singularity in gabor textures: Effect of signal density and spatial frequency. Vision Research 30, 1377-1388. 35. Adini, Y. and D. Sagi (1992). Parallel processes within the "spot-light" of attention. Spatial Vision 6, 61-77. 36. Posner, M. I. (1980). Orienting of attention. Quarterly Journal of Experimental Psychology 32, 3-25. 37. Moran, J., and R. Desimone (1985) Selective attention gates visual processing in the extrastriate cortex. Science 229, 782-784. 38. Duncan, J. and G. W. Humphreys (1989). Visual search and stimulus similarity. Psychological Review 96, 433-458.

16 39. Ben-Av, M. B., D. Sagi and J. Braun (1992). Visual attention and perceptual grouping. Perception & Psychophysics 52, 277-294. 40. Mack, A., B. Tang, R. Tuma, S. Kahn, and I. Rock (1992). Perceptual organization and attention. Cognitive Psychology 24-, 475-501. 41. Fogel, I. and D. Sagi (1989). Gabor filters as texture discriminator. Biological Cybernetics 61J 103-113. 42. Landy, S. L. and J. R. Bergen (1991). Texture segregation and orientation gradient. Vision Research 31, 679-691. 43. Malik, J. and P. Perona (1990). Preattentive texture discrimination with early vision mechanisms. Journal of the Optical Society of America A 7(5), 923-932. 44. Rubenstein B. S. and Sagi D. (1990) Spatial variability as a limiting factor in texture discrimination tasks: Implications for performance asymmetries. Journal of the Optical Society of America A, 7, 1632-1643. 45. Kami, A. and D. Sagi (1993). The time course of learning a visual skill. Nature 365, 250-252. 46. Sagi, D. and B. Julesz (1986). Enhanced detection in the aperture of focal attention. Nature 321, 693-695. 47. Braun, J. and D. Sagi (1990). Vision outside the focus of attention. Perception & Psychophysics 4S, 45-58. 48. Koffka, K. (1935). Principles of Gestalt Psychology. New York: Harcourt Brace. 49. Ben-Av, M. B. and D. Sagi (1995). Perceptual grouping by similarity and proximity: Experimental results can be predicted by intensity autocorrelations. Vision Research, 35, 853-866. 50. Malsburg, C. von der (1985). Nervous structures with dynamical links. Physical Chemestry 95, 703-710. 51. Braun, J., E. Niebur, H.G. Schusler and C. Koch (1995). Perceptual contour completion: a model based on local, anisotropic, fast-adapting interactions between oriented filters. Society of Neurosciences Abstracts 20, 1665. 52. Kovacs, I. and B. Julesz (1993). A closed curve is much more than an incomplete one: Effect of closure in figure-ground segmentation. Proceedings of the National Academy of Science USA 90, 7495-7497. 53. Field, D. J., A. Hayes, and R. F. Hess (1993). Good continuation and the association field: evidence for local feature integration in the visual system. Vision Research 33, 173-193. 54. Polat, U., and D. Sagi (1995) Plasticity of spatial interactions in early vision. In B. Julesz & I. Kovacs (Eds.), Maturational Windows and Adult Cortical Plasticity, SFI Studies in the Sciences of Complexity, Vol XXIV, Addison-Wesley, Reading MA. 55. Ishai, A., and D. Sagi (1995). Common mechanisms of visual imagery and perception. Science 268, 1772-1774. 56. Aertsen A.M.H.J., G. Gerstein, and P. Johannesma (1986). From neuron to assembly: Neuronal organization and stimulus representation. In G. Palm fc A. Aertsen (Eds.), Brain Theory, Springer-Verlag, Berlin.

17

Figure 5. A hidden figure: solution.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) 1996 Elsevier Science B.V.

19

Psychophysical Mapping of Orientation Sensitivity in the Human Cortex JOHANNES M . ZANKER ^^^

&

VALENTINO BRAITENBERG ^

^ Max-Planck-Institut fur biologische Kybemetik, D-72072 Tubingen, Germany ^ Istituto per la Ricerca Scientifica e Tecnologica, 1-38050 Trento/Povo, Italia

Based on anatomical and physiological knowledge about the representation of oriented stimuli in primate visual cortex, two attempts were made to analyse cortical orientation sensitivity maps in humans by means of psychophysics. (i) The minimum requirement to detect the orientation of a line in various regions of the visual field is characterised in terms of stimulus contrast and size. It turns out that, at sufficiently high contrast levels, a line needs only to span about 0.2 mm on the cortical surface in order to be recognised as oriented, independent of the actual eccentricity at which the stimulus is presented. This indicates that human orientation detection approaches the physical limits, requiring hardly more than approximately four input elements in order to be detected in its orientation, (ii) Starting from this observation, line segments were generated which were shorter in their cortical representation than the diameter of a cortical hypercolumn. They were used in a first attempt to investigate the spatial distribution of orientation preferences. Indeed, maps emerge from the data which resemble the orientation maps known from optical recordings of the cortical surface. However, due to technical limitations these preliminary results should as yet be treated with some caution.

1. INTRODUCTION Visual space is represented in the cortex in a highly ordered manner. Each position in the visual field (defined in angular co-ordinates of the eye) corresponds to a well defined region in the cortical area VI, and similarly - though less precisely - at several subsequent stages of the visual stream. Such retinotopic representations led to the notion of cortical maps which, of course, do not produce isometric images of the scenery, but are considerably distorted by the architecture of the cortical surface (Daniel & Whitteridge, 1961; Mallot, 1985; Fox, Miezin, Allman, Essen, & Raichle, 1987; Tootell, Switkes, Silverman, & Hamilton, 1988; Horton & Hoyt, 1991). Furthermore, there is a systematic change of scale when going from the centre of the visual field, the fovea, to the periphery. The so-called *M-scaling' or cortical magnification factor indicates how the number of neural elements - from retinal ganglion cells to cortical * present address: Department of Psychology, University College London, Gower Street, London WCIE 6BT, England

20

neurones - responsible for one degree of visual field decreases with increasing eccentricity according to a power function (Tynan & Sekuler, 1982; Cowey & Rolls, 1974; Rovamo & Virsu, 1979; Pointer, 1986; Wassle, Griinert, Rohrenbeck, & Boycott, 1989). This reflects the well-known fact that the spatial grain decreases from the fovea (which in fact is defined as the region of highest spatial resolution) to the periphery: A 1 mm x 1 mm region of cortical surface corresponds to a small part of the visual field in the fovea (about 1/8 x 1/8 degree^) and to a large part in the periphery (about 1 x 1 degree^ at 20°, estimated after Rovamo, Virsu, & Nasanen, 1978). In addition to this, there is ample physiological and anatomical evidence that certain stimulus features at a given position, such as ocularity, colour, local spatial frequency, or orientation are not randomly distributed over the cortex, but organised in spatial maps as well (Orban, 1984; Hubel & Livingstone, 1987; Livingstone & Hubel, 1988). For the representation of the local orientation of visual stimuli it is widely agreed that vertical columns of identical preferred orientation are arranged in two-dimensional arrays in which orientation changes systematically over the cortical surface (for review, see Hubel & Wiesel, 1977). Originally, this cortical orientation map was assumed to be best described by the *ice cube model' in which columns tuned to one orientation are arranged in rows with a continuous rotation of the preferred orientation. A set of orientation columns comprising a complete rotation in the orientation domain was labelled 'hypercolumn' in this model, and was understood as the comprehensive representation of all stimulus features at a given location in the visual field. This basic scheme was later challenged by the idea of a concentric arrangement of different orientations around regularly spaced singularities (Seelen, 1970; Braitenberg & Braitenberg, 1979). Such an-organisation may be realised in two specific versions (Gotz, 1987; Gotz, 1988; Braitenberg, 1992), depending on what organisational principle is proposed to determine the orientation specificity (see figure 1, top). These considerations about the regular arrangement of orientation in the cortex were originally based on linear tracks of extracellular recordings from neurones in cortical area VI which revealed orientation constancy or systematic changes in orientation, depending on the angle of the track relative to the plane of the cortex. Later these experiments were nicely complemented by optical recordings in which the surface arrangement of orientation preference was monitored in parallel (Blasdel & Salama, 1986; Tso, Frostig, Lieke, & Grinvald, 1990; Blasdel, 1992; Bonhoeffer & Grinvald, 1993). Such data revealed orientation maps close to those expected on the basis of the more recent models. They even allowed the singularities emerging in these maps to be related to the cytochrome-oxidase blobs known from histology in which neurones appear to be only weakly orientation tuned (Livingstone & Hubel, 1988). When several features are mapped on the same surface, the question arises as to how the different maps are kept apart, and whether they are functionally related to each other. One major question which seems not to be answered by the experiments published so far is how the representation of position and of orientation interfere with each other. Is a cortical unit which represents a certain orientation at a certain position in the visual field excited from the same region in space as its neighbours which represent different orientations, or do different orientations also imply different locations in the visual field?

21

cortical orientation map

-1 -

T

I

—

\

—

\

\

—

I y

receptive field organization

input sheet

patchy cooperation

Figure 1: Basic properties of cortical orientation maps. Top left and right: electrophysiological recordings suggest concentric arrangements of the preferred orientation of cortical units. In one variant each orientation (indicated by horizontal, vertical, or oblique bars) is represented twice (left square, after Braitenberg), in the other variant only once (right square, after Gotz) when moving along a fiill circle around singularities in which no orientation is preferred (indicated by dots). In general, the concentric arrangement corresponds to optical recording data (top centre, after Bonhoeffer & Grinvsild 1993; the pseudo-colour orientation map is converted to grey-scaled image, singularities are indicated by dots). However, realistic orientation maps differ from the ideaUsed schemes by irregularities in the fine structure as demonstrated by overlaying bars of the four basic orientations over the grey-scaled orientation map at the positions where the preferred orientation is expressed most strongly. Bottom: two altematives to interpret the large receptive fields of local neurones which are tuned to a specific orientation. Left: Each element (for example, centre element indicated by shading in the upper layer) could receive input from a large sheet of small, but densely arranged elements with the same preferred orientation (*line detectors', indicated by shaded elements in the lower layer), leading to a large, continuous receptive field represented at the given position. Right: A number of line detectors tuned to the same orientation (indicated by shaded elements in the single layer), but distributed across a region much bigger than the receptive field of the single element, could operate in an associative network thus forming a large - though patchy - effective receptive field for either element by co-operation.

At first, one may be tempted to dismiss the latter altemative since the receptive field as defined by electrophysiologists, i.e. the region in the visual field in which the element under consideration is excited by a visual stimulus, is much bigger than an orientation column (Hubel & Wiesel, 1977). In fact, receptive fields are bigger than a hypercolumn when their size is

22

measured in terms of cortical co-ordinates (see figure 1, bottom left). Thus the overlap between the receptive fields of neurones with different preferred orientation is considerable. However, such a view would have to be challenged if line segments much smaller than the classic receptive field could also be perceived as having a certain orientation. This would imply that each receptive field is in reality composed of a set of smaller - or at least narrower orientation sensitive elements (Zanker, in prep.) all tuned to the same orientation but with different locations in the visual field. For simplicity, such small oriented receptive fields will be called here 'line detectors', irrespective of whether they respond best to lines or edges. How could large receptive fields be constructed from such line detectors? Given that the set of pyramidal cells acting as an associative network is a plausible theory of the cortex (Braitenberg & Schiiz, 1991), one would not be surprised to find that elementary line detectors tuned to the same direction within one region of the cortex join up in a Hebbian cell-assembly through a process of associative learning. If this were so, what the electrophysiologist records from a single cell would be the collective activity of all neurones which excite the neurone under observation, with a corresponding large collective receptive field. In consequence, a certain orientation is only represented at a certain position in space but not at those being represented by the neighbouring units representing different orientations (figure 1, bottom right). Such a *patchy' cooperation being responsible for orientation mapping would not lead to serious consequences under normal viewing conditions because stimuli usually provide oriented edges or lines which are much longer than a hypercolumn and thus would excite several units of the same preferred orientation simultaneously. Furthermore, involuntary but permanent eye movements (Ratliff & Riggs, 1950), such as slow fixational drifts or tremor-like instabilities, will prevent small stimuli from being constantly centred on a single line detector. For these reasons, one easily could mistake a patchy co-operation of line elements for a hard-wired receptive field organisation in which an input sheet densely packed with line detectors feeds into the orientation sensitive cortical unit under observation (figure 1, left side). It should be possible, however, to determine the functional properties of the small subfields of conventional receptive fields either with electrophysiological recording (Schiller, Finlay, & Volman, 1976; Camarda, Peterhans, & Bishop, 1985a; Camarda, Peterhans, & Bishop, 1985b) or with psychophysical experiments. If the elementary receptive fields of the line detectors turn out to be small enough, smaller than the diameter of the hypercolumn, a possible link between their orientations and their respective positions within the hypercolumn could be unravelled. The experiments which we describe here were designed to examine such a fine-grain orientation mapping. There were two preliminary questions to be asked: (i) What is the minimal length of a line in order for it to be seen as oriented? Varying the retinal eccentricity of an oriented stimulus, this question will deal with the problem of how the visual field is scaled to the cortex, (ii) Can the unsteadiness of fixation, which would make a finegrain analysis quite impossible in the fovea, be overcome by mapping orientation in the more peripheral portions of the visual field? There each hypercolumn corresponds to a large region of the visual field (more than 1° visual angle) and the random errors due to the rotational movements of the eye bulb (which are the same for the whole field, about 10 min.arc after Ratliff & Riggs, 1950) should be less disturbing.

23

2. MATERIAL AND METHODS Human observers were seated in front of a computer screen (SUN workstation, 1200 x 1000 image pixels are displayed on the 36 x 30 cm screen, 66 Hz frame rate, 82% phosphor decay in 4 ms) at a viewing distance of 32.5 cm. The head of the subject was fixed by means of a head-chin rest which prevented gross head movements and guaranteed a constant viewing distance (see figure 2 top). Eye movements were restricted by instructing the observer to look constantly at the fixation target which was permanently present on the screen. In order to stimulate the right eye monocularly, i.e. to investigate only one of the two possibly overlapping orientation maps originating from either eye in isolation, the subject's left eye was covered by a black eye patch. The experiments were carried out in a quiet room illuminated by dim light. The subjects were asked to settle in the most comfortable position, to concentrate on the screen, and to report their decisions about the stimuh by pressing the computer's mouse buttons. All subjects were male colleagues of the authors, most of them naive with respect to the purpose of the experiments, and their average age was 29 years (between 23 and 36 years). No subject had any obvious ophthalmic disorder, some of them were wearing their usual optical corrections. The stimuli were produced on the computer screen within a window of 1100 x 500 pixels (corresponding to 33 x 15 cm) which was filled with dark pixels (average intensity Imin usually about 4 cd/m^). A fixation target was displayed at a constant position within the dark background throughout the whole experiment. It consisted of a frame of 16 x 16 bright pixels and a single bright pixel which was presented in its centre during the intervals between the stimulations (Imax usually about 100 cd/m^). After an acoustic signal, the test stimulus was presented for 45 msec. During this period the central bright pixel of the fixation target was replaced by a bright line: a row of eight pixels, for instance. In synchrony with this 'reference bar' a second line of the same length, the 'test bar', appeared at another position, i.e. in the periphery of the observer's visual field who was fixating the reference bar (see figure 2 bottom). The length of the reference bar, and the length and brightness of the test bar were treated as parameters for a series of experiments. The experimental variable was the orientation of either bar which could be horizontal (0°), vertical (90°) or oblique (45° or 135°). Note that the number of bright pixels may change for oblique bars, as compared to bars parallel to the pixel raster axes, when the absolute length of the bar is kept constant. After the presentation interval the fixation target reappeared on a otherwise dark screen, and was displayed for at least one second before the next stimulus was presented. The 16 possible combinations of the orientations of the two bars for each test position, and the test positions, were presented in random order. The subjects were asked simply to report whether the orientation of test and reference bar was the same or different. When all 4 x 4 combinations of the orientations of the two bars are displayed in random order, the same orientation is presented in one out of four cases, on average. This means that chance level - when the orientation is not perceived reliably - would be at 25% correct decisions, when subjects would always press the 'same' button, or 75% when the 'different' button is selected instead. Intermediate or floating strategies would lead to intermediate values, and 50% correct decisions are reached for decisions taken at random under those conditions. Since under certain experimental conditions (see below) the test bars

24

were hardly visible at all, the subjects were allowed to vote for a third option described to them as 'undecided or invisible'. This deviation from the standard two-alternative forced-choice paradigm extends the dynamic range for the psychophysical measurements to a baseline of 0% correct responses for honest and careful subjects, but still allows for higher scores reported by more self-confident subjects even when the percept is actually weak. Thus the results of this three-altemative-choice procedure with unbalanced presentation probabilities have to be regarded with appropriate care.

display observer viewing distance

mouse

orientation

, , eccentricity ^'^ation *^''9®* ^ ^ square

size

test bar

Figure 2: Experimental set-up and stimulus configuration. Top: The subject was seated in front of a computer monitor and the head position was fixed by means of a head-chin rest. The subject observed the display monocularly at a viewing distance of 32.5 cm and reported the decisions by pressing the buttons of the computer mouse. Bottom: Within a dark background a reference bar was presented in the fixation square, together with a test bar displayed at a variable eccentricity. The orientation and the size (length) of the target was treated as experimental variable.

reference bar

In addition to calculating the average performance over all orientations, the preferred orientation was estimated for each tested position in the visual field. For this purpose the number of correct responses was registered separately for each orientation of the test bar. Then a regression was calculated between the percentages of correct responses to the four bar orientations (at angular intervals of 45°) and a rectified cosine function for which the phase was shifted in 15° steps relative to the data points. The angular position of the best fitting cosine, i.e. that leading to the largest regression coefficient, was taken as an estimate of the preferred orientation. In the corresponding figures the preferred orientation is symbolised by a bar plotted at the angle leading to the best fit, with its length indicating the regression coefficient as a measure of the reliability of this estimate. It should be noted that in some cases no preferred orientation can be estimated (dots in the graphs), namely when bars with orthogonal orientation are detected by the subject with the same success.

25

3. OMENTATION DETECTION AND CORTICAL MAGNIFICATION The first experiment was designed to determine how far out in the peripheral visual field the orientation of a small line segment can be detected. For this purpose the fixation target was centred in the stimulus window, and a test bar was presented along the horizontal meridian to the left or to the right of this spot, i.e. in the nasal or temporal part of the visual field. The eccentricity was varied between 150 and 500 pixels away from the fixation point, corresponding to 7.5°-25° visual angle. The size of the reference and the test bar was 8 screen pixels (2.4 mm, corresponding to 0.42° in the fovea), and the orientations and the position of the test bar were randomised as described before. This basic experiment was repeated with four different stimulus contrasts to vary the strength of a given stimulus without changing its geometry. The Michelson contrast, defined as (Imax-Imin)/(Imax+Imin), was set to 40%, 63%, 82% and 94%, respectively. Each orientation combination was tested twice in two successive blocks, and the correct responses of a subject were pooled for all tested combinations. This led to 32 decisions for each of 2 x 8 possible test bar positions in the two hemi-fields. The average number of correct decisions from 5 subjects, each deciding 512 times for one stimulus contrast, is plotted in figure 3 as a function of stimulus eccentricity.

100 h

(0 0) (0

co a (0

S> o

o -10

0

10

30

eccentricity [deg] Figure 3: Orientation detection for various line contrasts. The percentage of correct decisions (n=5 subjects), pooled for all test bar orientations, is plotted as function of stimulus eccentricity (angular position in the visualfield)for both visual hemi-fields (nasal: negative eccentricity; temporal: positive eccentricity). The Michelson contrast of the test bar was set to 40% (circles), 63% (squares), 82% (up triangles), and 93% (down triangles). In general, performance is reduced with increasing eccentricity and decreasing contrast; in the temporalfieldperformance drops to about zero in the blind spot (16°-18° eccentricity).

26 In the left hemi-field of the right eye, i.e. in the nasal visual field, the percentage of correct decisions depends on stimulus eccentricity in a very clear pattern. At high and medium contrasts the orientation is detected reliably when the bar is presented close to the fovea, and performance drops with increasing eccentricity. Higher contrasts always lead to a better performance, meaning that at lower contrast the curve deviates from 100% correct responses closer to the fovea, and approaches lower values in the farther periphery. For the lowest contrast the overall performance is reduced in general. In this case the percentage of correct responses remains below 100% even close to the fovea, and drops to zero already at about 20° in the periphery, whereas at high contrast the performance deviates from 100% correct responses only in the far periphery. Altogether, the curves resemble a set of psychometric functions, which either can be interpreted as having different slopes and saturation levels (cf. curves for 40% and 63% contrast in figure 3), or as being shifted along the abscissa (cf. curves for 63% and 93% contrast in figure 3). Which of the two alternatives is more plausible, or whether they both hold for different contrast ranges, cannot be decided based on this limited set of data. In the right hemi-field of the right eye, i.e. in the temporal part of the visual field, this basic pattern of orientation detection performance is overruled by an additional peculiarity appearing at eccentricities between 15° and 18°. In this range, the performance is reduced to zero values for all stimulus contrasts. This behaviour was to be expected from the fact that the blind spot is located at this position in the visual field, and therefore the subjects should not be able to detect the stimuli at all in this region. This fact can be used to control for the reliability of the subjects' fixation by placing one test bar position in the blind spot. In concentrated and honest subjects, the detection probability should always be zero in the blind spot when it is stimulated from time to time during each experiment. Two basic conclusions can be drawn from the experimental results presented so far. The performance for orientation detection of a bar of a given size decreases (i) with decreasing stimulus contrast, and (ii) with increasing eccentricity of its presentation within the visual field. The latter result could either be due to contrast sensitivity which might be reduced in the periphery, and therefore could lead to a lower percentage of correct decisions, or to the fact that performance is limited by the bar size which has to be scaled with the spatial grain of the visual system as discussed in the introduction. The projection of a test bar of given physical length is larger in cortical coordinates when it is presented near the fovea than farther out in the periphery. To test this cortical scaling option, the physical bar size has to be varied in such a way as to keep the size of its cortical projection constant. In a second series of experiments, performance for orientation detection was therefore measured at various eccentricities with bars which were 4, 8, or 16 screen pixels long (1.2 mm, 2.4 mm, and 4.8 mm on the screen correspond to approximately 0.21°, 0.42°, and 0.85° visual angle, respectively, in the fovea). In these experiments we concentrated on the nasal visual field where orientation detection is not disturbed by the blind spot and steady performance curves have to be expected. We also included one test position close to the fovea and one within the blind spot in the temporal visual field to make sure that the subjects did not move their gaze or focus of attention. Targets appearing from time to time in the blind spot could be detected by the subjects only if their gaze were shifted from thefixationpoint during the experiments. The

27

two measurements in the nasal visual field thus allowed to appreciate coarsely the quality of fixation and the decision strategy of the subject in having one control position of maximum and one of minimum detection probability. The results for seven subjects are plotted in figure 4. Again the number of correct decisions was pooled for all target orientations, giving the average orientation detection probability as function of test bar eccentricity, for three different bar sizes and two contrast values (94% on the left, and 63% on the right side of fig. 4, respectively) treated as stimulus parameters.

100

80 (0

§ 60 o.

60

(0

40

2 high contrast -20

-10

0

eccentricity [deg]

10

1

20

low contrast -20

-10

0

I

\

0

10

eccentricity [deg]

Figure 4: Orientation detection for various line lengtiis. The percentage of correct responses for orientation detection is plotted as function of stimulus eccentricity for high (left side: c=94%, n=7 subjects) and low contrast (right side: c=63%, n=5 subjects). The length of the bar was set to 1.2 mm (dots), 2.4 mm (squares), or 4.8 mm (triangles), corresponding to 0.21° - 0.85° visual angle in the fovea. At either contrast level, performance at a given eccentricity is poorer for smaller bars than for larger bars. It is obvious for either contrast that performance decreases with increasing eccentricity, just as it was observed in the first experiment. In addition, the three curves are clearly arranged with respect to stimulus size. Orientation detection reaches a given level at smaller eccentricities when shorter test bars are used, as compared to longer bars. In other words, the orientation of a longer bar can be detected further out in the periphery than that of a shorter bar. This basic behaviour holds for both stimulus contrasts, but the performance is generally reduced for the low contrast stimuH. Again, it is not perfectly clear whether this is an effect of different slopes of the psychometric curves or of different positions along the abscissa. This combined dependence of the detection probability on stimulus eccentricity and size is to be expected from the knowledge about the spatial grain of cortical organisation mentioned in the introduction. According to the cortical magnification a given part of the visual field is represented by a certain number of ganglion cells in the retina and a certain area on the cortex (and a corresponding number of cortical neurones) which decreases according to a power function. This means that a Hne of constant length will excite more neural elements when presented in the fovea, as compared to peripheral stimulation. The M-scale factor indicates

28 what distance on the cortex corresponds to what angular distance in the visual field for a given position within the visual field. For the nasal meridian, for instance, 1 mm on the cortex surface covers a viewing angle of about 8' at 0° eccentricity (i.e., in the fovea) and about 1° at about 20° eccentricity. According to Rovamo and Virsu (1979), the magnification can be well approximated by the polynomial MF = 7.99 / (1 + 0.33 • E + 0.00070 • E^) in the nasal visual field, with E being the eccentricity, and MF the cortical size of one degree linear distance within the visual field, given in mm/°. This formula can be utilised to estimate for our experiments the size of the test bar as it is projected on the cortex. Following this line of thought, the performance of orientation detection is replotted in figure 5, now showing the percentage of correct decisions about the orientation of a bar as function of its cortical size. It can be seen immediately that the curves for the three bar sizes collapse when the bars are presented at the high contrast (left side). Irrespective of the physical bar size, which was varied as experimental parameter, the data points seem to describe a single function, sharply increasing from a value of about 40% at a cortical size of roughly 0.1 mm to approach 100% saturation at about 0.5 mm cortical size. This indicates that at high contrast the length of the cortical representation of a stimulus is the decisive variable which determines orientation detection performance. Further inspection of the same graph reveals an important feature of this size dependence of orientation detection: A performance between the minimum and the maximum level, i.e. about 60% - 80% correct decisions, may be regarded as a reasonable criterion to estimate threshold cortical size which indicates a reliable detection of orientation. This level is reached for cortical sizes of 0.2 mm - 0.3 mm, which is definitely less than the typical diameter of a hypercolumn being roughly 1 mm at any region of the primate cortex. This means that there is a good chance to investigate orientation detection in small regions which only cover a part of a hypercolumn when short bars are used as stimuli. Using bars which span about 0.2 mm on the cortical plane will allow stimulation of local groups of orientation columns without touching neighbouring columns responding to different orientations, and therefore an important precondition for investigating orientation maps seems to be fulfilled. Another surprising conclusion can be drawn from inspecting figure 5 when the density of input fibres on a given cortical area is considered. A rough estimate of the average number of input lines feeding from the retina onto a patch of cortical surface yields a value of 380 optic nerve fibres per mm^ striate cortex (1.01 • 10^ fibres project on 2613 mm^ cortical surface of area 17; according to Blinkov & Glezer, 1968). This leads to a value of about 20 x 20 fibres from the eyes spreading into an average rectangular cortex patch of 1 mm side-length. Similar figures of 10 to 15 input elements covering a distance of about 1 mm in the cortex emerge from more elaborate considerations of visual functions and cortical anatomy in the monkey (Braitenberg, 1985). If 0.2 mm cortex are sufficient to detect the orientation of a bar, it turns out that no more than 4 input elements appear to be necessary to detect orientation, assuming that our crude approximation of input density is appropriate. This is close to the theoretical limit, regarding each input fibre as a 'cortical image pixel': If the receptive fields of retinal

29 ganglion cells, and hence of geniculo-cortical fibres, are circular and not elongated, the minimal set of inputs necessary to recognise the orientation of a line is obviously two.

r

80)

I

8

I

r • •• • 1.2 mm bar • • •• • 2.4 mm bar I •• -^ ••• • 2.4 mm bar I • 4.8 mm bar

-.,^p. .^'

80

c " Q. Q »-

I

•

A.

•

^

.•

!•••••

60 h

• — •

^ •

; 4

A

m

*

4 •*

high contrast I

1

0.4

i

1

0.6

.•

^

f

^'

.'

1—

0.8

cortical size [mm]

—

1.0

••

*

A.

^ : A

low contrast —

0.2

.

C S

I

A'

• M .*

1

-I 60 1

H 40

»ari •

i80

A-

h

. ;

• - T •

.-

A a m m Kar I

.'•

fiO

40

] 100

. A- • • • . A

100

—

•

—

.

0.4

—

1

_

—

0.6

.

—

0.8

1

—

.

—

1

2 *-

"S 1 20 £ o o j I0

—

1.0

cortical size [mm]

Figure 5: Orientation detection and cortical size. The percentage of correct responses (same data and conventions as shown in figure 4, but note different scales on the ordinates) is plotted as function of cortical size by scaling the measurementsfromthe nasal visual field with the magnification factor. At high contrast, the performance curves for the three physical test bar sizes collapse to a single psychometric function, indicating that under these circumstances cortical size is the critical variable for orientation detection. When stimulating at low contrast (right side of fig. 5), the curves for the different bar sizes do not overlap as nicely as for high contrast. This may indicate that other factors besides stimulus size, such as local contrast enhancement features, may become relevant or even Umiting for the detection of orientation when contrast is lowered and may approach the threshold for the detection of a target per se. Under these conditions it also has to be taken into account that not only Une length but also its width (determined by the size of a single screen pixel) will vary in its cortical image size with eccentricity. Hence the cortical width may be larger for a physically smaller bar at a smaller eccentricity than for a longer bar which has the same cortical length due to larger eccentricity. According to this consideration the physiologically effective contrast may vary with the aspect ratio, leading to better detection of the bar (irrespective of its actual orientation) when the width-to-length fraction is larger. In fact, a better performance for smaller bars seems to emerge from the data shown on the right side of figure 5. Independent of this possible shift of the psychometric functions with cortical bar width, the performance at low contrast reaches half-maximum values (about 50%), in the region between 0.2 and 0.5 mm cortical size of the stimulus. Allowing for some amount of scatter in the data, this is close to the observation for high contrast values. In any case the saturation performance level is reached with a cortical stimulus size of 0.5 - 0.7 mm, still corresponding to less than the diameter of a hypercolumn (about 1 mm after Braitenberg, 1985). As reflected by these rather rough but conservative estimates, for low contrast bars the increase of performance with increasing cortical size (i.e. decreasing eccentricity) is less steep, and overall performance is worse. Because the base level of about 0% correct responses

30

(please note the different scales of the ordinates in left and right part of figure 5) corresponds to the subject's decision 'invisible', it may be suggested that at low contrast the detection of the stimulus per se could indeed be limiting the performance, instead of the detection of its orientation. At least, it cannot be excluded that different mechanisms might be responsible for orientation detection in the low and high contrast domains.

4. IN SEARCH OF CORTICAL ORIENTATION MAPS As mentioned before, the diameter of a hypercolumn is about 1 mm on the cortex (Braitenberg, 1985), independent of the actual position in the visual field as expressed by the cortical magnification. This means that a patch of cortex covering all orientations, corresponding to the average distance between two blobs in which no preferred orientation is observed, has the same size all over the striate cortex, but represents an increasing part of the visual field with increasing eccentricity. Knowing that we can detect the orientation of bars which in their cortical projection are much smaller than a hypercolumn, we felt motivated to measure the regional variation of orientation detection, looking for maps of preferred orientation. Using the same set-up and stimulus configuration as in the last experiment, we stimulated in the far periphery, around 29° within the nasal visual field. Since at this eccentricity a 1 x 1 mm^ cortex patch (i.e. a hypercolumn) corresponds to more than 1 x 1 degree^ within the visual field, we tested orientation detection at 5 x 5 positions within a region of about 1.6 X 1.6 degree^. The subjects were stimulated with bars consisting of 8 pixels in a line (corresponding to 0.32° visual angle or 0.21 mm of cortex) at high contrast (c=94%). Each of the 25 positions and each of the 16 orientation combinations was tested twice in two successive blocks in a single experiment which demands up to two hours of full concentration from the subject. The results of this experiment run on two experienced subjects can be seen in figure 6. The proportion of correct responses was about 50% on average (34%-69%) for the first subject, and about 65% (43%-81%) for the second, and did not vary systematically within the area tested. Therefore overall performance is not shown in the figure, and only the preferred orientation and the reliability of the estimate is indicated by the orientation and the length of the bars plotted into the 25 test fields, as described in the methods section. In both examples shown in figure 6 the bars seem to be grouped in some order. First of all, there are no specific preferred orientations which are not represented in the test field, basically all orientations appearing at some position. At some locations, on the other hand, no preferred orientation can be estimated (indicated by dots in figure 6), or the orientation selectivity is very weak (short bars). Most spatial changes between the preferred orientations are smooth, in that orientations in neighbouring test positions resemble each other, whereas they may differ considerably when the test positions are further apart. In this respect, the psychophysical measurements seem to resemble an 'orientation map'. Furthermore, there happens to be a central position without clear orientation preference, a singularity, for which the preferred orientations of the neighbouring test positions are arranged very similar to a tangential circle (shaded test fields). The size of this vortex-like pattern even matches the expected hypercolumn size which has a diameter corresponding to about 1° viewing angle in

31 this part of the visual field. Of course, there are distortions which impede the basic pattern extracted from these first measurements, but this also resembles to some extent the physiological observations. In the cortical orientation maps visualised by means of optical recording techniques, the pattern of preferred orientations can have local irregularities, as can be seen in the examplefi*omthe literature data sketched in figure 1.

-30

^2 (0 0)

1

"

-29

1

1

-28

>

I I

.^^/^'^/^"^/^""^ .^"^ H

-30

-29

-28

1

+1

OMzOO000l HvQQQQ©] (ZX30®® 1 s •0GXZXEXS) >

O

E

«*E o

0)

0

o c

(0 (0 T3

75 o '*^

>

i ^ ' A A A ^ A ^ A A A ^

•®0000 .^000© 1

.

1

,

-1 11

horizontal distance from fovea [°]

- 00G)0Oi

-^^^k\ 00000]

V.

»^o o c CO

*•* w 'S 75 o

r > horizontal distance from fovea [°] 1

1

1

1

1 1

Figure 6: Psychophysical orientation map in the peripheral visual field. The preferred orientations and the reliability of the estimate are shown for each test position as orientation and length of a bar plotted in a circle indicating roughly the size of the test bar. Orientation detecticHi was tested in two subjects (JMZ and HV) monocularly at 25 positions around 30® eccentricity (abscissa: horizontal position in the visual field) close to the meridian (ordinate: vertical position) in the nasal visual field. Note that the hexagonal grid is slightly distorted (vertical distances are increased by roughly 10%) due to the flat screen geometry.

However, given the inevitable data scatter, there is a danger that the exemplary images of the orientation preference in the peripheral visual field which are shown in figure 6 may be misleading. Instead of depicting the cortical orientation map they could simply reflect a pattern which emerges by chance in the data analysis, from essentially irregular distributions of responses. Taking into account the duration of the experiment, there may be doubts about the stability of attention and fixation over a period of about two hours. This also makes it difficult to map bigger regions at the same resolution with this psychophysical method, searching for a clearer image of the vortex-like topological structure in a larger patch of the perceptual map. So the question arises how reliable such a map actually may be, beyond the first impression that it seems to meet the expectations. Thus the reproducibility of such measurements was investigated in a control experiment. A horizontal row of 7 test positions in the far nasal periphery was presented to the subjects five

32 times at about one hour intervals. From the 5 consecutive maps of one subject which are shown in figure 7 exemplarily, it is immediately clear that distortions of the data are considerable. Whereas the orientations in some test fields are well reproducible (leftmost position, for instance), in other positions they vary strongly (see switch between almost orthogonal preferred orientations between trial 2 and 3 in rightmost position, for instance). Thus it is not entirely clear whether the measured orientation specificity maps really reflect cortical orientation preferences or simply emerge from the imponderabilities of the test procedure. When the average orientation difference between consecutive trials at the same position is compared quantitatively to the average orientation difference between two positions separated by 1° in space, it turns out that these two measures are quite close for the two subjects who took part in several control experiments of this kind (32.5-39.5° and 37.8-44.3°, respectively).

-30

' + .5h

-29

-28

I ' I ' I ' trial 1 A

0 -.5

I I I I I I I + .5 0 -.5 + .5 0 -.5

trial 2 4

h000©(Z)©(S)-^ I I I I I I I trials

I I I I I I I

trial 4

+ .5 0 -.5

hasxz)®®®®-!

+ .5

I 11 I I I I trials

o|-0©6®(Z)®®-|

•o

"(5

Figure 7: Replication of orientation mapping. For a single row of test positions of the field shown in figure 6 (on the horizontal meridian around 29° eccentricity in the nasal visualfieldof the right eye) the experiment was repeatedfivetimes, in order to appreciate the reliability of the preferred orientation measurements (conventions as in figure 6). Whereas at some positions the preferred orientation is quite stable in this example (compare the bars in the leftmost test field for all trials), at other positions they may change considerably (see jump from trial 2 to trial 3 at rightmost test position).

• ' •from ' fovea • horizontalI distance [T

In summary, from this limited set of data, the quesrion cannot be fmally settled whether a psychophysical orientation map such as depicted in figure 6 reliably reflects the cortical organisation of orientation preference in humans. Future research thus has to concentrate on the question of how psychophysical mapping can be made more reliable. It has to be considered whether the experimental setup and strategies can be optimised, and data analysis can be improved. Strongest emphasis should be placed on attempting to stabilise the visual stimulation on the retina, because involuntary eye movements could be a major source of

33

irregularities in the maps. Image stability could be controlled by measurements of eye movements throughout the experiments, and in advanced studies one could even try to generate fixed retinal images by elaborate optical methods (Kelly, 1979). 5. CONCLUSIONS Detecting the orientation in the retinal image of the surroundings is a very basic task for visual systems. It is extremely important on various spatial scales, from the detection of object orientation (such as reading the hands of a watch), through outlining object contours (as is necessary for recognising specific shapes), to texture based image segmentation which requires fine grain analysis of local orientation. Thus one is not surprised that orientation sensitivity is one of the first and most general steps of visual processing in the cortex which is extensively described physiologically (Hubel & Wiesel, 1977; Orban, 1984; Bonhoeffer & Grinvald, 1993, for instance), and has attracted some theoretical attention (Braitenberg, 1985; Gotz, 1988; Bauer & Dow, 1989; Malsburg, 1973; Swindale, 1991; Linsker, 1986, for instance). In the present context, we investigated psychophysical^ human performance for detecting the orientation of small bars, and related the geometrical constraints to cortical anatomy. It turned out that bars can be made surprisingly small while still being reliably detected in their orientation. Taking the cortical magnification into account, it can be shown that at high contrast not the physical size (i.e. the angular size in visual space), but the elongation of the stimulus in the cortical projection is limiting the performance of orientation detection. This suggests that a critical number of input lines - or 'cortical pixels' - are required to resolve the orientation of a local contour. Assuming a homogeneous cortical architecture, the literature values for the number of fibres in the optic nerve and for the surface area of the primary visual cortex lead to the notion that no more than about 4 input lines coming from the eye are needed to detect the local orientation. This is close to the theoretical limit of two cortical pixels which are the absolute minimum to discriminate orthogonal orientations. It is obvious that more pixels are required to achieve a higher differential orientation sensitivity. This performance limit has serious consequences for the mechanism which has to be proposed to underlie orientation detection. In particular, the data support the idea of a largely localised elementary mechanism. With large receptive fields - for instance of the Gabor type which is often suggested to describe VI neurones appropriately (Daugman, 1985; Heitger, Rosenthaler, Heydt, Peterhans, & Kubler, 1992) - it would be difficult to detect reliably the orientation of stimuli which are small compared to the receptive field size (Zanker, in prep.). Knowing about the local character of orientation detection, we felt encouraged to put some effort into tackling psychophysically the question of the basic cortical organisation of orientation sensitivity. Although the results of these attempts to describe maps of preferred orientation are of limited significance on statistical grounds, the basic patterns of orientation preference seem to indicate map-like structures. This should be taken as a promising starting point for future work in which such maps are investigated with higher precision. Taken together, the experimental results presented here can be interpreted as supporting the ideas about cortical organisation outlined in the introduction. Such organisation would explain the electrophysiological and anatomical data on the basis of very simple inhibitory mechanisms

34

within the hypercolumns which would lead to the observed patterns (Braitenberg, 1985; Braitenberg & Schiiz, 1991). Besides these speculations prompted immediately by our psychophysical results, in future work it has to be considered how to extend the coarse view on cortical organisation which we have had to adopt so far for reasons of simplicity. In order to characterise the elementary mechanism underlying orientation sensitivity, it would be valuable not only to test the detection of bars which are at least 45° apart in their orientation, but also to investigate orientation discrimination with smaller differences in the test bar angles. The orientation fine tuning would provide important information to analyse models with reference to their differential sensitivity. In this context, it has also to be noted that there is no general reason to restrict the observed efficiency of orientation processing to the first cortical stages, as was done here because it would be the most simple functional principle. Of course, higher processes which are not addressed by the experiments so far could be responsible for specific features of orientation detection. In a similar sense, different mechanisms in the fovea cannot be excluded, which for instance could improve orientation detection by special pooling mechanisms. Such a mechanism could be the reason for the casual observation that at very low contrasts the orientation of bars can be still be seen in the fovea, although they would disappear totally in the periphery, even if presented with the same cortical size. In summary, the experiments presented here may be interpreted as a first step into a large field of experimental questions which relate orientation selectivity to cortical organisation, and therefore may turn out to be relevant for our understanding of basic mechanisms of brain function. Acknowledgements: We want to thank A. Aertsen, A. Borst, B. Crespi, M. Fahle, C. Furlanello, and L. Stringa for many valuable discussions and carefully reading of earlier versions of the manuscript, and C. Clifford for correcting the English.

REFERENCES Bauer, R. & Dow, B.M. (1989). Complementary global maps for orientation coding in upper and lower layers of the monkey's foveal striate cortex. Experimental Brain Research, 76, 503-509. Blasdel, G.G. (1992). Orientation selectivity, preference, and continuity in monkey striate cortex. The Journal of Neuroscience, 12, 3139-3161. Blasdel, G.G. & Salama, G. (1986). Voltage-sensitive dyes reveal a modular organization in monkey striate cortex. Nature, 321, 579-585. Blinkov, S.M. & Glezer, I.I. (1968). Das Zentralnervensystem in Zahlen und Tabellen. Jena: VEB Gustav Fischer Verlag. Bonhoeffer, T. & Grinvald, A. (1993). The Layout of Iso-orientation Domains in Area 18 of Cat Visual Cortex: Optical Imaging Reveals a Pin wheel-like Organization. The Journal ofNeuroscience, 13, 4157-4180.

35 Braitenberg, V. (1985). Charting the Visual Cortex. In A. Peters & E.G. Jones (Eds.), Cerebral Cortex (pp. 379-414). Plenum Publishing Corporation. Braitenberg, V. (1992). How Ideas Survive Evidence to the Contrary: A Comment on Data Display and Modelling. In A. Aertsen & V. Braitenberg (Eds.), Information Processing in the Cortex (pp. 447450). Berlin: Springer. Braitenberg, V. & Braitenberg, C. (1979). Geometry of Orientation Coloumns in the Visual Cortex. Biological Cybernetics, 33, 179-186. Braitenberg, V. & Schiiz, A. (1991). Anatomy of the Cortex. Statistics and Geometry. Berlin: Springer Verlag. Camarda, R.M., Peterhans, E., & Bishop, P.O. (1985a). Spatial organization of subregions in receptive fields of simple cells in cat striate cortex as revealed by stationary flashing bars and moving edges. Experimental Brain Research, 60, 136-150. Camarda, R.M., Peterhans, E., & Bishop, P.O. (1985b). Simple cells in cat striate cortex: responses to stationary flashing and to moving light bars. Experimental Brain Research, 60, 151-158. Cowey, A. & Rolls, E.T. (1974). Human Cortical Magnification Factor and its Relation to Visual Acuity. Experimental Brain Research, 21,447-454. Daniel, P.M. & Whitteridge, D. (1961). The representation of the visual field on the cerebral cortex in monkeys. Journal of Physiology, 159, 203-221. Daugman, J.G. (1985). Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. Journal of the Optical Society of America, A 2, 1160-1169. Fox, P.T., Miezin, F.M., AUman, J.M., Essen, D.C.v., & Raichle, M.E. (1987). Retinotopic organization of human visual cortex mapped with positron-emission tomography. The Journal of Neuroscience, 7, 913-922. Gotz, K.G. (1987). Do "d-blob" and "1-blob" Hypercolumns Tesselate the MOnkey Visual Cortex? Biological Cybernetics, 56, 107-109. Gotz, K.G. (1988). Cortical templates for the self-organization of orientation-specific d- and 1hypercolumns in monkey and cats. Biological Cybernetics, 58, 213-223. Heitger, F., Rosenthaler, L., Heydt, R.v., Peterhans, E., & Kiibler, O. (1992). Simulation of neural contour mechanisms: From simple to end-stopped cells. Vision Research, 32, 963-981. Horton, J.C. & Hoyt, W.F. (1991). The Representation of the Visual Field in Human Striate Cortex. Archives of Ophthalmology, 109, 816-824. Hubel, D.H. & Livingstone, M.S. (1987). Segregation of form, color, and stereopsis in primate area 18. The Journal of Neuroscience, 1, 3378-3415. Hubel, D.H. & Wiesel, T.N. (1977). Functional architecture of macaque monkey visual cortex. Procedings of the Royal Society London, B 198, 1-59.

36 Kelly, D.H. (1979). Motion and vision. I. Stabilized images of stationary gratings. Journal of the Optical Society of America, 69, 1266-1274. Linsker, R. (1986). From basic network principles to neural architecture: Emergence of orientation columns. Proceedings of the National Academy of Sciences USA, 83, 8779-8783. Livingstone, M.S. & Hubel, D.H. (1988). Segregation of Form, Color, Movement, and Depth: Anatomy, Physiology, and Perception. Science, 240, 740-749. Mallot, H.A. (1985). An Overall Description of Retinotopic Mapping in the Cat's Visual Cortex Areas 17, 18, and 19. Biological Cybernetics, 52,45-51. Malsburg, C.v. (1973). Self-Organization of Orientation Sensitive Cells in the Striate Cortex. Kybemetik, 14, 85-100. Orban, G.A. (1984). Neuronal Operations in the Visual Cortex. Berlin: Springer Verlag. Pointer, J.S. (1986). The cortical magnification factor and photopic vision. Brain Research, 61, 97-119. Ratliff, F. & Riggs, L.A. (1950). Involuntary motions of the eye during monocular fixation. Journal of Experimental Psychology, 40, 687-701. Rovamo, J. & Virsu, V. (1979). An Estimation and Application of the Human Cortical Magnification Factor. Experimental Brain Research, 37,495-510. Rovamo, J , Virsu, V., & Nasanen, R. (1978). Cortical magnification factor predicts the photopic contrast sensitivity of peripheral vision. Nature, 271, 54-56. Schiller, P.H., Finlay, B.L., & Volman, S.F. (1976). Quantitative Studies of Single-Cell Properties in the Monkey Striate Cortex, I. Spatiotemporal Organization of Receptive Fields. Journal of Neurophysiology, 39, 1288-1319. Seelen, W.v. (1970). Zur Informationsverarbeitung im visuellen System der Wirbeltiere. Kybemetik, 1, 89-106. Swindale, N.V. (1991). Coverage and the design of striate cortex. Biological Cybernetics, 65,415-424. Tootell, R.B., Switices, E., Silverman, M.S., & Hamilton, S.L. (1988). Functional Anatomy of Macaque Striate Cortex, n. Retinotopic Organization. The Journal ofNeuroscience, 8,1531-1568. Tso, D.Y., Frostig, R.D., Lieke, E.E., & Grinvald, A. (1990). Functional Organization of Primate Visual Cortex Revealed by High Resolution Optical Imaging. Science, 249,417-420. Tynan, P. & Sekuler, R. (1982). Motion processing in peripheral vision: reaction time and perceived velocity. Vision Research, 22, 61-68. Wassle, H., Griinert, U., Rohrenbeck, J., & Boycott, B.B. (1989). Cortical magnification factor and the ganglion cell density of the primate retina. Nature, 341, 643-646.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) 1996 Elsevier Science B.V.

37

Multiple Parietal Representations of Space Carol L. Colby, Jean-Ren6 Duhamel and Michael E. Goldberg Laboratory of Sensorimotor Research, National Eye Institute, Building 49, Room 2AS0, National Institutes of Health, Bethesda, MD 20892 1. Introduction We live in an ever-changing sensory world. As we move our eyes and move through the environment, new images are continuously presented to the brain. Given such constantly changing input, it is remaiicable how easily we are able to keep track of where things are. We can reach for an object, or look at it, or even kick it without making a conscious effort to assess its location in space. The traditional view of spatial perception, strongly supported by subjective experience, is that we "know where things are** in some absolute, world-based frame of reference and use this spatial information to guide our movements. In this standard and intuitively plausible view, spatial perception is a monolithic process: the brain forms a single spatial representation of each object regardless of what action is going to be performed in relation to that object. A new and somewhat counterintuitive view is that the brain represents the spatial location of an object many times over in different cortical areas and each representation is suited to certain kinds of behavicmal responses. Neurophysiological research carried out in monkeys indicates that in areas responsible for controlling head movements, visually sensitive neurons encode the location of an object relative to the head (21,6,8). In contrast, areas involved in limb movement have neurons which encode object location relative to Umb position (17) and, in areas controlling eye movements, visually sensitive neurons encode the location of an objectrelativeto the center of gaze (16,15,13). Two such areas have been discovered in the parietal cortex of rhesus monkeys (6). Neurons in each area are selective for particular stimulus dimensions and particular regions of space. They signal not only where an object is but how to act on it. The representation of space in a given area reflects a particular motor output by which a stimulus can be acquired or avoided. These results suggest that parietal cortex contains multiple action-based spatialrepresentations.In the scheme emerging from these studies, spatial perception is a modular process. A single object may be multiplyrepresentedin terms of the actions that can be performed on it. While this new view seems implausible at first glance, it is exacdy analogous to current views of visual perception. When we see a large red bouncing bsdl, we perceive a single object, even though its size, shape.

38

color and direction of motion arc analyzed separately. Likewise, beneath the apparent unity of subjective spatial experience, lies a diversity of spatial representations, each with specific knowledge of how to act on an object. 2. Parietal Visual Areas Posterior parietal cortex is divided into a number of separate areas. Unlike lower level extrastriate visual areas, these parietal areas do not typically contain simple retinotopic maps and their borders cannot be defined with reference to ventral or horizontal meridian representations. Parietal areas have instead been initially identified on the basis of their connections with other cortical areas (1,2,9,19). Connections alone, however, are not sufficient to define an area. For instance, both the lateral and the ventral intraparietal areas (LIP and VIP) receive strong projections from area MT, in the superior

AREAS

AREA 7

•

SACCADE / TONIC / FOVEAL / FIXATION / ORIENTATION

H VISUAL / NOT DIRECTION SELECTIVE I

I

PASSIVE SOMATOSENSORY

§

ACTIVE SOMATOSENSORY / REACH

VISUAL/DIRECTION SELECTIVE

Figure 1. Distribution of neuronal response properties in rhesus monkey intraparietal sulcus. Each column represents a single 10 mm penetration along the lateral or medial bank of the sulcus. Two rows of penetrations spaced 1 mm apart are shown for each bank. The anterior part of the sulcus is shown at the top of the figure. The posterior portion of the sulcus is shown at the bottom of the figure, where the banks of the sulcus have been separated.

39 temporal sulcus (23). The most reliable guide to aieal boundaries in parietal cortex is the response properties of the neurons. Because many features of parietal neurons are observable only in alert animals (20,22,4,5) we have done a behavioral mapping of parietal cortex in which neural activity is examined in relation to a large set of tasks. Our standard protocol includes tests for visual and somatosensory responsiveness, attentional modulation of these responses, and for oculomotor and somatomotor activity. Previous efforts to map parietal cortex have also used alert animals and shown clear evidence for a regional distribution of distinct functional cell types (18). In the current experiments, we controlled the monkey's behavior by rewarding the animal for fixation or eye movements or selective attention to particular stimuli. A second advance over previous mapping experiments is the use of a recording grid which allows us to record from identified locations repeatedly and allows precise reconstruction of the location of recording sites (10). This reconstruction is especially important because recordings are carried out over a long period of time and because these mapping experiments have focused on cortex buried in the intraparietal sulcus. Accurate reconstruction of recording sites is critical for establishing the location of borders between physiologically defined areas. An overall view of the functional organization of intraparietal sulcus is presented in Figure 1. Of the many different kinds of cells observed, each has a restricted distribution within the sulcus. In moving from one region of the sulcus to the next, the probability of encountering any particular cell type changes systematically. In some regions of the sulcus, such as that near the fundus, the borders between areas are sharply defined by sudden changes in the predominant cell type. In other regions, such as on the medial bank, the boundaries are apparently less sharp and there is a more gradual shift in the response characteristics of the cells. Two areas in the intraparietal sulcus, LIP and VIP, have been defined physiologically. The following sections will describe response properties of neurons in each area and how they contribute to a different spatial representations. 3. Spatial Representation in Area LIP Neurons in the lateral intraparietal area are active in relation to both visual and oculomotor events (16,7,14). They discharge when a visual stimulus appears in the receptive field and, for about half the population, discharge again when the monkey executes a saccade to the location cued by the stimulus (Fig. 2). The strength of the visual response is modulated by the behavioral set induced by the task. When the monkey must attend to the stimulus, the amplitude of the on response is enhanced, compared to the response seen when the stimulus is irrelevant for the monkey's behavior. In addition to these visual, attentional and motor signals, many LIP neurons also carry a memory signal (14,6). These tonically active neurons continue to respond to a visual stimulus in the remembered saccade task during the interval between the appearance of the stimulus and the onset of the saccade. There is no visual stimulus present in the receptive field during this interval. In order to perform the task accurately, the monkey must retain an image of the stimulus location during the delay interval. Tonic activity during the delay reflects a memory trace of stimulus location. These results on multiple sources of activation in LIP underscore the importance of studying neurons in different behavioral tasks. If we had used only a saccade task we

40

might have concluded that the neurons were driving eye movements. But they cannot just be guiding eye movements because they are active in tasks in which saccades aie either inelevant or forbidden. Conversely, if we had used only a fixation task we might have concluded that the neurons were simply visual. But they cannot be performing a purely visual analysis because they consistently respond in circumstances in which there is no stimulus. The results from multiple tasks indicate that the responses of LIP neurons do not depend exclusively on either vision or movement The single point of intersection of the various activations observed is the receptive field itself.

RF

FP

FP

Vr

RF L. HMIIII ! • • • II i n

RF

I

• • • H nil I I 11 a m i I B M i • • • I III I INI I • i l I I 11 H MIH

I M

••11 I

II • • • • • • • § • III nil IN laili^Hii I I I • I M II I I id • • • • i i H i i m i i ^ i M n BIIHI M I l i l l l ^ 11 I l l l i n ^ I H I • INIilli I I n i l I • • milil^ • I M B N I I I • • I • I I I a n II wm i i a a i iii • • • I I I ••imii I I w i l l I ••IMia I I I I I B I i I I IIIBI • i i m III iiiMMii I I • IHIIMHH • • • I • I • « • III I I ill

• •••iM^iiw in Mil I I I I II I II MHIW § • • • I I l 4 I ••Mil mil I I ••HI I I I I i H H i a n I i H i I I IN I II i tmit III I

I I I • • • • • • a i ^ I I I III I II 11 I I I I H B M I I I I I II I ri I 11 • • i i i a i i i H i i i w I • i i m H I i i i i m II I I I i i H i IIII l a i i II M i l la II iif

I

^

•••IIH

HII I B I ^ I H

•man

r

ii ina iii

I H I III •liiim

II

•

•

I B BI^HI^ l l l l i H I l I I III I I • » • • I II l l i l l i

I

•iiiimHiH H M i • I mm iii

I

64151-14

Stimulus Appearance

Saccade Onset

Figure 2. Response of an LIP neuron in a remembered saccade task. The cartoon above each diagram shows the relative locations of thefixationpoint (FP) and the receptive field (RF). The time lines show vertical eye position (V) and the onset and offset of a stimulus in the RF. Each tic mark in the raster diagram signifies a single action potential. Successive trials are shown on successive lines, synchronized (vertical line) on the event indicated below the histogram. The calibration bar at left signifies a response rate of 1(X) spikes per second. In this task, the monkey must fixate while a stimulus is briefly presented in the RF. After a variable delay, the fixation point is extinguished and the monkey saccades to the location where the stimulus appeared. Separate visual and motor bursts are seen in each trial, as well as tonic activity during the memory period.

41 We hypothesize that l i P neurons encode spatial locations. Further, their activity is modulated by attention to a spatial locus that is defined not by a stimulus or by a movement, but by the spatial vector that could describe either. In essence, the activity of an LIP neuron encodes an attended spatial location. If LEP is encoding space, rather than visual perception or specific behavion, it is necessary to understand die coordinate system in which it operates. There are three plausible coordinatefiramesfcH* representing stimulus position. For a neuron operating in retinal coordinates, neural activity signals where the stimulus is on the retina. In a headcentered coordinate firame, neural activity signals where the stimulus is relative to the head, regaidless of where the eyes are looking. In oculocentnc coordinates, neural activity signals the saccade necessary to foveate the stimulus. We have been able to discriminate among these three possibilities by observing what happens to memory-related activity in LIP when the monkey makes a saccade (13). Every time an eye movement occurs, the projection of the visual world changes on the retina and, by implication, in all the retinotopically mapped areas of the brain. If it is to be useful, visual information currendy being processed in LIP must be remapped in conjunction with each saccade. We have discovered that LIP neurons remap the memory trace of a previous stimulus event (Fig. 3). While the monkey fixates, a stimulus is briefly presented (50 msec) at a location well outside the receptive field of the neuron (Fig. 3B). A new fixation point appears, and the monkey makes a saccade to it Because the stimulus flash is so brief, the stimulus is no longer present at the time of the saccade. The effect of the saccade is to bring the receptive field to the location that was previously stimulated. If the neuron had continued to encode events at the original receptive field location (i.e., in head-centered coordinates), it would remain silent after the saccade because there was never any stimulus at that location. Likewise, if the neuron had access only to retinal information (retinal coordinates) it would also remain silent, since no stimulus appeared in its retinal receptive field. The results show instead that LIP neurons do respond when the receptive field is brought to land on a previously stimulated location (Fig. 3B). There is no stimulus present on the screen, so the neuron can only be responding to a memory trace of the stimulus. Control experiments confirm that neither the stimulus alone (Fig. 3C) nor the saccade alone (Fig. 3D) can drive the neuron. We conclude that the neuron is responding to a remapped memory trace of the stimulus which is encoded in oculocentric coordinates. The specific spatial problem which LIP must solve is how to signal a spatial location with reference to the current fovea. By remapping memory traces, parietal cortex constructs a spatial representation that encodes stimulus location in terms of distance and direction from the current center of gaze. While neurons in LIP have retinotopic receptive fields, visual information in LIP is dynamically updated in conjunction with eye movements to produce an oculocentric representation: stimuli are coded in terms of their distance and direction from the fovea. LIP neurons maintain an oculocentric representation of target position by using a corollary discharge from the eye movement command to update retinotopic visual information. This remapping of stimulus location serves to maintain an alignment between the external world and the internal representation of it. Remapping also provides the oculomotor system with continuously accurate information about the vector of the saccadic eye movement necessary to acquire the stimulated location. Remapping is the means by which a coordinate transformation is effected from retinotopic to oculocentric coordinates.

42

B.SaccadetoFP2 Stim flashed for 50 msec

A. Stimulus In Receptive Field No Saccade

'I*

H t

StIm FP r

FP2

r-

Stlm

n_

FP J I I I III i

1_

III

untt 64151

Stimulus Onset

Beginning of Saccade

C. Stimulus Outside Receptive Field No Saccade

D. Saccade to FP 2 No Stimulus

'I*

pp. V H f_

FP2_

H r FP2

StIm.

sum

•

.FP2

~r

1

FP

III 4

!• I

I J.H

< .nil—I

Stimulus Onset

l«ii.M !••»*• I • !

•!••
4* •Ml

0

20

Figure 7. Trajectory selectivity in a VIP neuron. Top row: stimuli arc moved toward the monkey's brow while the monkey fixates a central FP on the tangent screen. Bottom row: the same stimuli are moved toward the monkey's chin, evoking a much larger response. The projected point of contact of the stimulus is more strongly related to response rate than either the absolute direction of motion (straight toward vs. down and toward) or the portion of the visual field stimulated (upper vs. lower).

48 Toward Forehead from Above with Eyes Up

'f*r f'hf "t 64-062-07

200MS

Toward Chin from Above with Eyes Up

I J Toward Chin with Eyes Down

^ ^

iJHi

Li4.*Jl>LK

-a

20

Figure 8. Head-centeied spatial coordinates in VIP. Changes in eye position do not change selectivity for stimuli moving towards the chin.

49 Cortex dorsal to VIP in the anterior portion of the medial bank is puiely somatosensory with an emphasis on hand representation (Fig. 1). Near the fundus of the sulcus, there is a sudden transition to a region of strong visual responsiveness. This visual area extends from the medial bank across the fundus to the lateral bank. VIP neurons are well driven by moving visual stimuli and most are selective for direction of stimulus motion. Other properties similar to those found in areas MT and MST, such as speed tuning and responsiveness to whole-field motion, are also observed, consistent with its inputs from these areas (23,3). These visual response properties set VIP apart from the surrounding cortex. Two special types of VIP neurons are of interest with regard to spatial representation. The first is the ultranear cells. These visual neurons respond only to stimuli presented very close to the the animal, within a few centimeters of the face (Fig. 6). These neurons may signal the presence of a stimulus that can be acquired by reaching with the mouth. The second special type is trajectory neurons. These cells respond selectively to a stimulus moving towards or away from the animal. For these neurons, the absolute direction of stimulus motion is less important than the anticipated point of contact of the stimulus. In the example shown, a stimulus moved toward the chin elicited a much stronger response than the same stimulus moved toward the brow (Fig. 7). This result suggests that something other than a simple retinal coordinate frame is used to represent space in VIP. This suggestion was confirmed by having the monkey change its gaze direction (Fig. 8). The neuron continued to respond best to a stimulus moving towards the chin regardless of eye position. This insensitivity to eye position indicates that the stimulus is encoded in head-centered coordinates and not retinal coordinates. A surprising feature of VIP is that most neurons can be independendy driven by somatosensory stimulation (11,12). The somatosensory receptive fields are found primarily on the face and head. The visual and somatosensory receptive fields correspond to one another in location, in size and in directional selectivity, as illustrated in Figure 9. This neuron responds both to a peripheral visual stimulus moved toward the fovea and to a cutaneous stimulus moved across the face toward the mouth. VIP neurons with visual receptive fields in the upper hemifield have somatosensory receptive fields on the upper part of the face and brow, while neurons with lower field visual receptive fields have somatosensory receptive fields on the lower part of the face. Strikingly, VIP neurons with foveal visual receptive fields have somatosensory receptive fields around the mouth, as though the mouth were the "fovea" of the facial somatosensory system. For bimodal trajectory sensitive neurons, the visual response is tied to the location of the somatosensory receptive field. Finally, some VIP neurons with very large visual receptive fields have somatosensory receptive fields that include the hand and arm as well as the head. These findings on VIP response properties are consistent with a spatial representation in head-centered coordinates. Both the ultranear and the trajectory sensitive neurons appear to encode stimulus location in a head-centered coordinate frame. The bimodal neurons may have a special role in hand, eye and mouth coordination. Overall, visual targets may in coded in VIP in terms of how they can be acquired by reaching with the head and mouth.

50

X

y 1

I m m i l III III

IIHIHtll I I

1

II

1 1 11 1

iimaiiiiii I I

I I I Hill I I I IIIIMIII MM IIIIHII n i l

1 1 1II 1 1 1 p •

II 1

I

I ai I I I I H iiiiiii I

I

I

I II

II H I

II I I I I I I I

i i i i i i i III i l l I I I n i l I III m i l i i a i i i I I II nil I I I MM ! • 111111111 n i l II i l l I I m i l nil I I I I I I I I I I mil I I I II I I III 11 I I I III I III I I I n I III I l i I I I II III IIII II I I I I I II I I I I I I I I I I I I I I I II I I I I till III I I I I I I I

I

II

I I III M I I I

I I

I n i l

II

n i l RIM 11 I I I 11 I miiiii • nil

1

1 1 11

1

H 1 1 1 1 1 1 1 1 1 1 1 mil II 1

1

Ui I

1

•

'

ljui|--l k-4-

1 1

nil 1 1

1

I I I I iii I i I Bllll I II II II I 11 I |i I I II I I I I I I I I I g I I I I I I I I B I I I I I I I I I I 11 II I I II II II I I I I I I III I I I I I I II r I I I I II H I I I I il 1111 11 I nil I 111 I I I I I III II I I I I I I I I I I I I I I 11 I I I I I i I I I III

ti Figure 9. Bimodal sensory responsiveness in a VIP neuron. Top panel shows the location of the visual RF and a directionally selective response to a stimulus moved through the RF. Bottom panel shows the location of the somatosensory RF for the same single neuron and a directionally selective response to a somatosensory stimulus moved across the RF (tested with eyes closed).

5. Conclusions Parietal cortex contains multiple spatial representations, two of which have been described here. Neurons in area LIP encode stimulus location in oculocentric coordinates, while some VIP neurons encode stimulus location relative to the head. These multiple representations are presumably tailored for guiding specific kinds of actions, namely eye movements and head movements. The function of parietal cortex is to signal the location of attended objects. It does so in order to allow the perceiver to act on its environment, and different kinds of actions are likely to be supported by different spatial representations.

51

References 1. Andersen, R. A., Asanuma, C , and Cowan, M. Callosal and prefrontal associadonal projecting cell populations in area 7a of the macaque monkey: a study using retrogradely transported fluorescent dyes. / . Comp, Neurol. 232: 443-455, 1985. 2. Andersen, R. A., Asanuma, C, Essick, G., and Siegel, R. M. Corticocortical connections of anatomically and physiologically defined subdivisions within the inferior parietal lobule. / . Comp, Neurol 296: 65-113, 1990. 3. Boussaoud, D., Ungerleider, L. G., and Desimone, R. Pathways for motion analysis: Cortical connections of the medial superior temporal and fundus of the superior temporal visual areas in the macaque. / . Comp, Neurol. 296: 462-495, 1990. 4. Bushnell, M. C , Goldberg, M. E., and Robinson, D. L. Behavioral enhancement of visual responses in monkey cerebral cortex: I. Modulation in posterior parietal cortex related to selective visual attention. / . Neurophysiol. 46: 755-772, 1981. 5. Colby, C. L. The neuroanatomy and neurophysiology of attention. / . Child Neurol. 6: S88-118, 1991. 6. Colby, C. L. and Duhamel, J.-R. Heterogeneity of extrastriate visual areas and multiple parietal areas in the macaque mcHikey. Neuropsychologia 29: 497-515, 1991. 7. Colby, C. L., Duhamel, J.-R., and Goldberg, M. E. The analysis of visual space by the lateral intraparietal area of the monkey: the role of extraretinal signals. In: Progress in Brain Research, Vol, 95, edited by T. P. Hicks, S. Molotchnikoff, T. Ono,, 1993, pp. 307-316. 8. Colby, C. L., Duhamel, J.-R., and Goldberg, M. E. Ventral intraparietal area of the macaque: Anatomic location and visual response properties. / . Neurophysiol, 69: 902-914, 1993. 9. Colby, C. L., Gattass, R., Olson, C. R., and Gross, C. G. Topographic organization of cortical afferents to extrastriate visual area PO in the macaque: a dual tracer study. / . Comp. Neurol. 238: 1257-1299, 1988. 10. Crist, C. F., Yamasaki, D. S. G., Komatsu, H., and Wurtz, R. H. A grid system and a microsyringe for single cell recording. / . Neurosci. Methods 26: 117-122, 1988. 11. Duhamel, J.-R., Colby, C. L., and Goldberg, M. E. Congruent visual and somatosensory response properties of neurons in the ventral intraparietal area (VIP) in the alert monkey. Soc. Neurosci. Abstr. 15: 162, 1989. 12. Duhamel, J.-R., Colby, C. L., and Goldberg, M. E. Congruent representations of visual and somatosensory space in single neurons of monkey ventral intraparietal cortex (area VIP). In: Brain and Space, edited by J. Paillard, Oxford: Oxford University Press, 1991, pp. 223-236. 13. Duhamel, J.-R., Colby, C. L., and Goldberg, M. E. The updating of the representation of visual space in parietal cortex by intended eye movements. Science 255: 90-92, 1992. 14. Gnadt, J. W. and Andersen, R. A. Memory related motor planning activity in posterior parietal cortex of macaque. Exp. Brain Res. 70: 216-220, 1988.

52 15. Goldberg, M. E. and Bruce, C. J. Primate frontal eye fields. HI. Maintenance of a spatially accurate saccade signal. / . NeurophysioL 64: 489-508, 1990. 16. Goldberg, M. E., Colby, C. L., and Duhamel, J.-R. The representation of visuomotor space in the parietal lobe of the monkey. Cold Spring Harbor Symp, Quant, Biol. 55: 729-739, 1990. 17. Graziano, M. S., Yap, G. S., and Gross, C. G. Coding of visual space by prcmotor areas. Science 266: 1054-7, 1994. 18. Hyvarinen, J. Regional distribution of functions in parietal association area 7 of the monkey. Brain Res. 206: 287-303, 1981. 19. Maunsell, J. H. R. and Van Essen, D. C. The connections of the middle temporal visual area (MT) and their relationship to a cortical hierarchy in the macaque monkey. / . Neurosci. 3: 2563-2586, 1983. 20. Mountcasde, V. B., Lynch, J. C, Georgopoulos, A., Sakata, H., and Acuiia, C. Posterior parietal association cortex of the monkey: command functions for operations within extrapersonal space. /. NeurophysioL 38: 871-908, 1975. 21. Rizzolatti, G., Gentilucci, M., Luppino, L., Matelli, M., and Ponzoni-Maggi, S. Neurons related to goal-directed motor acts in inferior area 6 of the macaque monkey. Ejq). Brain Res. 67: 220-224, 1987. 22. Robinson, D. L., Goldberg, M. E., and Stanton, G. B. Parietal association cortex in the primate: Sensory mechanisms and behavioral modulations. /. NeurophysioL 41: 910-932, 1978. 23. Ungerleider, L. G. and Desimone, R. Cortical connections of visual area MT in the macaque. /. Con^. NeuroL 248: 190-222, 1986.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) © 1996 Elsevier Science B.V. All rights reserved.

53

NEURAL MECHANISM OF FIGURE-GROUND SEGREGATION AT OCCLUDING CONTOURS IN MONKEY PRESTRIATE CORTEX R. Baumaim, X. M. Sauvan, and E. Peterhans* Department of Neurology, University Hospital Zurich, Frauenklinikstr. 26, CH-8091 Zurich, Switzerland

1. ABSTRACT An important aspect of figure-ground segregation is the detection of occluding contours and the discrimination of figure and ground at such contours. In this paper we investigate the neural processing and the representation of occluding contours defined by occlusion cues. We trained rhesus monkeys on a visual fixation task that reinforced foveal viewing. During the periods of active visual fixation we recorded the responses of single neurons in areas VI and V2 of the visual cortex. The stimulus conditions that we used mimicked situations of spatial occlusion; usually an opaque, uniform rectangle (or tongue) which appeared to overlay a larger, rectangular grating texture. The neural responses were analyzed with respect to figure-groxmd direction and contrast polarity. Neurons in area VI either failed to respond to such stimuli, or they were sensitive to contrast polarity only; others responded imselectively to all types of contour. By contrast, more than one third of the neurons studied in area V2 were sensitive to the direction of figure and groimd at such contours. The majority of these neurons were not sensitive to the contrast polarity of these contours. Some rare neurons preferred a certain combination of figure-ground direction and contrast polarity. We explain these results in terms of a model that had been developed to explain neural signals of illusory contours, contours that often coincide with occluding contours. In conclusion, the results of the present paper suggest that in monkey visual cortex occluding contours and mechanisms for the segregation of figure and ground at such contours are first represented at the level of area V2.

*This research was supported by SNF 31-31970.91 (Esprit Insight-II 6019) and HFSP RG-31/93. X.S. was supported by SNF 31.31963.91 (Esprit Mucom-II 6615). We thank R. van der Zwan for comments on the manuscript. Please send correspondence to E. Peterhans at the address given above.

54

2. INTRODUCTION Object recognition requires consistent representations of object borders iQdependent of the background structures and viewing conditions. In the retinal image borders can be defined by a number of cues, most typically by discontinuities of luminance, color, texture, and motion, or by binocular disparity (1-3). In this paper we investigate the neural processing of borders defined by occlusion cues (4, 5). Figure 1 shows examples of artificial visual scenes illustrating die problem. This Figure induces the perception of occluding contours and surfaces, Figs. 1A and C a triangle, and Figs. IB and D a peanut shaped object, both of which appear to overlay objects of other shapes in die background. It is the terminations of diese background structures (line-ends, comers) which produce illusory contours at sites of fading, or missing contrast. These contours complete die gaps of die occluding contours and dius facilitate die perception of occluding objects. This mechanism may contribute to the segregation of figure and ground, particularly in cluttered ^ visual scenes where objects overlay one anodier. In diis light, illusory contours contribute to perceptual stability. A similar mechanism may also enhance the segregation of figure and ground in natural visual scenes (see Ref. 6 for further discussion). Figure 1 also shows diat it is die spatial alignment and the direction of the terminating background structures which determine the shape of the occluding contours and the depth order of the associated surfaces. This mechanism is independent of contrast polarity. Figures widi dark (A, B) and light terminations (C, D) induce identical shapes of die occluding objects and die same depth relations as long as they have the same spatial arrangement. We have investigated this process in the visual cortex of the alert monkey during behaviorally induced visual fixation. The responses of single neurons to stimuli akin to diose of single elements of Fig. IB and D were studied in areas VI and V2. The results confirm our previous findings that illusory contours are represented in area V2, but not in area VI (see also Refs. 7 and 8). In addition, diey indicate diat not only illusory contours, but also information about the direction of figure and ground at such contours are represented at this early level of processing. A report on the very first neurons found of this type has been published (9).

3. METHODS 3.1 Animal preparation Rhesus monkeys (Macaca mulatta) were trained on a visual fixation task that reinforced foveal viewing. The fixation target consisted of two vertical lines that

55

A

B

Figure 1. Illusory figures that mimic situations of spatial occlusion. It is the spatial arrangement of the occlusion cues (line-ends, corners) with determines the shape of the illusory contours, and the depth order of the associated surfaces. This mechanism is independent of contrast polarity; both dark (A, B) and light terminations (C, D) induce identical percepts of form and depth. (Adapted from Ref. 4).

were 7 min arc long, 1 min arc wide, and separated by 5 min arc from center to center. The animals could initiate a trial by pulling a lever. After a variable time interval (0.5-5 sec) the orientation of the fixation target was turned by 90 deg and the animal had to release the key within 0.4 sec. Correct responses were rewarded with a small amount of fruit juice or water. When the animals reached a performance rate greater than 90% they were prepared for recording. In successive operations, under general anesthesia, a head-holder and two recording chambers (one for each hemisphere) were mounted to the skull. The chambers were centered on the operculum over the area representing the central 1-6 deg of the lower visual

56

field. For a detailed description of our method of accessing areas VI and V2 see Ref. 10. 3.2 Visual stimiilatioii and recording We characterized the response properties of single neurons using conventional stimuli like bars, edges and square-wave gratings. Subsequendy we used illusorycontour stimuli as shown in Fig. 2B, C (a detailed description of the responses to such stimuli are given in Refs. 7 and 8), and occluding-contour stimuli as shown in Fig. 3A-D. In human observers the occluding-contour stimuli induced the perception of opaque, bright (or dark) rectangles bounded by illusory contours. Typically, we used moving stimuli which were perceived as rectangular tongues sliding back and forth over a grating texture. The width of the grating lines was 1-9 min arc, the line spacing 6-28 min arc. As shown in Fig. 3, the edge of the tongue was presented at the neuron's preferred orientation and the grating texture perpendicular to it. It was oscillated with constant speed and frequency over the neuron's response field (ellipse) which had been determined with a bar or an edge. These stimuli and the fixation target were generated by means of analog and digital circuits and displayed on a specially designed high resolution, flat faced oscilloscope equipped with a fast decaying phosphor (Ferranti A5, peak at 555 nm). By means of a half silvered mirror we added a uniformly illuminated background to the display, such that the final luminance of the background was 10 or 36 cd/m^, and that of the stimuli 20 or 72 cd/m^. Two images, one for each eye, were generated side by side on the oscilloscope screen at a rate of 100 Hz. They were presented to the animal via a stereoscope, for each neuron at its preferred depth. The responses of single neurons were recorded during the periods of active visual fixation. We used glass-coated platinum-iridium microelectrodes prepared according to Wolbarsht et al. (11), but without platinum-black coating. The signals were amplified, fed to a Schmitt-trigger, recorded on instant film in the form of a dot display, and stored by computer for immediate display and off-line analysis.

4. RESULTS In perception, the borders of objects can be perceived independent of the cue by which they are defined in the retinal image. For example, we perceive a triangle when its borders are defined by color or contrast edges, and similarly when these border are defined by occlusion cues (see Fig. 1). This generalization of contours which implies conversion of cues is an important step toward perceptual constancy. Earlier studies from our laboratory on the processing of contour in monkey visual cortex suggested that the process of contour generalization begins in area V2 (7, 8, 12). Those studies showed that neurons of area V2, but not of area VI, signalled

57

Unit 3GD5

27.7

B 4.6

4.0

D 0.0

0.5s (2°) Figure 2. Responses of a "contour neuron" of area V2. (A) shows the responses to a dark bar at optimal orientation, (B) and (C) the responses to illusory-contour stimuli. The ellipses indicate the neuron's response field as determined with a bar stimulus; the cross marks the fixation point of the monkey. In each stimulus condition (A-C) the contour was moved over the response field. The dot-displays show the responses to 24 motion cycles (frequency: 1 Hz), those in the forth sweep in the left, and those in the back sweep in the right half of the display. The responses were recorded in blocks of 8 motion cycles in an interleaved, pseudorandom order. Each dot marks an action potential, the figures represent mean numbers per motion cycle. The spontaneous activity was zero (D). (Reproduced with permission from Ref. 29).

58

Unit 7BD1

15.7

B

16.2

10.0

3.0

2.3

Figure 3. Sensitivity to figure-ground direction at occluding contours. Responses of a neuron of area V2. The line terminations were presented perpendicular to the neuron's preferred orientation, and moved back and forth over the neuron's response field as plotted with a bar stimulus (ellipses). Note that the position of the contour represents the starting point of the forward sweep. The dot displays (A-D) show the responses recorded during 24 motion cycles (frequency: IHz), the figures on the right indicate mean numbers of spikes per motion cycle. The bottom display shows spontaneous activity for a corresponding time interval.

59

the orientation of contours independent of whether these contours were defined by luminance contrast or occlusion cues. The responses of such a *'contour neuron"" of area V2 are reproduced in Fig. 2. This neuron preferred long dark bars with oblique orientations (A), but it also responded to two types of illusory-contour stimuli (B, C). In human observers these stimuli induced the perception of an illusory bar overlaying two bright rectangles in the background (B), and an illusory contour between abutting line-gratuigs (C). The ellipse illustrates the dimensions of the minimum response field as plotted with a dark bar. In each stimulus condition (A-C) the "contours" were moved back and forth over the neuron's response field. The dot displays on the right show the responses in the two directions of stimulus movement. The bottom display (D) shows spontaneous activity. The two illusorycontour stimuli evoked similar, though weaker responses than the solid bar. On the average, the response strength evoked by illusory-contour stimuli was about 60% of that evoked by solid bars or edges (7). Since illusory contours often coincide with occluding contours (see above), we asked ourselves whether neurons of area V2 that are sensitive to the orientation of illusory contours also carry information about the direction of figure and ground at such contours. Thus, we studied the responses of neurons of area V2, and for control also of neurons of area VI, in stimulus conditions that were akin to the single elements of Figs. IB and D. We used pairs of stimuli that had the same figure-ground direction at the contour, but opposite contrast polarity. They are shown in the stimulus insets of Fig. 3 (A-D). The combination of these two pairs of stimuli (A,B and C, D) allowed us to analyze separately the effects of figureground direction and contrast polarity. The types of responses that we predicted from these stimuli are listed in Table 1. Neurons sensitive to figure-ground direction were expected to respond preferentially to one type of contour pair, either (A and B) or (C and D), independent of the contrast polarity at the contour. Neurons sensitive to a certain combination of figure-ground direction and contrast polarity were expected to prefer one type of contour, either (A), (B), (C), or (D). Neurons not sensitive to figure-ground direction were expected to be sensitive to the contrast polarity at the contour (dark/light or light/dark) and thus to prefer contours (A and D) or (B and C), or to be unselective giving similar responses to all four types of contour. This paper is based on the responses of 146 neurons, 46 recorded in area VI, 100 in area V2.

60

Table 1. Responses predicted from occluding-contour stimuli

Neural sensitivity to

Effective contours*

Figure-ground direction

(A and B) or (C and D)

Figure-ground direction and contrast polarity

A or B or C or D

Contrast polarity

(A and D) or (B and C)

Unselective

A and B and C and D

*For an illustration of these contours see Fig. 3 A-D. Note that contour pairs (A and B) and (C and D) have opposite contrast polarity.

4.1 Sensitivity to figure-ground direction Figure 3 shows the responses of a neuron of area V2 that was sensitive to figureground direction. It gave similar responses to the first stimulus pair (A, B) for which the line-terminations indicated an occluding surface to the right of the contour. Much weaker responses were evoked by the second stimulus pair (C, D) with the opposite figure-groimd arrangement. Of course, the four stimuli are highly abstract conditions which reduce the problem of figure-ground direction to a single contour. At this simplified level, human perception can be ambiguous. This is evident from the stimulus insets of Fig. 3. Two percepts are possible, either the opaque tongue covering a grating texture, or a grating with a cut-out notch of the size of the tongue. In the figure the second percept can dominate due to the disturbing effect of the ellipse. However, in the actual stimulus, particularly when it was moving, the first percept dominated. Also, if one looks at a single element of Figs. IB or D the illusory contour persists, though perhaps somewhat weaker than in the complete figure (cf. DISCUSSION). In any case, independent of the actual percept, the information conveyed by the neuronal response is unambiguous - it indicates the local position and the direction of line-terminations. The signals of these neurons can provide the very first, local information necessary for mechanisms of figure-ground segregation from occlusion cues. This interpretation predicts two types of neurons at each orientation preferring opposite figure-ground directions. Figure 4 shows the responses of two neurons that were, relative to their

61 preferred orientation, sensitive to opposite figure-ground directions. For clarity, the preferred orientation of each neuron has been plotted vertical (orientation of the ellipse), and the mean responses are shown in the form of histograms. Besides the responses to occluding-contour stimuli (open bars), Fig. 4 also shows the responses

Unit 6CB1

Unit 6CQ4

1 ^ ±

JL JL

I

I

I

A

B

C

JL

J_

1^

D

A

B

C

D

en 13

0 E

F

G

H

E

F

G

CO

H

Figure 4. Sensitivity to opposite figure-ground directions of a pair of neurons of area V2. The first neuron (6CB1) gave stronger responses to contours having the occluding surface to the right (A, B), the second neuron (6CQ4) to contours having the occluding surface to the left (C, D). This result was independent of contrast polarity; all solid edges evoked similar responses (E-H). For each contour (A-H) the mean responses of 24 motion cycles are plotted in the form of histograms (frequency: 1.5 Hz, unit 6CB1; IHz, unit 6CQ4). The vertical bars indicate standard deviations; the ellipses show the response field of unit 6CQ4 as defined with a bar stimulus.

62

to the corresponding solid edges (filled bars). The first neuron (6CB1) gave stronger responses to contours (A) and (B), i.e. to the left stimulus pair, and weaker responses to contours (C) and (D), i.e. to the right stimulus pair. This result was independent of contrast polarity. Note that the contrast polarity at contour (A) was opposite to that of contour (B), and similarly with contours (C) and (D). By contrast, all solid edges evoked similar responses (E-H). Also, Fig. 4 shows that the response strength to occluding-contour stimuli varied from neuron to neuron - in some this response was stronger (6CQ4), in others (6CB1) it was weaker than the responses to solid edges. 4.2 Quantifieatioii of sensitivity tofigure-grounddirection Of the 146 neurons studied with occluding-contour stimuli, we obtained complete quantitative records for 63 neurons. For these we determined an index of sensitivity to figure-ground direction (Ig) from the mean responses to the four types of contour as shown in Fig. 3 (RA-D)' Is = I ( R A + R B ) - ( R C + R D ) / ( R A + R B + R C + R D ) I

Figure 5 shows the result separately for areas VI and V2. One can see that this index was never greater than 0.2 for neurons of area VI, thus indicating low sensitivity to figure-ground direction (Fig. 5A). In contrast to neurons of area VI, many neurons of area V2 had indices greater than 0.2 (Fig. 5B). During qualitative testing (i.e. by simply listening to the responses) neurons for which the subsequent analysis revealed an index lower than 0.2 were not recognized as being sensitive to figure-ground direction. Furthermore, the histogram of Fig. 5B suggests a bimodal distribution for the indices of neurons of area V2 with a trough at 0.2. Therefore, only neurons with indices greater than 0.2 were called sensitive to figure-ground direction. Neurons with indices equal or lower than 0.2 were called unselective. By this criterion, none of the neurons of area VI for which this index was available (0/18) was sensitive to figure-ground direction, whereas more than 40% of the neurons of area V2 (20/45) showed this property. After pooling quantitative and qualitative results, none of the neurons of area VI (0/46), and 36% of the neurons of area V2 (36/100) were classified as being sensitive to figure-ground direction. 4.3 Sensitivity tofigure-grounddirection and contrast polarity We also found neurons that were selective for a certain combination of figuregroimd direction and contrast polarity. These neurons clearly preferred one type of occluding contour, for example in Fig. 3 either contour (A), (B), (C), or (D). These neurons preferred the same contrast polarity at the occluding contour as they did for solid edges, either light/dark or dark/light. So far, only 4 of the 100 neurons studied in area V2 were of this type. They gave strong responses to one type of occluding-contour stimulus and comparatively weak responses or none to

63

VI 10 -I

N=18

0.4

0.8

V2

15 n

to CD N=45

n rn 0.4

0.8

Index ( L ) Figure 5. Quantification of sensitivity to figure-ground direction. (A) shows the distribution of the sensitivity index (!§) for neurons of area VI, (B) for neurons of area V2. Neurons with indices greater than 0.2 were called sensitive to figureground direction (open bars), neurons with indices equal or lower than 0.2 were called unselective (filled bars). For a definition of the index see text.

the other types. However, more experiments are needed to determine the details of the functional properties of these neurons. 4.4 Effect of stimulus moyement Since we typically used moving stimuli to study sensitivity to figure-ground direction one might argue that the result was affected by stimulus movement. In

64

order to separate the two stimulus parameters, that is figure-ground direction and direction of movement, we studied some neurons with both stationary and moving stimuli. An example of the responses of a neuron studied in both stimulus conditions is shown in Fig. 6. The neuron was recorded in area V2. It gave stronger responses when the occluding surface was to the left of the contour (B) and weaker responses when this surface was to the right of the contour (A). This selectivity was independent of contrast polarity. (The two contours (A and B) evoked 2.8 and 5.8 spikes/sec for the stimulus pair with dark occluding surfaces as shown in Fig. 6 (open bars), and 3.0 and 5.5 spikes/sec for the corresponding stimulus pair with bright occluding surfaces, not shown). We used the stimulus pair with the dark occluding surface for comparing the responses to moving and stationary stimuli. Note that the sweep amplitude in the moving condition was much larger (2 deg) than the fixational eye movements to be expected in our stimulus conditions (cf. DISCUSSION). As shown in Fig. 6, this neuron had the same preferred direction of figure and groimd both in the moving (open bars) and stationary stimulus condition (dotted bars). We also determined the preferred direction of stimulus movement for conventional stimuli for 19 of the 20 neurons studied quantitatively that showed sensitivity to figure-ground direction. This analysis revealed that more than half of these neurons (12/19) were not direction selective. They gave similar responses in both directions of stimulus movement. In the remaining 7 neurons, the preferred figure-groimd direction and the preferred direction of movement were either the same (N=2) or opposite (N=5). (For an illustration of the relationship between the two types of selectivity see Fig. 7). This suggests diat the sensitivity of neurons of area V2 to figure-ground direction is independent of the neurons' sensitivity to motion direction.

5. DISCUSSION The results described in the present paper suggest that mechanisms for the segregation of figure and groimd at occluding contours are implemented relatively early in visual processing. To our knowledge these are the first data to show that single neurons in monkey visual cortex can be sensitive to the direction of lineterminations at occluding contours, and thus indicate the direction of figure and ground at such contours. These neurons were found in area V2, but not in area VI. With very few exceptions, this sensitivity was independent of the contrast polarity at the contours. Furthermore, the preferred figure-ground direction of these neurons was not related to the preferred direction of stimulus movement which suggest that it is the spatial arrangement and the direction of the occlusion cues (line-terminations) which combined to produce the responses of these neurons.

65

Unit6CM1

^ 5 n O

0

'—11 — '

'—11 — '

A

B

Moving

1 T '•• t 1

A

•_

'

1

B

Stationary

Figure 6. Sensitivity to figure-ground direction in moving and stationary stimulus conditions. Responses of a neuron of area V2 that preferred the occluding surface to the left of the contour (B). This selectivity was similar in the moving (open bars) and in the stationary stimulus condition (dotted bars). In both conditions the neuron gave stronger responses to contour (B) than to contour (A). The contours were either moved back and forth over the neuron's response field (ellipse) at a frequency of 1.5 Hz, or were kept stationary in the center of this field during a corresponding time interval. The bars represent mean responses recorded during 24 motion cycles or during a corresponding time interval. Vertical bars show standard deviations.

66

5.1 Interpretation of the neuronal responses The results of the present paper suggest that contours defined by occlusion cues, and the depth order at such contours are first encoded in area V2. In perception, such contours are often perceived as illusory contours (13, for a review see Ref. 5), and it has been shown that it needs at least two such cues (line-terminations) in order to perceive an illusory contour (14). Increasing the number of lines strengthens the percept. A similar effect has been found in neurons of area V2 that were sensitive to illusory contours (7). A single line-end usually failed to evoke a response. Adding more lines strengthened the response. A stimulus of 8-13 lines, as shown in Fig. 2C, usually evoked maximum response. In the present study we were careful always to present several line-ends within the response field (as determined with conventional stimuli) in area V2, but also in area VI where the receptive fields can be very small. Thus, the occluding-contour stimuli which we used in these experiments were all adequate to induce the perception of illusory contours. Of course, the four stimulus conditions that we used (see inset of Fig. 3) were highly abstract, designed to reduce the problem of figure-ground direction to a single, straight contour suitable for a study of single neurons. The perception of these stimuli was sometimes ambiguous. Human observers could perceive a notch cut out of a grating pattern or an opaque tongue overlaying a grating texture. These two percepts could alternate, especially in the stationary stimulus condition. However, the neuronal signals as such were unambiguous - they carried specific

B

Figure 7. Possible relationships between figure-ground direction and preferred direction of movement (arrow) of cortical neurons. In (A) the two directions are the same, that is from right to left, in (B) they are opposite.

67

local information reflecting the spatial arrangement and the direction of lineterminations at a particular contour. This neuronal information may be one of the very first to be used by mechanisms of figure-ground segregation from occlusion cues. In neurons sensitive to figure-ground direction we found no relationship between stimulus movement and the preferred figure-ground direction. First, the majority of these neurons were only weakly or not selective for motion direction. This result agrees with an earlier study from this laboratory which showed that direction selectivity was not a critical stimulus parameter for neurons of area V2 (10). Second, in neurons that did show direction selectivity, the preferred direction for stimulus movement was more often opposite to the preferred figure-ground direction than the same. Similarly, it seems unlikely that fixational eye movements accounted for these results. Motter and Poggio (15) have measured the scatter of eye positions ia rhesus monkeys performing a binocular fixation task which was virtually identical to the task of our monkeys. The result revealed a standard deviation of 6-8 min arc for the horizontal and 7-13 min arc for the vertical components. Their study showed that the size of the fixation target and the orientation discrimination involved in this task reinforced foveal viewiag. A similar result has been obtained by Snodderly and Kurtz (16). The scatter of the responses as shown in the dot displays of Figs. 2 and 3 allows a rough estimate of the fixational eye movements in our stimulus conditions. One can see that they were, on the average, much smaller than the width of the response field, and they appeared to be randomly scattered (for further discussion see Ref. 8 page 1753). Because fixational eye movements occur both in the moving and in the stationary stimulus conditions, and because they vary randomly, they should not affect the comparison between the results recorded in these two conditions. 5.2 Hypothetical mechanism producing sensitivity tofigm*e-gromiddirection We explain the sensitivity of cortical neurons to figure-ground direction in terms of a model that has been proposed previously for neural mechanisms of illusory contours (for a review see Ref. 6). The scheme of the model is shown in Fig. 8. It assumes two input pathways for neurons of area V2. The first input (open symbols) produces sensitivity to occluding contours andfigure-groxmddirection, the second (filled symbol) produces sensitivity to contrast edges. The first pathway uses the signals of end-stopped cells with asymmetrical receptive fields. The excitatory parts of their receptive fields are thought to be of the complex type which implies that they respond similarly well to dark and to light line-terminations. The preferred orientation of the end-stopped cells is assumed to ht perpendicular to the occluding contour and their alignment parallel to it. For simplicity, only four end-stopped cells are shown, but they are thought to cover the receptive field densely. In the example of Fig. 8 completely asymmetrical fields are assumed (the hatched discs

68

A

B

Figure 8. Hypothetical mechanism producing sensitivity to illusory contours and figure-ground direction. We assume a dual input for neurons of area V2. The first input (open symbols) produces the signals of illusory contours and figure-ground direction. It groups the signals of end-stopped cells with asymmetrical receptive fields (horizontal ellipses; the hatched discs represent inhibitory end-zones). These neurons prefer orientations perpendicular to tiie illusory contour and respond to line-terminations as shown in A-C. The signals of distant pairs of such cells are combined by multiplications (X) and the summed output (2) produces the signal for illusory contours. Thus, it is the field asymmetry of the end-stopped cells which conveys information about the direction of occlusion cues to these neurons. The second input (filled symbol) adds the signals of neurons with simple or complex type receptive fields (vertical ellipse), and thus the signals for solid lines or edges. This model predicts three types of neurons in area V2: Neurons not sensitive to figure-ground direction (A), and neurons sensitive to either one (B) or the other figure-ground direction (C). It is assumed that all three types of neuron also receive the second input and thus respond to bars or edges.

69

indicate inhibitory end-zones). However, in visual cortex, end-stopped cells with all degrees of field asymmetry can be found (see below). The signals of distant pairs are combined by a multiplication (X) because it needs at least two line ends to perceive such a contour (14). Furthermore, a single line-end usually also failed to activate cortical neurons (7). The sum of the output of such pairs (S) produces the responses of contour neurons of area V2. As discussed above, this summation was also found in the responses of cortical neurons (7). The second pathway (filled symbol) adds to flie first. It uses the signals of neurons with simple or complex type receptive fields (vertical ellipse) which respond to solid bars or edges. 5.3 Simulatioii of the model The model outlined in Fig. 8 has been simulated keeping at all stages as closely as possible to physiology (Refs. 17-19). The simulation involved convolutions of the image with different types of filters which simulated the functions of simple, complex and end-stopped cells. The result paralleled perception: It reproduced the perceived illusory contours and the figure-ground direction at such contours. These results were obtained with binary and with gray-valued images (for examples see Refs. 18 and 19). Alternative models for mechanisms of illusory contours and the segregation of figure and ground at such contours have been proposed (Refs. 2024). The main differences between these models and ours are twofold. First, a basic step of our model is the detection of the position and direction of terminations (occlusion cues) which are explicitly encoded by filters simulatiag the fimction of end-stopped cells with asymmetrical receptive fields. Second, our model implies a one-directional, feed forward process including sets of different filters at different stages that are convolved with the image; no feedback control is required (for a more detailed discussion of the various model see Refs. 19, 20 and 25). Considering its simplicity, the model produces stable results, also in gray-valued images. More complex neuronal interactions such as dynamic synaptic changes and the temporal coherence of groups of neurons are not required (26-28). 5.4 End-stopped cells We have studied the receptive fields of end-stopped cells in the light of this model (29, 30). About half of the neurons recorded in areas VI and V2 responded to lines terminating in the receptive field when this line covered one half of the field, and only half of that response or less when it covered the other half. Such responses indicated asymmetrical receptive fields with a strong inhibitory zone at one end of the field and a weak one or none at the other end. The remainder of the end-stopped cells gave similar responses to line terminations covering either one or the other half of the receptive field. These responses indicated symmetrical receptive fields with inhibitory zones of about equal strength at both ends. The results of these experiments suggest that end-stopped cells do respond to lineterminations, even those with symmetrical receptive fields. However, only end-

70 stopped cells with asymmetrical fields carry information about the directions of such terminations. Little is known about the role of end-stopped cells in visual processing. Hubel and Wiesel who first described these neurons in cat and monkey visual cortex (31, 32) proposed that those with symmetrical receptive fields could be involved in curvature detection, and models simulating such cells have been proposed (33). Indeed, it has been found recently that end-stopped ceils in cat visual cortex can be sensitive to curvature (34, 35). However, these neurons responded about equally well to long lines of optimal curvature and to short straight lines of optimal length. Thus, the signals of these neurons seem to be ambiguous; they are not exclusively related to curvature. The function of end-stopped cells with asymmetrical fields is less clear. Hubel and Wiesel showed that they respond to tongues and to comers of various angles (31, 32). The studies from our laboratory suggest that these neurons are sensitive to terminations (line-ends and comers) and thus may be involved in the detection of occlusion cues (29, 30). Thus, in the simulation of the mechanism producing illusory contours and sensitivity to figure-ground direction, only endstopped cells with asymmetrical fields have been invoked (18, 19).

5.5 Psychophysics In a study of the phenomena of the da Vinci stereopsis Nakayama & Shimojo (36) discovered that unpaired, monocular cues can induce the perception of illusory contours. They showed that the figure-ground direction at such contours is determined by the eye of origin of the unpaired stimulus element. Since the signals of the two eyes converge early in visual processing (mostly in area V1), they concluded that neural mechanisms for the perception of illusory contours and the detection of figure and ground at such contours should be represented early in visual processing; in area V1 or its immediate targets of projection. Our findings fit with this hypothesis and show that in monkey visual cortex this representation begins in area V2. Furthermore, it has been shown that motion perception at occluding contours, as for example in the barber pool illusion, depends on the depth order invoked by occlusion (37), and that interaction between depth from occlusion and depth from binocular disparity can produce conflicting percepts (38, 39). This also suggests that mechanisms for figure-ground segregation from occlusion cues are implemented just as early as mechanisms contributing to the perception of depth from motion or stereoscopic cues. In synergy, these mechanisms may enhance the perception of occluding contours and stabilize the perceived depth order of the associated surfaces.

71

6. CONCLUSIONS The present paper proposes a neural mechanism for the segregation of figure and ground at occluding contours. Evidence for such a mechanism is provided by the neurophysiological data showing that neurons in area V2 of the monkey visual cortex can be sensitive to illusory contours and to the depth order associated with such contours. We explain the responses of these neurons in terms of a model that uses a feed forward mechanism based on the signals of end-stopped cells sensitive to the position and the direction of occlusion cues (line-ends, comers). The validity of this model has been tested by simulation and comparison of the results with perception. These results fit with evidence from psychophysics which suggests that grouping mechanisms for figure-ground segregation from occlusion cues are implemented early in the visual pathway, and that these mechanisms may coexist with mechanisms contributing to the perception of depth from motion or stereoscopic cues.

REFERENCES L P . Cavanagh, In: Neural Mechanisms of Visual Perception, D. M. K. Lam and C. D. Gilbert (eds). Portfolio Publishing Company, The Woodlands, TX, (1989) 261-279. 2. B. Julesz, Foundations of Cyclopean Perception, University of Chicago Press, Chicago, (1971). 3. K. Nakayama, S. Shimojo and G. H. Silverman, Perception 18 (1989) 55-68. 4. G. Kanizsa, Organization in Vision, Praeger, New York, (1979). 5. S. Petry and G. L. Meyer, The perception of illusory contours. Springer, Berlin, (1987). 6. E. Peterhans and R. von der Heydt, Trends Neurosci. 14 (1991) 112-119. 7. R. von der Heydt and E. Peterhaos, J. Neurosci. 9 (1989) 1731-1748. 8. E. Peterhans and R. von der Heydt, J. Neurosci. 9 (1989) 1749-1763. 9. R. von der Heydt, F. Heitger and E. Peterhans, Biomedical Research 14 (Suppl. 4) (1993) 1-6. 10. E. Peterhans and R. von der Heydt, Eur. J. Neurosci. 5 (1993) 509-524. 11. M. L. Wolbarsht, J. E. F. MacNichol and H. G. Wagner, Science 132 (1960) 1309-1310. 12. R. von der Heydt, E. Peterhans and G. Baumgartner, Science 224 (1984) 12601262. 13. S. Coren, Psychol. Rev. 79 (1972) 359-367. 14. F. Schumann, Zeitschrift Sk Psychologic 23 (1900) 1-32. 15. B. C. Motter and G. F. Poggio, Exp. Brain Res. 54 (1984) 304-314. 16. D. M. Snodderly and D. Kurtz, Vision Res. 25 (1985) 83-98.

72

17. F. Heitger, L. Rosenthaler, R. von der Heydt, E. Peterhans and O. Ktibler, Vision Res. 32 (1992) 963-981. 18. F. Heitger and R. von der Heydt, In: Proc. 4th Int. Conf. Computer Vision, Berlin, Germany. IEEE Computer Society Press (1993) 32-40. 19. F. Heitger, R. von der Heydt, E. Peterhans, L. Rosenthaler and O. Kxibler, Image & Vision Computing (1996) in press. 20. L.H. Finkel and G.M. Edehnan, J. Neurosci. 9 (1989) 3188-3208. 21. J. Skrzypek and B. Ringer, In: Proc. 3rd Int. Conf. Computer Vision Champaign, IL. IEEE Computer Society Press (1992) 681-683. 22. S. Grossberg and E. MingoUa, Percept. Psychophys. 38 (1985) 141-171. 23. S. Grossberg, Percept. Psychophys. 41 (1987) 117-158. 24. P. Sajda and L.F. Finkel, In: Proc. 3rd Int. Conf. Computer Vision Champaign, IL. IEEE Computer Society Press (1992) 688-691. 25. L. Finkel and P. Sajda, Neural Computation 4 (1992) 901-921. 26. O. Spoms, G. Tononi and G. M. Edelman, Proceedings of the National Academy of Sciences of the United States of America 88 (1991) 129-133. 27. R. Eckhom, R. Bauer, W. Jordan, M. Brosch, W. Kruse, M. Munk and J. H. Reitboeck, Biol. Cybem. 60 (1988) 121-130. 28. C. M. Gray, P. Konig, A. K. Engel, W. Singer, Nature 338 (1989) 334-337. 29. E. Peterhans and R. von der Heydt, In: Representations of Vision. Trends and Tacit Assumptions, A. Gorea, Y. Fregnac, Z. Kapoulis and J. Findlay (eds), Cambridge University Press, Cambridge, (1991) 111-124. 30. E. Peterhans and R. von der Heydt, Soc. Neurosci. Abstract 16 (1990) 293. 31. D. H. Hubel and T. N. Wiesel, J. Neurophysiol. 28 (1965) 229-289. 32. D. H. Hubel and T. N. Wiesel, J. Neurophysiol. 195 (1968) 215-243. 33. A. Dobbins, S. W. Zucker and M. S. Cynader, Vision Res. 29 (1989) 13711387. 34. A. Dobbins, S. W. Zucker, and M. S. Cynader, Nature 329 (1987) 438-441. 35. M. Versavel, G. A. Orban and L. Lagae, Vision Res. 30 (1990) 235-248. 36. K. Nakayama and S. Shimojo, Vision Res. 30 (1990) 1811-1825. 37. S. Shimojo, G.H. Silverman and K. Nakayama, Vision Res. 29 (1989) 619626. 38. J. P. Harris and R. L. Gregory, Perception 2 (1973) 235-247. 39. V. S. Ramachandran and P. Cavanagh, Nature 317 (1985) 527-530.

Brain Theofy - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) O 1996 Elsevier Science B.V. All rights reserved.

75

Microarchitecture of Neocortical Columns Rodney J. Douglas and Misha A. Mahowald and Kevan A.C. Martin* MRC Anatomical Neuropharmacology Unit, Mansfield Road, Oxford 0X1 3TH, United Kingdom The neocortical system, with its exquisite variety of function, is built on a series of column-like structures that aggregate to form slabs and pinwheel patterns. The basic unit of the column is a vertical chain of neurons where later stages of the chain reconnect with earlier stages to form a series of recurrent circuits. We present a simple electrical circuit analogy to represent this recurrent chain and show how stability in the circuit can be achieved through the known biophysical mechanisms of the neuron and synapses. The possible role of recurrent excitation and inhibition is then explored in the context of extracting a signal embedded in noise. The example demonstrates how the recurrent circuits of the neocortex, with neurons connecting on a nearest neighbour basis, provide a means of representing the signal in a relatively noise-free neural code and of allowing the restored signal to scale with the magnitude of the input from the periphery.

1. MAPS, AREAS AND COLUMNS A brief history of neurophysiological research on the neocortex would reveal three interrelated strands that dominated the research over many decades. The first strand is the work that established the existence of topographic maps of the sensory and motor world on the surface of the neocortex. The best known example is the topographic map described for primate area 17 by Daniel and Whitteridge [15]. From data derived from electrophysiological mapping of area 17 they were able to develop a simple mathematical model that predicted the both the form of the representation of the visual field upon the striate visual cortex of the monkey and the unfolded shape of area 17. Their map revealed that neighbouring regions in visual space were represented in neighbouring regions of the visual cortex and this principle remains true for all sensory and motor maps in the cortex. Through the concept of magnification factor, i.e. the factor that relates the surface of the cortex devoted to a unit size of the sensory space, they were able to suggest a direct relationship between visual acuity and the amount of cortex devoted to the presentation of the fovea. This relationship was a necessary precursor of the notion of a cortical 'hypercolumn' (see third strand below). *We thank Bashir Ahmed, John Anderson and Charmaine Nelson for their contributions to the work described in this chapter. We acknowledge the financial support of The Royal Society, the Medical Research Council, the US Office of Naval Research, the EC, and the Wellcome Trust.

76

The second dominant strand is the existence of multiple cortical areas devoted to a single sensory domain, such as vision or audition. These areas began to be mapped physiologically in some detail in the early part of this century [12,1]. Many of these areas were originally defined by the fact that they have a complete or partial topographic map of the sensory surface. In the visual system, for example, Cowey [13] was able to demonstrate the existence of a topographic map in area 18, and showed that the border of area 17 and 18 followed the same principle of nearest neighbour mapping. Obviously to achieve this, the visual field representation was mirrored along the border between the two areas. All sensorimotor systems have multiple representations in cortex. The size of the individual corlK al areas varies as does the grain of the maps in the different areas. The largest single

thalamus Q

^

Figure 1. Schematic of some of the elements of the layer 4 circuits of neocortex. The inhibitory neurons (small basket cells) are indicated in shaded profiles, the excitatory neurons in open profiles. The percentages refer to the proportion of synapses formed between the various elements and the spiny stellate neurons. Inhibitory and excitatory percentages calculated separately.

[2]. These connections are summarized in Fig. 1. 8. M I C R O C I R C U I T S OF L A Y E R 4 A microcircuit could now be assembled from these elements and their interconnections. This circuit is similar in concept to the 'elementary unit' of Lorente de No [42], which contained all the necessary elements in cortex for transmitting impulses from the afferent fibre to the efferent axon. Although only a subcircuit is being considered here, the same principle applies: we have identified the chain of neurons that are required to excite and inhibit the spiny stellate cells within a single column. From our electrophysiological work in the cat we know that the thalamic afferents excite monosynaptically the spiny stellate cells, the layer 6 pyramidal cells and the small layer 4 basket cells [50,51]. Our anatomical work showed that the layer 6 pyramids and the spiny stellates form excitatory synapses

85 with other spiny stellate cells, with layer 6 pyramidal cells, and with small basket cells. Translated into function, this elementary subcircuit has all the components required for significant recurrent excitation and inhibition. This intrinsic organization of the column is intriguing. It suggests that the majority of excitatory and inhibitory synapses on any neuron in the column are provided by neurons whose functional properties are very similar to those of the target neuron, i.e. they will be excited from the same part of visual space, have similar orientation selectivity and similar ocular dominance. Each neuron receives between 5-10,000 excitatory synapses [4]. On average, each excitatory synapse is provided by a different neuron [49,2]. For the inhibitory synapses, more than one, but no more than 10 on average are provided by the same neuron [39]. Thus, if the majority of the synapses formed with any one neuron are provided by neurons in the same column, a stimulus that activates any one neuron in a column must activate most of the neurons, both inhibitory and excitatory, in that column. This prediction is counter to general notions of how inhibition must work. It seems to make no sense to connect together inhibitory and excitatory neurons that are activated by the same stimulus since they will be competing against each other. Nonetheless, both the anatomy and the physiology provide no easy alternative conclusion. For example, the columnar model indicates that neurons of similar specificity are grouped together, regardless of their morphological type. We cannot suppose that the clutch cells are differentially active, at least with the set of stimulus parameters that optimally activate a given column. In confirmation, intracellular marking experiments [23,50] show that the receptive field properties of the smooth inhibitory neurons are indistinguishable from their neighbouring spiny neurons. The light and electron microscopic analyses of these same smooth neurons show that they have a dense meshwork of local axonal collaterals and that they form synapses with somata and proximal dendrites of neighbouring spiny and smooth neurons [39]. Similar considerations apply to the spiny neurons; they too excite their neighbours through local collateral arborizations. Thus, both the inhibitory and the excitatory neurons would be activated by similar stimuli. A review of the receptive field structure of the layer 4 neurons in the cat's visual cortex, however, indicates that such a relationship between inhibition and excitation must exist. The vast majority of these neurons have 'simple' receptive fields and a cardinal characteristic of simple receptive fields is that they have separate subfields that are activated by stationary flashed ON or OFF stimuli of the same orientation [30]. As with centresurround receptive fields of the retina and thalamus, the ON and OFF subfields of simple cells are mutually inhibitory. For this structure to exist, the inhibitory and excitatory elements must have the same selectivity of orientation and eye preference, i.e. they have to lie within the same column. 9. A N A L Y T I C S O F R E C U R R E N T C I R C U I T S The dynamics of such a cortical network and the usefulness of its functional organization is poorly understood. Nevertheless, it seems important to explore the basic principles of organization and function of this portion of the columnar circuit, particularly because the same organization seems to be replicated in other layers of cortex. Our approach has been to take the detailed biological results described above and use them in construct a

86 simple circuit that could be used to analyse two fundamental issues that arise from the view of the column presented by the biology. Firstly, how can a population of recurrently connected excitatory neurons be prevented from going into a catastrophic positive feedback instability? Secondly, how are the inhibitory neurons employed within this recurrent circuit? These two aspects are drawn together in the simplified circuit illustrated in Fig. 2a. The circuit consists of a population of identical spiny stellate neurons that receive the same thalamic input and are connected to each other with the same synaptic strength. They also excite a population of inhibitory interneurons that in turn inhibit the spiny stellate neurons. Since all the outputs of all the spiny stellates are identical, the network can be reduced to the circuit shown (Fig. 2a). In this circuit, the thalamic synapses provide the input current Un. The spiny stellates form synapses with each other and provide a recurrent current Irec Both /,„ and Irec are inward currents that depolarize the neuron. An inhibitory current linh^ is an outward current, which is provided by the small basket cell synapses. Every cortical neuron has a number of active and passive conductances, some of which are intrinsic properties of the neuron itself others of which are extrinsic and arise from the inhibitory and excitatory synapses. These conductances and the synaptic currents operating within the simplified circuit shown in Fig. 2a can be expressed in a simple equivalent electric circuit (Fig. 2b). In this abstraction, the intrinsic conductances of the neuron are collected together in a single input conductance, G. The action potential discharge of the neurons is proportional to the net excitatory current delivered at the soma/axon hillock. The rate of firing (F) is effectively equal to the voltage Ig/G. Conservation of current requires that the current entering the neuron is exactly equalled by the current leaving the neuron via the passive membrane conductances and, more importantly, the spike conductances, which are an order of magnitude larger than the passive membrane conductances. Thus the spikes themselves are a significant current sink in the circuit. The outward currents, which act to decrease excitation within the circuit, include the voltage and ligand gated potassium conductances, the spike conductances, and the GABA-mediated synaptic conductances which give rise to inhibitory current {linh)In the circuit, the synaptic inward (excitatory) current to a given neuron is the sum of the thalamic input current (tin) and the recurrent excitatory current (Irec)- Each neuron contributes a fraction of the recurrent current that each other spiny stellate receives (Fig. 2b). Because the spiny stellates are all identical, the feedback current to each neuron is proportional to its own output, Irec = ocF. a is an effective conductance, which we call a network conductance. Since the excitatory feedback is a positive feedback, the excitatory network conductance is negative. Conversely, the inhibitory neurons provide an outward feedback current Unh that contributes to a positive network conductance pF. The effective conductance of the neuron (Gefj) is the sum of G + /? — a. Since the output of all the neurons is proportional to the thalamic synaptic current (lin) the discharge rate ( F ) of any neuron is given by Iin/{Geff). Therefore as Geff decreases, the rate of firing of the neurons increases, and as Geff increases, the firing rate decreases. The recurrent circuit remains stable and gives an output that is proportional to lin, provided that the excitatory network conductance (a) does not exceed the sum oi G -\- (3, i.e. the feedback loop gain does not exceed one. Under these conditions the output of the network will

87 relax to zero if there is no thalamic excitatory current. The thalamic synaptic current is amplified by the recurrent excitatory network with a gain that is expressed as Iin-\'IrtdUn^ which can be alternatively expressed as GH-/?/Ge//This gain can be much greater than one and as the value of the excitatory network conductance (a) approaches G + /?, the output is largely due to the current delivered by the spiny stellate network of excitatory synapses (/rec) rather than the thalamic synapses. Thus, this circuit encapsulates and solves analytically the issue raised 45 years ago by Lorent do No: how does the activity of the cortical column influence the impulses entering the column from afferent systems like the thalamus? In this modern formulation, the output of the spiny stellates is always proportional to the thalamic excitation, but the magnitude of the eflFect of the thalamic synapses on the spiny stellates, and hence the columnar circuit, depends on the gain of the cortical network at that point in time, i.e. the factor by which the thalamic input is amplified. This gain factor is affected by the activity existing in the network. The gain is highest when all the neurons in the column are above threshold and its gain is zero when all the neurons are below. Many pre- and postsynaptic factors determine the state of activity of the network. Presynaptically, amongst the many diflFerent factors that need to be considered is the issue of the synaptic efficacy. With repeated stimulation a synapse may potentiate or depress. This process is also dependent on the rate the synapse is stimulated by action potentials. Postsynaptically, issues of receptor saturation, the concentration of ions in small compartments like the spine, and the processes of adaptation will all have an affect on the gain of the circuit. The adaptive processes may be especially significant. The probability that a given spiny stellate cell will produce an action potential will depend on when it last produced an action potential. The action potential discharge of the spiny stellate neurons adapts rapidly and this adaptation is due largely to a calcium-dependent potassium current that has a time constant of about 20ms. Thus, the production of just one action potential by a spiny stellate will affect its response to the next volley from the thalamic synapses. The number of active synapses on the spiny stellate will also have a considerable effect on the input conductance of the neuron [5]. These numerous factors are changing dynamically and their cumulative effects need to be assessed through more detailed models than that presented here. 10. WHAT RECURRENCE IS GOOD FOR: A NEW ORIENTATION The possible role of the columnar recurrent circuit can now be considered in the context of the attribute of orientation selectivity in the visual cortex. The property of orientation selectivity has been studied in much detail at the level of the receptive fields of single cells and in the geometric arrangement of columns described above. Orientation selectivity of single neurons is quite robust in the face of changes in spatial and temporal frequency and in stimulus contrast of the stimuli. Bars and gratings give similar tuning curves and the orientation selectivity of binocular neurons is the same tested through either eye. It is clear that these properties do not reside in the physiological properties of the thalamic afferents, nevertheless the geometry of the thalamic afferent synapses are thought to be a necessary condition in setting up orientation selectivity [30,6]. The issue of what the role of the intracortical circuitry is in this system is rather contentious. One view is that of Hubel and Wiesel [30], which is that the intracortical circuitry does not

a.

'/TO

£ > - iinh ^ < ^

iin

current gain = G/Geff Geff = G + p - a

Figure 2. Reduced spiny stellate microcircuit for layer 4. a. Spiny stellate neurons form synapses with thalamic afFerents and other spiny stellates and with inhibitory basket cells (shown in black). The excitatory synapses provide an inward current, Iin from thalamic afFerents and Irec for recurrent spiny neurons. The basket cells provide an outward current Iinh. Ig is the current flowing across total conductance of the spiny stellate and the output is given by the frequency of discharge F. b. Equivalent electrical circuit. The spiny stellate net conductance is G. Currents as in a. a is the network conductance of the excitatory portion of the circuit and /3 is the network conductance of the inhibitory portion of the circuit.

89 contribute to the basic receptive structure of simple and complex cells. A contrary view is that the thalamic afFerents provide a non-oriented or weakly oriented excitation that is shaped by inhibitory neurons in the cortex [62,63,25,6]. In this view the inhibitory neurons provide a powerful 'cross-orientation' inhibition that is the critical functional component producing orientation selectivity. We have previously reviewed the evidence for both mechanisms and will not review our conclusion that neither of these extremes gives a coherent account of the cortical mechanisms of orientation selectivity [49]. Instead, we start from an acceptance of the existence of recurrently connected columnar circuits and attempt to understand the manner in which these circuits might interact with the thalamic input to produce orientation selectivity in columns in cat visual cortex. Our tool for exploring these interactions is a simple model of layer 4. In this model (Fig. 3), 40 spiny stellate cells were connected together in a ring: these could be considered to be components of an orientation 'pin-wheel'. All the spiny stellate cells received monosynaptic excitation from a group of thalamic afFerents. The receptive fields of the group of thalamic afferents forming synapses with any single spiny stellate neuron were roughly arranged along an axis in visual space (Fig. 3b). The preferred axis of each array of thalamic neurons shifted in an orderly fashion so that the full 180 degrees of the orientation domain was spread across the 40 neurons. The intracortical connections of the spiny stellates were arranged so that nearest neighbours had the strongest connections with each other and more distant neurons were weakly interconnected. These connections were distributed according to a simple gaussian function (Fig. 3a). The spiny stellates were recurrently connected to a pool of inhibitory neurons, i.e. they provided a convergent excitatory input to the inhibitory neuron pool, which provided a divergent and equal strength inhibitory connection to all the spiny stellates. For simplicity this pool was considered as a single neuron (Fig. 3b, grey neuron). This provided for an interesting analysis of the role of intracortical inhibition in orientation specificity. Since we were not studying the dynamics of the circuit, we did not provide for a feedfoward inhibitory pathway driven by the thalamic afFerents. The orientation tuning of the population of spiny stellate neurons was tested under various conditions of connectivity. The 'recordings' are the results that would be obtained if the net activity of the whole ring of spiny stellates could be seen simultaneously as they were being stimulated with one orientation. This recording is in effect a one-dimensional optical recording of the voltage of the array of 40 neurons. In the first condition, the spiny stellate ring was connected only to the geniculate afFerents and the afferents were stimulated with a weak stimulus at one orientation. The resultant activity profile showed that the orientation tuning of the array was very broad and that the signal-to-noise ratio was poor (Fig 3c, dotted line). This is what would be expected from the 'jitter' in the thalamic afferent connectivity. A very different profile was obtained when the intracortical circuitry was engaged (Fig. 3c solid curve). Here the same weak, noisy stimulus gave a well-tuned and robust response. The explanation of this result derives directly from the analysis of the recurrent circuitry of the column (Fig. 2). _ The process is as follows: the oriented stimulus activates all the thalamic afferents. Those converging on the cells with a receptive field biased along the principle axis of the stimulus will be slightly more excited than those tuned to other orientations. The

90 neurons reaching threshold will produce action potentials and excite their neighbouring spiny stellate cells, which in turn will excite the inhibitory neuron pool. The inhibitory neuron pool, because it connects to all the spiny stellates, will apply the same inhibition to all neurons ((Fig. 3c; inhibition 'strength' i = O^t = oo). Weakly driven spiny stellates will be completely inhibited, but more strongly activated spiny stellates will continue to fire and provide positive feedback to their neighbours. Neurons that are non-optimally activated will become more inhibited and fall silent, while the positive excitatory feedback between the optimally activated neurons will amplify the weak and noisy thalamic afferent signal. The result is a relatively noise-free and robust signal. The mechanism of action of the inhibitory neuron in this process is very interesting. It acts in at least two modes, depending on the state of the network. Initially, it acts as a thresholding device to extract the best estimate of the noisy input signal. As the network converges to the optimal solution, the inhibitory neuron pool will be strongly activated and will therefore be orientation tuned. In the final state, the inhibition is proportional to the degree of excitation of the active population of spiny stellates. This proportional inhibition stabilizes the co-operative excitation established within the ring. The neurons in the model circuit act co-operatively [27,64,18] to vote on their best decision as to the orientation of the stimulus. Although this co-operative action is in some senses a democratic one, it is not the democracy of the ballot box, where each neuron makes its own independent decision before adding its individual secret vote to the box. Instead the voting is done on the town hall model, where a show of hands decides the issue. Here each member is subject to the influence of its fellow's vote. A member (in this case a neuron) intending to vote differently from their immediate neighbours will be influenced by neighbours to change their vote to agree with those of its neighbours. This peer pressure is not the only factor. Unlike the town hall, in this cortical model there is active suppression of members whose local support is small. Nevertheless, as in most democracies, the winners take all.

11. C O L L E C T I V E M E M O R Y A N D M O D E L S It is important to note that the connectivity of the model circuit predisposes it to behave in the selective way described. It acts as a correlation detector for a predetermined set of patterns, amplifies the correlated signal and suppresses the noisy uncorrected signal. Thus, even before the weight of the synapses is considered, the 'weight' of the specific connections is having a powerful influence on the result. This embedding of an expectation of the nature of the stimulus in the hardware of the neocortex is not too long a long march from Craik's view that the brain constructs of a working model of reality [14]. Thus, the principle of organization and function of the cortical columnar systems outlined here could apply equally well to most of the other processes we know about in the cortex, whether they be sensory or motor, hardwired or plastic. The same architecture could be used to generate coherent action to take the noisy and ambiguous individual signals arising from the sense organs, shape it into some coherent form according to previous experience, and generate an appropriate response.

91

a.

proximity

current

inhibition t=oo A 5

'"-'

NvV-v

t=0

45deg

cell number

Figure 3. Reduced model of orientation map. a gives the distribution of excitatory connections of a given spiny stellate neuron, b. 'Ring' of 40 spiny stellate cells interconnected according to distribution given in a. Shaded symbol in centre is an inhibitory neurons to which all spiny stellates are recurrently connected. Boxes indicated topographical distribution of receptive fields of thalamic afferents connecting to spiny stellates indicated. All spiny stellates were 'stimulated' with bar indicated by shaded rectangle, c. Activity profile of 40 spiny stellate neurons when connected only to thalamic afferents (dotted line) and with spiny stellate intereconnections engaged (solid line). Magnitude of inhibition shown by horizontal lines at time t = 0 and in steady state ^ = oo .

92 REFERENCES 1. 2.

3.

4.

5.

6. 7.

8. 9. 10.

11. 12. 13. 14. 15. 16. 17.

18.

E.D. Adrian. Afferent discharges to the cerebral cortex from peripheral sense organs. J. Physiol (London), 100:159-191, 1941. B. Ahmed, J.C. Anderson, R.J. Douglas, K.A.C. Martin, and C. Nelson. Polyneuronal innervation of spiny stellate neurons in cat visual cortex. J, Comp. Neurol, 341:39-49, 1994. Y. Amir, M. Harel, and A. Grinvald. Cortical hierachy reflected in the organization of intrinsic connections in macaque monkey visual cortex. J. Comp. Neurol, 334:19-46, 1993. J.C. Anderson, R.J. Douglas, K.A.C Martin, C. Nelson, and D. Whitteridge. Synaptic output of physiologically identified spiny neurons in cat visual cortex. J. Comp. Neurol, 341:16-24, 1994. O. Bernander, R.J. Douglas, K.A.C. Martin, and C. Koch. Synaptic background activity influences spatiotemporal integration in single pyramidal cells. Proc. Natl Acad. ScL USA, 88:11569-11573, 1991. P.O. Bishop, J.S. Coombs, and Henry G.H. Receptive fields of simple cells in the cat striate cortex. J. Physiol (London), 231:31-60, 1973. G.G. Blasdel, D.S. Lund, and D. Fitzpatrick. Intrinsic connections of macaque striate cortex; axonal projections of neurons outside lamina 4c. J. Neurosci., 5:3350-3369, 1985. G.G. Blasdel and G. Salama. Voltage sensitive dyes reveal a modular organization in the monkey striate cortex. Nature, 218:438-441, 1986. J. Bolz and C D . Gilbert. Generation of end-inhibition in the visual cortex via interlaminar connections. Nature, 320:362-365, 1986. T. Bonhoeffer and A. Grinvald. The layout of iso-orientation domains in area 18 of cat visual cortex: optical imaging reveals a pinwheel-like organization. J. Neurosci., 13:4157-4180, 1993. V. Braitenberg and A. Schiiz. Anatomy of the Cortex. Springer-Verlag, Berlin, Germany, 1991. T.G. Brown and C.S. Sherrington. Observations on the localization in the motor cortex of the baboon {papio anubis). J. Physiol (London), 43:209-218, 1911. A. Cowey. Projection of the retina on to striate and prestriate cortex in the squirrel monkey (saimiri sciureus). J. Neurophysiol, 27:366-396, 1964. K.J.W. Craik. The Nature of Explanation. Cambridge University Press, Cambridge, UK, 1943. P.M. Daniel and D. Whitteridge. The representation of the visual field on the cerebral cortex in monkeys. J. Physiol (London), 159:203-221, 1961. J. DeFelipe and E.G. Jones. Cajal on the Cerebral Cortex. An annotated translation of the complete writings. Oxford University Press, New York: NY, 1988. C. Dehay, G. Horsburgh, M. Berland, H. Killackey, and H. Kennedy. Maturation and connectivity of the visual cortex in the monkey is altered by removal of retinal input. Nature, 337:265-267, 1989. R.J. Douglas, M.A. Mahowald, and K.A.C. Martin. Hybrid analog-digital architectures for neuromorphic systems. In IEEE International Conference on Neural Net-

93 works, pages 1848-1853, Orlando, 1994. 19. R.A. Fisken, L.J. Garey, and T.P.S. Powell. The intrinsic and commissural connections of area 17 of the visual cortex. Proc. Roy. Soc. Lond. B, 272:487-536, 1975. 20. D. Fitzpatrick, J.S. Lund, and G.C. Blasdel. Intrinsic connections of macaque striate cortex: Afferent and efferent connections of lamina 4c. J. Neurosci., 5:3329-3349, 1985. 21. T.F. Freund, K.A.C. Martin, P. Somogyi, and D. Whitteridge. Innervation of cat visual areas 17 and 18 by physiologically identified x- and y- type thalamic afferents. ii. identification of postsynaptic targets by gaba immunocytochemistry and golgi impregnation. J Comp Neurol, 291:242-275, 1985. 22. K.C Gatter and T.P.S. Powell. The intrinsic connections of the cortex of Area 4 of the monkey. Brain, 101:513-541, 1978. 23. C.D. Gilbert and T.N. Wiesel. Morphology and intracortical projections of functionally characterised neurons in the cat visual cortex. Nature, 280:120-125, 1979. 24. A. Grinvald, E. Lieke, R.P. Frostig, C. Gilbert, and T.N. Wiesel. Functional architecture of cortex revealed by optical imaging of signals. Nature, 324:361-364, 1986. 25. P. Heggelund. Receptive field organization of simple cells in cat striate cortex. Exp. Brain Res., 42:89-98, 1981. 26. S.H.C. Hendry, E.G. Jones, H.D. Schwark, and J. Yan. Numbers and proportions of gaba immunoreactive neurons in different areas of monkey cerebral cortex. J. Neurosci., 7:1503-1519, 1987. 27. J. J. Hopfield. Neural networks and physical systems with emergent collective computational abilities. Proc. Natl. Acad. Sci. USA, 79:2554-2558, 1982. 28. J. Horton and D.H. Hubel. Regular patchy distribution of cytochrome oxidase staining in primary visual cortex of macaque monkey. Nature, 292:762-764, 1980. 29. C.R. Houser, S.H.C. Hendry, E.G. Jones, and J.E. Vaughn. Morphological density of immunocytochemically identified gaba neurons in monkey sensory motor cortex. J. NeurocytoL, 12:617-638, 1983. 30. D. H. Hubel and T. N. Wiesel. Receptive fields, binocular interaction, and functional architecture in the cat's visual cortex. Journal of Physiology (London), 160:106-154, 1962. 31. D. H. Hubel and T. N. Wiesel. Receptive fields and functional architecture in two non-striate visual areas (18, 19) of the cat. J. NeurophysioL, 28:229-289, 1965. 32. D.H. Hubel and T.N. Wiesel. Receptive fields and functional architecture of monkey striate cortex. J. Physiol. (London), 195:215-243, 1968. 33. D.H. Hubel and T.N. Wiesel. Uniformity of monkey striate cortex: A parallel relationship between field size, scatter, and magnification factor. J. Comp. Neurol., 158:295-306, 1974. 34. D.H. Hubel and T.N. Wiesel. The functional architecture of the macaque visual cortex, the ferrier lecture. Proc. Roy. Soc. Lond. B, 198:1-59, 1977. 35. D.H. Hubel, T.N. Wiesel, and M.P. Stryker. Anatomical demonstration of orientation columns in macaque monkey. J. Comp. Neurol., 177:361-380, 1978. 36. A.L. Humphrey, M. Sur, D.J. Ulrich, and S.M. Sherman. Projection pattern of individual X- and y-cell axons from the lateral geniculate nucleus to cortical area 17 in the cat. J. Comp. A^ewro/., 233:159-189, 1985.

94 37. H. Kennedy and C. Dehay. Cortical specification of mice and men. Cerebral Cortex, 3:171-186, 1993. 38. H. Kennedy, C. Dehay, and G. Horsburgh. Striate cortex periodicity. Nature, 348:494, 1990. 39. Z.F. Kisvarday, K.A.C. Martin, D. Whitteridge, and P. Somogyi. Synaptic connections of intracellularly filled clutch neurons, a type of small basket neuron in the visual cortex of the cat. J. Comp. Neurol., 241:111-137, 1985. 40. E.A. Lachica, P.B. Beck, and V.A. Casagrande. Parallel pathways in macaque monkey striate cortex: anatomically defined columns in layer III. Proc. Natl. Acad. Sci. USA, 89:3566-3570, 1992. 41. M.S. Livingstone and D.H. Hubel. Anatomy and physiology of a colour system in the primate visual cortex. J. Neurosci., 4:309-356, 1984. 42. R. Lorente de No. Cerebral cortex: architecture, intracortical connections, motor projections. In J.F. Fulton, editor. Physiology of the Nervous System, chapter 15, pages 288-315. Oxford University Press, New York, NY, 1949. 43. J.S. Lund, G.H. Henry, C.L. Macqueen, and A.R. Harvey. Anatomical organization of the primary visual cortex (Area 17) of the cat. a comparison with Area 17 of the macaque monkey. J. Comp. Neurol., 184:599-618, 1979. 44. J.S. Lund, R.D. Lund, A.E. Bunt Hendrickson, and A. Fuchs. The origin of efferent pathways from the primary visual cortex, area 17, of the macaque monkey. J. Comp. Neurol, 164:265-285, 1975. 45. J.S. Lund, T. Yoshioka, and J.B. Levitt. Comparison of intrinsic connectivity in different areas of macaque monkey cerebral cortex. Cerebral Cortex, 3:148-162., 1993. 46. R. Malach. Dendritic sampling across processing streams in monkey striate cortex. J. Comp. Neurol, 315:303-312, 1992. 47. R. Malach. Cortical columns as devices for maximising neuronal diversity. TINS, 17:101-104, 1994. 48. K.A.C. Martin. Neuronal circuits in cat striate cortex. In E.G Jones and A. Peters, editors. Cerebral Cortex, Vol. 2, Functional Properties of Cortical Cells, volume 2, pages 241-284. Plenum Press, New York, 1984. 49. K.A.C. Martin. The Wellcome Prize Lecture, from single cells to simple circuits in the cerebral cortex. Q. J. Exptl Physiol, 73:637-702, 1988. 50. K.A.C. Martin, Somogyi P., and D. Whitteridge. Physiological and morphological properties of identified basket cells in the cat's visual cortex. Exp. Brain Res., 50:193200, 1983. 51. K.A.C. Martin and D. Whitteridge. Form, function and intracortical projection of spiny neurones in the striate visual cortex of the cat. J. Physiol. (London), 353:463504, 1984. 52. B.A. McGuire, J.-P. Hornung, C. Gilbert, and T.N. Wiesel. Patterns of synaptic input to layer 4 of cat striate cortex. J. Neurosci., 4:3021-3033, 1984. 53. V.B. Mount castle. Modality and topographic properties of single neurons of the cat's somatic sensory cortex. J. Neurophysiol, 20:408-434, 1957. 54. A. Peters. Thalamic input to the cerebral cortex. TINS, 2:183-185, 1979. 55. A. Peters. Number of neurons and synapses in primary visual cortex. In E.G. Jones and A. Peters, editors, Cerebral Cortex 6: Further aspects of cortical function including

95 hippocampus, volume 6, pages 267-294. Plenum Press, New York, NY, 1987. 56. A. Peters and Feldman M.L. The projection of the lateral geniculate nucleus to area 17 of the rat cerebral cortex. I. general description. J. NeurocytoL, 5:63-84, 1976. 57. C.G. Phillips. Actions of antidromic pyramidal volleys on single betz cells in the cat. Q. J. Exptl Physiol, 44:1-25, 1959. 58. T.P.S. Powell. Certain aspects of the intrinsic organisation of the cerebral cortex. In 0 . Pompeiano and C. Ajmone Marsan, editors. Brain mechanism and perceptual awareness, pages 1-19. Raven Press, New York, NY, 1981. 59. P. Rakic. Specification of cerebral cortical areas. Science, 241:170-176, 1988. 60. P. Rakic, I. Suner, and R.W. Williams. A novel cytoarchitectonic area induced experimentally within the primate visual cortex. Proc. Natl. Acad. Sci. USA, 88:20083-2087, 1991. 61. A.J. Rockel, R.W. Hiorns, and T.P.S. Powell. The basic uniformity in structure of the neocortex. Brain, 103:221-244, 1980. 62. A.M. Sillito. The contribution of inhibitory mechanisms to the receptive field properties of neurones in the striate cortex of the cat. J. Physiol. (London), 250:305-329, 1975. 63. A.M. Sillito. Inhibitory processes underlying direction specificity of simple, complex, and hypercomplex cells in cat's striate cortex. J. Physiol. (London), 271:699-720, 1977. 64. H. Sompolinsky, D. Golomb, and D. Kleinfeld. Cooperative dynamics in visual processing. Physics Review A, 43(12):6990-7011, 1991. 65. H. Van der Loos and T.A. Woolsey. Somatosensory cortex: structural alteration following early injury to sense organs. Science, 175:395-398, 1973. 66. D.J. Willshaw and C. von der Malsburg. How patterned neural conenctions can be set up by self organization. Proc. Roy. Soc. Lond. B, 194:431-455, 1976.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) © 1996 Elsevier Science B.V. Allrightsreserved.

97

FUNCTIONAL TOPOGRAPHY OF HORIZONTAL NEURONAL NETWORKS IN CAT VISUAL CORTEX (AREA 18) Z.F. Kisvarday^ T. Bonhoeffer'^, D.-S. Kim^, U.T. Eysel^

Abteilung fur Neurophysiologie, Medizinische Fakultat, Ruhr-Universitat Bochum, Universitatsstrasse 150, 44801 Bochum, Germany Max-Planck Institute fiir Psychiatry, Am Klopferspitz 18A, 82152 Miinchen-Martinsried, Germany Max-Planck Institute fiir Himforschung, Deutschordenstrasse 46, 60528 Frankfiirt, Germany

1. INTRODUCTION An intriguing and general feature of the cerebral cortex is that it contains fiinctional units of cell assemblies arranged in a columnar manner. The actual „columnarity" of the cortex was first observed in fi-ontal and sensorymotor areas (Mountcastle, 1957; Szentagothai, 1965; Szentagothai and Arbib, 1974; Goldman and Nauta, 1977), nevertheless, it is fair to say that since those early observations the most thoroughly studied regions in this respect have been the visual cortices, in particular primary and secondary visual areas of cat and monkey. Although anatomical studies revealed that corticocortical connections link distinct neuronal groups they could not reveal what might be the fiinctional role of these connections. On the other hand physiological experiments showed that visual cortical neurons selective e.g. for orientation were regularly distributed; in the same cortical colunm each cell had similar orientation preference and neighbouring columns showed a gradual shift in the preferred orientation (Hubel and Wiesel, 1962, 1963; Albus, 1975) resulting in a probable spatial mosaic of orientation selectivity (Braintenberg and Braintenberg, 1979). A causal link between the anatomical and physiological observations had been anticipated well before direct evidence was available. In a theoretical study, Mitchison and Crick (1982) suggested that corticocortical connections run between groups of neurons sharing similar physiological attributes; in terms of orientation selectivity iso-orientation columns should be linked (iso-model). During ensuing years this concept had become so much favoured that the report of Matsubara et al. (1985) came as a surprise. They decided to test the model proposed by Mitchison and Crick using a * Acknowledgement: The authors thank Ms E. Toth and Mr F. Brinkmann for their excellent technical assistance, and Ms D. Strehler for photography. This work was supported by the Deutsche Forschungsgemeinschaft (Ey8/17-1) and the European Communities (SCI 0329-C). We dedicate this work to Janos Szentagothai who died in September 1994.

98 combination of physiological mapping and anatomical tracing in the same cortical region, area 18 of the cat. What they found was all but supporting the iso-model. Notably, the corticocortical connections they visualized linked sites whose orientation preferences simply did not match. A few years later, the study of Gilbert and Wiesel (1989) revisited the same issue, this time in area 17 of the cat. Their findings, however, rather favoured the iso-model. Hitherto, these two studies represented the only genuine attempts in which anatomy and physiology were combined to address the central question whether corticocortical connections prefer to link sites of similar or dissimilar orientations. Clearly, at best, the above results established a balance between the two options. Thus, all the more surprising that an almost unanimous consensus exists in the literature favouring the iso-model. An explanation to this may derive from the fact that the iso-model better suits to the current view of parallel processing of visual information (Stone et al, 1979; Lennie, 1980; Livingstone and Hubel, 1988; for review see Merigan and Maunsell, 1993). Furthermore, support for the iso-model stems from studies measuring correlated neuronal activity whereby positive cross correlation could be detected between laterally displaced cells of similar orientation preferences (Michalski et al, 1983; Nelson and Frost, 1985; Ts'o et al, 1986). This technique, however, without anatomical back-up has serious inherent limitations in terms of interpreting the results (Aertsen and Gerstein, 1985). Additional contribution to the broad acceptance of the iso-model is probably lent from observations in primates on the specificity of lateral connections. Notably, the cytochrome oxidase compartments stationing cells of special physiological properties showed a qualitatively positive tendency although not selectivity of connecting each other (Livingstone and Hubel, 1984). We think that looking at the available pro- and contraevidences a quantitative analysis is imminent to reveal whether lateral connections abide by the iso-model in the visual system. Therefore we carried out experiments using a combination of detailed physiological and anatomical approaches in area 18 of the cat. The present chapter shall attempt to provide an account on the achieved results and highlight some of the implications.

2. CELLULAR COMPONENTS OF THE HORIZONTAL SYSTEM 2.1. Lateral excitatory connections Although the main neuronal composition of corticocortical connections can be inferred from Golgi-studies showing that certain pyramidal cells possess long intracortical axons full details about the types of neurons and axonal arborization patterns could not be obtained until modem extra- and intracellular-labelling techniques were available. Using bulk injections of retrograde and anterograde tracers it has been demonstrated in a number of cortical regions and in a number of mammalian species that the population of corticocortical connections originating from any given location formed a lattice-like pattern, often called patches, distributed over several mm laterally around the injection site (Rockland and Lund, 1982; Matsubara et al 1985, 1987; Gilbert and Wiesel, 1989; Luhmann et al 1989; Burkhalter et al, 1989, 1990; Yoshioka et al, 1992; Levitt et al, 1994). Obviously, the constituting neuronal elements of these connections could be best studied using the intracellular labelling technique whereby pyramidal cells in the superficial and deep layers were found to emit horizontal axons and bursts of collaterals at quasi-regular intervals (Gilbert and Wiesel, 1979, 1983; Martin and Whitteridge, 1984). A direct link between the population and the single cell results could be witnessed in a three-dimensional reconstruction often pyramidal cells in area 17 of the cat (Kisvarday and

99 Eysel, 1992). Interestingly, in this study, the patchy axons of pyramidal cells made up a network that was also patchy covering an area of 6.5x3.5 mm elongated in antero-posterior direction. Boutonal counts of the reconstructed neurons revealed that a single pyramidal cell may provide an average of 80 boutons per patch of its 300-1200 boutons in total. Obviously, these numbers should be considered as lower estimates. Using these and adequate quantitative data from the literature, we estimated that a single pyramidal cell received less than 0.1% of its total excitatory input from the same patchy axon (Kisvarday and Eysel, 1992). Furthermore, we found that each patchy axon innervated only 1-3% of all neurons in a given patch. These values are very much in line with those probability values obtained in dual intracellular recordings for monosynaptic connections between remote pyramidal cells (Mason et al, 1991, Thomson et al, 1988). They also agree with previous anatomical observations demonstrating one or only a few direct contacts from one pyramidal cell to another strongly supporting the view that any pyramidal cell is under the influence of a vast number of other cortical neurons (Szentagothai, 1975). Thorough examination of individually labelled pyramidal cells revealed an additional interesting connectivity feature. Namely, when their projection patterns were compared with each other's it was found that many of them were reciprocally connected. The mode of reciprocity was not, however, mutual between pairs of individual pyramidal cells that could obviously lead to abnormal hyperactivity in such a network. Instead, this kind of relationship did exist at the plopulation level that is a pyramidal cell could receive input from neurons of the very same patch it innervates but only from those neurons to which the pyramidal cell is not presynaptic. Although reciprocity seems to be a fundamental connectivity rule within the patchy excitatory network there is strong support to acknowledge that this is not always the case. Clearly some of the patches labelled from the very same injection site contained either labelled somata or labelled boutons but not both (Boyd and Matsubara, 1991; Kisvarday and Eysel, 1992). Certainly this pattern could emerge only if the neighbourhood relationship between colunms contains some degree of discontinuity (Amir et al, 1993). Although pyramidal cells with patchy axons represent the chief component of horizontal connections morphological analyses unambiguously proved that other excitatory cell types also have to be taken into account. For example, in the cat striate cortex, some spiny stellate neurons of layer IV were shown to establish axonal patches in layer III, others had patchy axonal arborization within layer IV (Martin and Whitteridge, 1984). In the monkey striate cortex, spiny stellate neurons in layer IVCP could provide periodic projection pattern into the superficial layers and to the IVCa and P subdivisions (see Lund et al, 1994). Unfortunately, the amount of projection by layer IV cells contributing to the patchy system of superficial layers is not known in either species. 2.2. Lateral inhibitory connections Examination of Golgi-pitp^ations and intracellularly labelled cells revealed a broad variety of non-pyramidal local circuit neurons (Cajal, 1899; Szentagothai, 1973; Jones, 1975; Lund et al, 1979, 1988; Peters and Regidor, 19^81) which are thought to be GABAergic and inhibitory in fijnction. Fine structural studies showed that of the many types of GAB A neurons only a few had lateral axons comparable in extent to that of the horizontal pyramidal cell network. These neurons were invariably identified as large basket cells in layers III-V (for review see Kisvarday, 1992). Commonly they have smooth dendrites and 3-5 thick myelinated axon collaterals running up to 1-2 mm parallel with the cortical surface. A very important characteristic of large basket cells is that they provide lateral input only to certain regions

100 within their reach. Notably, the 3-5 main axon collaterals follow a relatively straight course and en route regularly give off radial segments laden with boutons. This axonal arborization pattern resembles narrow vertical slabs or wedge-shaped fields that can be best appreciated in reconstructions viewed fi'om the cortical surface as shown in figures 2a and 3. Combined lightand electronmicroscopy proved that large basket cells target chiefly the perisomatic region of pyramidal cells (Somogyi et al, 1983) and non-pyramidal cells (Kisvarday et al, 1993), including other large basket cells, at a ratio of approximately 8-13:1. Recent estimates put the minimum number of boutons per basket cell between 2000 and 3000, implying that each basket cell could directly influence 300-600 other neurons (Martin 1988; Kisvarday et al, 1993). With regard to the latter figure large basket cells thus seem to resemble pyramidal neurons, each member of both groups could contact similar numbers of other neurons. At this point it might be relevant to ask whether there is sufficient number of large basket cells to play a significant role in lateral networks? Excitatory cells are known to outnumber inhibitory neurons by a ratio of 4:1 (Gabbott and Somogyi, 1986). Although exact measures are still not available on this issue a recent calculation puts the minimum occurrence of basket cells, including clutch cells and columnar basket cells, to be around 17% of all GABAergic neurons (Kisvarday, 1992). From this it follows that the proportion of large basket cells could be at least a few percent of all cortical neurons. While this seemingly sparse lateral inhibition suggests weak fiinctional relevance it should be remembered that lateral excitatory connections originate only fi-om an estimated 3-11% of cortical cells (Albus and Wahle, 1994). With this in mind and that large basket cells contact a fimctionally very potent region, proximal to the generation site of action potentials, of the target cells the actual significance of lateral inhibition could be as high as that of lateral excitation. Indeed, preliminary intracellular data revealed that in the hippocampus stimulation of a basket cell could delay or even suppress the repetitive discharge of action potentials of its targeted pyramidal cell (Miles et al, 1994). Furthermore, experiments using GABA-induced inactivation of sites remote fi'om the recording site showed strong influence of lateral inhibition, most likely to be mediated by large basket cells, on orientation (Eysel et al 1990; Crook et al, 1991; Crook and Eysel, 1992) and direction selectivity (Eysel et al, 1988; Crook, Kisvarday and Eysel, unpublished results). These findings suggest that basket cells in the neocortex could be extremely influential in determining the output of their target neurons.

3. FUNCTIONAL TOPOGRAPHY OF LATERAL CONNECTIONS In this section, we examine the topographical relationship between orientation selectivity and lateral connectivity. Contrary to previous studies we used methods that allowed us to quantify both the physiological and the anatomical results. In one set of experiments orientation maps were obtained by recording multiunit activity in layer III at intervals of 100-300 \xm in a region of 1-2 mm^. At each penetration, orientation selectivity of a small cluster of cells was determined for the dominant eye using computer controlled stimuli (see details in Kisvarday and Eysel, 1993). The location of penetrations was marked on an enlarged photograph of the exposed area for which surface blood vessels were used as landmarks. In addition the precise stereotaxic position of each penetration was registered with a resolution of 10 ^im. Once the exact location of the recording electrode can be identified in the tissue this method provides reliable information about the orientation preference of neurons in the close neighbourhood of the electrode. It has to be mentioned, however, that this kind of mapping suffers fi-om two major limitations. First, due to the relatively small number of penetrations the resulting

101 orientation map has a relatively low spatial resolution. Secondly, to collect sufficient amounts of quantitative data even from a small region may take several days. Hence, in another set of experiments we employed optical imaging of intrinsic signals to map orientation selectivity (Kisvarday et al, 1994). The major advantage of applying this technique is that it provides information from large cortical areas at relatively high spatial resolution. We routinely obtained orientation maps from regions of 8-19 mm^, a noticeable tenfold increase compared to our physiological mapping with electrodes. The image resolution of our maps was kept at 5002500 iim^ per pixel resulting in a confidence range of ±(20-50) jam in lateral direction from any given point in the map. It should not be forgotten, however, that although optical imaging is faster and provides better resolution than mapping with electrodes, it utilizes averaged signals originating from the upper 600-900 \xm of cortical tissue. Furthermore, since this method detects changes in oxygen metabolism and not directly the electrical activity of neurons the results need to be treated with caution. Other problems may arise in interpreting the results of optical imaging if, for example, the spatial architecture of orientation columns is perturbated most likely at convoluted regions. In order to minimize the number of such errors we carried out our experiments in largely flat zones of area 18. After completion of either electrophysiological or optical mapping of orientation selectivity the underlying anatomical connections were revealed with iontophoretic delivery of biocytin (see details in Kisvarday and Eysel, 1993). Because the orientation maps were obtained in a plane tangential to the cortical surface we sectioned the injected cortical regions in a plane as parallel with the cortical surface as possible. Apparently a critical issue of this study was to find the best match between the physiological and the anatomical images. To overcome this problem each section was osmium treated and resin embedded to retain the fiill threedimensional structure of the tissue. We also found that this procedure provided little and more even tissue shrinkage than other histological treatments. Another unportant advantage of osmication is that any tissue damage caused by electrode penetrations that otherwise go unnoticed can be detected in the light microscope. This has proved advantageous in correlating the layout of recording electrode holes marked on the physiological charts and their actual location in the tissue. In this way, we were able to find a precise match between our orientation maps and histological sections with an overall error of 30-40 |im. Figure 1 shows a typical example of a biocytin injection site and its surrounding labelling. In general, our injection parameters were set to produce small injections with a core diameter of less than 200 [im in diameter. Although, in the core region, the reaction deposit was often opaque rendering any tracing of labelled elements impossible, in the surrounding tissue, individual axons could be readily traced for several mm. The labelled axons often formed clusters or patches which occasionally contained retrogradely labelled neurons at various labelling intensity level. Interestingly not all retrogradely labelled cells were found in patches, they could also be found in inter-patch zones. On the basis of dendritic morphology two main neuronal types of retrogradely labelled cells could be distinguished; (i) non-pyramidal cells with smooth dendrites, (ii) pyramidal cells with spiny dendrites. We found that non-pyramidal cells constituted a smaller proportion of labelled somata than can be anticipated by their known proportion in the cortex (Gabbott and Somogyi, 1987). A similar observation was reported recently using bulk injection of retrograde tracers combined with GABA-immunostaining (Matsubara and Boyd, 1992; Albus and Wahle, 1994). Since we used biocytin that often reveals recurrent axon collaterals of retrogradely labelled neurons, in a number of instances, we were able to determine the type they belong to. Much to our surprise, virtually all retrogradely labelled non-pyramidal neurons with traceable axons showed characteristics of the family of

102 large basket cells. This phenomenon suggests that, at least in the cat, biocytin is selective in revealing inhibitory connections of the long-range type. Unfortunately, tracing the fine axon collaterals of individual pyramidal neurons proved to be more difficult in these materials. Their thin axons were either too faint or arborized in regions which were too dense for detailed tracing. 3.1 Analysis of single large basket cells Axons In this section, we present anatomical data on the axonal distribution of individual large basket cells obtained in area 18 that had been mapped for orientation using the optical imaging technique. For the analysis presented here, two large basket cells in layer III were selected on the basis of completeness of their axonal arbors one of which is shown in figure 2a. It is well established that the perisomatic termination of basket cell boutons strongly indicate the actual physical location of their target neuron. We utilized this phenomenon together with the notion that signal transduction between neurons takes place at synaptic junctions. Thus, by plotting the spatial distribution of basket cell boutons according to orientations a quantitative estimate could be made for their inhibitory contribution on the orientation map. In order to do this we dissected the orientation map into 16 orientation divisions each of which represented 11.25 degrees along the orientation axis. Then the relative occurrence of boutons in each of these divisions was displayed (Fig. 2c). Looking at the results there are a few inFigure 1. Tangential view of a biocytin injection into layer III of area 18 of the cat. Hie dark area in the bottom represents tiie injection site showing a core diameter of 150 pm. Arrowheads indicate labelled main axons whose terminal boutons form a patch in the upper part of the picture. Bar: lOOpm.

•V'

•% \

c-

il

^ -" -

*• V *

*^''.

103 teresting points that are worth to be considered. Firstly, boutons of the same basket cell provide input to all orientations, a finding that is very much in line with previous assumptions on the broad scale of lateral inhibition by large basket cells to orientation sites (Martin, 1988; Kisvarday and Eysel, 1993). Secondly, although the overall shape of the distribution is rather symmetrical it is nevertheless skewed with respect to the preferred orientation at the soma location. We do not know yet whether this phenomenon is due to a genuine bias of the axons to provide input to these orientations or due to incomplete reconstruction of the axonal field, most likely at the densely labelled injection site. Thirdly, the above quantitative analysis allowed us to compare our findings directly with previous results on the same issue. In a recent analysis of area 17, we compared the distribution of individually traced large basket cells with orientation maps obtained with electrophysiological mapping (Kisvarday and Eysel, 1993). Applying a semi-quantitative method basket cells were found to provide approximately the same amount of input to orientation sites defined as iso- (±30^, oblique- (±[30-60]°) and cross-orientation (±[60-90]°) with respect to the basket cell's own orientation preference. When we used a similar scheme in area 18, the results averaged for two large basket cells resembled those of area 17, 42.6% of all boutons occupied regions of iso-orientation sites, and oblique- and cross-orientations were represented by 35.4 and 22.0%, respectively. In essence, the above findings provide direct evidence that in the visual cortex of the cat individual large basket cells could mediate lateral inhibition to virtually all orientations. A direct application of these results is that in addition to iso-orientation inhibition there is now fijnctional-topographical evidence for non-iso-orientation inhibition in the strictest sense. In this context, one should not forget that the results underestimate the real contribution of lateral inhibition to non-iso-orientation sites because our categories of iso-, oblique- and crossorientation preferences are artificial. Recording of single cells show that, in most cases, a 30° oflfeet of the visual stimulus fi-om the preferred orientation of a neuron represents an already non-optimal orientation. With this in mind our result are consistent with extracellular experiments demonstrating strong inhibitory influence fi'om non-optimal orientations (Sillito, 1975, 1979; Morrone et al, 1982; Ramoa et al 1986; Eysel et al, 1990) and some intracellular experiments showing the broad band nature of inhibitory postsynaptic potentials (Benevento et al, 1972; Douglas et al, 1991; Volgushev et al 1993). Dendrites The method that we applied for unravelling the relationship between the axonal topography of basket cells and orientation maps can also be used for a similar analysis of their dendritic fields. Assuming that each point in the orientation map defines an orientation specific input to the underlying structures, e.g. somata and dendrites, a logical question that one may ask is whether the dentritic field of a basket cell receives input fi'om a narrow or a broad range of orientations. Hence we used the same approach as for the axons and counted the occurrence of basket cell dendrites in orientation pixels. The results showed that „input" to the dendrites is more narrowly tuned than the boutonal „output" of the same basket cell. Using a simple numerical analogy of the data, 86% of the dendrites were found to branch at iso-orientation sites, and oblique- and cross-orientation sites were represented by 11 and 3 %, respectively. These values might be argued to follow fi'om the fact that dendrites have smaller lateral extent than axons although they occasionally reach distances up to 300 [im fi'om the parent soma. It would be interesting to see how the orientation tuning of physiologically characterized basket cells correlates with the "dendritic input" calculated with the above method. In any case, our

104

a

Single basket cell

^>^\\\ \ I / I

//^^^

I

b

Population of basket axons

^ X \ \ \ \ I / ///X-

Figure 2. Reconstructions of axon collaterals emitted by a single basket cell (in a) and by a population of eight basket cell collaterals (in b) viewed from the cortical surfece. The parent soma (in a) and the injection site (in b) are marked by asterisk. Notice that the single basket cell and the eight basket axons innervate only certain regions. In (c) and (d), frequency distributions of boutons provided by the basket cell shown in (a) andti&epopulation of basket axons shown in (b) are displayed as a function of orientation preferences. It is evident that the boutonal distribution of tiie single basket cell is normal-like and skewed to the soma location (arrowed) and the distribution of the pooled axons is highly irregular. In (d), the injection site is indicated by asterisk. Bar: (a) and (b), 250 pm. Figure 3. (in facing page) Tangential view of reconstructed axonal fields of two large basket cells, one in layer III (in black) and one in layer V (in white), in the same cortical column. Somata of the two cdls are indicated by arrows. Notice that both axonal fields tend to overlap, nonetheless, they terminate'in different laminae. Bar: 500 ^mi.

105

106 finding of narrow orientation distribution of basket cell dendrites matches well with electrophysiological data (Martin et al, 1983). 3.2 Population of basket cell axons While the analysis of single basket cells could disclose subtle details about the functional topography of lateral inhibition it could by no means completely reveal the situation at the population level. Accordingly we studied how the population of large basket axons originating from a circumscribed region of the cortex is distributed along the orientation map revealed with optical imaging. Therefore, in one animal, all labelled basket axons (8) which projected more than 400 |Lim laterally and originated from the centre of the injection site were reconstructed in area 18. Laminar analysis showed that each of them arborized mainly in layer III with occasional branching into upper layer IV. Figure 2b demonstrates the boutonal distribution of the eight basket axons in a tangential plane. Interestingly, we found that their composite field occupied only certain sectors and did not project into other regions. In this respect the population of basket axons resembled individual basket cells shown above. An implication of such a topography could be that the population of basket cells of a given cortical column may provide input only to certain visuotopic locations in the cortex. This assumption is strengthened by the observation of two basket cells whose somata were found in the same cortical column, one in layer III and the other one in layer V (Fig.3). Superimposed their reconstructed axons revealed that both basket cells were in quasi-register by innervating corresponding sectors of different laminae. A similar, spatially specific arborization pattern has been described for layer III pyramidal cells providing patchy input to layers III and V (Gilbert and Wiesel, 1983; Martin and Whitteridge, 1984; Kisvarday et al, 1986). It thus seems reasonable to speculate that lateral inhibition may also be selectively distributed along the visual cortical map although the governing rules may well be very different from that of the excitatory system. The similar arborization pattern of individual basket cells and the population of basket axons suggests that their orientation distributions might also be similar. Indeed, when we applied the same quantitative method that was used for single basket cells the results showed that the boutons of pooled basket axons also covered the entire range of orientations (Fig.2d). However, while single basket cells showed preference for one or two orientations the orientation distribution for the population of basket axons was rather irregular. Again, it is possible that we underestimated the proportion of orientation preferences at the core of the injection site hence the data needs to be critically viewed. 3.3 Lateral patchy connections in area 18 of the cat Connectivity of orientation domains The topography of lateral intracortical connections has been commonly visualized with bulk injection of anterograde and retrograde tracers. While pure anatomical data on this issue are widely available, there are few attempts in the literature, mainly due to technical difficulties, in which the anatomical findings could be directly correlated with function. As pointed out above, the patchy character of iso-orientation domains is thought to be directly linked to the patchiness of lateral connections. Experimental data obtained in areas 17 and 18, however, led to results showing an apparent discrepancy. Here we examine how the distribution of lateral connections in area 18 relates to the topography of orientation. To explore the underlying connectivity rules we used a combination of optical imaging of intrinsic signals and focal injection of biocytin (see details in Kisvarday et al, 1994).

107

Figure 4. Anterogradely labelled boutons reconstructed from adjoining tangential sections. The biocytin injection was made into layer III in area 18 resulting in labelled patches. The labelled area is elongated in antero-posterior Erection. Right = anterior, down = medial. Bar: 1 mm. While previous attempts relied on qualitative evaluation of the resuks in determining topographical relationships we decided to apply quantitative approaches as follows. We chose the location of labelled boutons as the most informative measure for a quantitative interpretation of the anatomical signals. Accordingly, each labelled bouton in large consecutive sections was registered and digitized using the computer software Neurolucida (MicroBrightField) and a personal computer attached to a light microscope (Leitz) with a motorized stage. A typical example for such a reconstruction is shown in figure 4 where the labelled boutons were compiled from the entire cortical thickness. For further analysis, boutonal distributions were converted into boutonal density maps using the same pixel resolution as that of the corresponding orientation maps. In this way the actual distribution structure of the connections could be treated quantitatively. A direct comparison between an orientation map and the underlying boutonal density map can be seen in figure 5. In this example, biocytin was injected into the centre of an orientation domain showing a clear preference to horizontal stimulus orientation. As expected, the injection site and its immediate neighbourhood showed the highest density of labelling and at a lateral distance of 1.3-1.4 mm from the core of the injection site patches of labelled boutons occurred. When the topography of the patches was analysed in terms of orientation specificity they revealed an obvious tendency, the preferred orientations of the patches were similar to that of the injection site, i.e. the patches and the injection site covered mainly blue zones in the orientation map in figure 5. We wanted to know exactly to what orientations the patches corresponded. Therefore we calculated the average orientation preference in a circular region (200 jim in diameter) of each

108

109 patch centred on its density maximum. The actual differences between the averaged orientation preferences of the patches and that of the injection site were then determined (Fig. 5b). It is clear that none of the patches differed more than ±10 degrees from the average orientation preference of the injection site. The most simple application of this result is that patchy connections in area 18 of the cat predominantly link sites of similar orientations. From this it might follow that the connectivity rule in area 18 is the same as in area 17. However, as is often the case, the situation regarding lateral connectivity turned out to be much more complex. Detailed inspection of the orientation map and the anatomical reconstruction shown in figure 5 revealed two important facts. Namely, in this particular experiment, the injection site was confined to the centre of an orientation domain where orientation gradients are obviously low compared to orientation centres. It is tempting to speculate that lateral connections in regions of high orientation gradient, orientation centre, maybe different. Thus we carried out a critical test on this issue by injecting biocytin into regions containing orientation centres or „pinwheels" and the results are discussed in the following section. A second important fact is the presence of labelled boutons in interpatch zones irrespective of the topographical location of the injection site. It was noticed that although these boutons constituted a relatively small proportion of the total labelling they provided a continuum between the patches so that no regions within the labelled area was exempt from boutonal labelling (see brown regions in figure 5b). At present little is known about the exact origin of these connections. Clearly their majority is provided by a mixture of pyramidal type of axons and, as we have seen above, to some extent by basket cell collaterals. Connectivity of orientation centres In the previous paragraph, we have shown that injections into orientation domains of area 18 label regions which on average have similar orientation preferences. In this paragraph, we provide comparative data on injections made into zones including orientation centres. Qualitatively, injecting into an orientation centre resulted in patchy labelling although the patches were less remarkable than after injecting into orientation domains. This feature can be appreciated in 3-dimensional representations of boutonal density maps as shown in figure 6. Apparently the orientation centre-injection in the lower panel of figure 6 resulted in strong labelling of interpatch regions that hinders to recognize individual patches. Nevertheless, on the basis of contour plots, whereby the contours of iso-density locations were displayed, we were able to differentiate 5 distinct patches around the injection site. Comparing the distribution of the labelled patches resulting from orientation domain- (Fig. 6, upper panel) and from orientation centre-injections (Fig. 6, lower panel) we found no appreciable difference between their overall number and centre-to-centre spacing. Again, we asked whether the orientation preferences of the labelled patches were similar or dis-

Figure 5. Orientation map obtained with optical imaging of intrinsic signals (a) and boutonal density map (b) of tiie same region of area 18. The orientation map was computed from single condition maps (Bonhoeffer and Grinvdd, 1993) according to the color scheme on theright-handside. Asterisk indicate tiie site of iontophoretically delivered biocytin into layer HI. In b, the boutonal density distribution is shown in the very same area as in (a). Note the patehy distribution of the labelled boutons around the injection site. Comparison between ^e optical and the anatomical images revealed that the labelled patches occupy regions possessing like-orientations to that of the injections site. The difference in orientation preferences between each pateh and the injection site was numerically estimated by calculating the average orientation preference in areas of 200 \mi in diameter centred on each pateh and tiie injection site. The actual difference in orientation preference between the injection site and the patehes are indicated in degrees. Bar: 1 mm.

no

Orientation domain-injection

Orientation centre-injection

Ills I s ct^ 4^ yii|i|P|;ftir'Ji ,

ifilfillii.

Figure 6. Shaded surface representations of the density distribution of labelled boutons in cat visual cortex area 18. In the upper panel, the core of the injection site is situated in the middle of an orientation domain. In the lower panel, the core of the injection site is situated in an orientation centre so-called pinwheel, where many orientations meet in a small region. Note the obvious patchy distribution of labelling in the upper distribution as opposed to the less obvious patchy labelling in the lower distribution. In both distributions, notice the preponderance of local connections up to a lateral distance of 300-400 |Lim from the core of the injection site.

Ill

l l l i l l l ^ ^ % 20 15

rtl :;:v; : ; ^:; ; ; :; : :i^

Patch 2

TO |y^

% 20 15 10 0

^^^^^^^i^^^^^^^

ll|il|iI|MFv^^

;:^-^^.^^yy/iim

% 20

PafcH3

';;,MM

15

id 0

-•-^•>>;/;/::^/:.^I/;A;>\:\.;N;*^^>^I-:.;'

• :i^^^:^:^::/,i/;;::;/:;:;i:;:fr^t

.^MVA..,^^^^

.r^'

100 ms

\M.

100 ms

Figure 7. Synaptic events associated with visually evoked periodic activity Records from two cells (left and right panels) stimulated with drifting bars, showing two forms of oscillatory behavior analyzed in current clamp at rest (upper half) and voltage clamp held close to the resting potential (lower half). Left: in this cell, recorded in a 9 week old kitten, periodic events observed in current clamp consisted of depolarizing potentials separated by pauses during which the membrane potential was stabilized close to the resting level ( - 66 mV). A similar pattern was observed when the cell was voltage clamped at - 66 mV (bottom), demonstrating further that periodic potentials stemmed from bursts of synchronized inward synaptic currents. Right: In this complex cell, recorded in a 8 week old kitten, periodic events in current clamp consisted of alternate depolarizing and hyperpolarizing potentials shifted in phase by 45° (top), corresponding to a fixed sequence of inward and outward synaptic currents when the cell was voltage clamped at - 60 mV (bottom).

173 Another characteristic pattern present in EPSCs recordings is that despite variability, a clear asymmetry between the rising and the falling phase is preserved. In most cases, the falling phase follows the rising phase immediately, without any apparent plateau. The rising phases last on average 17.7 +/- 12.2 ms (20% - 80% rise time, n=196 EPSCs). A theoretical development (to be published elsewhere) shows that the rise time probably gives an upper limit boundary (because of imperfect space clamp) of the duration of overlap of presynaptic activity, i.e. of the degree to which presynaptic spikes are synchronized (Bringuier, 1995). The shorter rise times are 7-10 ms, which corresponds to about one tenth of the period of the oscillation, suggesting a highly structured temporal activity. The possibility that presynaptic spikes are not exactly synchronized but might be slightly spread in time has been explored by several theoretical studies: Diesmann and colleagues have developed a formalism to characterize the efficiency of the transmission of such "pulse packets" through synaptic connections (Diesmann et al., 1995). Bernander and collaborators showed that for a postsynaptic neuron showing a refractory period, pre-synaptic spikes will tend to be more efficient in eliciting several spikes in a row from the common target cell if their arrival times show some temporal jitter (Bernander et al., 1994). If clustering of synaptic events appears optimal in terms of transfer of information in the network, episodes of synchrony do not need to reoccur periodically, as illustrated in Figure 6a. This conclusion is supported by reports based on crosscorrelation studies showing non-oscillatory synchronization (Kriiger and Aiple, 1988; Nelson et al., 1992; Schwarz and Bolz, 1991; Toyama, 1988; Ts'o et al, 1986). It strengthens the view that multiple sudden depolarizations and repolarizations can be observed at a variety of frequencies resulting in a propagation of pulse packets in a nonperiodic manner. The packets of 15-100 presynaptic spikes as indicated above are probably conveyed by a comparable number of fibers for the following reasons: i) synchronization of input results in a barrage of activity impinging on the postsynaptic cell within a temporal window lasting for 10-20 ms; ii) in our recordings visually evoked bursts rarely contained more than 3 action potentials, and iii) morphological reconstruction of identified geniculo-cortical or intracortical axons shows that each of these makes only a few contacts per postsynaptic cell (Anderson et al., 1994; Thomson and Deuchars, 1994). In our view, measuring large EPSCs rise times is a procedure somewhat analog to cross correlation. Although, as we mentioned earlier, it gives an image of pre-synaptic synchronization that might not be extremely sharp, it has the advantage over classical cross-correlation measurements of being more functional, in the sense that the monitored activity is indeed postsynaptically integrated.

174

10 ms

20 ms Figure 8. currents

Profile and variability of periodic visually evoked inward synaptic

The analysis of synaptic currents recorded in a Complex cell of a nine week old kitten was restricted to responses evoked by repeated sweeps of an optimally oriented stimulus at a fixed velocity across the RF, while the cell was held in voltage clamp at - 80 mV. (a): Superposition of 28 periodic EPSCs extracted from four trials synchronized with the maximum of the inward peak. A large variability is apparent, in both in the rising and falling phases of the compound currents. In spite of this variability, a clear asymmetry differentiates the slopes of the rising and the falling phases. (b): The twenty eight EPSCs have been clustered according to the trial from which they were extracted. Variability seen between individual synaptic events appears to be present both within and between trials.

175 6. SPATIO-TEMPORAL INTERACTIONS IN THE ASSOCIATION FIELD As mentioned briefly earlier , theories of perceptual binding based on correlational schemes presuppose the existence of "Hebbian-like" plasticity in order to account for cell assembly formation. However changes in functional links between coactive cells are hypothesized to operate reversibly on a much faster time scale (ms) than synaptic changes described during development, i.e. they are compatible with the minimal integration time necessary to separate a figure from its background or for recognizing objects as parts of a more complex scene (Von der Malsburg, 1981; Von der Malsburg and Bienenstock, 1986). It was assumed, in particular, that "synfire chains" or graphs grow and wane during recognition because of rapid activity-dependent changes in synaptic efficacy (Abeles, 1991; Bienenstock and Doursat, 1989). In spite of this declared quest for "fast Hebbian" synapses, the Hebbian temi should not be taken too literally: reversible modulation of functional links, if it exists, does not in theory require a recapitulation of the elementary subcellular processes responsible for Hebbian associative LTP and heterosynaptic LTD, in order to share a similar phenomenology accelerated on a 10^ to 10^ faster time-scale. An obvious non-Hebbian form of "fast" changes in synaptic efficacy may be found in the rather general observation, that the evoked response in cortical cells elicited by presynaptic spikes activating the same synapse less than 50 ms apart can undergo fast upand down-regulations in amplitude. It is indeed well established, at least in vitro , that two successive stimulations of the same presynaptic fiber induce paired pulse depression (Wilcox and Dichter, 1994) or potentiation (Thomson et al., 1993b) of the test response (review in Magleby, 1987; Thomson, et al., 1993a). Such fast changes in synaptic gain are thought to be predominandy of presynaptic origin, and linked to free calcium accumulation or depletion and alterations in vesicular recruitment which will affect subsequent liberation of neurotransmitter (Zucker, 1989). Recent evidence obtained in identified synapses between pairs of neurons recorded intracellularly suggests that the sign of the change might be dependent on the initial synaptic efficacy (Debanne et al., 1996), the weaker the synapse, the more readily it will be susceptible to be facilitated and thus help to propagate the chain of synchronous activity. Nevertheless the implication of postsynaptic regulatory mechanisms cannot be excluded at both excitatory and inhibitory synapses and might involve receptor desensitization or fast-acting regulation of second messengers (Marty and Llano, 1995). Another class of regulatory processes which cause strong non-linearities in the final summation and integration of synaptic events carried out by the postsynaptic neuron, is linked to the activation of voltage-dependent and temporally gated conductances allowing

176 the cell to boost or damp the response to a test synaptic input during a fixed temporal window. For instance the in vitro work of Deisz et al. (1991) in somatosensory cortex shows a strong APV-resistant voltage-dependency of synaptic potentials evoked by white matter activation. These authors demonstrated that a low threshold calcium current is responsible for the increase in size and duration of the initial EPSP appearing for more depolarized membrane potential values (Figure 9a). The most remarkable feature of this process is that the amplification gain depends on the past history of the membrane potential, and acts over a critical temporal window, in the order of 100 to 200 ms following the onset of depolarization, during which the boosting of the postsynaptic response can be expressed independently of an actual change in synaptic transmission. When the cell has been kept depolarized for a longer period, the arrival of synaptic input, out of phase with the onset of the change in membrane potential, no longer benefits from the calcium inward current which is now inactivated (Figure 9b). How general is this process in cortical cells? It is now well established, at least in the pyramidal neurons of CAl, that apical dendrites possess voltage sensitive calcium conductances (Magee and Johnston, 1995a-b). The currents are mainly of the T type (Christie et al., 1995), as found by Deisz et al. in neocortical neurons (Deisz et al., 1991). At the level of dendritic spines and shafts, back-propagating action potentials elicit intracellular calcium transients that can cooperatively enhance the calcium accumulation resulting from synaptic activity (Yuste and Denk, 1995). In the apical dendrites of pyramidal cells of the primary visual cortex, voltage dependent calcium currents are also present and activated by synaptic input (Markram et al., 1995; Yuste, et al., 1994). These have even been implicated in the induction of LTP following repeated activation of the same afferent pathway in the presence of APV (Komatsu et al., 1991). Other mechanisms of fast regulation of EPSPs should be considered. In principle, a major source of voltage dependent amplification should be observed following NMD A receptor activation (Stem et al., 1992, Thomson, 1986), and visual responses elicited at rest have been shown to contain an APV-sensitive component (Miller et al., 1989). An other type of conductance which might prolong the depolarization evoked by an excitatory input is that linked to persistent sodium channels and has been described in vitro in somatosensory and visual cortical neurons (Hirsch and Gilbert, 1991, Stafstrom et al., 1984). A local iontophoretic application of glutamate on the apical dendrites of somatosensory cortex layer 5 cells induces an inward current, the amplitude of which increases with depolarization (Schwindt and Crill, 1995). Because this faciUtation is not impaired by voltage clamping the soma, these authors conclude that the site of interaction must be rather distal. However,

177 Stuart and Sakmann, using local control and simulation of the postsynaptic depolarization while recording simultaneously from the soma and dendrite of the same layer 5 pyramidal cell, have shown in visual cortex that the TTX sensitivity of the depolarization-induced facilitation of the EPSPs is spatially restricted to a region close to the soma (Stuart and Sakmann, 1995). In spite of this discrepancy, one cannot exclude that very distal dendrites have enough sodium channels to amplify significantly local synaptic inputs. As far as a graded regulation of synaptic gain is concerned, the conductance dependent gain control hypothesis presents a definite advantage over a scenario based on pure synaptic plasticity: it does not require reversibility of the established changes, whereas fast Hebbian schemes have to be accompanied by complementary depression rules to reset connectivity back to its initial state, i.e. before the start of the recognition process. However the diversity of conductances participating to the postsynaptic machinery offers symmetrical associative ways of damping afferent input, thus providing a normalization of synaptic weights, certain synapses being boosted whereas others might be transiently weakened. An example can be found in non-cortical networks, such as the solitary complex (Figure 9 c-d). In this structure which is a caudal bulbar relay of the brainstem involved in respiratory activity (Bianchi et al., 1988; Ezure, 1990), it has been shown that low threshold calcium currents present before birth (which in neocortical neurons are responsible for the associative boosting) are replaced during the first postnatal week of life by potassium currents at the same time as GABA-ergic innervation appears (Schweitzer et al., 1992). Associative protocols similar to those applied in neocortical neurons reveal a transient damping of synaptic input at that age (Fortin, 1993). This opposite change in the modulation of the apparent synaptic gain is due to a potassium outward current, probably of the IA type, which shortens the time course of EPSP if the depolarization has been applied within the last 100ms, but is inactivated for longer delay periods (Figure 9d). In order to test for the existence of boosting or damping of subthreshold inputs in visual cortical neurons, we decided in collaboration with F. Chavane and J, Lorenceau to replay the protocol devised by Deisz et al. in a more functional framework. The aim of this work was several fold: i) to study the spatial and voltage dependency properties of subthreshold responses in the periphery of the MDF, and iii) to determine how synchronization of one input with somatic current depolarization, or of different sources of subthreshold activity, could facilitate their postsynaptic integration. The experimental protocol was devised the following way: the same visual stimulus was shown twice, one second apart, and the two test responses of the cell were recorded and compared at the same membrane potential (bottom part in Figure 10). The holding potential was varied from one

178 trial to the next to reveal a possible voltage dependency. For a given trial, the only distinction between the two successive visual stimulation epochs was their relationship with the recent history of the membrane potential: at the beginning of the trial, the cell was held at rest; then, the onset of the first presentation of the visual stimulus was paired with the intracellular injection of a constant subthreshold current, the application of which was maintained throughout the second presentation of the stimulus (Bringuier et al, 1995). The complete "Current-Vision association" protocol consisted in randomly interleaved pairings of dual sequential visual stimulations in different parts of the visual field (MDF, NEAR, FAR) with an intracellular subthreshold current step. The value of the current for a given trial was a fraction of the threshold intensity needed to reach spike initiation. Both polarities of current and spatial locations of the test stimuli were randomly chosen in order to avoid long-lasting effects of the pairing procedure. The onset of the current pulse was matched in phase with the ON response to the first visual stimulus such as that the

Figure 9. Role of slowly inactivating conductances in the modulation of the effective synaptic gain in vitro . a and b: data adapted from Deisz et al, 1991, with permission. c and d: data adapted from Fortin, 1993 with permission. (a): Voltage dependency of EPSPs recorded in a rat sensorimotor cortex layer 2-3 cell and evoked by stimulation of white matter. The superimposed traces correspond to recordings with and without orthodromic stimulation at different levels of depolarization. EPSPs evoked at a depolarized level, 50 to 100 ms after the current onset are markedly larger than control EPSPs evoked at resting potential or evoked when the cell has been kept depolarized for a long time. The current step value is chosen just below the level of activation of the low threshold calcium current. (b): Superimposed traces of EPSPs evoked in independent trials and positioned at different delays after the onset of the depolarizing current pulse. A selective enhancement of the EPSP is observed only for that evoked a short delay after the onset of the current pulse. (c): Voltage dependency of EPSPs recorded in a cell of the solitary complex evoked by stimulation of the solitary tract. The superimposed traces correspond to recordings with and without orthodromic stimulation at different levels of depolarization. EPSPs evoked at a depolarized level, 50 to 100 ms after the current onset are markedly shorter in duration than control EPSPs evoked at resting potential or evoked when the cell has been kept depolarized for a long time. (d): Superimposed traces of EPSPs evoked in independent trials and positioned at different delays after the onset of the depolarizing current pulse. The comparison of the shape of all recorded EPSPs, shown in the lower inset, demonstrates the selective shortening of the falling phase of the EPSP, due to a potassium current (see text) activatable only for a short period following the onset of the current pulse.

>

^ ^ » o ^






1

y

> s

SYNAPTIC DAMPING

s

o

% ^-*

/

/

^•^^i^

> s o

n

/\

(2T~ ^/ ^ v-_

I|"'-^>s.^i

xaidwoD AHvxnos

179

180 membrane potential depolarization would start concomitantly with the first evoked visualresponse, i.e. would be applied at a delay precisely adjusted to compensate for the latency of visual cortical responses. The second visual stimulus was presented out of phase with respect to the pulse onset, i.e. 1 second later, at a delay where temporal interaction no longer occurs between successive stimuli (Nelson, 1991). Three types of effects were observed: - i) One concemed thefiringfrequency of the neuron: in certain cells the response of the cell to the first stimulus became larger and more tightly tuned temporally than the response to the second stimulus. This observation might appear surprising since increases in the duration of the postsynaptic potential under the influence of calcium conductances do not predict an improvement in the synchronization of temporally overlapping inputs. - ii) A second finding was that in some of the cells, peripheral input leads to a synaptic potential whose amplitude and duration was favored during the first visual stimulus presentation, thus reproducing in a functional context the observations made by Deisz and collaborators (1991). - iii) A third observation was that occasionally IPSPs could also benefit from such boosting. In summary, cells in visual cortex seem to possess the non-linear conductances needed to selectively boost either excitatory or inhibitory synaptic input. These conductances could be turned on during specific functional situations, such as the contiguous presentation of a visual stimulus in the MDF of the recorded cell and of a contextual pattern in the periphery of the RF. However the associative requirements between the test stimulus onset and the somatic intracellular depolarization used in the current-vision protocol impose peculiar timing relationships during natural visuo-visual associations: in order to benefit from the temporal selectivity in inactivation kinetics of postsynaptic conductances, non-homogeneities in response latencies of responses originating from different parts of the visual field (see Figure 5) have to be accounted for and precisely balanced. For instance, we predict a more efficient boosting when the stimulation of the central part of the RF by the inner grating of a cross-oriented bipartite stimulus is preceded by the peripheral appearance of the outer grating by a few tens of milliseconds (see also Zipser, 1995). A possible test situation would be the use of a collapsing annulus (centripetal motion) which should be more efficient than expanding circles (centrifugal motion) in favoring postsynaptic summation. In that respect, it would be interesting to compare the spatial distribution in the latencies of subthreshold responses found in primary cortex with those observed in non-primary visual areas, such as the Clare-Bishop area of the cat, where a

181

iJHl

A,

20 ap/s

[lOniV 500 ms

VISUAL

T 2/3 T 1/3 T CURRENT

"IT

0 -1/3T

Figure 10. Boosting of visually evoked lateral £PSPs by temporal coincidence with post synaptic depolarization Complex cell, recorded in area 17 of an adult cat at a resting potential of - 73 mV. The association protocol is schematized in the lower part of the figure: an identical visual stimulus was shown twice in the same trial, each time for a duration of 400 ms, separated by a 600 ms delay. Meanwhile, a step of current was injected so that its onset coincided with the beginning of the visual response (which required an initial measurement of its latency). The two successive presentations of the stimuli differed only by then: delay with respect to the current pulse. From trial to trial, the parameters which were varied were the amount of current injected (five levels, e>q)ressed as a fraction of the intensity (T) needed to elicit a significant change in the spontaneous level of spiking) and the retinal location of the visual stimulus (MDF, NEAR, FAR, see text). The two upper graphs represent the response to the association of the largest intracellular current (+ 0.27 nA) with a NEAR stimulation, using a moving grating restricted to the side bands of the MDF. The top histogram is the PSTH averaged over 16 trials and the lower trace corresponds to the averaged membrane potential after spike filtering. The second presentation of the test grating evoked almost no response, whereas the first one elicited a large membrane depolarization and a significant increase in firing, the duration of which lasted for that of the stimulation.

182 large proportion of lateral suprasylvian sulcus cells prefer centrifugal motion away from the area centralis (Rauschecker, et al, 1987; but see Toyama, et al., 1990). A possible research line in these different cortical areas could be to look for optimal temporal association sequences of spatially distributed patches of elementary visual patterns derived from the knowledge of the distribution of subthreshold input latencies across the visual field.

7. THE MAGIC RING In summary our data suggest that the receptive field of a visual cortical cell should not be considered as a fixed entity but more as a dynamic field of integration and association. Two types of dynamics can be argued for: - i) The central core of the receptive field (MDF) can be profoundly reorganized at least during development and most probably during selective phases of learning under the control of activity-dependent mechanisms. Adaptive changes in visual responses are thought to reflect long-lasting potentiation and/or depression of synaptic efficacies. - ii) During sensory processing, reconfiguration of synaptic weights may be achieved on a much faster time-scale and linked to non-linear properties of the postsynaptic membrane as well as that of recruited networks. Association of information available in the central part of the RF and of input coming from the reputed "unresponsive" regions surrounding it, or arising simultaneously from different parts of the visual field, might be suppressive in certain cases and capable of boosting hidden responses in other cases, depending on the global stimulus configuration. We conclude from these two lines of evidence that the spatial extent of cortical receptive fields could vary on at least two distinct time scales. The slower one, compatible with time constants of LTP and LTD, would reflect memory traces of the past activity of the network in which the cell is embedded. The faster one, compatible with recognition processes, would depend on the kinetics of inactivation of specific conductances and the recent history of the postsynaptic membrane potential. A remarkable observation is that the SLF of cortical cells assessed intracellularly is much larger than that expected from classical extracellular methods. The unmasking of peripheral responses at the suprathreshold level is often observed when the postsynaptic cell is depolarized and/or when certain combinations of inputs distributed across the visual field are used. This process could require synchronization of converging subliminal inputs evoked by the simultaneous presentation of co-linear stimuli or stimuli sharing the same direction or orientation across the visual field, which might increase the gain of horizontal

183 excitatory connectivity. A puzzling question is to determine if intrinsic oscillating behavior or resonance of recurrent inputs that have been described in cortical networks participate in building up synchrony. Our own data suggest that a cortical cell does not need to undergo a rhythmic pattem of activity to be able to synchronize itself with others cells. Rhythmicity is predominantly observed for a restricted set of visual inputs, and which are not obligatorily optimal for the cell firing. Periodic composite synaptic potentials reflect most probably the working signature of recurrent circuits proned to resonate. However the detailed study of the temporal profiles of visually evoked PSPs and PSCs leads us to conclude that even at the single cell level it is not possible to separate oscillatory behavior from network synchrony. On one hand, no convincing evidence has been presented in the literature that oscillations participate in the postsynaptic elaboration of synchronous firing. Complex stimulus dependency does not argue for hierarchical models in which in-phase depolarizations evoked by various features of the same complex object summate preferentially on hypothetical grandmother cells. On the other hand, detailed analysis of temporal patterns of membrane potential trajectories indicate that often oscillations seen at the single unit level are composed of synchronous barrages of EPSPs, or of phase-locked EPSPs and IPSPs separated by pauses of synaptic activity. Thus, the oscillatory behavior of cortical cells results predominantly from synchronized packets of afferent input. If rhythmicity in oscillations does not appear to be instrumental in building synchrony, the altemation of pauses at the resting membrane potential with abrupt changes in the polarization level of the neuron, might be enough to do the job: the onset of a depolarization plateau could allow the activation of slowly inactivating conductances which might help to boost transiently incoming input. The end of the burst and the associated AHP would de-inactivate the conductances and reset the input gain control mechanism before the next burst. To summarize our view, it is likely that repetition, as Von der Malsburg proposed, might be necessary for the synchrony to be reliably distinguished from the single episodes of correlated activity that are likely to occur by chance, but no strong evidence has been so far provided showing unambiguously that periodicity in the reoccurring synchronizing wave is also required. As stated above, an attractive cellular process that might reinforce synchronous transmission of converging inputs consists in the selective ampUfication of sensory responses when the stimulus appearing in the central part of the receptive field differs strongly from that presented in the periphery ("pop-out effect") and evokes already a subthreshold change

184 in the polarization state of the neuron, favorable for the activation of boosting conductances. According to the literature reviewed in section 6, this would allow the convergence and the synchronization of various inputs during a temporal window of 100 to 300 ms . As alluded to earlier, particular predictions can be applied to the filtering ability of visual cortical cells in response to (normally subliminal) contextual input presented in the periphery at the same time as they are depolarized for instance by the presentation of the sensory stimulus in a central part of the receptive field. Our intracellular subthreshold data can be compared to the recent observations of "pop-out" effects in visual cortical receptive fields where two types of protocols have been used: Knierim and Van Essen (1992) varied the orientation of the texture of the background while stimulating the MDF with a fixed orientation, and Sillito and colleagues used bipartite stimuli where the outer grating had an orientation orthogonal to that of the inner disk covering the center of the receptive field (Sillito et al., 1995). The latter group recorded from cells whose level of firing became selectively enhanced when the two orientations (central and peripheral) differed by 90° independently of their absolute values (configuration D in Figure 11). The fact that the level of firing for the bipartite stimulus became higher than that evoked by the presentation of the optimal inner grating alone (configuration A in Figure 11) suggests that the optimal feature encoded by some cortical cells is not the orientation per se shown in the MDF but the contrast in orientation between center and periphery. A plausible regulatory mechanism derived from our experiments could be that the transmission of the peripheral input, which is by itself mainly excitatory (when remaining within the tuning subthreshold preference of the cell), benefits from the sudden concomitant presentation of the stimulus in the central part of the receptive field. This boosting effect would especially predominate when the inner stimulus is cross-oriented or distinct from the preferred orientation of the cell, i.e. when by itself it evokes only a subliminal depolarization, equivalent in our case to subthreshold somatic current injection (Figure 10). In contrast, stimulus co-alignment across the visual field when the central input has the same (preferred) orientation as that of the peripheral stimulus, might trigger (as we have observed) a nonlinear suppressive interaction, responsible for hypercomplexity, and mediated by intracortical inhibitory interneurons. The reduction in postsynaptic firing of hypercomplex cells would be expressed only when the total excitatory drive, provided by the sum of the feedforward and lateral inputs, exceeds the threshold needed to fire the inhibitory interneuron (Si, third column in Figure 11). The spiking threshold is assumed to require a higher level of convergence in inhibitory cells (Si) than in excitatory cells (Se). These two simple mechanisms (inhibitory interneuron, difference in spiking thresholds), which were in part already suggested by Sillito (1977), would result in a differential increase in the

185 response, thus allowing a cell to detect local orientation contrast between its preferred orientation and that shown in the peripheral field (Figure 11). Related models of a biphasic control of the cortical response gain as a function of the global drive (feedforward, recurrent, lateral) have been proposed (Somers et al., 1994; Stemmling et al., 1995). However their predictions do not reproduce some of Sillito's findings, in particular where higher levels of discharge may be observed in response to high contrast cross-oriented bipartite stimuli as compared to that evoked by the optimal stimulation of the MDF alone. Figure 11 illustrates an hypothesis of a cortical circuit for the detection of orientation contrast. The orientation contrast selective cell is assumed here to be a second-order neuron. This cell receives a much stronger excitation from the feedforward input than from the lateral input, because of initial amplification of feedforward input by first-order cells. The lateral connectivity agrees with our observation of combined excitatory and inhibitory subthreshold input originating from the periphery of the MDF. All cortical cells are assumed to amplify their own response through recurrent excitatory and inhibitory local circuits of the type described by Douglas and colleagues (1995). As in previous models (Sommers et al., 1994; Stemmler et al., 1995) the convergence of feedforward and lateral excitatory drive is required to fire a specific class of inhibitory interneurons. Their suprathreshold activation (which occurs mostly for stimulus C in Figure 11) will induce a down-regulation of the cortical cell's response to an optimally oriented grating covering the full extent of the visual field. In order to account for the facilitating effect of cross-oriented stimuli, we make the specific assumption that the AND-gate interneurons are activated below threshold for only one type of input (feedforward in A, and lateral in B), and below or near threshold for orthogonal orientations shown simultaneously in the center and the periphery of the RF (cross-oriented bipartite stimulus in D, see Figure 11). A possible way of achieving this requirement independently of the absolute orientations of the gratings is to further assume that, in contrast to most cortical cells, feedforward and lateral inputs provide a similar subthreshold orientation tuned contribution to the AND-gate intemeuron. The sum of these two similar tuning curves shifted one from each other by 90° (cross-oriented stimulation) may be considered as poorly oriented (with an appropriate scaling in their width of tuning) and kept below or near the threshold of activation of the interneuron. Consequently, as long as the threshold of the inhibitory intemeuron is not trespassed, the recurrent circuitry would provide a higher level of response to the bipartite configuration in D than that evoked in the same cell by the preferred stimulus restricted to the MDF (configuration C), and this for three reasons:

186 - i) the AND-gate circuit is poorly activated by the cross-oriented stimuli, - ii) the orientation contrast selective cell receives in addition to the feedforward input an excitatory lateral drive, - iii) the temporal association of the inner / outer stimuli recruits the boosting of synaptic gain suggested by our own experiments. Comparison between response levels evoked by configurations C and D still predicts a larger response for D than for C for nonoptimal stimuli, but quantitative simulations have to be made taking into account the dependency of the boosting mechanism itself on pre- and postsynaptic parameters. In addition, in the case of cross-oriented stimuli where an optimal orientation is shown in the MDF, it is conceivable that the addition of a lateral drive will overcome the Si threshold and trigger modest activity in the interneuron, thus providing a negative regulation of the peak response. This, in turn, would result in making the output of the cell less dependent on the absolute orientation of that shown in the central part of the RF. Although such models put a strong emphasis on an apparent dichotomy between horizontal (lateral) and vertical (feedforward) cortical connectivity, it remains to be ascertained to which degree such a clear cut separation is indeed present. A first series of argument is based on neuroanatomical evidence: the pattern of terminals formed by axons of spiny stellate first-order cells and of layer n/in pyramidal cells show a spatial spread

Figure 11. A cortical circuit for the detection of orientation contrast. Top, schematic view of the microcircuit. Excitatory cells (E-cells) are symbolized by empty triangles, and inhibitory cells (I-cells) by filled black circles. The recorded cell (top) is selective to orientation contrast. Bottom, input and output activities corresponding to the E- and I-cells in response to different configurations of stimulation centered on the aggregate RF of the cortical column. In A, only feedforward pathways (and consequently recurrent local intracortical loops) are activated by restricting the presentation of an oriented stimulus to the MDF of the recorded E-cell. In B, only lateral activation is provided to the same cell by presenting an oriented stimulus in the "unresponsive" periphery of its RF. In C, both central and surround stimuli of the same orientation are presented simultaneously and coaligned without spatial offset or intermediate gap. In D, central and surround stimuli still cover the whole visual field, but are cross-oriented, i.e. their respective orientations differ by 90°. The two top rows of diagrams correspond to the orientation tuning curves of inputs (synaptic conductance) to the recorded E-cell (upper row) and to the inhibitory interneuron (lower row). Both lateral and feedforward inputs are assumed to show the same orientation bias (OPT: optimal orientation). Se and Si stand for the postsynaptic firing threshold of the excitatory and inhibitory neurons, respectively, the former being reached for lower absolute global input levels than the latter. The two bottom row diagrams correspond to the orientation tuning curves of the outputs (mean spike frequency) of the same two cells. See text for details.

187 RECURRENT \ LATERAL J INPUT RECURRENT

FEEDFORWARD INPUT

D

B lllililllii iiiiiiiiii FEED-FORWARD

-90° OPT +90° ORIENTATION

LATERAL

OPT

ISO-ORIENTED

CROSS-ORIENTED

OPT

OPT

n contained within an hypercolumn, on the order of 500 jim (Douglas, et al., 1995). But the distribution is clearly bimodal with one set of boutons forming proximal intracolumnar contacts (within 300 |im) and a second one connecting more distant columns possibly analyzing the same part of the visual field. In contrast, the disinhibitory field appears more continuously distributed over larger distances in the order of one millimeter, at least in area 18 (Kisvarday, et al., 1993). This pattern of connectivity is further complicated by the dendritic coverage of supragranular pyramids, which extends over 200 [im and whose overlap increases diversity in sampling (Malach, 1994). Another type of evidence is gained from the electrophysiological comparison of the postsynaptic effects evoked respectively by intracolumnar (or radial) and lateral activation. A first issue concerns the EPSP/IPSP balance in the evoked response. Supragranular pyramids show systematic IPSPs in response to intracolumnar or radial feedforward stimulation when the cell is sufficiently depolarized, in vitro (Jones and Baughman, 1988, Sutor and Hablitz, 1989a) and in vivo (Douglas, et al., 1991). This effect is not as pronounced when the stimulation is lateral, either located in layer 1 (CauUer and Connors, 1994; but see Nakajima, et al., 1988) or in layer 2/3 (Hirsch and Gilbert, 1991): IPSPs are sometimes absent even when applying strong stimulation intensity, but when present, are of polysynaptic origin (Hirsch and Gilbert, 1991). It has also been observed that the intensity threshold required to spike an action potential is higher for lateral stimulations than for radial ones. It is not yet clear if these conclusions also apply to layer 5 cells (CauUer and Connors, 1994; Nakajima, et al. 1988). A third issue concerns putative excitatory transmitters and receptors. Both AMPA and NMDA types of receptors mediate responses to glutamate after radial stimulation (Jones and Baughman, 1988; Sutor and Hablitz, 1989b). However, lateral depolarizing responses seem to be largely dominated by AMPA receptor and almost devoid of NMDA mediated responses (Hirsch and Gilbert, 1991; Murakoshi et al., 1993; Cauller and Connors, 1994; but see Nakajima, et al., 1988). Cauller and Connors have however observed APV sensitive, long-latency EPSPs in response to lateral stimulation. A last issue concerns the enhancement of EPSPs through the activation of inward rectification. This phenomenon, which is probably due to a persistent sodium channel, has been observed both for radial EPSPs (Sutor and Hablitz, 1989b) and laterally evoked EPSPs (Hirsch and Gilbert, 1991). Indeed, the probable perisomatic location of the responsible sodium current (Stuart and Sakmann, 1995) would tend to make this type of enhancement rather unspecific with respect to the input.

189 In conclusion, in spite of an unclear status, a trend in computational neuroscience is to attribute distinct roles to lateral and feedforward input and include non-linearities brought upon by local reverberating activity into a Meta-neuron replacing the considered cell and its local environment. Apparently further work is needed to justify a dissection of associative properties in neuronal integration by compartmentalizing the incoming input into two or three distinct classes of afferent connections. Nevertheless a new Receptive Field concept seems to emerge from an increasing wealth of data gathered at the cellular and neuroanatomical level, which suggest the existence of a critical frontier separating the Platonical RF center from its peripheral association shrine. Discontinuities in contrast, orientation, colinearity of contours shown on each side of this still hypothetical boundary would trigger non-linear discrimination between what the "inner eye" of the cortical cell sees and the contextual prediction established by the rest of the network and transmitted by lateral connections.

Acknowledgments: V/e thank Ralph Freeman for providing stimulus generation algorithms, AdAertsen for sharing his expertise on reverse correlation techniques. The electrophysiological in vivo work reviewed in this chapter was done in collaboration with Cyril Monier, Frederic Chavane, Larry Glaeser Jean Lorenceau, Attila Baranyi, Dominique Debanne and Daniel Shulz. Research was funded by grants to Y.F.from the CNRS (ATIPE Cognisciences), HFSP (RG-69/93) and the Conseil de I'Essonne. V.B. was supported during his PhD. by MRES, Fondation Fouassier and Fondation des Aveugles de France fellowships. Off-line data analysis was performed with a specialized home-made program Acquisl (developed by Gerard Sadoc and commercialized by ANVAR CNRS, Dipsi) and further processed using MatLab Signal Processing Library. Thanks are due to Dr. Lyle Borg-Graham and Cyril Monier for helpful comments on the manuscript, and to Dr. Kirsty Grant for help with the English.

190 REFERENCES: Abeles, M. (1982). "Local cortical circuits. An electrophysiological study." (New-York: Springer-Verlag), 101. Abeles, M. (1991). "Corticonics: neuronal circuits of the cerebral cortex." (Cambridge: Cambridge University Press), 280. Adrian, E.D. (1935). "The mechanism of nervous action." (Philadelphia: University of Pennsylvania Press), Adrian, E.D. (1941). Afferent discharges to the cerebral cortex from peripheral sense organs. J. Physiol. (Lond.). 100,159-191. Aertsen, A.M.H.J., Gerstein, G.L., Habib, M.K. and Palm, G. (1989). Dynamics of neuronal firing correlation: modulation of "effective connectivity". J. Neurophysiol. 61, 900917. Agmon-Snir, H. and Stgew, I. (1993). Signal delay and input synchronization in passive dendritic structures. J. Neurophysiol. 70, 2066-2085. Albus, K. (1975). A quantitative study of the projection area of the central and the paracentral visual field in area 17 of the cat. I. The precision of the topography. Exp. Brain Res. 24, 181-202. Anderson, J.C, Douglas, R.J., Martin, K.A.C. and Nelson, J.C. (1994). Map of the synapses formed with the dendrites of spiny stellate neurons of cat visual cortex. J. Comp. Neurol. M l , 25-38. Ballard, D.H. (1986). Cortical connections and parallel processing: structure and function. Behav. Brain Sci. 9,67-120. Baranyi, A,, Debanne, D., Shulz, D. and Fregnac, Y. (1991). Postsynaptic membrane potential regulates potentiation and depression of visually evoked synaptic potentials in kitten cortical neurons recorded in vivo. Soc. Neurosci. Abstr. 17, 1470. Barlow, H.B. (1953). Summation and inhibition in the frog's retina. J. Physiol. (Lond.). 119. 69-88. Barlow, H.B. (1972). Single units and sensation: a neurone doctrine for perceptual psychology? Perception. 1, 371-394. Barlow, H.B., Blakemore, C. and Pettigrew, J.D. (1967). The neural mechanism of binocular depth discrimination. J. Physiol. (Lond.). 193. 327-432. Barnes, C , Baranyi, A., Bindman, L., Dudai, Y., Fr6gnac, Y., Ito, Y., Knopfel, T., Lisberger, S.G., Moulins, M., Morris, R.G.M., Movshon, J.A., Singer, W. and Squire, L.R. (1994). Group report: relating activity-dependent modifications of neuronal function to changes in neural systems and behavior. In "Cellular and Molecular Mechanisms Underlying Higher Neural Functions", eds. A. Selverston and P. Ascher J. Wiley and Sons), 81-110. Berman, N.J., Douglas, R.J. and Martin, K.A.C. (1989). The conductances associated with inhibitory postsynaptic potentials are larger in visual cortical neurones in vitro than in similar neurones in intact, anaesthetized rats. J. Physiol. (London). 418, 107. Berman, N.J., Douglas, R.J., Martin, K.A.C. and Whitteridge, D. (1991). Mechanisms of inhibition in cat visual cortex. J. Physiol. (Lond.). 440, 697-722. Bernander, O., Koch, C. and Douglas, R.J. (1994). Amplification and linearization of distal synaptic input to cortical pyramidal cells. J. Neurophysiol. 72, 2743-2753. Bianchi, A.L., Grelot, L., Iscoe, S. and Remmers, J.E. (1988). Electrophysiological properties of rostral medullary respiratory neurones in the cat: an intracellular study. J. Physiol. (London). 407, 293-310. Bienenstock, E., Cooper, L.N. and Munro, P. (1982). Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32-48, Bienenstock, E. and Doursat, R. (1989). Elastic matching and pattern recognition in neural networks. In "Neural networks: from models to applications.", eds. L. Personnaz and G. Dreyfus (Paris: IDSET), 472-482.

191 Bishop, P.O., Coombs, J.S. and Henry, G.H. (1971). Response to visual contours: spatio temporal aspects of excitation in the receptive fields of single striate neurons. J. Physiol. (Lond.). 219, 625-657. Bishop, P.O., Coombs, J.S. and Henry, G.H. (1973). Receptive fields of simple cells in the cat striate cortex. J. Physiol. 221, 31-60. Bishop, P.O. and Henry, G.H. (1972). Striate neurons: Receptive field concepts. Invest. Ophthahnol. U , 346-354. Blakemore, C. and Tobin, E.A. (1972). Lateral inhibition between orientation detectors in the cat's visual cortex. Exp. Brain Res. 15,439-440. Bolz, J. and Gilbert, CD. (1986). Generation of end-inhibition in the visual cortex via interlaminar connections. Nature. 320, 362-365. Borg-Graham, L. (1991). Modelling the non-linear conductances of excitable membranes. In "Cellular Neurobiology.", Eds. J, Chad and H. Wheal (IRL Press at Oxford University Press), 247-275. Borg-Graham, L. and Grzywacs, N.M. (1992). A model of the direction selectivity circuit in retina: transformations by neurons singly and in concert. In "Single Neuron Computation.", eds. T. McKenna, J. Davis and S.F.Zornetzer (Academic Press), 347375. Borg-Graham, L., Monier, C, Bringuier, V. and Fregnac, Y. (1995). Analysis of simple and complex synaptic input in cat area 17 with in-vivo whole-cell patch recordings. Soc. Neurosc. Abstr. 21, 17.1, p.21. Boven, K.H. and Aertsen, A. (1990). Dynamics of activity in neuronal networks give rise to fast modulations of functional connectivity. In "Parallel Processing in Neural Systems and Computers", eds. R. Eckmiller, G. Hartmann and G. Hauske (Elsevier Science Publishers), 53-56. Bringuier, V. (1995). Oscillations et integration neuronale dans le cortex visuel primaire. Doctorat d'Universite, Universite Paris VI, Sciences Cognitives, Bringuier, V., Chavane, P., Monier, C , Glaeser, L., Fregnac, Y. and Lorenceau, J. (1995). Role of voltage-dependent inactivating conductances in the control of the visual integration field profile in cat area 17. Soc. Neurosc. Abstr. 21, 1647. Bringuier, V., Fregnac, Y., Baranyi, A., Debanne, D. and Shulz, D. (submitted). Synaptic origin and stimulus dependency of neuronal oscillatory activity in the primary visual cortex of the cat. Bringuier, V., Fr6gnac, Y., Debanne, D., Shulz, D. and Baranyi, A. (1992). Synaptic origin of rhythmic visually evoked activity in kitten area 17 neurons. NeuroReport. 3., 10651068. Casanova, C. (1993). Responses of cells in cat's areal7 to random dot patterns: influence of stimulus size. NeuroReport. 4, 1011-1014. Casanova, C , Nordman, J.P. and Molotchnikoff, S. (1991). Le complexe noyau lateral post6rieur. Pulvinar des mammiferes et la fonction visuelle. J. Physiol. (Paris). 85» 4457. CauUer, L.J. and Connors, B.W. (1994). Synaptic physiology of horizontal afferents to layer I in slices of rat SI neocortex. J. Neurosci. 14,751-762. Christie, B.R., Eliot, L.C., Ito, K.-L, Miyakawa, H. and Johnston, D. (1995). Different Ca^"' channels in soma and dendrites of hippocampal pyramidal neurons mediate spikeinduced Ca^"" influx. J. Neurophysiol. 22* 2253-2557. Cleland, B.G., Dubin, M.W. and Levick, W.R. (1971). Sustained and transient neurones in the cat's retina and lateral geniculate nucleus. J. Neurophysiol. 217.473-496. Cotterill, R.M.J, and Nielsen, C. (1991). A model for cortical 40 Hz oscillations invokes inter-area interactions. NeuroReport. 2, 289-292. Creutzfeldt, O., Innocenti, G.M. and Brooks, D. (1974). Vertical organization in the visual cortex (area 17) in the cat. Exp. Brain Res. 21, 315-336. Creutzfeldt, O. and Ito, M. (1968). Functional synaptic organization of primary visual cortex neurones in the cat. Exp. Brain Res. 4 324-352.

192 Creutzfeldt, O.D., Kuhnt, V. and Benevento, L.A. (1974). An intracellular analysis of visual cortical neurons to moving stimuli: responses in a co-operative neuronal network. Exp. Brain Res. 21, 251-274. Crick, F.H.C. (1984). Function of the thalamic reticular complex: the searchlight hypothesis. Proc. Natl. Acad. Sci. USA. 81,4586-4590. Cudeiro, J. and Sillito, A.M. (1996). Spatial frequency tuning of orientation-discontinuitysensitive corticofugal feedback to the cat lateral geniculate nucleus. J. Physiol. (London). 49Q, 481-492. Damasio, A.R. (1989). The brain binds entities and events by multiregional activation from convergence zones. Neural Comput. 1,123-132. De Angelis, G.C., Freeman, R.D. and Ohzawa, I. (1994). Length and width tuning of neurons in the cat's primary visual cortex. J. Neurophysiol. 21, 347-374. De Angelis, G.C., Ohzawa, L and Freeman, R.D. (1995). Receptive-field dynamics in the central visual pathways. T.I.N.S. 18,451-458. Debanne, D., Guerineau, N.C., Gahwiler, B.H. and Thompson, S.M. (1996). Paired-pulse facilitation and depression at unitary synapses in rat hippocampus: quantal fluctuation affects subsequent release. J. Physiol. (London). 490,713-727. Debanne, D., Shulz, D. and Fregnac, Y. (1995). Temporal constraints in associative synaptic plasticity in hippocampus and neocortex. Canad. J. Physiol. Pharmacol. 73, 1295-1311. Debanne, D., Shulz, D. and Fr6gnac, Y. (submitted). Activity-dependent regulation of ON and OFF responses in cat visual cortical neurons. Deisz, R.A., Fortin, G. and Zieglgansberger, W. (1991). Voltage dependence of excitatory postsynaptic potentials of rat neocortical neurons. J. Neurophysiol. 65, 371-382. Deppisch, J., Bauer, H.U., Schillen, T., Konig, P., Pawelzik, K. and Geise, T. (1993). Alternating oscillatory and stochastic states in a network of spiking neurons. Network. 4, 243-257. Diesmann, M., Gewaltig, M. and Aertsen, A. (1995). Characterization of synfire activity by propagating 'pulse packets'. Caltech: 1-6. Douglas, R.J., Koch, C , Mahowald, M.A., Martin, K.A.C. and Suarez, H.H. (1995). Recurrent excitation in neocortical circuits. Science. 269, 981-985. Douglas, R.J., Martin, K.A.C. and Whitteridge, D. (1988). Selective responses of visual cortical cells do not depend on shunting inhibition. Nature. 332, 642-644. Douglas, R.J., Martin, K.A.C. and Whitteridge, D. (1991). An intracellular analysis of the visual responses of neurones in cat visual cortex. J. Physiol. (Lond.). 440, 659-696. Dreifuss, J.J., Kelly, J.S. and Krnjevic, K. (1969). Cortical inhibition and gammaaminobutyric acid. Exp. Brain Res. 9, 137-154. Eckhorn, R., Bauer, R., Jordan, W., Brosch, M., Kruse, W., Munk, M. and Reitbock, H.J. (1988). Coherent oscillations: A mechanism of feature linking in the visual cortex? Biol. Cyb. 60, 121-130. Eggermont, J.J., Johannesma, P.LM. and Aertsen, A.M.H.J. (1983). Reverse-correlation methods in auditory research. Quart. Rev. Biophys. 16, 341-414. Engel, A.K., Konig, P., Kreiter, A.K., Schillen, T.B. and Singer, W. (1992). Temporal coding in the visual cortex: new vistas on integration in the nervous system. Trends Neurosci. 15, 218-226. Ezur6, K. (1990). Synaptic connections between medullary respiratory neurons and considerations on the genesis of respiratory rythm. Prog. Neurobiol. 25,429-450. Ferster, D. (1986). Orientation selectivity of synaptic potentials in neurons of cat primary visual cortex. J. Neurosci. 6, 1284-1301. Ferster, D. (1988). Spatially opponent excitation and inhibition in simple cells of the cat visual cortex. J. Neurosci. 8,1172-1180. Ferster, D. and Lindstrom, S. (1983). An intracellular analysis of geniculo-cortical connectivity in area 17 of the cat. J. Physiol. (London). 342. 181-215. Fetz, E., Toyama, K. and Smith, W. (1991). Synaptic interaction between cortical neurons. In "Cerebral Cortex", eds. A. Peters and E. G. Jones (New-York: Plenum Press), 9, 147.

193 Fiorani, M.J., Rosa, M.G.P., Gattas, R. and Rocha-Miranda, C.E. (1992). Dynamic surrounds of receptive fields in primate striate cortex: a physiological basis for perceptual completion? Proc. Natl. Acad. Sci. USA. 89, 8547-8551. Fischer, B. and Krtiger, J. (1974). The shift-effect in the cat's lateral geniculate neurons. Exp. Brain Res. 21,225-227. Fischer, B., Krtiger, J. and Droll, W. (1975). Quantitative aspects of shift effect in cat retinal ganglion cells. Brain Res. 83, 391-403. Fortin, G. (1993). Le r6seau neuronal du complexe solitaire: roles respectifs des connexions synaptiques, des propri^tes mebranaires et du m^tabolisme intracellulaire. These de Doctorat de rUniversit6 Paris VI, University Paris VI, Fr^gnac, Y. (1995). Comparative and developmental aspects of hebbian synaptic plasticity. In "Handbook of Brain Theory and Neural networks", eds. M. Arbib (MIT Press), 459-464. Fr6gnac, Y., Bringuier, V. and Baranyi, A. (1994a). Oscillatory neuronal activity in visual cortex: a critical re-evaluation. In "Temporal Coding in the Brain, Research and Perspectives in Neurosciences", eds. G. Buzsaki, R. Llinas, W. Singer, A. Berthoz and Y. Christen (Berlin: Springer-Verlag), 81-102. Fr6gnac, Y., Burke, J., Smith, D. and Friedlander, M.J. (1994b). Temporal covariance of pre and postsynaptic activity regulates functional connectivity in the visual cortex. J. Neurophysiol. 71, 1403-1421. Fregnac, Y. and Debanne, D. (1993). Potentiation and depression in visual cortical neurons: a functional approach to synaptic plasticity. In "Brain Mechanisms of Perception and Memory: from Neuron to Behavior", eds. T. Ono, L. Squire, M. E. Raichle, D. I. Perrett and M. Fukuda (Oxford University Press), 533-561. Fr6gnac, Y., Debanne, D., Shulz, D. and Baranyi, A. (1994c). Does membrane potential regulate functional plasticity in kitten visual cortex? In "Long-term Potentiation", eds. M. Baudry and J. L. Davis (Cambridge: MIT Press), 2, 227-264. Fr6gnac, Y. and Shulz, D. (1994). Models of synaptic plasticity and cellular analogs of learning in the developing and adult vertebrate visual cortex. In "Advances in Neural and Behavioral Development", eds. V. Casagrande and P. Shinkman (New Jersey: Neural Ablex Publ), 4,149-235. Fregnac, Y., Shulz, D., Thorpe, S. and Bienenstock, E. (1988). A cellular analogue of visual cortical plasticity. Nature. 333, 367-370. Fr6gnac, Y., Shulz, D., Thorpe, S. and Bienenstock, E. (1992). Cellular analogs of visual cortical epigenesis: I. Plasticity of orientation selectivity. J. Neurosci. 12, 1280-1300. Gilbert, CD. (1977). Laminar differences in receptive field properties of cells in cat primary visual cortex. J. Physiol. (London). 268, 391-421. Gilbert, CD. (1992). Horizontal integration and cortical dynamics. Neuron. 9, 1-13. Glaeser, L., Bringuier, V., Fregnac, Y., Borg-Graham, L., Monier, C and Fleury, G. (1995). Reverse correlation mapping of subthreshold synaptic potentials in cat primary visual cortex. Soc. Neurosc. Abstr. 21, 648.1, p. 1647. Gray, CM., Konig, P., Engel, A.K. and Singer, W. (1989). Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties. Nature. 338, 334-337. Grieve, K.L. and Sillito, A.M. (1991). A re-appraisal of the role of layer VI of the visual cortex in the generation of cortical end inhibition. Exp. Brain Res. 87, 521-529. Grinvald, A., Lieke, E.E., Frostig, R.D. and Hildesheim, R. (1994). Cortical point-spread function and long-range lateral interactions revealed by real-time optical imaging of macaque monkey primary visual cortex. J. Neurosci. 14, 2545-2568. Hansel, D. and Sompolinsky, H. (1992). Synchronization and computation in a chaotic neural network. Phys. Rev. Lett. 68,718-721. Hartline, H.K. (1938). The responses of single optic nerve fibers of the vertebrate eye to illumination of the retina. Am. J. Physiol. 212,400-415. Hartline, H.K. (1940). The receptive fields of the optic nerve fibers. Amer. J. Physiol. 130, 690-699.

194 Hebb, D.O. (1949). The organization of behavior. (New-York: J. Wiley and Sons), 337 p. Henry, G.H., Goodwin, A.W. and Bishop, P.O. (1978). Spatial summation of responses in receptive fields of single cells in cat striate cortex. Exp. Brain Res. 32, 245-266. Hirsch, J.A., Alonso, J.M. and Reid, R.C. (1995). Visually evoked calcium action potentials in cat stiate cortex. Nature. 378, 612-616. Hirsch, J.A. and Gilbert, CD. (1991). Synaptic physiology of horizontal connections in the cat's visual cortex. J. Neurosci. H , 1800-1809. Hoffmann, K.P. and Stone, J. (1971). Conduction velocity of afferents to cat visual cortex: a correlation with cortical receptive field properties. Brain Res. 32,460-466. Hubel, D.H. (1988). Eye, brain and vision. (Scientific American Library), Hubel, D.H. and Wiesel, T.N. (1962). Receptive fields, binocular interaction and functional architecture in the cat's visual cortex. J. Physiol. (London). 160. 106-154. Hubel, D.H. and Wiesel, T.N. (1965). Receptive field and functional architecture in two nonstriate visual areas (18 and 19) of the cat. J. Neurophysiol. 2S, 229-289. Hughes, CM. and Peters, A. (1992). Symmetric synapses formed by callosal afferents in rat visual cortex. Brain Res. 583, 271-278. Ikeda, H. and Wright, M.J. (1972). Functional organization of the periphery effect in retinal gangHon cells. Vis. Res. 12, 1857-1879. Innocenti, G.M. and Fiore, L. (1974). Post-synaptic inhibitory components of the responses to moving stimuli in area 17. Brain Res. 80, 122-126. Jagadeesh, B., Gray, CM. and Ferster (1992). Visually evoked oscillations of membrane potential in cells of cat visual cortex. Science. 257, 552-554. Jagadeesh, B., Wheat, H.S. and Ferster, D. (1993). Linearity of summation of synaptic potentials underlying direction selectivity in simple cells of the cat visual cortex. Science. 262,1901-1904. Jones, H.E. and Sillito, A.M. (1994). Directional asymmetries in the length-response profiles of cells in the feline dorsal lateral geniculate nucleus. J. Physiol. (London). 479, 475486. Jones, J.P. and Palmer, L.A. (1987). The two-dimensional spatial structure of simple receptive fields in cat striate cortex. J. Neurophysiol. 58,1187-1211. Jones, K.A. and Baughman, R.W. (1988). NMDA- and non-NMDA-receptor components of excitatory synaptic potentials recorded from cells in layer V of rat visual cortex. J. Neurosci. 8, 3522-3534. Kapadia, M.K., Ito, M., Gilbert, CD. and Westheimer, G. (1995). Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in VI of alert monkeys. Neuron. 15, 843-856. Kisvarday, Z.F., Beaulieu, C and Eysel, U.T. (1993). Network of GABAergic large basket cells in cat visual cortex (area 18): implication for lateral disinhibition. J. Comp. Neurol. 327, 398-415. Knierim, J.J. and Van Essen, D.C (1992). Neuronal responses to static texture patterns in area VI of the alert macaque monkey. J. Neurophysiol. 67, 961-980. Koechlin, E. and Burnod, Y. (1995). Dual population coding in the neocortex: a model of interaction between representation and attention in the visual cortex. J. Cogn. Neurosc. in press. Komatsu, Y., Nakajima, S. and Toyama, K. (1991). Induction of long-term potentiation without participation of N-methyl-D-aspartate receptors in kitten visual cortex. J. Neurophysiol. 65, 20-32. Kriiger, J. and Aiple, F. (1988). Multimicroelectrode investigation of monkey striate cortex: spike train correlations in the infragranular layers. J. Neurophysiol. 60,798-828. Kuffler, S.W. (1953). Discharge patterns and functional organization of the mammalian retina. J. Neurophysiol. 16, 37-68. Li, CY. and Li, W. (1994). Extensive integration field beyond the classical receptive field of cat's striate cortical neurons: Classification and tuning properties. Vision Res. 34. 2337-2355.

195 Linsker, R. (1986a). From basic network principles to neural architecture: emergence of spatial opponent cells. Proc. Natl. Acad. Sci. USA. 83,7508-7512. Linsker, R. (1986b). From basic network principles to neural architecture: emergence of orientation selective cells. Proc. Natl. Acad. Sci. USA. 82, 8390-8394. Linsker, R. (1986c). From basic network principles to neural architecture: emergence of orientation columns. Proc. Natl. Acad. Sci. USA. 83, 8779-8783. Llinas, R. (1988). The intrinsic electrophysiological properties of mammalian neurons: insights into central nervous system function. Science. 242, 1654-1664. Llinas, R.R., Grace, A.A. and Yarom, Y. (1991). In vitro neurons in mammalian cortical layer 4 exhibit intrinsic oscillatory activity in the 10- to 50-Hz frequency range. Proc. Natl. Acad. Sci. USA. 88, 897-901. Mac Kay, D.J.C. (1995). Bayesian methods for supervised neural networks. In "The Handbook of Brain Theory and Neural Networks", ed. M. A. Arbib (Cambridge: The MIT Press), 144-149. Maffei, L. and Fiorentini, A. (1976). The unresponsive regions of visual cortical receptive fields. Vision Res. 16,1131-1139. Magee, J.C. and Johnston, D. (1995a). Characterization of single voltage-gated Na"^ and Ca^"*" Channels in apical dendrites of rat CAl pyramidal neurons. J. Physiol. (London). 487.67-90. Magee, J.C. and Johnston, D. (1995b). Synaptic activation of voltage-gated channels in the dendrites of hippocampal pyramidal neurons. Science. 268. 301-304. Magleby, K.L. (1987). Short-term changes in synaptic efficacy. In "Synaptic Function", eds. G. M. Edelman, W. E. Gall and W. M. Cowan (New-York: J. Wiley), 21-56. Malach, R. (1994). Cortical columns as devices for maximizing neuronal diversity. Trends Neurosc. 12,101-104. Malach, R., Tootell, R.B.H. and Malonek, D. (1994). Relationship between orientation domains, cytochrome oxidase stripes, and intrinsic horizontal connections in squirrel monkey area v2. Cereb. Cortex. 4,151-165. Markram, H., Helm, P.J. and Sakmann, B. (1995). Dendritic calcium transients evoked by single back-propagating action potentials in rat neocorical pyramidal neurons. J. Pysiol. 4SS, 1-20. Marrocco, R.T., Mc Clurkin, J.W. and Young, R.A. (1982). Modulation of lateral geniculate nucleus cell responsiveness by visual activation of the corticogeniculate pathway. J. Neurosci. 2, 256-263. Martin, K.A.C. and Whitteridge, D. (1984). The relationship of receptive field properties to the dendritic shape of neurones in the cat striate cortex. J. Physiol. (London). 356. 291302. Marty, A. and Llano, I. (1995). Modulation of inhibitory synapses in the mammalian brain. Curr. Op. Biol. 5, 335-341. Mason, A., NicoU, A. and Stratford, K. (1991). Synaptic transmission between individual pyramidal neurons of rat visual cortex in vitro. J. Neurosci. i l , 72-84. Mc Clurkin, J.W., Optican, L.M. and Richmond, B.J. (1994). Cortical feedback increases visual information transmitted by monkey parvocellular lateral geniculate neurons. Vis. Neurosc. 11, 601-607. Mc Cormick, D.A., Connors, B.W., Lighthall, J.W. and Prince, D.A. (1985). Comparative electrophysiology of pyramidal and sparsely spiny stellate neurons of the neocortex. J. Neurophysiol. 54,782-806. Mc Cormick, D.A., Gray, C. and Wang, Z. (1993). Chattering cells: a new physiological subtype which may contribute to 20-60 hz oscillations in cat visual cortex. Soc Neurosci. Abstr. 12, 869. Mc Donald, C.T. and Burkhalter, A. (1993). Organization of long-range inhibitory connections within rat visual cortex. J. Neurosc. H , 768-781. Mc lUwain, J.T. (1964). Receptive field of optic tract axons and lateral geniculate cells: peripheral extent and barbiturate sensitivity. J. Neurophysiol. 22, 1154-1173.

196 Merzenich, M.M. and Sameshima, K. (1993). Cortical plasticity and memory. Curr. Opin. Neurobiol. 2,187-196. Miller, K.D. (1990). Correlation based models of neural development. In "Neuroscience and Connectionist", eds. M. A. Gluck and D. E. Rumelhart (Hillsdale: L. Erlbaum), 267353. Miller, K.D., Chapman, B. and Stryker, M.P. (1989). Visual responses in adult cat visual cortex depend on N-methyl-D-aspartate receptors. Proc. Natl. Acad. Sci. USA. 86» 5183-5187. Milner, P.M. (1974). A model for visual shape recognition. Psychol. Rev. 81, 521-535. Molotchnikoff, S. and Cerat, A. (1992). Responses from outside classical receptive fields of dorsal lateral geniculate cells in rabbits. Exp. Br. Res. 92, 94-104. Mountcastle, V.B. (1957). Modality and topographic properties of single neurons of cat's somatic sensory cortex. J. Neurophysiol. ^ , 408-434. Movshon, A.J., Adelson, E.H., Gizzi, M.S. and Newsome, W.T. (1986). The analysis of moving visual patterns. Pont. Acad. Sci. Scrip. Var. 54» 117-152. Movshon, J.A., Thompson, J.D. and Tolhurst, D.J. (1978). Spatial summation in the receptive fields of simple cells in the cat's striate cortex. J. Physiol. (Lond.). 283, 5377. Murakoshi, T., Quo, J.-Z. and Ichonose, T. (1993). Electrophysiological identification of horizontal synaptic connections in rat visual cortex in vitro. Neurosc. Lett. 163, 211214. Murphy, P.C. and Sillito, A.M. (1987). Corticofugal feedback influences the generation of length tuning in the visual pathway. Nature. 329, 727-729. Nakajima, S., Komatsu, Y. and Toyama, K. (1988). Synaptic action of layer I fibers on cells in cat striate cortex. Brain Res. 457, 176-180. Nelson, D.A. and Katz, L.C. (1995). Emergence of junctional cirvuits in ferret visual cortex visualized by optical imaging. Neuron. 15, 23-34. Nelson, J.I. and Frost, B.J. (1978). Orientation-selective inhibition from beyond the classic visual receptive field. Brain Res. 139, 359-365. Nelson, J.I. and Frost, B.J. (1985). Intracortical facilitation among co-oriented, co-axially aligned single cells in cat striate cortex. Exp. Brain Res. 61, 54-61. Nelson, J.I., Salin, P.A., Munk, M.H.J., Arzi, M. and Bullier, J. (1992). Spatial and temporal coherence in cortico-cortical connections: a cross-correlation study in areas 17 and 18 in the cat. Vis. Neurosci. 9, 21-37. Nelson, S., Toth, L., Sheth, B. and Sur, M. (1994). Orientation selectivity of cortical neurons during intracellular blockade of inhibition. Science. 265, 774-777. Nelson, S.B. (1991). Temporal interactions in the cat visual system: I. Orientation-selective suppression in the visual cortex. J. Neuroscience. H , 344-356. NicoU, A. and Blakemore, C. (1993). Single fibre EPSPs in layer 5 of rat visual cortex in vitro, NeuroReport. 4, 167-170. Nowak, G. (1995). Etude electrophysiologique des aspects temporels du traitement de I'information dans le n6ocortex visuel. Docorat de I'Universite, University Claude Bernard - Lyon I, Orban, G.A. (1984). Neuronal operations in the visual cortex. (Berlin Heidelberg: SpringerVerlag), 365 p. Orban, G.A., Gulyas, B. and Vogels, R. (1987). Influence of a moving textured background on direction selectivity of cat striate neurons. J. Neurophysiol. 52, 1792-1812. Orban, G.A., Kato, H. and Bishop, P.O. (1979). End-zone region in receptive fields of hypercomplex and other striate neurons in the cat. J. Physiol. (London). 42, 818-832. Palmer, L.A. and Davis, T.A. (1981). Receptive field structure in cat striate cortex. J. Neurophysiol. 46, 260-295. Pei, X., Vidyasagar, T.R., Volgushev, M. and Creutzfeldt, O.D. (1994). Receptive field analysis and orientation selectivity of postsynaptic potentials of simple cells in cat visual cortex. J. Neurosci. 14,7130-7140.

197 Pei, X., Volgushev, M., Vidyasagar, T.R. and Creutzfeldt, O.D. (1991). Whole-cell recording and conductance measurements in cat visual cortex in vivo. NeuroReport. 2, 485-488. Rauschecker, J.P., Grunau, M.W. and von Poulin, C. (1987). Centrifugal organization of direction preferences in the cat's lateral suprasylvian cortex and its relation to flow processing. J. Neurosc. 7,943-958. Rizzolati, G. and Camarda, R. (1977). Influence of the presentation of remote visual stimuli on visual responses of cat area 17 and lateral suprasylvian area. Exp. Brain Res. 29, 107-122. Rosenquist, A.C. (1985). Connections of visual cortical areas in the cat. In "Cerebral Cortex", eds. A. Peters and E. G. Jones (New York: Plenum), 3, 81-117. Salin, P.A. and BuUier, J. (1995). Corticocortical connections in the visual system: structure and function. Physiol, rev. 75, 107-154. Schillen, T.B. and Konig, P. (1991). Stimulus-dependent assembly formation of oscillatory responses: desynchronization. Neur. Comput. 3, 167-178. Schuster, H.G. and Wagner, P. (1990). A model for neuronal oscillations in the visual cortex. 1. Mean-field theory and derivation of the phase equations. Biol. Cyb. 64, 7782. Schwarz, C. and Bolz, J. (1991). Functional specificity of a long-range horizontal connection in cat visual cortex: a cross-correlation study. J. Neurosci. H , 2995-3007. Schweitzer, P., Fortin, G., Beloeil, J.C. and Champagnat, J. (1992). In vitro study of newbom rat brain maturation. Neurochem. Interntl. 20, 109-112. Schwindt, P.C. and Crill, W.E. (1995). Amplification of synaptic current by persistent sodium conductance in apical dendrite of neocortical neurons. J. Neurophysiol. 74, 2220-2224. Sejnowski, T.J. (1977a). Statistical constraints on synaptic plasticity. J. Theor. Biol. 69, 387389. Sejnowski, T.J. (1977b). Storing covariance with non-linearly interacting neurons. J. Math. Biol. 4, 303-321. Sejnowski, T.J. and Tesauro, G. (1989). The Hebb rule for synaptic plasticity: algorithms and implementations. In "Neural models of plasticity", eds. J. H. Byrne and W. O. Berry (Academic Press), 94-103. Shulz, D., Bringuier, V. and Fregnac, Y. (1993b). Complex-like structure of simple visual cortical receptive fields is masked by GABA^ intracortical inhibition. Soc. Neurosci. Abstr. 19, 638. Shulz, D. and Fregnac, Y. (1992). Cellular analogs of visual cortical epigenesis: H. Plasticity of binocular integration. J. Neurosci. 12, 1301 -1318. Sillito, A.M. (1975). The contribution of inhibitory mechanisms to the receptive field properties of neurones in striate cortex of the cat. J. Physiol. (London). 250, 305-329. Sillito, A.M. (1977). The spatial extent of excitatory and inhibitory zones in the receptive field of superficial layer hypercomplex cells. J. Physiol. (London). 273,791-803. Sillito, A.M., Grieve, K.L., Jones, H.E., Cudeiro, J. and Davis, J. (1995). Visual cortical mechanisms detecting focal orientation discontinuities. Nature. 378, 492-496. Sillito, A.M., Jones, H.E., Gerstein, G.L. and West, D.C. (1994). Feature-linked synchronization of thalamic relay cell firing induced by feedback from the visual cortex. Nature. 2S2,479-482. Sillito, A.M. and Versiani, V. (1977). The contribution of excitatory and inhibitory inputs to the lengh preference of hypercomplex cells in layer II et III of the cat's striate cortex. J. Physiol. (Lond.). 271,775-790. Silva, L.R., Chagnac-Amitai, Y. and Connors, B.W. (1991). Intrinsic oscillations of neocortex generated by layer V pyramidal neurons. Science. 251,432-435. Singer, W. (1990). Search for coherence : a basic principle of cortical self-organization. Concepts in Neurosci. 1, 1-26. Singer, W. (1993). Synchronization of cortical activity and its putative role in information processing and leaming. Annu. Rev. Physiol. 55» 349-374.

198 Singer, W. and Gray, CM. (1995). Visual feature integration and the temporal correlation hypothesis. Annu. Rev. Neurosci. 18, 555-586. Somers, D., Nelson, S.B. and Sur, M. (1994). Effects of long-range connections on gain control in an emergent model of visual cortical orientation selectivity. Soc. Neurosc. Abstr. 20,1577 (646.7). Somogyi, P., Kisvarday, Z.F., Martin, K.A.C. and Whitteridge, D. (1983). Synaptic connections of morphologically identified and physiologically characterized large basket cells in the striate cortex of cat. Neurosci. 10, 261-294. Stafstrom, C.E., Schwindt, P.C. and Crill, W.E. (1984). Repetitive firing in layer V neurons from cat neocortex in vitro. J. Neurophysiol. 52, 264-277. Stemmler, M., Usher, M. and Niebur, E. (1995). Lateral interactions in primary visual cortex: a model bridging physiology and psychophysics. Science. 269.1877-1880. Stent, G. (1973). A physiological mechanism for Hebb's postulate of learning. Proc. Natl. Acad. Sci. USA. 70, 997-1001. Stern, P., Edwards, F.A. and Sakmann, B. (1992). Fast and slow components of unitary EPSCs on stellate cells elicited by focal stimulation in slices of rat visual cortex. J. Physiol. (London). 449, 247-278. Stryker, M.P. (1989). Cortical physiology: is grandmother an oscillation? (News and Views). Nature. 338, 297-298. Stuart, G.J. and Sakmann, B. (1995). Amplification of EPSPs by axomatic sodium channels in neocortical pyramidal neurons. Neuron. 15, 1065-1076. Sutor, B. and Hablitz, J.J. (1989a). EPSP's in rat neocortical neurons in vitro. L Electrophysiological evidence for two distinct EPSP's. J. Neurophysiol. 61, 607-620. Sutor, B. and Hablitz, J.J. (1989b). EPSP's in rat neocortical neurons in vitro. IL Involvement of N-methyl-D-aspartate receptors in the generation of EPSPs. J. Neurophysiol. 61, 621-634. Tanifuji, M., Yamanaka, A., Sunaba, R. and Toyama, K. (1993). Propagation of excitation in the visual cortex studied by the optical recording. Jpn. J. Physiol. 43, S57-S59. Thomson, A.M. (1986). A magnesium-sensitive post-synaptic potential in the rat cerebral cortex resembles neuronal response to N-methylaspartate. J. Physiol. (London). 370, 531-549. Thomson, A.M. and Deuchars, J. (1994). Temporal and spatial properties of local circuits in neocortex. Trends Neurosci. 17, 119-126. Thomson, A.M., Deuchars, J. and West, D.C. (1993a). Large, deep layer pyramid-pyramid single axon EPSPs in slices rat motor cortex display paired pulse and frequencydependent depression, mediated presynaptically and self-facilitation, mediated postsynaptically. J. Neurophysiol. 70, 2354-2369. Thomson, A.M., Deuchars, J. and West, D.C. (1993b). Single axon excitatory postsynaptic potentials in neocortical interneurons exhibit pronounced paired pulse facilitation. Neuroscience. 54, 347-360. Thomson, A.M. and Radpour, S. (1991). Excitatory connections between CAl pyramidal cells revealed by spike triggered averaging in slices of rat hippocampus are partially NMDA receptor mediated. Eur. J. Neurosci. 3, 587-601. Toyama, K. (1988). Functional connections of the visual cortex studied by cross-correlation techniques. In "Neurobiology of Neocortex", eds. P. Rakic and W. Singer (New-York: J. Wiley and Sons), 203-217. Toyama, K., Fujii, K. and Umetani, K. (1990). Functional differentiation between the anterior and posterior Clare-Bishop cortex of the cat. Exp. Brain Res. 81, 221-233. Toyama, K., Matsunami, K., Ohno, T. and Takashiki, S. (1974). An intracellular study of neuronal organization in the visual cortex. Exp. Brain Res. 21,45-66. Traub, R.D., Whittington, M.A., Colling, S.B., Buszaki, G. and Jefferys, J.G.R. (in press). Analysis ofgamma rythms in the rat hippocampus in vitro and in vivo. J. Physiol. (London). Treisman, A. (1977). Focused attention in the perception and retrieval of multidimensional stimuli. Percept, and Psychophys. 22, 1-11.

199 Treisman, A.M. and Gelade, G. (1980). A feature-integration theory of attention. Cogn. Psychol. 12, 97-136. Ts'o, D.Y., Gilbert, CD. and Wiesel, T.N. (1986). Relationships between horizontal interactions and functional architecture in cat striate cortex as revealed by crosscorrelation analysis. J. Neurosci. 6,1160-1170. Volgushev, M., Pei, X., Vidyasagar, T.R. and Creutzfeldt, O.D. (1993). Excitation and inhibition in orientation selectivity of cat visual cortex neurons revealed by whole-cell recordings in vivo. Vis. Neurosci. JjO, 1151-1155. Von der Malsburg, C. (1981). The correlation theory of brain function. Internal report, MaxPlanck Institute for Biophysical Chemistry Goettingen RFA, Von der Malsburg, C. and Bienenstock, E. (1986). Statistical coding and short-term synaptic plasticity: a scheme for knowledge representation in the brain. In "Disordered Systems and Biological Organization.", eds. E. Bienenstock, F. Fogelman-Souli6 and G. Weisbuch. (Berlin: Springer-Verlag), 247-272. Von der Malsburg, C. and Schneider, W. (1986). A neural cocktail-party processor. Biol. Cyb. 54, 29-40. Von der Malsburg, C. and Singer, W. (1988). Principles of cortical network organization. In "Neurobiology of Neocortex", eds. P. Rakic and W. Singer (New-York: J. Wiley and Sons), 69-99. Wang, X.J. (1993). Ionic basis for intrinsic 40 Hz neuronal oscillations. NeuroReport. 5, 221-224. Wassle, H. and Boycott, B.B. (1991). Functional architecture of the mammalian retina. Physiological Reviews. 71, 447-480. Whitehead, S.D. and Ballard, D.H. (1990). Active perception and reinforcement learning. Neural Comp. 2,409-419. Wilcox, K.S. and Dichter, M.A. (1994). Paired pulse depression in cultured hippocampal neurons is due to a presynaptic mechanism independent of GABAg autoreceptor activation. J. Neurosci. 14, 1175-1788. Wilson, H.R. and Cowan, J.D. (1972). Excitatory and inhibitory interactions in localized populations of model neurons. Biophys J. 12, 1-24. Whittington, M.A., Traub, R.D. and Jefferys, J.G.R. (1995). Synchronized oscillations in interneuron networks driven by metabotropic glutamate receptor activation. Nature. 373. 612-615. Yang, G. and Masland, R.H. (1992). Direct visuaUzation of the dendritic and receptive fields of directionalloy selective retinal ganglion cells. Science. 258, 1949-1992. Yuste, R. and Denk, W. (1995), Dendritic spines as basic functional units of neuronal integration. Nature. 225, 682-684. Yuste, R., Gutnick, M.J., Saar, D., Delaney, K.R. and Tank, D.W. (1994). Ca^"" accumulations in dendrites of neocortical pyramidal neurons: an apical band and evidence for two functional compartments. Neuron. 13, 23-43. Zipser, K. (1995). Measuring the delay of the onset of extra-receptive field modulation in VI neurons. Soc. Neurosci. Abstr. 21,1751 (689-2). Zucker, R.S. (1989). Short-term synaptic plasticity. Annu. Rev. Neurosci. 12, 13-31.

Brain Theory - Biological Basis and Computational Principles A. Aertsen and V. Braitenberg (Editors) © 1996 Elsevier Science B.V. All rights reserved.

201

On the Role of Neural Synchrony in the Primate Visual Cortex Andreas K. Kreiter and Wolf Singer Max-Planck-Institut fiir Hirnforschung, Deutschorden Strafie 46, D-60528 Frankfort, Germany

Neuronal representation of visual stimuli - The role of single neurons The question how sensory signals are processed and represented is still unresolved. Traditional approaches assume that information is contained mainly in the response rate of individual neurons. Elevated firing of feature selective neurons is thought to signal the presence of particular stimulus configurations. There is indeed ample evidence from all sensory systems that stimulus features selectively modulate the response rate of single cells. Well documented examples are found in the primate visual system. In the retina, the lateral geniculate nucleus, and primary visual cortex the spike rate of individual cells reflects simple stimulus properties like spatial location, spatial extent, orientation, spectral composition, the direction of motion, binocular disparity and several others (Hubel and Wiesel, 1959; Hubel and Wiesel, 1962; Zeki, 1975; Orban, 1984; Desimone et al. 1985; Desimone, 1991; Henry, 1985; Maunsell and Newsome, 1987; Livingstone and Hubel, 1988). Within extrastriate areas of the macaque 1 visual cortex rate coded responses occur in addition to more complex stimuli such as optical flow fields, non-cartesian gratings, illusory contours, specific arrangements of simple geometric forms and characteristic features of faces and body parts (Gross et al. 1972; Bruce et al. 1981; Perrett et al. 1985; Saito et al. 1986; Desimone, 1991; Sakai and Miyashita, 1991; Rolls, 1992; Gallant et al. 1993; Miyashita, 1993; Tanaka, 1993). Further support for the functional significance of rate codes comes fi'om the evidence that the sensitivity of single cortical neurons for certain stimulus properties reaches the psychophysical threshold. This suggests a close relation between the activity level of single nexu-ons and perception. Thus, in the monkey the grating acuity and contrast sensitivity of the most sensitive neurons in VI can reach the threshold found for human observers (Parker and Hawken, 1985; Hawken and Parker, 1990). Other studies concluded that contrast sensitivity (Tolhurst et al. 1983) and orientation selectivity in VI come close to psychophysically observed thresholds but do not reach them (Vogels and Orban, 1990). Within the motion sensitive medio-temporal area (MT, V5) good

202

correspondence was found between single unit activity evoked by dynamic random dot stimuli and the simultaneously estimated psychophysical performance of monkeys engaged in a motion discrimination task (Newsome et al. 1989; Britten et al. 1992). Furthermore, it was shown that electrical stimulation of local groups of neurons in MT can influence the decision of monkeys judging the direction of motion in random dot displays (Salzman et al. 1990; Salzman et al. 1992; Murasugi et al. 1993). These and many other results clearly indicate that the response rate of single neurons contains information which contributes to the representation of perceptual objects. However, theoretical considerations and experimental findings suggest that in general the full description of a visual object cannot be provided by the activity of a single neuron. Object representation by single units would require at a final processing stage a dedicated neuron for every distinguishable pattern (Fig. 1, left column). Because of the exceedingly large number of possible configurations the limited reservoir of cortical neurons seems to exclude that this strategy is the only option to represent patterns. The combinatorial problem is somewhat reduced in the model suggested by Horace Barlow in 1972 in which patterns are thought to be represented by generalizing "cardinal" cells at the top of the processing hierarchy. These cells are assumed to represent objects as complex as an apple or a mouse, objects that can be described with a single word (Fig.l, middle column). Their responses are thought to be context independent and invariant to changes in size, position and minor modifications of appearance of the respective objects. For the response of such generalizing cells it would for example not matter whether an apple is green or red, with or without brown patches, as long as it can be perceived as an apple. According to this model, the complete perceptual situation is represented exclusively by the small set of vigorously activated cardinal cells. Such generalization reduces the combinatorial problem to some extent. However, the cardinal cell model cannot easily cope with the requirement to express relations between different attributes of the represented objects. This is because the cardinal cells represent by definition abstractions which lack details. If a cardinal cell responds for example to a mouse the response does not signal whether the mouse is patchy or uniformly coloured since a "mouse" cardinal cell would have to respond to any mouse, even if it is partially hidden, pinkish and striped. Therefore, the mouse would be recognized but it would lack any specific property. The existence of a 'patchy'-cell would also not help since this attribute could also be related to the apple just eaten by the mouse. Of course a 'patchy mouse'-cell would resolve the problem but this leads again into the combinatorial explosion mentioned above. Thus, the main advantage of cardinal cells, their ability to generalize, turns out to be their major problem and prevents effective object representation by such individual neurons. Stimulus representation by single, highly specific object detector cells (or gnostic units) is not only problematic fi-om a theoretical point of view but has also

203

o oooo

oo

ooooooooo OOQpOOOOOO oo®oooooo

ooooooooo

oioodoo'iriiiiT

OCiOQppilcR)

oooooooooo QOpOQQOOO

OOOOSCTOOOO

Pontifical Cells

Cardinal Cells

iOQ

o" o^o°^p o Neural Assembly

Fig. 1. Three different models for stimulus representation in the cerebral cortex. Pontifical cells (left column) are at the top level of a hierarchical feed forward network which extracts at each level progressively more complex stimulus constellations from the sensory input. At the top level a single activated neuron out of a huge set of pontifical cells represents the complete scene. Cardinal cells (middle column) are also builtfi*oma feed forward network. They are not specific for an entire scene but detect certain objects. A scene is represented by the small group of activated cardinal cells which code for individual objects. In both models the lower level neurons in the feed forward network do not participate in the representation of perceived objects. Neural assemblies (right column) are formed in feedback networks of neurons which behave like coarse filters rather than precise detectors. An object is represented by a population of cooperatively interacting neurons.

204

not received much experimental support. Neurons selective for the most complex stimuli are found in the inferior temporal cortex and the anterior superior temporal sulcus. They respond to hands, faces or behaviourally relevant objects (Gross et al. 1972; Bruce et al. 1981; Perrett et al. 1985; Saito et al. 1986; Desimone, 1991; Sakai and Miyashita, 1991; Rolls, 1992; Gallant et al. 1993; Miyashita, 1993). However, the results obtained by Tanaka and colleagues (Tanaka, 1993) show that most of these cells are tuned to certain geometrical arrangements of features that are indeed more complex than bars or gratings but much simpler than what is needed to distinguish unambiguously different objects that share certain figural aspects. Therefore, these cells cannot be regarded as gnostic units. Even face cells are not selective for individual faces, nor do they respond indiscriminately to every face as it would be required for a general face detector. Rather they are tuned to certain properties of faces like geometrical relations between features of faces (Yamane et al. 1988) from which the identity, the direction of gaze (Perrett et al. 1985), or the emotional expression of a face (Hasselmo et al. 1986; Hasselmo et al. 1989) can be inferred. (For review see Desimone (1991), Young and Yamane (1992), Rolls (1992)). Thus, even the cells with the most complex stimulus requirements described so far in the visual cortex do not have the properties required for stimulus representations based on single units. Problems with single cell codes do not only occur at the level where complex objects have to be represented. Also quite simple stimulus properties such as orientation or direction of motion are diflBcult to extract from the responses of single cells tuned to these properties. The reason is that the responses of neurons in the visual cortex are broadly tuned and influenced by features belonging to different dimensions. Different stimuli can elicit responses of equal strength in the same neuron. The cells are smoothly tuned to a region in the high dimensional stimulus space, and their responses exhibit no unequivocal relation between firing rate and stimulus properties (Gerstein and Gochin, 1992). Therefore single neurons cannot signal reliably the presence of a particular feature. Taken together, theoretical considerations and experimental results suggest that representations of stimulus properties cannot be achieved solely by single cell codes. How can this conclusion be reconciled with the close relation between single unit activity and psychophysical performance? The experiments mentioned above have in common that they did not require to exploit the frill coding capacity of the system. First, comparisons between single neurons and psychophysical performance were based on tasks requiring only decisions on very simple stimulus properties for which specifically tuned neurons exist. This circumvents the problem that complex constellations cannot be encoded selectively by single units. Second, the psychophysical measurements were done with detection or discrimination tasks tailored to one specific aspect of the stimuli. This eliminates potential difficulties with the complete representation of stimuli by single units

205 because the task can be solved without a full description of the stimulus. Third, in part of the studies the stimuli which had to be distinguished were very different as e.g. motion to the left versus motion to the right or grating present versus absent. The subjects' task was not to differentiate between fine differences within the same stimulus dimensions, as for example between two similar directions of motion. It was sufficient to simply detect the stimuli and therefore the difficulty to derive parametric judgements from the responses of broadly tuned single units was excluded. Fourth, the stimuli were presented in very reduced displays containing only the basic feature which had to be judged. Under such conditions there is no problem to identify cells which deliver the relevant information since most of the activated neurons are directly related to the behaviourally relevant aspect of the relevant stimulus. Fifth, the stimuli were well known to the subjects due to prior instruction or overtraining. Through learning this could have facilitated the identification of the neurons with the most discriminative responses. This reduces the problem of ambiguities in response rates of single neurons. Thus, tasks based on restricted and well known sets of simple, qualitatively different stimulus conditions do not necessarily require full descriptions of the stimuli. It is sufficient to know the neurons which respond differentially in the few stimulus conditions to be discriminated and this ability can with all likelihood be acquired by learning. Even under such well defined conditions it is unlikely that a decision is based on a single cortical neuron. The information content of single cell responses as assessed by signal detection techniques which assume an optimal observer is rather impressive. Natural decision-making structures have certainly a much more limited computational precision and are in addition affected by noise from other sources than the neurons tuned optimally to the respective stimulus. The finding that psychophysically determined motion detection thresholds were often above the threshold of single MT neurons (Britten et al. 1992) supports the assumption that the decision-making mechanisms of psychophysical observers have only suboptimal access to individual, highly discriminative neurons. Since the behavioural performance reveals a comparable amount of information as the average single unit activity, the suboptimal access to this information must be compensated by pooling responses from multiple neurons. This increases the signal to noise ratio even though the possible improvements are limited by correlations between the response amplitudes of different neurons (Gawne and Richmond, 1993; Zohary et al. 1994; Kreiter and Singer, 1995b). Simulations confirmed that the motion discrimination performance of monkeys can be explained by the opposing effects of pooling on the one hand, and response rate correlation and noise from suboptimally driven neurons on the other (Britten et al. 1992). Thus, even under conditions in which task design and informational content of single unit activity would permit decisions based on the responses of a single cell, the limitations of neural information processing suggest that groups of neurons are required to extract the necessary information.

206 Distributed processing and population coding The perceptually and behavioxirally restricted and predetermined conditions in psychophysical experiments represent an extreme case in a rich spectrum of perceptual situations to which the visual system is usually exposed. More often it has to cope with complex visual scenes containing many, partially overlapping objects in front of a varying background. The specific constellation, identity and behavioxu-al relevance of objects contained in the scene are typically not known in advance and usually there is a huge set of different possibilities. Under such conditions the ambiguous message of an individual neuron cannot be disambiguated by precise expectations and a restricted set of possible stimulus constellations. Rather the meaning of an individual neuronal response has to be inferred from relations with the responses of other neurons, especially of those activated by the same stimulus feature or perceptual object. Unambiguous information about stimuli is only contained in the pattern of responses of multiple neurons. Therefore, it is commonly held that population coding is a indispensable principle for cortical representation of sensory information (Hebb, 1949; Braitenberg, 1978; Edelman and Mountcastle, 1978; Grossberg, 1980; Abeles, 1982; Abeles, 1991; Hopfield, 1982; Aertsen et al. 1986; Gerstein et al. 1989; Georgopoulos, 1990; Palm, 1990; Singer, 1990a; Singer, 1990b; Zipser and Andersen, 1988; Rolls, 1992; Young and Yamane, 1992). Theoretical considerations suggest many advantages of population over single unit codes. Because of the combinatorial natxire of population codes their representational capacity is much higher than that of the simi of the involved single cells. The loss of specific information in the generalizing responses of cardinal units is prevented by the flexible association of many different cells responding to different aspects of the same stimulus. A further advantage of population codes is the flexibility with which new representations can be generated. If representations consisted of individual detector neurons the formation of a new representation would require that a formerly unused neuron is newly connected to the already existing circuitry. This is much more demanding than inducing gradual changes of S5maptic weights in a network of distributed but cooperating neurons. Also other associative capabilities such as generalization, pattern completion and fault tolerance are advantageous properties associated typically with distributed representations (Palm, 1982; Rumelhart and McClelland, 1986; Hopfield and Tank, 1991).

The problem of concurrently activated representations Distributed or population codes represent information about a certain stimulus or content by the pattern of graded activity in many different units. If only one stimulus is represented at the same time, then the distributed information can be

207

interpreted unambiguously since all responses are related to the same content. If however several distinct stimuli are present simultaneously, their representations may become ambiguous since there is no explicit information which identifies the neurons activated by the same stimulus (Fig. 2). Because of the high degree of divergence and convergence in the cortical circuitry each neuron will in almost all complex stimulus configurations receive synaptic inputs evoked by different stimuli in an unpredictable and unresolvable mixture. Thus, the requirement for joint processing of signals related to a particular stimulus could become compromised by inclusion of responses evoked by other stimuli. In the visual system this problem arises if several objects are present within the same scene at the same time and in particular when they are spatially contiguous or overlapping. In this case the patterns of activated cells evoked by different stimuli will fiise to an undifferentiated compound pattern which needs to be decomposed into components related to the various individual objects.

B

o o o POL o

ooooogo Ooooo o

op_oX^o^ ooo Pox o

=\ Oo9^oOOo °;m^'

+ °o°o°SoO»

OQOOO O

Qo

o OoOOO O

Fig. 2. Different parts of a visual scene (A) will activate different populations of neurons (B) which represent their properties. If these populations reside in overlapping cortical regions there is no possibility to distinguish responses related to different populations (C). Thus, the population coded stimulus descriptions cannot be accessed if several populations are active simultaneously.

208 Therefore, mechanisms are required to identify responses evoked by the same stimulus and to distinguish them from those related to other stimuli. In addition, effective interactions between neurons have to be restricted to those processing related contents to prevent unintended disturbances by unrelated signals. Retinotopic mapping has been considered as a solution to this problem because it permits the segregation of responses evoked from different locations. In the same way ordered maps for stimulus features other than location can contribute to the association of similar and the separation of different features. This segmentation mechanism may be sufficient if stimuli are widely separated in space and the efferent projections of the activated populations have no common target. Then processing within the respective populations is largely independent and false conjunctions are excluded. In general, however, such anatomical mapping is not a sufficient mechanism to segregate responses related to different stimuli and to link those related to the same content. The broad tuning of the neurons in stimulus space and the scatter within maps cause populations related to different stimuli to overlap in the map if the respective properties of the stimuli are similar. In the case of retinotopic maps, for example, the spatial extent of the receptive fields and their scatter between different cells at the same cortical site (Van Essen et al. 1984) predict that the same cells will often be activated by different but adjacent or overlapping stimuli. In addition size and scatter of receptive fields rise quickly in the ascending visual pathway until the receptive fields cover most of the visual field and retinotopy is almost absent. Therefore, it becomes progressively difficult to relate active cells to certain parts of the scene on the basis of retinotopic mapping. Similar problems arise in the maps for other features. Furthermore, the location of neurons in a feature map and their connections are fixed and cannot be changed at the relevant time scale. The set of neurons related to a certain stimulus is, however, permanently changing when stimulus configurations change. Two neurons which are related to the same stimulus may become activated by different stimuli in another scene. Such changing associations of different cells can of course not be expressed by the invariant spatial positions of the neurons in a feature map. While mapping is certainly useful to attenuate binding problems by concentrating signals which are possibly but not necessarily related because of their proximity along certain coordinates of stimulus space, it cannot be used to represent the actual relations between responses. This clearly requires a djmamic mechanism that binds responses in changing constellations as stimulus configurations change. A more flexible mechanism is suggested by models of distributed processing which assume cooperative interactions in neuronal assemblies and competition between them (Fig. 1, right colimin). The relation of neurons to an activated assembly is expressed by their enhanced firing level. The stronger sjmaptic couplings between neurons of the same assembly mediate this cooperative enhancement while the activity of other neurons which do not belong to the active assembly is reduced by an inhibitory mechanism which limits the global

209

excitation of the network. The responses of neurons representing a single stimulus can then be differentiated because their enhanced activity separates them from responses arising in neurons which are not part of the activated assembly. However, if two assemblies get activated simultaneously their neurons are again indistinguishable because their enhanced activities permit segregation only from neurons not participating in any active assembly but not from each other. Theoretically it might be possible to distinguish different assemblies by different activation levels. A disadvantage of this strategy is that it destroys the information contained in the response amplitudes because these would have to be the same for all members of a particular activated assembly.

Temporal synchronization as a possible solution It has been suggested on the basis of theoretical considerations that relations between the responses of different neurons could be expressed by temporal patterning rather than by activity levels (Milner, 1974; Grossberg, 1980; Abeles, 1982; Abeles, 1991; von der Malsburg, 1985; von der Malsburg, 1986; von der Malsburg and Schneider, 1986; von der Malsburg and Singer, 1988; Gerstein et

Bo

oO o o o o o o p oo o o °?)OgO OOQO o O O

o

OOOQO

o oo o

+

=>

ooo°°o® OO ® 0 ^ O o o o ^Oo o o o o( oo w o ooo^o

o o o oo o

Fig. 3. Labelling of populations by s5nichronous discharge. The superposition of different populations can be resolved if neurons related to the same population discharge synchronously and avoid synchronization with other populations. Different populations may therefore be active simultaneously without confoimding their stimulus descriptions.

210 al. 1989; Singer, 1990b; Singer, 1993; Singer and Gray, 1995). The hypothesis discussed here assumes that responses which need to be bound together for further joint processing become organized in time by synchronization. Neurons engaging in episodes of synchronous discharge would thereby signal their participation in the representation of the same stimulus. Consequently, several distinguishable populations can coexist at the same time if synchronization between the neurons related to different populations is avoided (Fig. 3). This can be achieved most easily if synchronous firing is defined in the millisecond range since it requires not more than to adjust the precise time of discharge in relation to the discharge pattern of other neiu^ons and makes rate changes unnecessary. The main effect of this mechanism is that synchronous EPSPs elicited by neurons of the same population tend to simimate more effectively in target cells (Abeles, 1991). Thus, the saliency of the distributed signals related to the same stimulus would be enhanced and their joint processing favoured. Due to the lack of synchronization between the discharges of cells belonging to different populations the processing of different contents becomes separated in time and this reduces the chance of false conjunctions and unwanted signal exchange between unrelated populations.

Experimental evidence for temporal synchrony in primate visual cortex If neuronal assemblies are distinguished by the temporal coherence of activity evoked in their constituting neurons then episodes of synchronized discharges should be observable in response to coherent stimuli. Even though only a few studies have investigated cross-correlations between different neurons in the primate visual system there is clear evidence that different neurons can synchronize their discharges with a precision in the range of a few milliseconds. In the striate cortex of anaesthetized monkeys synchronization was found preferentially but not exclusively between cells with similar selectivity for ocular dominance, orientation, color and spatial location (Krtiger and Aiple, 1988; Ts'o and Gilbert, 1988; Livingstone, 1991). Under certain circumstances also larger groups of neurons get organized into a synchronous pattern of activity. This is suggested by the observation of oscillatory local field potentials (Fig. 4) in VI of awake fixating monkeys (Kreiter, 1992; Eckhom et al. 1993). Correlations have also been found across area boundaries between areas VI and V2 (BuUier et al. 1992; Munk et al. 1993; Frien et al. 1994) Correlation analysis has also been performed in visual areas beyond VI and V2 such as the inferiotemporal cortex (IT) and the middle temporal area (MT) of the superior temporal sulcus (STS) of awake macaque monkeys. Within area MT strong synchronization between adjacent neurons has been observed. Different neurons recorded simultaneously with the same electrode were found to synchronize their discharges with a precision of a few milliseconds over short epochs of 100 to 300 ms duration (Fig. 5). These grouped discharges are

211 separated by silent intervals with almost no activity and repeat every 15 to 35 ms. Neither the grouped discharges nor the episodes in which they occur are time locked to the stimulus (Kreiter and Singer, 1992).

50 Frequency [ Hz ]

Time [ s ]

100

c J 4

0)

f

CO

Hn r^wFw

n

±dq +80

Time [ s ]

Time [ m s ]

Fig. 4. Local field potentials in VI of an awake fixating macaque monkey. The stimulus was a small striped (3 cyclesT) square moving over the RF (17s). (A) Time course of the power between 40 and 80 Hz contained in the field potential. Note the parallel time course of the PSTH (C) which indicates that the LFP must be generated by neurons with RFs extending not more in space than those of the simultaneously recorded cells. (B) Power spectrum and (D) auto-correlogram of the LFP in the stimulated epoch (continous line) marked by the vertical lines in (A) and a nonstimulated epoch (dashed line).

212

+ 100

Time [ ms ]

Time [ ms ]

Fig. 5. Two examples of locally synchronized activity evoked with a moving bar in area MT of an awake fixating macaque monkey. Spikes of different cells recorded with the same electrode tend to occur in short clusters separated by silent intervals. The auto-correlograms below reflect these clusters by the broad centr^ peak and the initial trough. Additional side peaks as in (A) indicate that the sjnichronous discharges occurred in regular intervals. The dashed line indicates the trigger level.

Neuronal synchronization within area MT is not confined to directly adjacent neurons but has also been observed between spatially separate recording sites (Figs. 6,7,8) whereby the direction of motion preferred by the neurons did not have to be similar. Cases with more than 90" difference between the preferred directions of motion could show substantial synchronization if they were activated with a single moving bar stimulus. The temporal precision of coincident firing was in the range of 3 to 5 ms as indicated by the width of the peaks in the cross-correlograms which almost always straddled the origin. Within inferiotemporal cortex Gochin et al. (1991) observed synchronization between directly adjacent neurons recorded with the same electrode. The width of the peaks varied between 1 and more than 400 ms and straddled the origin in about 60% of the cases. Taken together, neurons in various areas of the primate visual cortex are able to s3aichronize their discharges. The temporal precision of S3nichronization is oflien in the millisecond range and thus sufficient for the purpose of d5niamic grouping of neurons into distinguishable populations.

213 D e p e n d e n c e of synchrony on stimulus configuration One of the important features of population coding is that a particular neuron can participate in the representation of different contents by joining at different times different assemblies. If synchronization is used to bind neurons into assemblies this predicts that neurons must be able to change the partners with which they synchronize when changing stimulus configurations require them to join different assemblies. This prediction has been tested in area MT of awake fixating monkeys with simple bar stimuli (Kreiter and Singer, 1995a). The assimiption is that neurons activated by a single moving bar stimulus contribute to the neuronal representation of this stimulus and synchronize their responses. If, however, the same group of neurons is activated by two independently moving bars, neurons should regroup themselves into two different assemblies according to their respective preferences for the two bars, and cells belonging to different assemblies should no longer exhibit synchronized discharges despite of being simultaneously active. A condition where neurons at two recording sites either join the same assembly or split into two different assemblies can be created by recording fi"om sites which have overlapping receptive fields but different preferences for the direction of stimulus motion (e.g. Fig. 6.A). Strong S3nichronization between the responses of neurons at both sites was observed if they were evoked with a single bar moving into a direction intermediate between the two preferred directions of motion (Fig. 6.C). In contrast, simultaneous stimulation with two different bars, moving into different directions close to the respective preferred directions of the two recording sites (dual bar condition. Fig. 6.B), resulted in a disruption or strong attenuation of synchronization (Fig. 6D). This effect was robust since it occurred in all tested pairs which exhibited correlations. The average reduction of synchronization in the dual bar condition as compared to the single bar condition was by a factor of six (n=19. Fig. 6.E). The change of synchronization strength could not be explained by changes in activation levels. Despite of the better fit of the stimuli to the preferred directions of motion in the dual bar condition, the average spike rate in the single bar condition was only 15% less and was in several cases even higher (Fig. 6.F). So far the result is in agreement with the predictions made by the correlation hypothesis and confirms similar experiments in anaesthetized cats (Gray et al. 1989; Engel et al. 1991a; Engel et al. 1991b). It is conceivable that the breakdown of synchrony in the dual bar condition is unrelated to assembly coding and simply a consequence of the interference caused by the second bar. This possibility was ruled out by an experiment in which two moving bars were crossing each other as in the dual bar condition, but in such a way that the nevirons at both recording sites were activated only by one of the two stimuli (Crossing bar condition; Fig. 7.B). In this situation the neurons at both recording sites are expected to synchronize with each other despite of the second bar since they are all activated by the same stimulus and should be part of the same assembly. As predicted, comparison of the correlations observed for the crossing

214

1

1

1 m pJli;

+63

^k ,—, 120• ^ n=19 o^ 100-

o z 1

80-

/

o ••6

c o O m

m

(D O)

c c/)

//

/

/

/

/

• ••

0-n/

/

/

0 -63 0 5 Time [ ms ] Time [ ms ]

250i

/

n=38

C N

OX

/

200 /

O 0 150

60- ! . . ' ^

0 25 50 75 100 Dual Bar Condition-NC [ % ]

0

—

I

1

1

1

1

—

^

0

100 200 Dual Bar ConditionSpike Frequency [ Hz ]

Fig. 6. Dependence of synchronization on single and dual bar configuration: (A,B) plots of the receptive fields and stimulus configurations. The dot marked 'F' corresponds to the fixation point. Arrows within the RF plots indicate the preferred direction of motion for the neurons at the respective recording sites and arrows at stimulus bars the direction of motion. (A) depicts the single and (B) the dual bar configuration. Cross-correlogram and PSTHs obtained for the single bar condition are shown in (C) and for the dual bar condition in (D). The thin vertical lines in the PSTHs mark the window over which the cross-correlograms were computed. The scale bars correspond to 40 spikes/s. Note the pronounced s)mchronization in the single bar condition and the absence of s)m.chronization in the dual bar condition. Scatter plots of normalized correlation values (NC, defined as the peak amplitude above offset divided by the offset) and firing rates obtained for the single bar condition (ordinate) against those obtained for the dual bar configuration (abscissa) in 19 cases are shown in (E) and (F), respectively. The dashed line indicates the region of equal values for both conditions. In all cases synchronization is considerably stronger for the single bar condition while response rates are similar.

215

0) CD

Q-

^ +63

0 -63 0 5 Time [ ms ] Time [ ms ]

Time [ ms ]

1

wl

2

^b.?^-^

Time [ ms ]

E 100-

^

n=11

—

•

o zc

.o

•D

80-

/

..r*

C

o O

^c« _ ca CD C CO

(/) o ^ 1 _

/

//

/

60-

• // / •

4020/

/

/

2501 n=22

/

o^

1

•/ / /•

0^ / / 0 20 40 60 80 100 Single Bar Conditlon-NC [ % ]

c o

H-^

•D

C

• / /

1—1

N

X

.

200

o o O CcD 150

V

X—

CO D

m CT 0) iL rx LL c

100

( 0 CD CO ^

2Q_ QC/D

50

i

••/

0-L 0 100 200 Single Bar ConditionSpike Frequency [ Hz ]

Fig. 7. Comparison of synchronization in the single and crossing bar configuration: (A,B) plots of RFs and stimulus configurations. (A) shows the single and (B) the crossing bar condition. Crosscorrelogram and PSTHs obtained for the single bar condition are shown in (C) and for the crossing bar condition in (D). The scale bars for the PSTHs correspond to 40 spikes/s. In this case the normalized correlation (NC) was 56.5% for the single bar configuration, 58.8% for the crossing bar configuration and 4.4% for the dual bar configuration (data not shown). (E,F) Scatter plots of normahzed correlation (NC) and firing rates obtained for the crossing bar configuration (ordinate) against those obtained for the single bar condition (abscissa) in 11 cases are shown in (E) and (F), respectively. Note that the additional bar in the crossing bar configuration causes no major reduction of sjmchronization as compared to the reduction found for the dual bar configuration. Conventions as in Fig. 6.

216 and the single bar condition revealed no significant differences (Fig. 7). The second stimulus did not disturb the synchronization that occurred when only the single bar was presented. Desynchronization in the dual bar condition can therefore not be attributed to unspecific effects such as the mere number of simultaneously present stimuli. Stimulus configurations associated with S5mchronization have in common that the bar used in both, the single and the crossing bar configuration activates the neurons at both recording sites. It might thus be that synchronization is critically dependent on the presence of a bar whose orientation and direction of motion is just intermediate between the preferences of the neurons at both recording sites. The synchronization hypothesis predicts that sjntichronization should not depend on the precise parameters of the stimulus but should occur as long as the neurons respond to the same stimulus and contribute to its representation. This prediction was tested with single bars moving in different directions over the receptive fields (Fig.8). Synchronization remained similar despite of considerable changes of the direction of motion as long as both recording sites remained activated. Even if one of the two bars of the dual bar condition was presented alone sjnichronization was often as strong as in the original single bar condition. Thus, the orientation and direction of motion of an individual bar is not critical for the synchronization of the spike trains at both sites, as long as both sites are sufficiently activated by the same bar. Furthermore, the reduction of synchronization in the dual bar condition cannot be attributed to the particular parameters of the two bars since each of them, when presented alone can induce s5aichronization. The conclusion from these control experiments is therefore that the critical variable determining s3mchronisation is whether the responses at the two recording sites are evoked by a common stimulus or by two different stimuli. Further support for this conclusion is provided by the quantitative analysis of synchronization in the dual bar condition. In part of the cases correlation was only attenuated but not abolished completely (Fig. 6.E). Such a residual correlation is expected if at least one of the simultaneously present stimuli causes residual activation of neurons at the respective non-optimal site. In this case the neurons at both sites contribute in a graded way to the representation of the same stimulus and should therefore exhibit some residual correlations. The correlation hypothesis therefore predicts a relation between the strength of the residual correlations and the extent to which each of the two bars coactivates also neurons at the respective non-optimal site. To describe this relation quantitatively coactivation was estimated as the fraction of activity evoked at the same site by the non-optimal bar of the dual bar stimulus over the activity evoked by the dual bar stimulus at the same site. These coactivation values from both sites were subsequently averaged and compared with the correlation strength in the dual bar condition, the latter being expressed as a fraction of the correlation strength measured in the single bar condition. The analysis revealed that the residual correlations in the dual bar condition are positively correlated

217

iilil \ iji

w

A,

d^

F?

150-

k

/

•—'

O

z

Time [ ms ]

100-

//

CD

I

\

Jl

4

CO

w

Q

o IMsv

Time [ ms ]

0

c O

»' 50-

//

> //

/

/

.

>4-

• 0- / • >> 50 100 150 0/ Sinc)le Bar Condition-NC [ %

Fig 8. Dependence of synchronization on different directions of motion: Stimulus configurations are indicated in the plots above the respective cross-correlograms and PSTHs. Other conventions as in Fig. 6. The single bar configuration (A, repeated in E) resulted in a strong correlation (NC = 73.9% in A and 52.1% in E) which essentially disappeared in the dual bar configuration (B, NC = 3.8%). Changing orientation and direction of motion of the single bar by 15° resulted only in a minor change of correlation (C, NC = 60.6%). In (D) the orientation and direction of motion of the single bar is changed further so that it equals the right bar of the dual bar configuration shown in (B) and stimulates site 1 only poorly. Note that the NC value remains similar as in the original single bar condition (NC = 56.8%). The scale bars for the PSTHs correspond to 40 spikes/s. The scatter plot (F) of NC values obtained for presentation of one of the bars of the dual bar configuration (ordinate) versus those obtained for the single bar configuration (n=15) indicates that only in a few cases the single bar condition resulted in a stronger correlation.

218

Fig. 9. Relation between the residual correlation found in part of the cases for the dual bar condition and the extent of coactivation of the two sites by the respective nondominant bar. Coactivation is expressed as the average of the two ratios between the respective nondominant responses and the responses evoked at corresponding sites by the dual bar stimulus. Residusd correlation is expressed as the ratio of the NC values measured in the dual bar condition over those measured in the single bar condition. The residual correlation increases with increasing coactivation of both sites by the same bar (r=0.846, p