No title

ARTICLE Communicated by Peter Dayan Cortical Map Reorganization as a Competitive Process Granger G. Sutton I11 James ...

Author: MIT Press

11 downloads 756 Views 26MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

ARTICLE

Communicated by Peter Dayan

Cortical Map Reorganization as a Competitive Process Granger G. Sutton I11 James A. Reggia Steven L. Armentrout C. Lynne D’Autrechy Departinriit of Computer Scicmc, A. V. Willinins BIdg., Uiiiversity of Morylaiid, College Park, M D 20742 U S A

Past models of somatosensory cortex have successfully demonstrated map formation and subsequent map reorganization following localized repetitive stimuli or deafferentation. They provide an impressive demonstration that fairly simple assumptions about cortical connectivity and synaptic plasticity can account for several observations concerning cortical maps. However, past models have not successfully demonstrated spontaneous map reorganization following cortical lesions. Recently, an assumption universally used in these and other cortex models, that peristimulus inhibition is due solely to horizontal intracortical inhibitory connections, has been questioned and an additional mechanism, the competitive distribution of activity, has been proposed. We implemented a computational model of somatosensory cortex based on competitive distribution of activity. This model exhibits spontaneous map reorganization in response to a cortical lesion, going through a two-phase reorganization process. These results make a testable prediction that can be used to experimentally support or refute part of the competitive distribution hypothesis, and may lead to practically useful computational models of recovery following stroke. 1 Introduction

Feature maps in primary sensory cortex are highly plastic in adult animals: they undergo reorganization in response to deafferentation (Merzenich et al. 1983; Kaas 1991), de-efferentation (Sanes et al. 1988), localized repetitive stimuli (Jenkins et al. 19901, and focal cortical lesions (Jenkins and Merzenich 1987). During the past few years there have been several efforts to develop computational models of such cortical map self-organization and map refinement (Obermeyer et al. 1990; Pearson et al. 1987; Grajski and Merzenich 1990; Sklar 1990; von der Malsburg 1973; Ritter et a/. 1989). For example, models of the hand region of primary somatosensory cortex (SI) have demonstrated map refinement, map reorganization after localized repetitive stimulation and deafferentaNritrnl Cotnputntian 6, 1-13 (1994)

@ 1993 Massachusetts lnstitute of Technology

2

Granger G. Sutton 111 et al.

tion (Pearson et al. 1987), and the inverse magnification rule (Grajski and Merzenich 1990). These computational studies show that fairly simple assumptions about network architecture and synaptic modifiability can qualitatively account for several fundamental facts about cortical map self-organization. However, it is known from limited animal experiments that focal cortical lesions also produce spontaneous map reorganization (Jenkins and Merzenich 1987). Developing a model of map reorganization following cortical lesions is important not only for the insights it may provide into basic cortical physiology, but also because it could serve as a model of nervous system plasticity following stroke (Reggia et al. 1993). In the only previous computational model we know of that simulated a focal cortical lesion, map reorganization would not occur unless it was preceded by complete rerandomization of weights (Grajski and Merzenich 1990). Map reorganization following a cortical lesion is fundamentally different from that involving deafferentation or focal repetitive stimulation. In the latter situations there is a change in the probability distribution of input patterns seen by the cortex, and such a change has long been recognized to result in map alterations (Kohonen 1989). In contrast, a focal cortical lesion does not affect the probability distribution of input patterns, so other factors must be responsible for map reorganization. Past computational models of cortical map self-organization and plasticity have all assumed that the sole mechanism of intracortical inhibition is horizontal (lateral) inhibitory connections. Recently, the validity of this assumption has been called into question and an additional mechanism, the competitive distribution ofactivify, has been proposed for some intracortical inhibitory phenomena (Reggia et al. 1992). We recently implemented a model of cerebral cortex and thalamocortical interactions based on the hypothesis that competitive distribution is an important factor in controlling the spread of activation at the level of the thalamus and cortex (Reggia et al. 1992). Because of the flexible nature of the competitive cortex model, we hypothesized that a version of this model augumented by making thalamocortical synapses plastic would not only demonstrate map formation and reorganization as with previous models, but would also demonstrate spontaneous map reorganization following cortical lesions. In the following we describe simulations that show that this is correct. Our computational model of somatosensory (SI) cortex uses competitive distribution of activity as a means of producing cortical inhibitory effects (Sutton 1992). This model, augmented with an unsupervised learning rule, successfully produces cortical map refinement, expansion of cortical representation in response to focal repetitive stimulation, and map reorganization in response to focal deafferentation. More importantly, our model exhibits something that previous models have not yet produced: map reorganization following focal cortical damage without the need to rerandomize weights. This reorganization is a two-phase process, and

Cortical Map Reorganization

3

results in a testable prediction that can be examined experimentally. We conclude that competitive distribution of activity can explain some features of cortical map plasticity better than the traditional view of cortical inhibition. 2 Methods

We augmented the original competitive distribution cortical model with an unsupervised learning rule that modifies synaptic strengths over time. The intent was to examine how attributing peristimulus inhibition in cortex to competitive distribution of activity rather than to lateral inhibitory connections affected map formation and reorganization. We refer to our augmented model as the competitive S l model because it is a crude representation of a portion of the thalamus and primary somatosensory cortex (area 3b of SI), specifically portions of those structures receiving sensory input from the hand. This area was chosen because of its topographic organization, the availability of interesting experimental data (Jenkins and Merzenich 1987; Merzenich et al. 19831, and to allow comparison with some previous models of SI which make more traditional assumptions about intracortical inhibition (Pearson et al. 1987; Grajski and Merzenich 1990). The competitive SI model is constructed from two separate hexagonally tesselated layers of 32 x 32 volume elements representing the thalamus and the cortex. Each element represents a small set of spatially adjacent and functionally related neurons. To avoid edge effects, opposite edges of the cortical sheet are connected to form a torus. All connections are excitatory and competitive. Each thalamic element connects to its corresponding cortical element and the 60 surrounding cortical elements within a radius of four. With probability 0.5 a thalamocortical connection is initially assigned the minimum weight value 0.00001; otherwise, the weight is chosen uniformly randomly between this minimum and 1.O. Each cortical element connects to its six cortical neighbors; all corticocortical weights are equal (their magnitude then has no effect due to the activation rule). Each element’s functionality is governed by a competitive activation rule (Reggia et al. 1992). The activation a l ( t ) of cortical element j, representing the mean firing rate of the neurons contained in element j, is governed by

where in,(t) = C,out,,(t) with i ranging over all thalamic and cortical elements sending connections to cortical element j. Activation of thalamic element j is also determined by equation 2.1, but its in,(t) term represents only input from sensory receptors. Equation 2.1 bounds al(t)between

Granger G. Sutton 111 et a].

4

zero and a constant M. An output dispersal rule provides for competitive distribution of activation: (2.2)

for both thalamic and cortical elements. The small predefined, nonnegative constant q serves two purposes: it dampens competitiveness and it prevents division by zero. The competitive learning rule for changing weight wjIon the connection to cortical element j from thalamic element i is Azql(t) = c[al(t) wIl(t)]a,( t). To maintain normalized incoming weight vectors, an explicit weight renormalization step is needed after the weight update takes place. Simulations were run both with and without this explicit weight renormalization step, and only minor differences were observed. The results reported here are for the normalized model to be consistent with Grajski and Merzenich (1990). Random, uniformly distributed hexagonal patches were used as input stimuli because of their simplicity, intuitive appeal, and similarity to stimuli used with some past models of SI cortex. To evaluate topographic map formation, we defined the measures Total response:

rj

(2.3)

= I

Center:

X,

Moments: wx,

WYj

=

=

=

(2.5)

Here a,, is the activation level of cortical element j when a point stimulus (input of 1.0 to a single thalamic element) is applied at thalamic element i, rI is the total response of cortical element j summed over the thalamic point stimuli, xI and yl are the x and y coordinates of thalamic element i, f, and y, are the x and y coordinates for the center of cortical element j’s receptive field, and wx,and wy,are the x and y moments of cortical element j’s receptive field. The x and y moments of the cortical receptive field do not indicate the entire extent of the receptive field, but rather are measures of its width analogous to the standard deviation. 3 Results

Figure 1 shows the coarse topographic map that existed before training due to initially random weights and the topographic projection of thalamocortical connections. The topographic map is plotted by placing


5

a.

b.

Figure 1: Cortical receptive field plots for the model before learning: (a) receptive field centers with nearest neighbors connected, and (b) receptive field centers and moments. points at the computed centers of the cortical elements’ receptive fields and connecting any two points which represent cortical elements that are nearest neighbors (Fig. la). The x and y cortical receptive field moments are plotted by drawing ellipses in sensory/thalamic space centered on the computed centers (Fig. lb). The lengths of the x and y axes of each ellipse represent the x and y moments (not the full receptive fields). Anal-

6


ogous plots can be defined for thalamic response fields (each thalamic point stimulus generates a response across the cortical layer called its response field), cortical incoming weight vectors, and thalamic outgoing weight vectors. Figure 2 shows that with training a finely tuned, uniform topographic map appeared, and receptive field moments became uniform and decreased in size. Incoming weight vectors of cortical elements also became very uniform compared to their initial random state; after training with hexagonal patches of radius two, the incoming weight vectors became roughly bell-shaped when the weights are plotted as a surface. When the fingertips of attending monkeys are stimulated much more frequently than the rest of the hand, the region of SI cortex which represents the fingertips increases in size (Jenkins et al. 1990). This increase in size is mostly at the expense of neighboring regions, but also at the expense of more distant regions. The receptive fields in the expanded fingertips region of cortex also show a decrease in size. For our model, these results were simulated by first performing topographic map formation as above. After the map formed (Fig. 2), the input scheme was changed so that an 8 x 16 region of the thalamic layer designated as the repetitively stimulated finger (second finger region from the left), was now seven times more likely to be stimulated than other regions. After the topographic map reorganized due to this change in the input scheme, a number of effects observed in animal studies occur in the competitive SI model (Fig. 3). The number of cortical elements whose receptive field centers were in the repetitively stimulated finger representation increased dramatically, more than doubling. Thus, there was a substantial increase in the cortical magnification for the repetitively stimulated finger, as is observed experimentally. The neighboring finger representations decreased in size and shifted, and even more distant finger representations were reduced in size. Following repetitive stimulation, for the enlarged representation of the second finger shown in Figure 3b the mean receptive field size did not decrease. However, receptive field size did decrease for a large number of the cortical elements whose receptive field centers lie toward the edges of the repetitively stimulated finger representation, consistent with the inverse magnification rule in these regions. To simulate an afferent lesion, our model was trained and then a contiguous portion of the thalamic layer corresponding to a single finger was deprived of sensory input. Input patterns within the deafferented finger region ceased to occur. With continued training, some cortical receptive fields which were in the deafferented finger region shifted outside of this region, forcing a resultant shift of surrounding cortical receptive fields. Some cortical elements near the center of the deafferented region received insufficient activation to reorganize and remained essentially unresponsive to all input patterns. Much reorganization in biological cortex involves replacement of deafferented glabrous representation by


7

a.

b.

Figure 2: Cortical receptive field plots for the model after training with randomly positioned hexagonal patches of radius two as input stimuli: (a) receptive field centers with nearest neighbors connected and (b) receptive field centers and moments. Contrast with Figure 1. The refined topographic map (b) has small, roughly equal-size receptive fields [actual receptive fields extend beyond the ovals drawn in (b) and overlap]. As a reference for discussion, boundaries for four "fingers" and a "palm" are added. In all the simulations described in this paper cs = -2.0 (self-inhibition), cp = 0.6 (excitatory gain), M = 3.0, 9 = 0.0001 in equations 2.1 and 2.2, and a time step of 0.5 is used, for cortical elements. The same values are used for thalamic elements except cp = 1.0. A learning rate of t = 0.01 is used. Distance between neighboring receptive fields here and in subsequent figures is 1.0.


8

a.

b.

Figure 3: Cortical receptive field moments plotted in cortical space and filled to show regional location of receptive field centers (a) after training with uniformly distributed stimuli, and (b) after subsequent repeated finger stimulation. The horizontal line indicates the breadth of the cortical representation of the selectively stimulated finger before (a) and after (b) repetitive stimulation. The x and y moments of the cortical receptive fields are represented by ellipses centered at the physical location of the cortical element in the cortex rather than the location of the receptive field center in sensory/thalamic space. The general location of receptive field centers is shown by filling ellipses with different gray scale patterns to indicate the different finger and palm regions.


9

new/extended representation of dorsal hand surfaces (Kaas 1990); this could not happen in our model as no sensory input corresponding to the dorsum of the hand was present. To simulate a focal cortical lesion in our model, a contiguous portion of the trained cortical layer (elements representing the second finger from the left) was deactivated after training (i.e., activation levels clamped at 0.0). After lesioning, the topographic map showed a two-phase reorganization process. Immediately after lesioning and before any retraining, the receptive fields of cortical elements adjoining the lesioned area shifted towards the second thalamic finger and increased in size (Fig. 4a). This immediate shift was due to the competitive redistribution of thalamic output from lesioned to unlesioned cortical elements. The second phase of map reorganization occurred more slowly with continued training and was due to synaptic weight changes (Fig. 4b).' Cortical representation of the "lesioned finger" was reduced in size (reduced magnification). The mean receptive field x moment (y moment) prior to the lesion was 0.626 (0.627) for the entire cortex. Following the cortical lesion and subsequent map reorganization, the mean receptive field x moment (y moment) increased to 0.811 (0.854) for elements within a distance of two of the lesion site (mostly shaded black in Fig. 4b), consistent with an inverse magnification rule. 4 Discussion

It has recently been proposed that competitive distribution of activity may underlie some inhibitory effects observed in neocortex (Reggia et al. 1992). The present study supports this hypothesis in two ways. First, we have shown that a computational model of cerebral cortex based on the competitive distribution hypothesis, starting from a coarse topographic map, can simulate the development of a highly-refined topographic map with focused, bell-shaped receptive fields. Once such a map was formed, changing the probability distribution of input stimuli resulted in substantial map reorganization. With repetitive stimuli to a localized region of the sensory surface, the cortical representation of that region increased dramatically and changes occurred that were in part consistent with an inverse magnification rule as has been observed experimentally (Jenkins et al. 1990). With deafferentation of a localized region many of the cortical elements originally representing the deafferented region came to represent surrounding sensory surface regions, as has been described experimentally (Kaas 1991). Of course, neither our model nor previous ones, being tremendous simplifications of real cortex ( e g , having very limited radii of interconnectivity and modeling only a small cortical 'Results reported here are for unchanged uniformly random stimuli. If stimuli frequency in the region originally represented in the lesioned cortex was increased, reorganization was even more pronounced.

P


11

region), can account for all experimental data related to these phenomena. For example, none of these models, including ours, adequately accounts for the almost immediate changes in receptive fields observed following deafferentation (Calford and Tweedle 1991; Kaas et al. 1990; Chino et al. 1992; Gilbert and Wiesel 1992) nor for long-term “massive” reorganization (Pons et al. 1991). Second, our competitive SI model exhibited dramatic map reorganization in response to a focal cortical lesion. Reorganization following a cortical lesion is fundamentally different from repetitive stimulation and deafferentation as there is no change in the probability distribution of input stimuli. To our knowledge, the only previous cortical model which has tried to simulate reorganization following a focal cortical lesion is the three-layer model of Grajski and Merzenich (1990). Following a focal cortical lesion, map reorganization did not occur with this earlier model unless the synaptic strengths of all intracortical and cortical afferent connections that remained intact were randomized, that is, unless the model reverted to its completely untrained state (it was also necessary to enhance cortical excitation or reduce cortical inhibition). In our SI model no special procedure such as weight randomization was required: sensory regions originally represented in the lesioned cortex spontaneously reappeared in cortex outside the lesion area. This spontaneous map reorganization is consistent with that seen experimentally following small cortical lesions (Jenkins and Merzenich 1987). Further, the model’s receptive fields increased in size in perilesion cortex as has also been described experimentally (Jenkins and Merzenich 1987). Demonstration that the competitive SI model reorganizes after a cortical lesion provides a potential computational model of stroke.2 Map reorganization following a cortical lesion to the competitive SI model involved a two-phase process where each phase, rapid and slow, is due to a different mechanism. Immediately after a cortical lesion, competitive distribution of activation caused some finger regions orginally represented by the lesioned area of cortex to ”shift outward” and appear in adjacent regions of intact cortex. This result provides a specific testable prediction for half of the competitive distribution hypothesis: if competitive distribution of activity is present from thalamus to cerebral cortex, then significant shifts of sensory representation out of a lesioned cortical area should be observed right after a cortical lesion. The second, slower phase of additional map reorganization is due to synaptic plasticity, and is apparently triggered by the first phase. It is not yet clear whether a model based on more traditional intracortical inhibitory connections can produce spontaneous reorganization following a cortical lesion. Further computational studies should determine whether the difficulties encountered in obtaining such reorganization are a general property of cortical models using inhibitory connections or whether they 2We have recently extended this work to a model of proprioceptive cortex based on

length and tension input from muscles in a model arm (Cho d ~ 1 1993). .

12


reflect specific details of the one computational model studied so far (Grajski and Merzenich 1990).

Acknowledgments Supported by NINDS Awards NS 29414 a n d NS 16332. The authors are also with the Department of Neurology a n d the Institute for Advanced Computer Studies at the University of Maryland.

References Calford, M., and Tweedle, R. 1991. Acute changes in cutaneous receptive fields in primary somatosensory cortex after digit denervation in adult flying fox. j . Neurophys. 65, 178-187. Chino, Y., Kaas, J., and Smith, E., et al. 1992. Rapid reorganization of cortical maps in adult cats following restricted deafferentation in retina. Vision Res. 32, 789-796. Cho, S., Reggia, J., and DAutrechy, C. 1993. Modelling map formation in proprioceptive cortex. Tech. Rep. CS-TR-3026, Dept. of Computer Science, Univ. of Maryland. Gilbert, C., and Wiesel, T. 1992. Receptive field dynamics in adult primary visual cortex. Nature (London) 356, 150-152. Grajski, K., and Merzenich, M. 1990. Hebb-type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy. Neural Comp. 2, 71-84. Jenkins, W., and Merzenich, M. 1987. Reorganization of neocortical representations after brain injury: A neurophysiological model of the bases of recovery from stroke. In Progress in Brain Research, Vol. 71, F. Seil, E. Herbert, and B. Carlson, eds., pp. 249-266. Elsevier, Amsterdam. Jenkins, W., Merzenich, M., Ochs, M., Allard, T., and Guic-Robles, E. 1990. Functional reorganization of primary somatosensory cortex in adult owl monkeys after behaviorally controlled tactile stimulation. 1. Neurophys. 63, 82-1 04. Kaas, J. 1991. Plasticity of sensory and motor maps in adult mammals. Ann. Reu. Neurosci. 14, 137. Kaas, I., Krubitzer, L., Chino, Y., et a/. 1990. Reorganization of retinotopic cortical maps in adult mammals after lesions of the retina. Science 248, 229-231. Kohonen, T. 1989. Self-Organization and Associatizw Memory. Springer-Verlag, Berlin. Merzenich, M., Kaas, J., et al. 1983. Topographic reorganization of somatosensory cortical areas 3b and 1 in adult monkeys following restricted deafferentation. Neuroscience 8, 33-55. Obermayer, K., Ritter, H., and Schulten, K. 1990. A neural network model for the formation of topographic maps in the CNS: Development of receptive


13

fields. Proceedirigs C J Iiiteriintioiinl ~ joint Coriferciice 017 Neirral Networks, Vol. 11, pp. 423-429. San Diego, CA. Pearson, J., Finkel, L., and Edelman, G. 1987. Plasticity in the organization of adult cerebral cortical maps: A computer simulation. j . Neurosci. 7, 42094223. Pons, T., Garraghty, I?, Ommaya, A., e t a / . 1991. Massive cortical reorganization after sensory deafferentation in adult macaques. Science 252, 1857-1860. Reggia, J., DAutrechy, C., Sutton, G., and Weinrich, M. 1992. A competitive distribution theory of neocortical dynamics. Neural Comp. 4, 287-317. Reggia, J., Berndt, R., and DAutrechy, C. 1993. Connectionist models in neuropsychology, Hatidbook of Neuropsycholog!y, Vol. 9, Elsevier, in press. Ritter, H., Martinetz, T., and Schulten, K. 1989. Topology-conserving maps for learning visuo-motor-coordination. Neirral Networks 2, 159-168. Sanes, J., Suner, S., Lando, J., and Donoghue, J. 1988. Rapid reorganization of adult rat motor cortex somatic representation patterns after motor nerve injury. Proc. Natl. Acnd. Sci. U.S.A.85, 2003. Sklar, E. 1990. A simulation of cortical map plasticity. Proc. IjCNN 111, 727-732. Sutton, G. 1992. Map formation in neural networks using competitive activation mechanisms. Ph.D. Dissertation, Department of Computer Science, CS-TR2932, Univ. of Maryland. von der Malsburg, C. 1973. Self-organization of orientation sensitive cells in the striate cortex. K!ybernetic 14, 85-100.

Received January 4, 1993; accepted June 1, 1993.

This article has been cited by: 1. Charles E. Martin, James A. Reggia. 2010. Self-assembly of neural networks viewed as swarm intelligence. Swarm Intelligence 4:1, 1-36. [CrossRef] 2. Jared Sylvester, James Reggia. 2009. Plasticity-Induced Symmetry Relationships Between Adjacent Self-Organizing Topographic MapsPlasticity-Induced Symmetry Relationships Between Adjacent Self-Organizing Topographic Maps. Neural Computation 21:12, 3429-3443. [Abstract] [Full Text] [PDF] [PDF Plus] 3. Reiner Schulz , James A. Reggia . 2005. Mirror Symmetric Topographic Maps Can Arise from Activity-Dependent Synaptic ChangesMirror Symmetric Topographic Maps Can Arise from Activity-Dependent Synaptic Changes. Neural Computation 17:5, 1059-1083. [Abstract] [PDF] [PDF Plus] 4. Reiner Schulz, James A. Reggia. 2004. Temporally Asymmetric Learning Supports Sequence Processing in Multi-Winner Self-Organizing MapsTemporally Asymmetric Learning Supports Sequence Processing in Multi-Winner Self-Organizing Maps. Neural Computation 16:3, 535-561. [Abstract] [PDF] [PDF Plus] 5. Svetlana Levitan , James A. Reggia . 2000. A Computational Model of Lateralization and Asymmetries in Cortical MapsA Computational Model of Lateralization and Asymmetries in Cortical Maps. Neural Computation 12:9, 2037-2062. [Abstract] [PDF] [PDF Plus] 6. Dean V. Buonomano, Michael M. Merzenich. 1998. CORTICAL PLASTICITY: From Synapses to Maps. Annual Review of Neuroscience 21:1, 149-186. [CrossRef] 7. Eytan Ruppin , James A. Reggia . 1995. Patterns of Functional Damage in Neural Network Models of Associative MemoryPatterns of Functional Damage in Neural Network Models of Associative Memory. Neural Computation 7:5, 1105-1127. [Abstract] [PDF] [PDF Plus] 8. Manfred Spitzer, Peter Böhler, Matthias Weisbrod, Udo Kischka. 1995. A neural network model of phantom limbs. Biological Cybernetics 72:3, 197-206. [CrossRef]

NOTE

Communicated by Michael Hines

An Efficient Method for Computing Synaptic Conductances Based on a Kinetic Model of Receptor Binding A. Destexhe Z. F. Mainen

T. J. Sejnowski The Howard Hughes Medical lnstitute and The Salk Institute, Computational Neurobiology Laboratory, 10010 North Torrey Pines Road, La lolla, C A 92037 USA

Synaptic events are often formalized in neural models as stereotyped, time-varying conductance waveforms. The most commonly used of such waveforms is the 0-function (Rall 1967):

where gsynis the synaptic conductance and to is the time of transmitter release. This function peaks at a value of l / e at t = to + T , and decays exponentially with a time constant of T . When multiple events occur in succession at a single synapse, the total conductance at any time is a sum of such waveforms calculated over the individual event times. There are several drawbacks to this method. First, the relationship to actual synaptic conductances is based only on an approximate correspondence of the time-course of the waveform to physiological recordings of the postsynaptic response, rather than plausible biophysical mechanisms. Second, summation of multiple waveforms can be cumbersome, since each event time must be stored in a queue for the duration of the waveform and necessitates calculation of an additional exponential during this period (but see Srinivasan and Chiel 1993). Third, there is no natural provision for saturation of the conductance. An alternative to the use of stereotyped waveforms is to compute synaptic conductances directly using a kinetic model (Perkel eta!. 1981). This approach allows a more realistic biophysical representation and is consistent with the formalism used to describe the conductances of other ion channels. However, solution of the associated differential equations generally requires computationally expensive numerical integration. In this paper we show that reasonable biophysical assumptions about synaptic transmission allow the equations for a simple kinetic synapse model to be solved analytically. This yields a mechanism that preserves the advantages of kinetic models while being as fast to compute as a single tr-function. Moreover, this mechanism accounts implicitly for satN p m l Computation 6, 14-18 (1994)

@ 1993 Massachusetts Institute of Technology

Computing Synaptic Conductances

15

uration and summation of multiple synaptic events, obviating the need for event queuing. Following the arrival of an action potential at the presynaptic terminal, neurotransmitter molecules, T , are released into the synaptic cleft. These molecules are taken to bind to postsynaptic receptors according to the following first-order kinetic scheme:

R

+

T $TR*

(2)

where R and TR* are, respectively, the unbound and the bound form of the postsynaptic receptor, N and 1) are the forward and backward rate constants for transmitter binding. Letting r represent the fraction of bound receptors, these kinetics are described by the equation

where [TI is the concentration of transmitter. There is evidence from both the neuromuscular junction (Anderson and Stevens 1973) and excitatory central synapses (Colquhoun et al. 1992) that the concentration of transmitter in the cleft rises and falls very rapidly. If it is assumed that [TI occurs as a pulse, then it is straightforward to solve equation 3 exactly, leading to the following expressions: 1. During a pulse ( t o < t < t,), [TI = T,,,

and r is given by

where

and

If the binding of transmitter to a postsynaptic receptor directly gates the opening of an associated ion channel, then the total conductance through all channels of the synapse is r multiplied by the maximal conductance of the synapse, gsyn.Response saturation occurs naturally as Y

A. Destexhe et al.

16

approaches 1 (all channels reach the open state). The synaptic current, Iscl,, is given by the equation Iwn(f)

r(t) [V,y,(t)

s,yn

1

-

(6)

Lyn]

where Vsyn is the postsynaptic potential, a n d E.,yn is the synaptic reversal potential. These equations provide a n easily implemented method for computing synaptic currents a n d have storage and computation requirements that are independent of the frequency of presynaptic release events. To simulate a synaptic connection, it is necessary only to monitor the state of the presynaptic terminal a n d switch from equation 5 to equation 4 for a fixed time following the detection of a n event. At each time step, this method requires the storage of just two state variables [either f,, a n d r(t,,) o r t , and r(tl)], a n d the calculation of a single exponential (either equation 4 or equation 5). This compares favorably to summing rr-functions, which requires storage of 17 release times a n d I I corresponding exponential evaluations, where 17 is the product of the maximum frequency of release events a n d the length of time for which the conductance waveform is calculated. The parameters of the kinetic synapse model can be fit directly to physiological measurements. For instance, duration of the excitatory neurotransmitter glutamate in the synaptic cleft has been estimated to be on the order of 1 insec at concentrations in the 1 mM range (Clements et nl. 1992; Colquhoun et nl. 1992). Figure 1 shows simulated synaptic

Figure I: Fnciricg page. Postsynaptic potentials from receptor kinetics. Presynaptic voltage, Vpre (mV); concentration of transmitter in the synaptic cleft, [T] (niM); fraction of open (i.e., transmitter-bound) postsynaptic receptors, r; synaptic current, Isyn(PA); and postsynaptic potential, VSyl1(mV), are shown for different conditions. (A) A single transmitter pulse evokes a fast, excitatory conductance ( r i = 2 msec-' mM /l = 1 msec Esyn = 0 mV). (B) A train of presynaptic spikes releases a series of transmitter pulses evoking excitatory synaptic conductances (parameters as in A). C and D correspond to A and 8,but with parameters set for slower, inhibitory synaptic currents (0 = 0.5 msec- I mM I , , j = 0.1 msec-', Esyn = -80 mV). For all simulations, the synaptic current was calculated using equations 4-6, with ysyn= 1 nS, T,,,, = 1 mM, and transmitter pulse duration ( t , to) = 1 msec. Membrane potentials were simulated using NEURON (Hines 1993). Presynaptic and postsynaptic compartments were described by single-compartment cylinders (10 pm diameter and 10 pm length) with passive (leak) conductance (specific membrane capacitance of 1 /rF/cm2, specific membrane resistance of 5000 61-cm2,leak reversal potential of -70 mV). Presynaptic action potentials were modeled by standard Hodgkin-Huxley kinetics. A transmitter pulse was initiated when Vpre exceeded a threshold of 0 mV, and pulse initiation was inhibited for 1 msec following event detection.

.',

~

',

Computing Synaptic Conductances

17

events obtained using these values. Figure 1A and B show fast, excitatory currents resulting from a single synaptic event and a train of four events. Note that the time course of the postsynaptic potential resembles an (Yfunction even though the underlying current does not. Figure 1C and D show the time courses of the same variables for a slower, inhibitory synapse. In this case the rates for ( t and /j were slower, allowing a more progressive saturation of the receptors. We have presented a method by which synaptic conductances can be computed with low computational expense using a kinetic model. The kinetic approach provides a natural means to describe the behavior of synapses in a way that handles the interaction of successive presynaptic events. Under the same assumption that transmitter concentration occurs as a pulse, more complex kinetic schemes can be treated

18

A. Destexhe et al.

in a manner analogous to that described above (Destexhe rt al. in preparation). The “kinetic synapse” can thus be generalized to give various conductance time courses with multiexponential rise a n d decay phases, without sacrificing the efficiency of the first-order model.

Acknowledgments This research was supported by the Howard Hughes Medical Institute, the U.S. Office of Naval Research, a n d the National Institutes of Mental Health. Z . F. M. is a Howard Hughes Medical Institute Predoctoral Fellow.

References Anderson, C. R., and Stevens, C. E 1973. Voltage clamp analysis of acetylcholineproduced end-plate current fluctuations at frog neuromuscular junction. 1. Physiol. (London) 235, 655-691. Clements, J. D., Lester, R. A. J., Tong, J., Jahr, C., and Westbrook, G. L. 1992. The time course of glutamate in the synaptic cleft. Scirrice 258, 1498-1501. Colquhoun, D., Jonas, P., and Sakmann, B. 1992. Action of brief pulses of glutamate on AMPA/KAINATE receptors in patches from different neurons of rat hippocampal slices. 1. Physiol. (Lotidon) 458, 261-287. Hines, M. 1993. NEURON-A program for simulation of nerve equations. In Neural Systems: Analysis and Modeling, F. Eeckman, ed., pp. 127-136. Kluwer Academic Publishers, Norwell, MA. Perkel, D. H., Mulloney, B., and Budelli, R. W. 1981. Quantitative methods for predicting neuronal behavior. Neurosci. 6, 823-827. Rall, W. 1967. Distinguishing theoretical synaptic potentials computed for different somedendritic distributions of synaptic inputs. I. Ncurophysiol. 30, 1138-11 68. Srinivasan, R., and Chiel, H. J. 1993. Fast calculation of synaptic conductances. Neural Conip. 5, 200-204.

Received March 31, 1993; accepted May 26, 1993.

This article has been cited by: 1. Samuel A. Neymotin, Kimberle M. Jacobs, André A. Fenton, William W. Lytton. 2010. Synaptic information transfer in computer models of neocortical columns. Journal of Computational Neuroscience . [CrossRef] 2. Ming Yi, Lijian Yang. 2010. Propagation of firing rate by synchronization and coherence of firing pattern in a feed-forward multilayer neural network. Physical Review E 81:6. . [CrossRef] 3. M. A. Komarov, G. V. Osipov, J. A. K. Suykens. 2009. Sequentially activated groups in neural networks. EPL (Europhysics Letters) 86:6, 60006. [CrossRef] 4. Sheng-Jun Wang, Xin-Jian Xu, Zhi-Xi Wu, Zi-Gang Huang, Ying-Hai Wang. 2008. Influence of synaptic interaction on firing synchronization and spike death in excitatory neuronal networks. Physical Review E 78:6. . [CrossRef] 5. Kalyan V. Srinivas, Sujit K. Sikdar. 2008. Epileptiform activity induces distance-dependent alterations of the Ca 2+ extrusion mechanism in the apical dendrites of subicular pyramidal neurons. European Journal of Neuroscience 28:11, 2195-2212. [CrossRef] 6. Rogerio R. L. Cisi, André F. Kohn. 2008. Simulation system of spinal cord motor nuclei and associated nerves and muscles, in a Web-based architecture. Journal of Computational Neuroscience 25:3, 520-542. [CrossRef] 7. Xu-Long Wang, Xiao-Dong Jiang, Pei-Ji Liang. 2008. Intracellular calcium concentration changes initiated by N-methyl-D-aspartic acid receptors in retinal horizontal cells. NeuroReport 19:6, 675-678. [CrossRef] 8. Romain Brette, Michelle Rudolph, Ted Carnevale, Michael Hines, David Beeman, James M. Bower, Markus Diesmann, Abigail Morrison, Philip H. Goodman, Frederick C. Harris, Milind Zirpe, Thomas Natschläger, Dejan Pecevski, Bard Ermentrout, Mikael Djurfeldt, Anders Lansner, Olivier Rochel, Thierry Vieville, Eilif Muller, Andrew P. Davison, Sami El Boustani, Alain Destexhe. 2007. Simulation of networks of spiking neurons: A review of tools and strategies. Journal of Computational Neuroscience 23:3, 349-398. [CrossRef] 9. Giuseppe Massobrio, Paolo Massobrio, Sergio Martinoia. 2007. Modeling and simulation of silicon neuron-to-ISFET junction. Journal of Computational Electronics 6:4, 431-437. [CrossRef] 10. Quan Zou, Alain Destexhe. 2007. Kinetic models of spike-timing dependent plasticity and their functional consequences in detecting correlations. Biological Cybernetics 97:1, 81-97. [CrossRef] 11. Gregory R. Stiesberg, Marcelo Bussotti Reyes, Pablo Varona, Reynaldo D. Pinto, Ramón Huerta. 2007. Connection Topology Selection in Central Pattern Generators by Maximizing the Gain of InformationConnection Topology Selection in Central Pattern Generators by Maximizing the Gain of Information. Neural Computation 19:4, 974-993. [Abstract] [PDF] [PDF Plus]

12. T Pereira, M. S Baptista, J Kurths. 2007. Detecting phase synchronization by localized maps: Application to neural networks. Europhysics Letters (EPL) 77:4, 40006. [CrossRef] 13. T. Pereira, M. Baptista, J. Kurths. 2007. General framework for phase synchronization through localized sets. Physical Review E 75:2. . [CrossRef] 14. Mario F. Simoni, Stephen P. DeWeerth. 2007. Sensory Feedback in a Half-Center Oscillator Model. IEEE Transactions on Biomedical Engineering 54:2, 193-204. [CrossRef] 15. Pablo Balenzuela, Javier Buldú, Marcos Casanova, Jordi García-Ojalvo. 2006. Episodic synchronization in dynamically driven neurons. Physical Review E 74:6. . [CrossRef] 16. Yuan Wu-Jie, Luo Xiao-Shu, Wang Bing-Hong, Wang Wen-Xu, Fang Jin-Qing, Jiang Pin-Qun. 2006. Excitation Properties of the Biological Neurons with Side-Inhibition Mechanism in Small-World Networks. Chinese Physics Letters 23:11, 3115-3118. [CrossRef] 17. Simon Durrant, Jianfeng Feng. 2006. Negatively correlated firing: the functional meaning of lateral inhibition within cortical columns. Biological Cybernetics 95:5, 431-453. [CrossRef] 18. M.F. Simoni, S.P. Deweerth. 2006. Two-Dimensional Variation of Bursting Properties in a Silicon-Neuron Half-Center Oscillator. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14:3, 281-289. [CrossRef] 19. Romain Brette. 2006. Exact Simulation of Integrate-and-Fire Models with Synaptic ConductancesExact Simulation of Integrate-and-Fire Models with Synaptic Conductances. Neural Computation 18:8, 2004-2027. [Abstract] [PDF] [PDF Plus] 20. P. Suffczynski, F. Wendling, J.-J. Bellanger, F.H. Lopes Da Silva. 2006. Some Insights Into Computational Models of (Patho)Physiological Brain Activity. Proceedings of the IEEE 94:4, 784-804. [CrossRef] 21. Dexter M. Easton. 2005. Gompertz kinetics model of fast chemical neurotransmission currents. Synapse 58:1, 53-61. [CrossRef] 22. Pablo Balenzuela, Jordi García-Ojalvo. 2005. Role of chemical synapses in coupled neurons with noise. Physical Review E 72:2. . [CrossRef] 23. William W. Lytton , Michael L. Hines . 2005. Independent Variable Time-Step Integration of Individual Neurons for Network SimulationsIndependent Variable Time-Step Integration of Individual Neurons for Network Simulations. Neural Computation 17:4, 903-921. [Abstract] [PDF] [PDF Plus] 24. Pablo Balenzuela, Jordi García-Ojalvo. 2005. Neural mechanism for binaural pitch perception via ghost stochastic resonance. Chaos: An Interdisciplinary Journal of Nonlinear Science 15:2, 023903. [CrossRef]

25. Katsuki KATAYAMA, Tsuyoshi HORIGUCHI. 2005. Synchronous Phenomena of Neural Network Models Using Hindmarsh-Rose Equation. Interdisciplinary Information Sciences 11:1, 11-15. [CrossRef] 26. A Volkovskii, S Brugioni, R Levi, M Rabinovich, A Selverston, H D I Abarbane. 2005. Analog electronic model of the lobster pyloric central pattern generator. Journal of Physics: Conference Series 23, 47-57. [CrossRef] 27. Toshiaki Omori, Tsuyoshi Horiguchi. 2004. Dynamical State Transition by Neuromodulation Due to Acetylcholine in Neural Network Model for Oscillatory Phenomena in Thalamus. Journal of the Physics Society Japan 73:12, 3489-3494. [CrossRef] 28. S. Martinoia, P. Massobrio, M. Bove, G. Massobrio. 2004. Cultured Neurons Coupled to Microelectrode Arrays: Circuit Models, Simulations and Experimental Data. IEEE Transactions on Biomedical Engineering 51:5, 859-864. [CrossRef] 29. Sang-Gui Lee, Shigeru Tanaka, Seunghwan Kim. 2004. Orientation tuning and synchronization in the hypercolumn model. Physical Review E 69:1. . [CrossRef] 30. Masaki Nomura , Tomoki Fukai , Toshio Aoyagi . 2003. Synchrony of Fast-Spiking Interneurons Interconnected by GABAergic and Electrical SynapsesSynchrony of Fast-Spiking Interneurons Interconnected by GABAergic and Electrical Synapses. Neural Computation 15:9, 2179-2198. [Abstract] [PDF] [PDF Plus] 31. Toshio Aoyagi , Takashi Takekawa , Tomoki Fukai . 2003. Gamma Rhythmic Bursts: Coherence Control in Networks of Cortical Pyramidal NeuronsGamma Rhythmic Bursts: Coherence Control in Networks of Cortical Pyramidal Neurons. Neural Computation 15:5, 1035-1061. [Abstract] [PDF] [PDF Plus] 32. Jan Reutimann , Michele Giugliano , Stefano Fusi . 2003. Event-Driven Simulation of Spiking Neurons with Stochastic DynamicsEvent-Driven Simulation of Spiking Neurons with Stochastic Dynamics. Neural Computation 15:4, 811-830. [Abstract] [PDF] [PDF Plus] 33. P. Tiesinga. 2001. Information transmission and recovery in neural communication channels revisited. Physical Review E 64:1. . [CrossRef] 34. Michael Rudolph, Alain Destexhe. 2001. Correlation Detection and Resonance in Neural Systems with Distributed Noise Sources. Physical Review Letters 86:16, 3662-3665. [CrossRef] 35. M. Eguia, M. Rabinovich, H. Abarbanel. 2000. Information transmission and recovery in neural communications channels. Physical Review E 62:5, 7111-7122. [CrossRef] 36. Elizabeth Thomas , Thierry Grisar . 2000. Increased Synchrony with Increase of a Low-Threshold Calcium Conductance in a Model Thalamic Network: A Phase-Shift MechanismIncreased Synchrony with Increase of a Low-Threshold Calcium Conductance in a Model Thalamic Network: A

Phase-Shift Mechanism. Neural Computation 12:7, 1553-1571. [Abstract] [PDF] [PDF Plus] 37. Michele Giugliano . 2000. Synthesis of Generalized Algorithms for the Fast Computation of Synaptic Conductances with Markov Kinetic Models in Large Network SimulationsSynthesis of Generalized Algorithms for the Fast Computation of Synaptic Conductances with Markov Kinetic Models in Large Network Simulations. Neural Computation 12:4, 903-931. [Abstract] [PDF] [PDF Plus] 38. Luis Lago-Fernández, Ramón Huerta, Fernando Corbacho, Juan Sigüenza. 2000. Fast Response and Temporal Coherent Oscillations in Small-World Networks. Physical Review Letters 84:12, 2758-2761. [CrossRef] 39. Michele Giugliano , Marco Bove , Massimo Grattarola . 1999. Fast Calculation of Short-Term Depressing Synaptic ConductancesFast Calculation of Short-Term Depressing Synaptic Conductances. Neural Computation 11:6, 1413-1426. [Abstract] [PDF] [PDF Plus] 40. BRIAN MULLONEY, FRANCES K. SKINNER, HISAAKI NAMBA, WENDY M. HALL. 1998. Intersegmental Coordination of Swimmeret Movements: Mathematical Models and Neural Circuitsa. Annals of the New York Academy of Sciences 860:1 NEURONAL MECH, 266-280. [CrossRef] 41. J. Köhn , F. Wörgötter . 1998. Employing the Z-Transform to Optimize the Calculation of the Synaptic Conductance of NMDA and Other Synaptic Channels in Network SimulationsEmploying the Z-Transform to Optimize the Calculation of the Synaptic Conductance of NMDA and Other Synaptic Channels in Network Simulations. Neural Computation 10:7, 1639-1651. [Abstract] [PDF] [PDF Plus] 42. Richard Bertram. 1997. A Simple Model of Transmitter Release and FacilitationA Simple Model of Transmitter Release and Facilitation. Neural Computation 9:3, 515-523. [Abstract] [PDF] [PDF Plus] 43. Alain Destexhe. 1997. Conductance-Based Integrate-and-Fire ModelsConductance-Based Integrate-and-Fire Models. Neural Computation 9:3, 503-514. [Abstract] [PDF] [PDF Plus] 44. M. Storace, M. Bove, M. Grattarola, M. Parodi. 1997. Simulations of the behavior of synaptically driven neurons via time-invariant circuit models. IEEE Transactions on Biomedical Engineering 44:12, 1282-1287. [CrossRef] 45. M. Bove, M. Grattarola, G. Verreschi. 1997. In vitro 2-D networks of neurons characterized by processing the signals recorded with a planar microtransducer array. IEEE Transactions on Biomedical Engineering 44:10, 964-977. [CrossRef] 46. Xing Fang Li, Jorge L. Armony, Joseph E. LeDoux. 1996. GABAA and GABAB receptors differentially regulate synaptic transmission in the auditory thalamo-amygdala pathway: An in vivo microiontophoretic study and a model. Synapse 24:2, 115-124. [CrossRef]

47. William W. Lytton . 1996. Optimizing Synaptic Conductance Calculation for Network SimulationsOptimizing Synaptic Conductance Calculation for Network Simulations. Neural Computation 8:3, 501-509. [Abstract] [PDF] [PDF Plus] 48. Bo Cartling. 1996. Dynamics control of semantic processes in a hierarchical associative memory. Biological Cybernetics 74:1, 63-71. [CrossRef] 49. François Chapeau-Blondeau , Nicolas Chambet . 1995. Synapse Models for Neural Networks: From Ion Channel Kinetics to Multiplicative Coefficient wijSynapse Models for Neural Networks: From Ion Channel Kinetics to Multiplicative Coefficient wij. Neural Computation 7:4, 713-734. [Abstract] [PDF] [PDF Plus] 50. M. Grattarola, M. Bove, S. Martinoia, G. Massobrio. 1995. Silicon neuron simulation with SPICE: tool for neurobiology and neural networks. Medical & Biological Engineering & Computing 33:4, 533-536. [CrossRef] 51. Alain Destexhe, Zachary F. Mainen, Terrence J. Sejnowski. 1994. Synthesis of models for excitable membranes, synaptic transmission and neuromodulation using a common kinetic formalism. Journal of Computational Neuroscience 1:3, 195-230. [CrossRef] 52. Gene V. Wallenstein. 1994. Simulation of GABA B -receptor-mediated K+ current in thalamocortical relay neurons: tonic firing, bursting, and oscillations. Biological Cybernetics 71:3, 271-280. [CrossRef] 53. Elizabeth ThomasModeling Individual Neurons and Small Neural Networks . [CrossRef]

Communicated by David Robinson

A Neural Network for Coding of Trajectories by Time Series of Neuronal Population Vectors Alexander V. Lukashin Apostolos P. Georgopoulos Brain Sciences Center, Department of Veterans Affairs Medical Center, Minneapolis, M N 55417 U S A , and Departments of Physiology and Neurology, University of Minnesota Medical School, Minneapolis, M N 55455 U S A

The neuronal population vector is a measure of the combined directional tendency of the ensemble of directionally tuned cells in the motor cortex. It has been found experimentally that a trajectory of limb movement can be predicted by adding together population vectors, tipto-tail, calculated for successive instants of time to construct a neural trajectory. In the present paper we consider a model of the dynamic evolution of the population vector. The simulated annealing algorithm was used to adjust the connection strengths of a feedback neural network so that it would generate a given trajectory by a sequence of population vectors. This was repeated for different trajectories. Resulting sets of connection strengths reveal a common feature regardless of the type of trajectories generated by the network namely, the mean connection strength was negatively correlated with the angle between the preferred directions of neuronal pair involved in the connection. The results are discussed in the light of recent experimental findings concerning neuronal connectivity within the motor cortex.

1 Introduction The activity of a directionally tuned neuron in the motor cortex is highest for a movement in a particular direction (the neuron’s preferred direction) and decreases progressively with movements farther away from this direction. Quantitatively, the change of neuron activity can be approximated by the cosine of the angle between the movement direction and the neuron’s preferred direction (Georgopoulos et al. 1982). The direction of an upcoming movement in space can be represented in the motor cortex as the neuronal population vector which is a measure of the combined directional tendency of the whole neuronal ensemble (Georgopoulos et al. 1983, 1986). If C; is the unit preferred direction vector for the ith neuron, Nrurd Computation 6, 19-28 (1994)


Alexander V. Lukashin and Apostolos P. Georgopoulos

20

then the neuronal population vector P is defined as the weighted sum of these vectors:

P(t)=

c

Vl(t)Cl

(1.1)

1

where the weight V , ( t ) is the activity (frequency of discharge) of the ith neuron at time bin t. The neuronal population vector has proved to be a good predictor of the direction of movement (for a review see Georgopoulos 1990; Georgopoulos et a/. 1993). Moreover, the population vector can be used as a probe by which to monitor in time the changing directional tendency of the neuronal ensemble. One can obtain the time evolution of the population vector by calculating it at short successive intervals of time or continuously, during the periods of interest. Adding these population vectors together, tip-to-tail, one may obtain a neural trajectory. It was shown that real trajectories of limb movement can be accurately predicted by neural trajectories (Georgopoulos et nl. 1988; Schwartz and Anderson 1989; Schwartz 1993). It was hypothesized (Georgopoulos et al. 1993) that the observed dynamic evolution of the neuronal population vector is governed by the interactions between directionally tuned neurons in motor cortex while extrinsic inputs can initiate the changes in activity and contribute temporarily or constantly to the ongoing activity. Two types of neural network models could be suggested in the framework of the hypothesis. Within a model of the first type, the movement is decomposed in piecewise parts, and local geometric parameters of a desired trajectory are introduced into the network by the mechanism of continuous updating of the current position (Bullock and Grossberg 1988; Lukashin and Georgopoulos 1993). The main disadvantage of this model is that it needs a mechanism for relatively fast local changes of synaptic weights during the movements. The second type of models may be treated as an opposite limiting case. It could be supposed that subsets of synaptic weights in the motor cortex permanently store information about possible trajectories or at least about its essential parts, and synaptic weights do not change during the movement. Then for realization of a particular trajectory, only one external command is needed: namely, a global activation of an appropriate neuronal subset. The purpose of the present paper is to simulate dynamic evolution of the neuronal population vector in the framework of the second model above. We consider a one-layer feedback network that consists of fully interconnected neuron-like units. In full analogy with experimental approaches, the neuronal population vector is calculated at successive instants of time in accordance with equation 1.1 as a vector sum of activities of units. A neural trajectory is computed by attaching these vectors tip-to-tail. The network is trained to generate the neural trajectory that coincides with a given curve, and its synaptic weights are adjusted until it does. This is repeated for different trajectories. It is obvious that practically any kind of reasonable dynamic evolution could be reached by

Neural Network for Coding Trajectories

21

appropriate learning procedure; for example, rather complex dynamics of trained neuronal ensembles have been demonstrated by Jordan (1986), Pineda (1987), Dehaene et al. (1987), Massone and Bizzi (1989), Pearlmutter (1989), Williams and Zipser (1989), Fang and Sejnowski (1990), and Amirikian and Lukashin (1992). For the same network design, learning different trajectories entails different sets of synaptic weights. Moreover, one and the same trajectory can be generated by the network with different sets of connection strengths. The main question we address in the present paper is whether these sets of connection strengths reveal common features. The results of this analysis are compared with experimental data (Georgopoulos et a / . 1993) concerning functional connections between directionally tuned neurons in the motor cortex. 2 Model and Learning Procedure

We consider a network of N neurons whose dynamics is governed by the following system of coupled differential equations: 1fU;

Tdt V,(t) El

-

(2.1)

-

=

tanh[ir,(t)]

(2.2)

=

COS(H

(2.3)

- (ti)

Argument t is shown for values which depend on time. The variable u l (t ) represents internal state (for example, the soma membrane potential) and the variable V , (t ) represents correspondingly the output activity (for example, firing frequency) of the ith neuron, T is a constant giving the time scale of the dynamics, and zu,, is the strength of interaction between neurons (j+ i). External input El (2.3) serves to assign preferred direction for the ith neuron. Indeed, in the simplest case, ZU,~ = 0, one has 1 4 , ( f >> T ) = El and V , FZ cos(0 - 0 , ) .Thus, if the angle H is treated as a direction of "movement" that is given externally, then the angle o, can be regarded as the preferred direction for the ith neuron. It is noteworthy that preferred directions of motor cortical neurons range throughout the directional continuum (Georgopoulos et al. 1988). The same type of distribution was obtained for a network that learns arbitrary transformations between input and output vectors (Lukashin 1990). Therefore, below we use random uniform distribution of angles ( t , . Once preferred directions are assigned, components of the neuronal population vector P can be calculated as the decomposition (equation 1.1) over preferred directions:

P,(t)

=

cV , ( t )

cos

I

(ti

(2.4)

22


where the time dependence of the V , values is determined by equations 2.1-2.3. Equations 2.4 may be interpreted as an addition of two output units with assigned synaptic weights. Let a desired two-dimensional trajectory be given as a sequence of points with coordinates X d ( t k ) , Y'l(tk), k = 1,. . . K . In accordance with the above consideration corresponding points X l l ( t k ) ,Yn(tk) of the actual trajectory generated by the network should be calculated by attaching successive population vectors: (2.5) The goal of a training procedure is to find a set of connection strengths wlIthat ensures that the difference between desired and actual trajectories is as small as possible. We minimized this difference by means of the simulated annealing algorithm (Kirkpatrick et al. 1983)treating the chosen cost function

as the "energy" of the system. The optimization scheme is based on the standard Monte Carlo procedure (Aart and van Laarhoven 1987) that accepts not only changes in synaptic weights wi/that lower the energy, but also changes that raise it. The probability of the latter event is chosen such that the system eventually obeys the Boltzmann distribution at a given temperature. The simulated annealing procedure is initialized at a sufficiently high temperature, at which a relatively large number of state changes are accepted. The temperature is then decreased according to a cooling schedule. If the cooling is slow enough for equilibrium to be established at each temperature, the global minimum is reached in the limit of zero temperature. Although the achievement of the global minimum cannot be guaranteed in practice when the optimal cooling rate is unknown, the simulated annealing algorithm seems to be the most adequate procedure for our specific purposes. We wish to extract the common features of the sets of synaptic weights ensuring different trajectories. In general, each given trajectory can be realized by different sets of synaptic weights. A complete analysis of the problem needs exhaustive enumeration of all possible network configurations that can be done only for sufficiently simple systems (Carnevali and Patarnello 1987; Denker et al. 1987; Baum and Haussler 1989; Schwartz et al. 1990). The advantage of the simulated annealing method is that during the procedure a treated system at each temperature (including zero-temperature limit) tends to occupy likeliest (in a thermodynamical sense) regions of the phase space (Kirkpatrick et nl. 1983; Aart and van Laarhoven 1987). Thus the algorithm provides a useful tool for obtaining likeliest or "typical" solution of the problem.


23

3 Results of Simulations

The minimal size of the network that still allows the realization of the desired dynamics is about 10 units. In routine calculations we used networks with number of neurons N from 16 to 48. Since in this range the size of the network was not an essential parameter, below we show the results only for N = 24. During the learning procedure, the randomly chosen set of preferred directions of was not varied. For each selected set of connection strengths w,/, the system of equations 2.1-2.3 was solved as the initial value problem, u,(O) = 0, using a fifth-order Runge-Kutta-Fehlberg formula with automatic control of the step size during the integration. Components of the neuronal population vector (equation 2.4), current positions on the actual trajectory (equation 2.51, and the addition to the cost function (equation 2.6) were calculated at time instances separated from each other by the interval ~ / 1 0 0 .The total running time ranged from 7 ( K = 100) to 57 ( K = 500). Below we show results for K = 300. The time constant, T , is usually thought of as the membrane time constant, about 5 msec. At this point one should take into account that the running time in the model is completely determined by the time that it takes for a desired trajectory to complete, and may be given arbitrarily. Since the crucial parameter for us was the shape of trajectories, we considered short (or fast) trajectories in order to make the training procedure less time-consuming. Slower trajectories can easily be obtained. Nevertheless, we note that a direct comparison of the velocity of the real movement and the velocity obtained in the model is impossible. The model operates with neural trajectories, and the question of how the "neural" length is related to the real length of a trajectory cannot be answered within the model. For each learning trial, the connection strengths w,/ were initialized to uniform random values between -0.5 and 0.5. The temperature at the initial stages of the simulated annealing was chosen so that practically all states of the system were accepted. During the simulated annealing procedure, values w,!were selected randomly from the same interval [-0.5,0.5] without assuming symmetry. The angle H (equation 2.3) was also treated as a variable parameter on the interval [ O , T ] . We used the standard exponential cooling schedule (Kirkpatrick et al. 1983): Tn+l = /jT,i, where T,, is the temperature at the nth step and the value 1 - /jis varied within the interval from 5 ~ 1 0 to -~ Each step of the simulated annealing procedure included a change of one parameter and the entire recalculation of the current trajectory. We checked the robustness of the results with respect to different series of random numbers used for the generation of particular sets of preferred directions It, and during the realization of the simulated annealing procedure (about 10 trials for each desired trajectory; data not shown). Figure 1 shows three examples of desired curves and trajectories produced by the trained network described above. It is seen that actual


24

a

b dellred

IClUd

Figure 1: The (X. Y)-plots of desired (upper) and actual (lower) trajectories. Arrows show directions of tracing. The actual curves shown were obtained after the following number of steps of the simulated annealing procedure: 2 x lo4 for the orthogonal bend (a), 9 x lo4 for the sinusoid (b), and 4 x lo5 for the ellipse with the relation between axes 3 : 1 (c). trajectories generated by the network reproduce the desired ones very well. The trajectories generated by the network (Fig. 1) d o not correspond to the global minimum of the cost function (equation 2.6). In all cases these are local minima. This is the reason why the corner in Figure l a is rounded and the curve in Figure l c is not closed. If allowed to continue, Figure l c would trace a finite unclosed trajectory. However, we have found that a limit cycle close to the desired elliptic trajectory can be obtained if the network is trained to trace twice the elliptic trajectory. To extract the common features of the sets of synaptic weights giving the dynamics shown in Figure 1 we calculated the mean value of the synaptic weight as a function of the angle between the preferred directions of the two neurons in a pair. Corresponding results are shown in Figure 2a, b, c for each trajectory presented in Figure la, b, c. Regardless of the type of trajectories generated by the network, the mean connection strength is negatively correlated with the angle between preferred directions: r = -0.86 for the orthogonal bend (Fig. 2a), -0.90 for the sinusoid (Fig. Zb), and -0.95 for the ellipse (Fig. 2c). 4 Discussion

Increasing efforts have been recently invested in neural network models for motor control (see, for example, Bullock and Grossberg 1988; Massone and Bizzi 1989; Kawato et al. 1990; Burnod et al. 1992; Corradini et 01. 1992). An important question is whether the neural networks that control different types of movements share many or few neuronal subsets. At one end of the spectrum, quite different behavior could be produced by


b

a 0.2:

0.25

0.oc

O

.

O

0

k

l

25

'

T

C 0.25

0.00

l

-0.25

.n

I1L

Llll

Figure 2: The dependence of the mean value (kSEM) of connection strength on the angle between preferred directions of neurons involved in the connection. The mean value of connection strength was calculated by averaging over connections between neurons the preferred directions of which did not differ from each other by more than 18". Straight lines are linear regressions. Connection strengths w,, used in the calculation of mean values were the same zu,, parameters that gave actual trajectories presented in Figure 1: (a) orthogonal bend, (b) sinusoid, and (c) ellipse.

continuous modulation of a single network. At the other end, different subsets could generate each type of movement or "movement primitive." Taking together in sequential chains or in parallel combinations these movement primitives may provide a variety of natural behavior. Both types of organization have been found experimentally (for a discussion see, for example, Alexander et al. 1986; Harris-Warrick and Marder 1991; Bizzi ef al. 19911. Clearly, intermediate cases involving multiple networks with overlapping elements are likely. The model we have used implies that synaptic weights d o not change during the movement. This means that at the level of the motor cortex different trajectories are realized by different neuronal subsets or by different sets of synaptic weights which store the information about trajectories. Our main result is that although different trajectories correspond to different sets of synaptic weights, all of these sets have clearly a common feature: namely, neurons with similar preferred directions tend to be mutually excitatory, those with opposite preferred directions tend to be mutually inhibitory, whereas those with orthogonal preferred directions tend to be connected weakly or not at all (see Fig. 2). Remarkably, the same structure of the synaptic weights matrix was obtained in modeling

26


studies of connection strengths that would ensure the stability of the neuronal population vector (Georgopoulos et a/. 1993). The results of this study are relevant to those obtained experimentally in the motor cortex (Georgopoulos et a / . 1993). In those studies, the connectivity between cells was examined by recording the impulse activity of several neurons simultaneously. The correlation between the firing times of pairs of neurons was examined. The correlation reveals the net effect of the whole synaptic substrate through which two neurons interact, including both direct and indirect connections; it represents the “functional connection” between the two neurons. The weight of a connection was estimated by calculating the “difference distribution” between the observed and randomly shuffled distributions of waiting times (for details see Note 34 in Georgopoulos ef a/. 1993). It was found that the mean connection strength was negatively correlated with the difference between preferred directions of the neurons in a pair ( r = -0.815). This result is in good agreement with the results of our calculations (Fig. 2). Although the weight of the functional connection estimated experimentally is not completely equivalent to the efficacy of single synapse that is implied in the model, our simulations show how this type of the organization of connections in the motor cortex can provide a dynamic evolution of the neuronal population vector during the limb movement. The correlations between the strength of interaction and a similarity among units observed in the experiments and in our simulations might reflect a general principle of the organization of connections in the central nervous system (for a discussion see Tso ef a/. 1986; Martin 1988; Sejnowski et a / . 1988; Douglas et a/. 1989; Georgopoulos et a / . 1993). Acknowledgments This work was supported by United States Public Health Service Grants NS17413 and PSMH48185, Office of Naval Research contract N00014-88K-0751, and a grant from the Human Frontier Science Program. A. V. Lukashin is on leave from the Institute of Molecular Genetics, Russian Academy of Sciences, Moscow, Russia. References Aart, E. H. L., and van Laarhoven, P. J. M. 1987. Simulated Annealing: A Review of the Theory nnd Applications. Kluwer Academic Publishers. Alexander, G. A., DeLong, M. R., and Strick, P. L. 1986. Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annu. Rev. Neurosci. 9, 375-381. Amirikian, B. R., and Lukashin, A. V. 1992. A neural network learns trajectory of motion from the least action principle. Bid. Cybern. 66, 261-264.


27

Baum, E. B., and Haussler, D. 1989. What size net gives valid generalization? Neural Comp. 1, 151-160. Bizzi, E., Musa-Ivaldi, F., and Giszter, S. 1991. Computations underlying the execution of movement: A biological perspective. Science 253, 287-291. Bullock, D., and Grossberg, S. 1988. Neural dynamics of planned arm movements: Emergent invariants and speed-accuracy properties during trajectory formation. Psychol. Rev. 95, 49-90. Burnod, Y., Grandguillaume, P., Otto, I., Ferraina, S., Johnson, P. B., and Caminiti, R. 1992. Visuomotor transformations underlying arm movement toward visual targets: A neural network model of cerebral cortical operations. J. Neurosci. 12, 1435-1453. Carnevali, P., and Patarnello, S. Exhaustive thermodynamical analysis of Boolean learning networks. Europhys. Lett. 4, 1199-1204. Corradini, M. L., Gentilucci, M., Leo, T., and Rizzolatti, G. 1992. Motor control of voluntary arm movements. Kinematic and modelling study. Biol. Cybern. 67, 347-360. Dehaene, S., Changeux, J. P., and Nadal, J. P. 1987. Neural networks that learn temporal sequences by selection. Proc. Natl. Acad. Sci. U.S.A. 84, 2727-2731. Denker, J., Schwartz, D., Wittner, B., Solla, S., Howard, R., Jackel, L., and Hopfield, J. 1987. Large automatic learning, rule extraction, and generalization. Complex Syst. 1, 877-922. Douglas, R. J., Martin K. A. C., and Whitteridge, D. 1989. A canonical microcircuit for neocortex. Neural Comp. 1, 480438. Fang, Y., and Sejnowski, T. J. 1990. Faster learning for dynamic recurrent backpropagation. Neural Comp. 2, 270-273. Georgopoulos, A. P. 1990. Neural coding of the direction of reaching and a comparison with saccadic eye movements. Cold Spring Harbor Symp. Quant. Biol. 55, 849-859. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R., and Massey, J. T. 1982. On the relation between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527-1537. Georgopoulos, A. P., Caminiti, R., Kalaska, J. F., and Massey, J. T. 1983. Spatial coding of movement: A hypothesis concerning the coding of movement by motor cortical populations. Exp. Brain Res. Suppl. 7, 327-336. Georgopoulos, A. P., Schwartz, A. B., and Kettner, R. E. 1986. Neuronal population coding of movement direction. Science 233, 1416-1419. Georgopoulos, A. P., Kettner, R. E., and Schwartz, A. B. 1988. Primate motor cortex and free arm movement to visual targets in three-dimensional space. 11. Coding of the direction of movement by a neuronal population. J. Neurosci. 8, 2928-2937. Georgopoulos, A. P., Taira, M., and Lukashin, A. V. 1993. Cognitive neurophysiology of the motor cortex. Science 260, 47-52. Harris-Warrick, R. M., and Marder, E. 1991. Modulation of neural networks for behavior. Annu. Rev. Neurosci. 14, 39-57. Jordan, M. I. 1986. Attractor dynamics and parallelism in a connectionist sequential machine. In Proceedings of the 1986 Cognitive Science Conference, pp. 531-546. Erlbaum, Hillsdale, NJ.

28


Kawato, M., Maeda, Y., Uno, Y., and Suzuki, R. 1990. Trajectory formation of arm movements by cascade neural network models based on minimum torque change criterion. Biol. Cyberrr. 62, 275-288. Kirkpatrick, S., Gelatt, C. D., and Vecchi, M. P. 1983. Optimization by simulated annealing. Scieiice 220, 671-680. Lukashin, A. V. 1990. A learned neural network that simulates properties of the neuronal population vector. Biol. Cyberrr. 63, 377-382. Lukashin, A. V., and Georgopoulos, A. P. 1993. A dynamical neural network model for motor cortical activity during movement: Population coding of movement trajectories. Bid. Cybertz., in press. Martin, K. A. C. 1988. From single cells to simple circuits in the cerebral cortex. Q. I. Ex". Physiol. 73, 637-702. Massone, L., and Bizzi, E. 1989. A neural network model for limb trajectory formation. B i d . Cybertr. 61, 417425. Pearlmutter, B. A. 1989. Learning state space trajectories in recurrent neural networks. Neural Cornp. 1, 263-269. Pineda, F. 1987. Generalization of backpropagation to recurrent neural networks. Phys. Rev. Lett. 19(59), 2229-2232. Schwartz, A. B. 1993. Motor cortical activity during drawing movements: Population representation during sinusoid tracing. ]. Neurophysiol., in press. Schwartz, A. B., and Anderson, B. J. 1989. Motor cortical images of sinusoidal trajectories. Soc. Neurosci. Abstr. 15, 788. Schwartz, D. B., Salaman, V. K., Solla, S. A., and Denker, J. S. 1990. Exhaustive learning. Neirml Cornp. 2, 374-385. Sejnowski, T. J., Koch, C., and Churchland, P. S. 1988. Computational neuroscience. Scieiice 241, 1299-1306. Tso, D. Y., Gilbert, C. D., and Weisel, T. N. 1986. Relationship between horizontal interactions and functional architecture in cat striate cortex as revealed by cross-correlation analysis. J. Neurosci. 6, 1160-1170. Williams, R. J., and Zipser, D. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural Coinp. 1, 270-280.

Received February 10, 1993; accepted June 9, 1993.

This article has been cited by: 2. Charidimos Tzagarakis, Trenton A. Jerde, Scott M. Lewis, Kâmil Uğurbil, Apostolos P. Georgopoulos. 2009. Cerebral cortical mechanisms of copying geometrical shapes: a multidimensional scaling analysis of fMRI patterns of activation. Experimental Brain Research 194:3, 369-380. [CrossRef] 3. Sohie Lee Moody , David Zipser . 1998. A Model of Reaching Dynamics in Primary Motor CortexA Model of Reaching Dynamics in Primary Motor Cortex. Journal of Cognitive Neuroscience 10:1, 35-45. [Abstract] [PDF] [PDF Plus] 4. Siming Lin, Jennie Si, A. B. Schwartz. 1997. Self-Organization of Firing Activities in Monkey's Motor Cortex: Trajectory Computation from Spike SignalsSelf-Organization of Firing Activities in Monkey's Motor Cortex: Trajectory Computation from Spike Signals. Neural Computation 9:3, 607-621. [Abstract] [PDF] [PDF Plus] 5. Shoji Tanaka, Noriaki Nakayama. 1995. Numerical simulation of neuronal population coding: influences of noise and tuning width on the coding error. Biological Cybernetics 73:5, 447-456. [CrossRef] 6. Alexander V. Lukashin, Apostolos P. Georgopoulos. 1994. Directional operations in the motor cortex modeled by a neural network of spiking neurons. Biological Cybernetics 71:1, 79-85. [CrossRef]

Communicated by Michael Arbib

Theoretical Considerations for the Analysis of Population Coding in Motor Cortex Terence D. Sanger MIT, E25-534, Cambridge, M A 02139 U S A Recent evidence of population coding in motor cortex has led some researchers to claim that certain variables such as hand direction or force may be coded within a Cartesian coordinate system with respect to extrapersonal space. These claims are based on the ability to predict the rectangular coordinates of hand movement direction using a ”population vector’’ computed from multiple cells’ firing rates. I show here that such a population vector can always be found given a very general set of assumptions. Therefore the existence of a population vector constitutes only weak support for the explicit use of a particular coordinate representation by motor cortex. 1 Introduction Recent results suggest that the representation of arm movement in motor cortex involves the simultaneous activity of many cells, and that the pattern of activation over the group of cells specifies the motion that occurs (Caminiti et a!. 1990; Kalaska and Crammond 1992, for review). These results have led many researchers to ask whether movement variables are coded internally in terms of a particular coordinate system such as a Cartesian or polar representation of extrapersonal space, a representation of muscle lengths around relevant joints, or some other set of coordinates. In the following, I distinguish between a coded variable (such as hand position) and the coordinates used to represent that variable (such as Cartesian coordinates). I will summarize certain experiments that demonstrate that hand movement direction is represented within motor cortex, but I claim that these experiments cannot be used to determine the coordinate system in which movements are coded. I discuss a set of experiments that investigated the relationship between cell firing rates during free arm movements in awake monkeys, and the direction in which the hand was moved to a target in space (Georgopoulos et al. 1988; Kettner et al. 1988; Schwartz ef al. 1988). These experiments led to the following results:

R1: The firing rate of 89.1% (486/568) of the cells tested in motor cortex varied consistently with the direction of hand motion within a Ntwral Cornpiitation 6, 29-37 (1994)


Terence D. Sanger

30

limited region of space, and many cells would be simultaneously active for any given direction. R2: A statistically significant component of the variance of the firing rate of 83.6% of the cells could be accounted for by a broadly tuned function of the form d;(M) zz b; + k;COS( 0; - OM)

(1.1)

where d,(M) is the firing rate of cell i for hand motion in the direction of a unit vector M, 0, is the direction of motion in which the cell has maximal response, 0, - OM is the angle' between the direction of hand motion OM and the cell's preferred direction, and b, and k, determine the average firing rate and modulation depth, respectively. (Here and in the following, capital letters indicate vector quantities.) R3: The preferred directions 0, are approximately uniformly distributed with respect to directions in the workspace. R4: The hand direction vector M can be approximated in Cartesian coordinates by a population vector P computed from a linear combination of the cell firing rates. R5: The coefficients of this linear combination are given by unit vectors C, along the preferred direction 0, for each cell, so that N

Mz P

=

C C,d,

(1.2)

i=l

where the di have been normalized to account for resting firing rate and response amplitude, and both the movement direction M and the preferred direction vectors Ci are given in Cartesian coordinates with respect to the external workspace. Together, these results might suggest that a Cartesian representation of the direction of hand motion is coded in motor cortex (Schwartz et a!. 1988). I will show that results R2, R4, and R5 are direct consequences of results R1, R3, and the experimental design. This in no way reduces the importance of these experiments, but rather emphasizes the fact that results R1 and R3 contain the most significant information. Although their importance was recognized in Georgopoulos et al. (19881, the fact that they imply the other results was not. Previous investigations have studied the conditions under which results R4 and R5 hold, and it has been shown that the population vector predicts the direction of hand motion if ' A difference of 3D angles is defined by B, both unit vectors.

- HM

= cos-'(C, . M ) ,where C, and M are

Population Coding in Motor Cortex

31

the tuning curve is symmetric and the distribution of preferred directions is uniform (Georgopoulos et al. 1988) or has no second harmonic components (Mussa-Ivaldi 1988). I derive a necessary and sufficient condition that is even broader, since it requires only that the three components of the preferred directions be uncorrelated with each other over the population. Before doing this, I will first show that the cosine tuning curves found in (Schwartz et al. 1988) may be an artifact of the analytic techniques used. 2 Single Unit Tuning Curves In Schwartz et al. (19881, the firing rate of each tuned cell d,(M)is approximated by a linear combination of the normalized Cartesian coordinates of the target toward which the monkey is reaching. These coordinates are relative to the initial hand position Xu and are given by a unit vector in the direction of motion M = (m,. my.mZ). The linear approximation is d l ( M )= b,

+ b,,m,. + b,,m, + blzmz

(2.1)

and an F test showed that the variance of 83.6% of all cells was at least partly accounted for by this linear regression. The preferred direction vector C , is calculated from

(2.2) (2.3) and we can now write d i ( M ) = bj

+ k,Cl . M

(2.4)

which is equivalent to equation 1.1. Note that cells with k, = 0 are not sensitive to the direction of movement and were not analyzed further, so k, f 0. To understand results R1 and R2, I perform a simplified analysis of movement in two dimensions (the extension to three dimensions is straightforward but complicates the notation significantly). For a fixed initial hand position and with all other variables held constant, consider any arbitrary firing rate function d ( O M ) that depends on the direction of hand movement OM. OM is a periodic variable, so the output of d ( O M ) will be periodic, and if eight uniformly spaced directions are tested then the complete behavior can be described by a discrete Fourier series with periods u p to 7r/2 4

(2.5) where 4k is the phase for each angular frequency component k. Note that for k > 1 the terms have no directional component, since they consist of

Terence D. Sanger

32

either two, three, or four "lobes" symmetrically placed around the circle. Thus a linear regression on the Cartesian coordinates x = COS(HM), y= sin(HM)will be unaffected by the values of {YZ? (13, and ( ~ and 4 will depend only on ckn and 01. To see this, note that linear regression computes the three expected values:

1

=

(F)cos(Q1)

where the expectation operator E [] is taken over all tested directions HM. The preferred direction is therefore equal to 411 and is independent of d ~ ~ . 4 (or '~, Even if more than eight directions are tested, the linear regression will respond only to the 4 ~ 1component. The "goodness of fit" to the linear regression is the extent to which the k = 0 and k = 1 terms capture the behavior of ~ ( H M ) . However, it is important to realize that a statistically significant F test does not indicate a good fit to a linear model in the sense of having small prediction error variance. Fit is determined by mean squared error, which distributes according to a ,y2 statistic. The F test estimates only the probability that the linear model accounts for some portion of the total variance. This is equivalent to testing if cvl is significantly different from 0. A significant F test does not imply that (v1 describes the dominant response behavior, and ( v 2 , 03, or (y4 might well be larger. If a set of tuning curves were generated randomly by selecting the coefficients o k independently from a normal distribution, then one would expect 95% of the tuning curves to have statistically significant values of (PI.Thus the observed value of 83.6% [93%in Caminiti etal. (1990)l does not support statistical arguments that the population has been "engineered" to have directional tuning. Since this method of analysis ignores terms for k > 1, it in effect low-pass filters the tuning curves. So the cosine tuning results from the method of analysis and may not be justified by the original data. These considerations show that result R2 does not provide any information beyond result R1, since R2 would be true for a randomly chosen set of tuning curves satisfying R1 that were analyzed in this way.

a4.


33

True cosine tuning could be verified by fitting equation 1.1 to data samples from many different directions and measuring the average meansquared approximation error over the population using a k 2 statistic. A similar test was done in the two-dimensional case, where it was found that 75%of 241 cells had a normalized mean-squared approximation error less than 30% of the total variance (Georgopoulos et nl. 1982). Although this is not a statistically good fit to the population, there may have been individual cells whose response was well predicted by cosine tuning. What is the significance of the cells that were well fit by a cosine tuning curve? As shown in equation 2.4, these cells have a response d that is approximately linearly related to the hand movement vector M . We can thus claim either that these cells are in fact linear in the movement direction, or else that they are linear in the testing region but may be nonlinear if tested in other regions of space. So if we write the response as d ( X 0 . X ) where X0 is the initial hand position and X is the target, then we know that d ( X 0 . X ) must be sufficiently smooth that it appears locally linear for the positions X that were tested. Over larger distances, d may not be well approximated linearly, but it can still be written as

d ( M ) z b + kC(X,,) . M

(2.6)

where C ( X O emphasizes ) that the preferred direction may become dependent on the initial position, as was indeed found in Caminiti et d . (1990). But equation 2.6 is a general representation for arbitrary smooth functions, so even an accurate fit to a locally linear function does not allow one to claim much beyond the fact that the preferred direction remains approximately constant over the tested region.

3 Population Vectors Result R4 that there exists a linear combination of the firing rates that can predict the Cartesian coordinates of hand motion follows as a direct consequence of well-known results on coarse coding and the theory of radial basis functions (Poggio and Girosi 1990, for example), since a raised cosine function of angle can be thought of as a local basis function centered on the preferred direction. An alternate way to prove this fact follows. Define an N x 3 matrix Q whose rows are the preferred direction vectors C,. Let D be an N-dimensional column vector formed from the firing rates of all the cells by the formula [ D ] ,= (d, - b , ) / k ,as in Georgopoulos ef al. (1988). Then from equation 2.4 we can write

DzQM

(3.1)

We seek a 3 x N weighting matrix H such that a population vector of the form H D predicts hand direction M according to M z HD

= HQM

(3.2)

Terence D. Sanger

34

There are many matrices H that will satisfy this equation. One possibility is to use linear least-squares regression, giving H

=

(3.3)

(QTQ)-'QT

where the inverse (QTQ)-' will always exist so long as there are three linearly independent preferred direction vectors. We now have M

FZ

HD

FZ

HQM

=

( Q T Q ) - ' Q T Q M= M

(3.4)

as desired. This equation means that so long as there exist three linearly independent direction vectors, the hand direction will be approximately linearly related to the cell firing rates d; in any coordinate system for M that satisfies equation 3.1. So far I have shown that result R1 implies both results R2 and R4, given the method of analysis. In Georgopoulos et al. (1988) the columns of H were not found by performing a regression of the cell firing rates against the hand direction according to equation 3.3, but instead were assumed a priori to be equal to the preferred direction Ci for each cell, so that H = QT. I now discuss under what conditions result R5 holds, so that this particular linear combination will give the right answer. The population vector is given by equation 1.2, which we can rewrite in vector notation as M sz P

=

QTDE QTQM

(3.5)

and if this holds for all directions M then we must have QTQ = 1. This is a necessary condition for the existence of a population vector. In Georgopoulos et al. (1988) a more restrictive sufficient condition satisfying equation 1.2 is that the distribution of preferred directions is uniform over the sphere. Another necessary and sufficient condition based on Fourier analysis of the distribution of preferred directions for the planar case is given in Mussa-Ivaldi (1988). To understand the meaning of equation 3.5, we can write each component of QTQ as

and I = QTQ implies that Cf"=,[Cl],[C,]k = 0 whenever j # k. This expression is the correlation of the jth and kth components of the preferred direction vectors C,, so a necessary and sufficient condition for equation 1.2 to work is that the x, y, and z components of these vectors are uncorrelated and have equal variance. The result that equation 1.2 is satisfied is thus implied by the approximately uniform distribution of cell preferred directions in result R3. Note that for other coordinate systems, even if the components of the C,s are correlated there will still exist a linear combination H # QT of the firing rates that will predict the desired values, although the matrix H may need to be found by regression using


35

equation 3.3. But if both results R1 and R3 hold, then result R5 must hold. Suppose that rather than using the predicted value Q M we use the true measured value D and this includes significant noncosine (nonlinear) terms. Then we have D=QM+E where E is a vector with components 4

e,(oM) =

Oik

cos(kflM + h k )

k=2

If the terms C?,k and djrk are distributed independently of the components of C,, then QTE = 0 and these terms will not affect the value of the population vector. So even if the individual cells d o not have cosine tuning, the population vector will correctly predict hand direction if the terms for k > 1 do not correlate with the terms for k = 1 in the expansion given in equation 2.5. If the experiments are repeated with differing initial positions as in Caminiti ef al. (1990), then the preferred directions C, may change. This will lead to a new matrix Q‘ so that D’= Q M . Population vector analysis under the new conditions will give P‘ = Q T Q M ,so again the requirement for success is that the components of the new preferred directions are uncorrelated. The fact that population vectors “proved to be good predictors of movement direction regardless of where in space the movements were performed” (Caminiti et al. 1990, p. 2039) provides no information beyond the knowledge that the components of the preferred directions remain uncorrelated as the initial hand position changes. 4 Coordinate-Free Representations

One might ask if the experiments described above could be modified to determine the ”true” coordinate system used by motor cortex to describe hand movement direction. However, I claim that for certain classes of distributed representation this is not a well-defined question. Distributed representations of measured variables can be coordinate-free in the sense that they do not imply any particular coordinate system. To see this, let X be any variable represented in cortex (such as hand movement direction), and let D ( X ) be a vector-valued function representing the outputs of a large set of basis functions d , ( X ) that describe the behavior of (motor) cortical cells. D ( X ) is then a distributed representation of the variable X. Now, consider a vector function T ( X ) that measures X in a particular coordinate system W ( X ) might give the three Cartesian components of hand movement direction, for example]. If there exists a matrix H such that H D ( X ) M T ( X ) ,then one can say that the distributed representation D codes the coordinate system T. Yet this will hold for any T ( X ) that is

36

Terence D. Sanger

close to the linear span of the basis functions d ; ( X ) ,so we cannot claim that D encodes any single coordinate system for X within this span better than another. 5 Conclusion

In this letter, 1 have extended the generality of previous results (Georgopoulos eta/. 1988; Mussa-Ivaldi 1988) to show that cosine tuning curves will be found for large classes of arbitrary response functions if they are analyzed according to the statistical techniques in Schwartz et a/. (19881, and that the existence of a population vector a s found in Georgopoulos eta/. (1988) is determined by very general necessary and sufficient conditions that depend only on the distribution of preferred directions rather than on any intrinsically coded coordinate system. The concept that a distributed representation codes a particular coordinate system may not be well-defined, since certain types of representations can be considered "coordinate-free." These considerations imply that experiments of the type described may yield population vectors which predict many different three-dimensional coordinates (such as Cartesian, polar, muscle lengths, or joint angles). It is important to understand that the considerations presented here in no way reduce the importance of the results reported in Schwartz et a/. (1988), Georgopoulos et al. (19881, Kettner et a/. (19881, Caminiti et a/. (1990), and elsewhere. The fact that results R2, R4, and R5 are direct consequences of R1 and R3 serves only to underscore the significance of these two results. They show that large populations of motor cortical cells respond to hand motion in a predictable way, and that the preferred directions are approximately uniformly distributed with respect to a Cartesian representation of extrapersonal space. No additional conclusions can be drawn from the population vector, since its existence is a mathematical consequence of these two facts. However, if the distribution of preferred directions is nonuniform with respect to other coordinate systems or if the distribution can be modified through experience, then this would provide significant information about cortical representations. In addition, if cosine tuning can be verified by explicitly fitting cell tuning curves to a linear regression model, then further studies may discover constraints that explain why more than 486 linear cells are needed to code for only 3 linearly independent components of hand direction. Acknowledgments

I would like to thank Sandro Mussa-Ivaldi, Ted Milner, Emilio Bizzi, Richard Lippmann, Marc Raibert, and the reviewers for their comments and criticism. This report describes research done within the laboratory of Dr. Emilio Bizzi in the department of Brain and Cognitive Sciences


37

at MIT. The author w a s supported during this work by the division of Health Sciences a n d Technology, a n d by N I H Grants 5R37AR26710 a n d 5ROlNS09343 to Dr. Bizzi.

References Caminiti, R., Johnson, P. B., and Urbano, A. 1990. Making arm movements within different parts of space: Dynamic aspects in the primate motor cortex. J. Neurosci. 10(7), 2039-2058. Georgopoulos, A. P., Kalaska, J. F., Caminiti, R., and Massey, J. T. 1982. On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2(11), 1527-1537. Georgopoulos, A. P., Kettner, R. E., and Schwartz, A. B. 1988. Primate motor cortex and free arm movements to visual targets in three-dimensional space. 11. Coding of the direction of movement by a neuronal population. 1.Neurosci. 8(8), 2928-2937. Kalaska, J. F., and Crammond, D. J. 1992. Cerebral cortical mechanisms of reaching movements. Science 255, 1517-1523. Kettner, R. E., Schwartz, A. B., and Georgopoulos, A. P. 1988. Primate motor cortex and free arm movements to visual targets in three-dimensional space. 111. Positional gradients and population coding of movement direction from various movement origins. J. Neurosci. 8(8), 2938-2947. Mussa-Ivaldi, F. A. 1988. Do neurons in the motor cortex encode movement direction? An alternative hypothesis. Neurosci. Lett. 91, 106-111. Poggio, T., and Girosi, F. 1990. Regularization algorithms for learning that are equivalent to multilayer networks. Science 247, 978-982. Schwartz, A. B., Kettner, R. E., and Georgopoulos, A. P. 1988. Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. Relations between single cell discharge and direction of movement. J. Neurosci. 8(8), 2913-2927.

Received April 28, 1992; accepted May 13, 1993.

This article has been cited by: 2. A. Moran, H. Bergman, Z. Israel, I. Bar-Gad. 2008. Subthalamic nucleus functional organization revealed by parkinsonian neuronal oscillations and synchrony. Brain 131:12, 3395-3409. [CrossRef] 3. S. Shoham, L.M. Paninski, M.R. Fellows, N.G. Hatsopoulos, J.P. Donoghue, R.A. Normann. 2005. Statistical Encoding Model for a Primary Motor Cortical Brain-Machine Interface. IEEE Transactions on Biomedical Engineering 52:7, 1312-1322. [CrossRef] 4. Paul Cisek. 2005. Neural representations of motor plans, desired trajectories, and controlled objects. Cognitive Processing 6:1, 15-24. [CrossRef] 5. Rony Paz, Eilon Vaadia. 2004. Learning-Induced Improvement in Encoding and Decoding of Specific Movement Directions by Neurons in the Primary Motor Cortex. PLoS Biology 2:2, e45. [CrossRef] 6. Pierre Baraduc , Emmanuel Guigon . 2002. Population Computation of Vectorial TransformationsPopulation Computation of Vectorial Transformations. Neural Computation 14:4, 845-871. [Abstract] [PDF] [PDF Plus] 7. Frank Bremmer. 2000. Eye position effects in macaque area V4. NeuroReport 11:6, 1277-1283. [CrossRef] 8. Sohie Lee Moody , David Zipser . 1998. A Model of Reaching Dynamics in Primary Motor CortexA Model of Reaching Dynamics in Primary Motor Cortex. Journal of Cognitive Neuroscience 10:1, 35-45. [Abstract] [PDF] [PDF Plus] 9. Bremmer, Pouget, K. -P. Hoffmann. 1998. Eye position encoding in the macaque posterior parietal cortex. European Journal of Neuroscience 10:1, 153-160. [CrossRef] 10. Alexandre Pouget, Terrence J. Sejnowski. 1997. Spatial Transformations in the Parietal Cortex Using Basis FunctionsSpatial Transformations in the Parietal Cortex Using Basis Functions. Journal of Cognitive Neuroscience 9:2, 222-237. [Abstract] [PDF] [PDF Plus] 11. Etienne Koechlin, Yves Burnod. 1996. Dual Population Coding in the Neocortex: A Model of Interaction between Representation and Attention in the Visual CortexDual Population Coding in the Neocortex: A Model of Interaction between Representation and Attention in the Visual Cortex. Journal of Cognitive Neuroscience 8:4, 353-370. [Abstract] [PDF] [PDF Plus] 12. Shoji Tanaka, Noriaki Nakayama. 1995. Numerical simulation of neuronal population coding: influences of noise and tuning width on the coding error. Biological Cybernetics 73:5, 447-456. [CrossRef] 13. Marco Idiart, Barry Berk, L. F. Abbott. 1995. Reduced Representation by Neural Networks with Restricted Receptive FieldsReduced Representation by Neural Networks with Restricted Receptive Fields. Neural Computation 7:3, 507-517. [Abstract] [PDF] [PDF Plus]

14. Alexandre Pouget, Lawrence H SnyderModeling Coordinate Transformations . [CrossRef]

Communicated by Stephen Lisberger

Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses Dean V. Buonomano* Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX 77225 U S A and Departamento de Matematica Aplicada, lnstituto de Matematica, Universidade Estadual de Campinas, Campinas, Brasil, and Laboratdrio de Psicobiologia, Universidade de Srio Paulo, Ribeirio Preto, Brasil Michael D. Mauk Department of Neurobiology and Anatomy, University of Texas Medical School, Houston, TX 77225 U S A Substantial evidence has established that the cerebellum plays an important role in the generation of movements. An important aspect of motor output is its timing in relation to external stimuli or to other components of a movement. Previous studies suggest that the cerebellum plays a role in the timing of movements. Here we describe a neural network model based on the synaptic organization of the cerebellum that can generate timed responses in the range of tens of milliseconds to seconds. In contrast to previous models, temporal coding emerges from the dynamics of the cerebellar circuitry and depends neither on conduction delays, arrays of elements with different time constants, nor populations of elements oscillating at different frequencies. Instead, time is extracted from the instantaneous granule cell population vector. The subset of active granule cells is time-varying due to the granuleGolgi-granule cell negative feedback. We demonstrate that the population vector of simulated granule cell activity exhibits dynamic, nonperiodic trajectories in response to a periodic input. With time encoded in this manner, the output of the network at a particular interval following the onset of a stimulus can be altered selectively by changing the strength of granule + Purkinje cell connections for those granule cells that are active during the target time window. The memory of the reinforcement at that interval is subsequently expressed as a change in Purkinje cell activity that is appropriately timed with respect to stimulus onset. Thus, the present model demonstrates that 'Present address: Keck Center for Integrated Neuroscience, University of California, San Francisco, CA 94143.

Neural Computation 6, 38-55 (1994)


Timing and the Cerebellum

39

a network based on cerebellar circuitry can learn appropriately timed responses by encoding time as the population vector of granule cell activity.

1 Introduction

Generating movements inherently involves producing appropriately timed contractions in the relevant muscle groups. Accordingly, an important aspect of motor control is the timing of motor commands in relation to internal (i.e., proprioceptive cues) and external stimuli. One clear and experimentally tractable example of the timing of responses with respect to an external stimulus is Pavlovian conditioning of eyelid responses. Learned responses are promoted in this paradigm by paired presentation of a cue or conditioned stimulus (CS) and a reinforcing unconditioned stimulus (US). For example, the presentation of a tone (the CS) is reinforced by the copresentation of a puff of air directed at the eye (the US). This promotes the acquisition of learned eyelid responses that are elicited by the CS. These learned eyelid movements are delayed so that they peak near the onset of the potentially harmful US (Schneiderman and Gormezano 1964; Gormezano et al. 1983). Previous studies have shown that the timing of the eyelid response is learned and that the underlying neural mechanism involves temporal discrimination during the CS (Mauk and Ruiz 1992). Since appropriately timed responses can be generated for CS-US intervals between 80 and 2000 msec, the neural mechanism appears to be capable of temporal discrimination in this range. Very little is known about how and where the nervous system encodes time. There is evidence that neurons can use axonal delays to detect temporal intervals. For example, axonal conduction delays appear to contribute to the detection of interaural time delays (Carr and Konishi 1988; Overholt et al. 1992). Furthermore, theoretical work (Koch et al. 1983) has suggested that dendritic conduction delays could contribute to the detection of temporal delays important for direction selectivity in the visual system. In both these instances, however, the relevant time intervals are below tens of milliseconds. Braitenberg (1967) suggested that the parallel fibers in the cerebellum could function as delay lines that would underlie the timing of movements. Given the conduction velocity of parallel fibers of approximately 0.2 mm/msec it is unlikely that such a mechanism could underlie timing in the tens of milliseconds to second range (Freeman 1969). Other models have been presented in which intervals above tenths of seconds could be stored by a group of oscillating neurons with different frequencies (Miall 1989; Church and Broadbent 1991; Fujita 1982; Gluck et al. 1990). As of yet, however, no such population of neurons has been described.

40

Dean V. Buonomano and Michael D. Mauk

It has been suggested that the cerebellum may play an important role in the timing of movements (Braitenberg 1967; Freeman 1969; Eccles 1973), in addition to being an important component for the learning of movements (Marr 1969; Albus 1971; Fujita 1982; Lisberger 1988). Indeed it has been shown that while cerebellar cortex lesions do not abolish the conditioned response, the timing of the responses is disrupted (McCormick and Thompson 1984; Perrett et nl. 1993). Whereas numerous studies indicate that output from the cerebellum via cerebellar nuclei is required for Pavlovian eyelid conditioning (Thompson 1986; Ye0 1991; however, see Welsh and Harvey 1989). One class of cerebellar afferents (mossy fibers) appears to convey to the cerebellum the presentation of the CS and a second class of cerebellar afferents (climbing fibers) appears to convey the presentation of the US (e.g., see Thompson 1986). These data suggest that (1) Pavlovian conditioning is a relatively simple, experimentally tractable paradigm for the study of the input/output properties of the cerebellum, (2) the cerebellum is necessary for the appropriate timing of conditioned movements, and thus, (3) Pavlovian conditioning is particularly well suited for the study of cerebellar mechanisms that mediate the ability to discriminate time with respect to the onset of an internal or external stimulus. The purpose of this paper is to use a neural network model to test a specific hypothesis suggesting how the known circuitry of the cerebellum could make temporal discriminations and mediate the timing of the conditioned response. The model tested here is based on the knoivn circuitry of the cerebellum and does not utilize delay lines, assume arrays of elements with different time constants, or assume arrays of elements that oscillate at different frequencies, but instead shows how the dynamics of cerebellar circuitry could encode time with the population vector of granule cell activity. 2 Structure of the Model

2.1 Cerebellar Circuitry. In comparison to most brain regions the neural circuitry, cell ratios, and synaptic convergence/divergence ratios of the cerebellum are fairly well established (Eccles e t a / . 1967; Palkovits et nl. 1971; Ito 1984). The principal six cell types are the granule (Gr), Golgi (Go), Purkinje (PC), basket, stellate, and cells of the cerebellar nuclei. The two primary inputs to the cerebellum are conveyed by mossy fibers (MF) and climbing fibers (CF). Figure 1A illustrates the known connectivity of these cells. 2.2 Classic Cerebellar Theories. Based on the known characteristics of cerebellar organization, Marr (1969) and later Albus (1971) proposed models of the cerebellum suggesting a mechanism to mediate motor learning. In these theories (1) the contexts in which movements take


GOLGI

A.

41

a 0 STELLATEBASKET

4 GRANULE

LEGEND OFSYNAPSES --O

-INHIBITORY

-i]

4 EXCITATORY

MOSSY FIBERS CLIMBING FIBER

D OUT

B.

0

.

0

Figure 1: Synaptic organization of the cerebellum. (A) A schematic diagram of the known synaptic connections in the cerebellum. Shown in bold and with solid lines are the components incorporated into the present computer model. (B) A schematic representation of the connectivity in the present model. Small representations of the granule (Gr) and Golgi (Go) layers are shown; there were lo4 and 900 Gr aiid Go cells, respectively, in the simulations. The 500 MFs and single PC are omitted for clarity. The arrows aiid shaded pyramids illustrate the spans or regions to which the cells are eligible to make a synaptic contact. Within the spans connections are made with a uniform distribution. The white cells in each span exemplify cells that receive a synaptic input from the presynaptic cell. Thus, the shape of the spans was designed to reflect the geometry of the projections of each cell type, whereas the number of cells within the span that actually received a connection was a reflection of the convergence and divergence ratios of the synaptic connections.

place are encoded by the population vector of G r cell activity, (2) the CFs convey error signals indicating that the response requires modification, a n d (3) this CF error signal modifies active G r i PC synapses such that subsequent motor performance in that context is improved. Marr also

Dean V. Buonomano and Michael D. Mauk

42

suggested that the Go cell negative feedback stabilizes the amount of Gr cell activity which could maximize the discriminability of the contexts.

2.3 Hypothesis. The hypothesis we test here is an elaboration of the Marr/Albus scheme which suggests that due to (1) the dynamic interactions between Gr and Go cell populations and (2) the complex geometry of their interconnections, the population vector of granule cell activity exhibits time-variant trajectories that permit both stimulus and temporal discriminations (see Mauk and Donegan 1991). Thus, the population vector of Gr cell activity encodes both the particular MF input pattern and the time since its onset. A particular periodic MF input pattern will activate a subset of Gr cells which will activate a subset of Go cells; these in turn will inhibit a second, partially overlapping subset of Gr cells. The Gr -+ Go Gr negative feedback loop will create a dynamic, nonperiodic population vector of Gr cell activity even when driven by a periodic input pattern of ME Thus, the population vector of Gr cell activity would encode not just the constellation of stimuli impinging on the organism, but also the time since the onset of the stimuli. In a manner consistent with the Marr/Albus theories, a given subset of Gr cells and thus a particular interval can be stored by changing the strength of the Gr PC connection of active Gr cells. The retrieval of the interval is expressed as a change in the activity level of the PC. -+

-+

2.4 Neural Network. The neural network consisted of lo4 Gr cells, 900 Go cells, 500 MF inputs, and one PC. The architecture is illustrated schematically in Figure 1B; the 500 MFs and the single PC are omitted for clarity. Each Gr cell received excitatory synaptic inputs from three MFs and inhibitory inputs from three Go cells. Each Go cell received excitatory inputs from 100 Gr cells and 20 MFs. The PC received inputs from all lo4 Gr cells. In scaling the network to computationally feasible dimensions, it is not possible to maintain both the empirically observed Gr/Go cell ratios and convergence/divergence ratios. The biological Gr/Go cell ratio is approximately 5000/1, and the Gr convergence/divergence ratio is approximately 0.4 (4/10, i.e., each Gr cell receives input from 4 Go cells and sends contacts to 5-10 Go cells; Eccles 1973; Palkovits et al. 1971). Thus, a network with lo4 Gr cells should have only 2 Go cells, yet each of the Gr cells should contact approximately 5-10 Go cells. The underlying assumption we employed in making the compromise between cell ratios and convergence/divergence ratios was that the latter is more important. Thus, the convergence/divergence ratios for the Gr and Go cells were maintained within an order of magnitude of observed experimental values. In the model the Gr/Go ratio was 11.1 (104/900) and the Gr convergence/divergence ratio was 0.33 (3/9).


43

The Gr and Go cells were simulated as modified integrate and fire elements. For example, the voltage of each Go cell (VF" was determined by

A spike (S$) was generated if threshold was reached.

sy= { 0.1.

V p 2 Thr;" V p 0 indicates an excitatory connection, and self-inhibition or self-excitation is permitted when i = j. Each element receives a tonic input s,, and uses b, to weigh the influence of the corresponding adaptation variable 2, such that for b, > 0 adaptation occurs, and for b, < 0 self-excitation results. c, weighs the influence of the RAS on tonic activity of the ith neuron. T is the activation time constant for the neural

60

Allan Gottschalk et a].

Figure 2: Diagram of the neural connections used by the simulations. These connections appear in tabular form in Table 1. Here, inhibitory connections are indicated by solid bars and excitatory connections are indicated by small open circles. The interaction of the neural elements with the reticular activating system (RAS) represents a component of s, to each neural element as indicated in Table 1. elements, and represents the time course for the activation of a population of synergistic neurons. T is the time constant governing adaptation in this population, and 7 is the time constant of the RAS. We began with the network model proposed by Richter et al. (1986), which was based on data of the type shown in Figure 1. The network depicted in Figure 2 was configured from the five identified neuron types, and included the post-inspiratory modulation of the tonic excitatory inputs described above and in Table 1. Initial estimates of the connection strengths and time constants were made on the basis of the trajectories of the intracellular recordings. The conditions under which the proposed network, when configured using equations 2.1-2.4, would oscillate were computed following Matsuoka, and these were used to guide the initial choices of tonic inputs s, and the levels of adaptation b;. Thereafter, these parameters were adjusted interactively until network trajectories most closely matched the physiologic data. All simulations were programmed using the ASYST scientific programming package (Keithley

Respiratory Pattern Generator

61

Asyst, Rochester) on an IBM PC/AT or compatible computer using the integration scheme suggested by MacGregor (1987). Eigenvalues of the linearized system were analyzed with PHSPLAN (0 Bard Ermentrout, Pittsburgh). 3 Results

The network parameters obtained as described above are given in Table 1, and the corresponding network outputs are shown in Figure 3. These trajectories are comparable to those of Figure 1, especially with respect to the cardinal three-phase feature of the physiologic network. We have not performed a quantitative point-by-point comparison of the trajectories of Figures 1 and 3 because the substantial heterogeneity within each physiological neuron class makes such a comparison inappropriate. It is especially important to appreciate that physiological recordings are available which span the range of differences between Figures 1 and 3. The level of adaptation in the expiratory (exp) neuron that was required to terminate the expiratory phase of the network model, although less than the other neurons of the model, is still greater than physiologically observed levels of adaptation of the exp neurons (Richter et nl. 1975,1986). The three-phase feature of the model outputs is preserved, and the finer features are minimally altered when parameters are varied about the set given in Table 1. The topological structure of the model is also preserved when individual parameters or combinations of parameters are varied. This was indicated by analysis of the signs of the eigenvalues of the linearized system, which consistently exhibited a single complex pair of eigenvalues with a positive real component accompanied by eigenvalues that were negative and real. We now evaluate the ability of the network model, which generated the outputs of Figure 3, to replicate the phase resetting studies of Paydarfar et nl. (1986, 1987) (see Glass and Mackey 1988 for a review of this approach). Briefly, these investigators determined the change in the phase of the phrenic output after either superior laryngeal nerve (SLN) stimulation, which is inhibitory to inspiration (Remmers et a / . 19861, or stimulation of the midbrain reticular formation, which is facilitatory to inspiration (Gauthier et a/. 1983). I n these studies, the important variables are the change in phase of the onset of phrenic nerve activity as a function of both the magnitude and timing of the stimulus. The results of Paydarfar et nl. are reproduced on the left of Figures 4A and B for the inhibitory and facilitatory stimuli, respectively. Relatively weak stimuli (top of figure) produce phase resetting curves whose average slope has an absolute value of one. This is referred to as Type 1 resetting. Relatively strong stimuli (bottom of figure) produce phase resetting curves whose average slope is 0. This is referred to as Type 0 resetting. Intermediate strength stimuli (middle of figure) produce a highly variable

Allan Gottschalk et al.

62

Phases

early-I

ramp-I

late-I

post -I

exp.

Figure 3: Normalized outputs of the five components of the network model for the parameterization given in Table 1. The horizontal line indicates the threshold for each neural element. The regions above the horizontal line indicate the period of superthreshold activity (yi), which is proportional to firing rate. The regions below the horizontal line correspond to the subthreshold pattern of activity ( x i ) . These patterns should be compared to the physiologically determined patterns of activity depicted in Figure 1, noting the presence of three well-defined phases of activity.


63

B) FACILITATORY PULSE

A) INHIBITORY PULSE EXPERIMENT

WEAK STIMULI

SIMULATION

'.. . . . . .... . .

. ". b.

5

8.

EXPERIMENT

-.

-.

* 0

SlMULArlON

'

INTER MEDlAT E STIMULI

i .............

STRONG STIMULI

t-_

..,.., ,;.*,;; 0

O

I

Z

O

OLD PHASE

I

Z

..............

.............. 0 05

1

OLD PHASE

Figure 4: Phase resetting studies for (A) superior laryngeal nerve (SLN) stimuli, which are inhibitory to ventilation, and (B) midbrain reticular formation stimuli, which are facilitatory to ventilation. The left-hand side of each component of the figure reproduces the physiologically determined phase resetting studies of Paydarfar et al. The right-hand side of each component of the figure depicts the results of phase resetting studies performed computationally on the network model. For each section of the figure, the top panel illustrates the Type 1 phase resetting behavior that accompanies relatively weak stimuli, the bottom panel illustrates the Type 0 behavior that accompanies relatively strong stimuli, and the middle panel illustrates the variable phase resetting that occurs for intermediate strength stimuli presented at the phase singularity of the oscillator (see text for details).

resetting, a n d the phase in the respiratory cycle at which this occurs is the phase singularity of the oscillator. Some nonlinear oscillators have a topologic structure such that stimulation with a n intermediate strength stimulus at the phase singularity will terminate the oscillations (Glass and Mackey 1988). Note that apneas were not produced in the phase resetting experiments of Paydarfar ef al. In o u r simulations we examined

64


the phase resetting of the onset of output by the inspiratory ramp ( 1 ~ ) neuron. The effects of SLN stimulation were simulated by us as excitation of the post-inspiratory neuron (Remmers et al. 1986) and inhibition of the exp neuron (Ballantyne and Richter 19861, and the corresponding set of phase resetting curves is shown to the right of Figure 4A, where the ratio of p-1:exp in the stimulus pulse was 1:3. Facilitatory stimuli were modeled as leading to excitement of the early-inspiratory neurons (Gauthier et al. 1983), and the corresponding phase resetting curves are shown to the right of Figure 4B. It is important to note that the location of the phase singularity is nearly the same in both the physiologic data and the corresponding simulations. However, there is a small, and possibly significant, difference in the location of the phase singularity with respect to the expiratory to inspiratory (E-I) junction in the simulations of the SLN data (Fig. 4A, right middle). Here, the phase singularity appears earlier in the respiratory cycle than in the experimental data. As in the phase resetting experiments, none of the simulations produced apnea when the resetting pulse was applied. A number of distinct qualitative changes in the pattern of phrenic activity can be induced by physiologic modifications of the network afferents. These include the two phase alternating inspiratory and postinspiratory rhythm described by Remmers et al. (1986), and the double firing of the p-I cells (Schwarzacher et al. 1991). Remmers et al. observed the two phase inspiratory/post-inspiratory rhythm with low frequency stimulation of the SLN. Modeling SLN stimulation as in the phase resetting studies, we observed a similar two phase rhythm with a 5% increase in the tonic input to the p-I neurons and an 18% decrease in the tonic input to the expiratory neurons. Cycles without a post-inspiratory phase have not been observed physiologically and could be produced in the model only by changes of over 300% in some of the tonic inputs. Double firing of the p-I cells, which is commonly observed (Schwarzacher et al. 19911, could be simulated with as little as a 10% increase in the tonic input to the p-I neuron of the model. 4 Discussion

Our results demonstrate that a network model of respiratory rhythmogenesis based on that proposed by Richter et al. (1986) can replicate the physiologically observed outputs of the brainstem respiratory neurons in significant detail. Although this result supports the network hypothesis, it is reasonable to expect that the network that was configured specifically to replicate the physiologic data should be capable of reproducing its trajectories in some detail. For this reason, and because of the variability in each physiologically defined neuron class, we turned to other physiological phenomena of respiratory rhythmogenesis to validate the behavior of the network model described above. In addition to support-


65

ing the network hypothesis, our computational experiments also indicate how a diverse array of physiologic data can be unified under a single hypothesis. Moreover, the topologic structure of the network, as indicated by localized eigenvalue analysis and the phase resetting studies, has important implications for the generation of respiratory arrhythmias. The failure of the network model to produce the E-I transition without a level of adaptation of the exp neurons in excess of that which is observed physiologically (Richter et al. 1975, 1986) suggests a limitation of the model. Additional computational evidence of the limitations of the model is seen in the simulations of the phase resetting data, where the phase singularity produced by the SLN pulses is not perfectly located with respect to the E-I junction. This difficulty cannot be overcome by simple parameter changes to the model. All of this suggests the possibility of an additional type of neuron that serves to mediate the E-I transition. Physiologic evidence of an additional respiratory neuron (phase-spanning expiratory-inspiratory), which fires at the E-I transition, and, thus, could help to terminate the expiratory phase, does exist (Cohen 1969; Smith et al. 1990; Schwarzacher et al. 1991). Preliminary computational data indicate that such a neuron could terminate the expiratory phase with a more physiologically appropriate level of adaptation to the exp neuron. In this manner, our computational results have aided physiologic investigation. Although apnea could be produced in the model by manipulations such as sustained and intense SLN activity which also produce apnea in physiologic preparations (Lawson 1981), we observed nothing to suggest that brief perturbations of any type to the network, as it is presently configured, could produce apnea. It has been postulated that infant apnea has its origins in the topologic structure of the respiratory pattern generator (Paydarfar et al. 1986, 1987), and that if the system is driven by a perturbation to a locally stable region of the state space, respiratory activity will cease. That the phase singularity of the oscillator should be located within such a region motivated the studies of Paydarfar et al. However, their work in adult animals did not reveal apneas at the phase singularity. Similar results were obtained in our model, and the presence of eigenvalues with positive real components when the network is linearized about its fixed point indicates that termination of ventilation with a well-timed pulse of any type is impossible in the model as it is currently configured. Thus, if infant apnea is the consequence of perturbation of the network to a region of stability, we would anticipate the observation in susceptible individuals of networks with substantially different properties than those described above, especially since it appears that neonatal respiratory control, at least in minipigs, is organized similarly to that of adult animals (Lawson et al. 1989). In summary, our computational studies have demonstrated how a group of virtually identical model neurons, when appropriately connected, can replicate the physiologically observed trajectories of the dif-

66

Allan Cottschalk et al.

ferent respiratory neurons as well as a large number of the rhythmic phenomena of the respiratory pattern generator. In addition to supporting the network hypothesis of respiratory pattern generation, o u r studies demonstrate that the topologic structure of the network is stable with respect to brief perturbations. These studies have also motivated the physiologic search for additional types of respiratory neurons. Acknowledgments

The work reported here was supported in part by NATO Grant RG 0834/86, SCOR Grant HL-42236, DFG Ri 279/7-11 a n d Training Grant GM-07612-16. References Anders, K., Ballantyne, D., Bischoff, A. M., Lalley, P. M., and Richter, D. W. 1991. Inhibition of caudal medullary expiratory neurones by retrofacial inspiratory neurones in the cat. ]. Pliysiol. (Loiidori) 437, 1-25 Ballantyne, D., and Richter, D. W. 1984. Postsynaptic inhibition of bulbar inspiratory neurones in cat. ]. Pli!ysiol. (Loridor?) 348, 67-87. Ballantyne, D., and Richter, D. W. 1986. The non-uniform character of expiratory synaptic activity in expiratory bulbospinal neurones of the cat. 1. Pliysiol. (Londoii) 370, 433456. Botros, S. M., and Bruce, E. N. 1990. Neural network implementation of the three-phase model of respiratory rhythm generation. Biol. Cybcrti. 63, 143153. Cohen, M. I. 1969. Discharge patterns of brain-stem respiratory neurons during Hering Breuer reflex evoked by lung inflation. 1. Ncirroph!ysio/. 32, 356-374. Cohen, M. I. 1979. Neurogenesis of respiratory rhythm in the mammal. Physiol. RPZl. 59(4),1105-1 173. Ezure, K. 1990. Synaptic connections between medullary respiratory neurons and considerations on the genesis of respiratory rhythm. Pros. Neirrohiol. 35, 429-450. Feldman, J. L., and Cleland, C. L. 1982. Possible roles of pacemaker neurons in mammalian respiratory rhythmogenesis. In Crllirlnr Pnccinnkm, Vol. 11, D. 0. Carpenter, ed., pp. 101-119. Wiley, New York. Gauthier, P., Monteau, R., and Dussardier, M. 1983. Inspiratory on-switch evoked by stimulation of mesencephalic structures: A patterned response. E q i . Brniri Res. 51, 261-270. Glass, L., and Mackey, M. C. 1988. From Clocks to Chnos, pp. 98-118. Princeton University Press, Princeton, NJ. Lawson, E. E. 1981. Prolonged central respiratory inhibition following reflexinduced apnea. ]. Appl. Physiol. 50, 874-879. Lawson, E. E., Richter, D. W., and Bischoff, A. 1989. Intracellular recordings of respiratory neurons in the lateral medulla of piglets. /. Appl. Pkysiol. 66(2), 983-988.


67

Lindsey, B. G., Segers, L. S., and Shannon, R. 1987. Functional associations among simultaneously monitored lateral medullary respiratory neurons in the cat. 11. Evidence for inhibitory actions of expiratory neurons. J. Neurophysiol. 57, 1101-1117. Lindsey, B. G., Segers, L. S., and Shannon, R. 1989. Discharge patterns of rostrolateral medullary expiratory neurons in the cat: Regulation by concurrent network processes. 1. Neurophysiol. 61, 1185-1196. MacCregor, R. J. 1987. Neural and Brain Modeling, pp. 250-254. Academic Press, San Diego, CA. MacCregor, R. J., and Tajchman, G. 1988. Theory of dynamic similarity in neuronal systems. J. Neurophysiol. 60, 751-768. Matsuoka, K. 1985. Sustained oscillations generated by mutually inhibiting neurons with adaptation. Biol. Cybern. 52, 367-376. Ogilvie, M. D., Gottschalk, A., Anders, K., Richter, D. W., and Pack, A. I. 1992. A network model of respiratory rhythmogenesis. A M .1. Physiol. 263, R962R975. Paydarfar, D., and Eldridge F. L. 1987. Phase resetting and dysrhythmic responses of the respiratory oscillator. Am. J. Physiol. 252 (Regulatory Integrative Comp. Physiol. 211, R55-R62. Paydarfar, D., Eldridge F. L., and Kiley J. P. 1986. Resetting of mammalian respiratory rhythm: Existence of a phase singularity. A m . J. Physiol. 250 (Regulatory Integrative Comp. Physiol. 19), R721-R727. Remmers, J. E., Richter, D. W., Ballantyne, D., Bainton C. R., and Klein, J. P. 1986. Reflex prolongation of stage I of expiration. Pf7ngers Arch. 407, 190-198. Richter, D. W. 1982. Generation and maintenance of the respiratory rhythm. 1. Exp. Biol. 100, 93-107. Richter, D. W., Heyde, F., and Gabriel, M. 1975. Intracellular recordings from different types of medullary respiratory neurons of the cat. J. Neurophysiol. 38, 1162-1171. Richter, D. W., Camerer, H., Meesmann, M., and Rohrig, N. 1979. Studies on the interconnection between bulbar respiratory neurones of cats. Pflugers Arch. 380, 245257. Richter, D. W., Ballantyne, D., and Remmers J. E. 1986. How is the respiratory rhythm generated? A model. NIPS 1, 109-112. Schwarzacher, S. W., Wilhelm, Z., Anders, K., and Richter, D. W. 1991. The medullary respiratory network in the rat. 1. Physiol. (London) 435, 631444. Segers, L. S., Shannon, R., and Lindsey, B. G. 1985. Interactions between rostra1 pontine and ventral medullary respiratory neurons. J. Neurophysiol. 54,318334. Segers, L. S., Shannon, R., Saporta, S., and Lindsey, B. G. 1987. Functional associations among simultaneously monitored lateral medullary respiratory neurons in the cat. I. Evidence for excitatory and inhibitory connections of inspiratory neurons. J. Neurophysiol. 57, 1078-1100. Smith, J. C., and Feldman, J. L. 1987. Involvement of excitatory and inhibitory amino acids in central respiratory pattern generation in vitro. Fed. Proc. 46, 1005. Smith, J. C., Greer, J. J., Liu, G., and Feldman, J. L. 1990. Neural mechanisms

68


generating respiratory pattern in mammalian brain stem-spinal cord in vitro. I. Spatiotemporal patterns of motor and medullary neuron activity. I. Neurophysiol. 64, 1149-1169. Smith, J. C., Ellenberger, H., Ballanyi, K., Richter, D. W., and Feldman, J. L. 1991. Pre-Botzinger complex: A brainstem region that may generate respiratory rhythm in mammals. Science 254, 726-729. Suzue, T. 1984. Respiratory rhythm generation in the in vitro brain stem-spinal cord preparation of the neonatal rat. I. Physiol. (London) 354, 173-183. Tank, D. W., Sugimori, M., Connor, S. A., and Llinas, R. R. 1988. Spatially resolved calcium dynamics of mammalian Purkinje cells in cerebellar slice. Science 242, 773-777.

Received May 28, 1992; accepted March 29, 1993.

This article has been cited by: 2. G.N. Borisyuk, R.M. Borisyuk, Yakov B. Kazanovich, Genrikh R. Ivanitskii. 2002. Models of neural dynamics in brain information processing the developments of 'the decade'. Uspekhi Fizicheskih Nauk 172:10, 1189. [CrossRef] 3. David Paydarfar, Frederic L. Eldridge, Joseph A. Paydarfar. 1998. Phase resetting of the respiratory oscillator by carotid sinus nerve stimulation in cats. The Journal of Physiology 506:2, 515-528. [CrossRef]

Communicated by John Rinzel

Subharmonic Coordination in Networks of Neurons with Slow Conductances Thomas LoFaro Nancy Kopell Department of Mathematics, Boston University, Boston, M A 02225 U S A

Eve Marder Department of Biology and Center for Complex Systems, Brandeis University, Waltham, M A 02254 U S A

Scott L. Hooper Department of Biology and Center for Complex Systems, Brandeis University, Waltham, M A 02254 U S A and Department of Biological Sciences, Ohio University, Athens, OH 45701 U S A

We study the properties of a network consisting of two model neurons that are coupled by reciprocal inhibition. The study was motivated by data from a pair of cells in the crustacean stomatogastric ganglion. One of the model neurons is an endogenous burster; the other is excitable but not bursting in the absence of phasic input. We show that the presence of a hyperpolarization activated inward current (ih) in the excitable neuron allows these neurons to fire in integer subharmonics, with the excitable cell firing once for every N 2 1 bursts of the oscillator. The value of N depends on the amount of hyperpolarizing current injected into the excitable cell as well as the voltage activation curve of ih. For a fast synapse, these parameter changes do not affect the characteristic point in the oscillator cycle at which the excitable cell bursts; for slower synapses, such a relationship is maintained within small windows for each N. The network behavior in the current work contrasts with the activity of a pair of coupled oscillators for which the interaction is through phase differences; in the latter case, subharmonics exist if the uncoupled oscillators have near integral frequency relationships, but the phase relationships of the oscillators in general change significantly with parameters. The mechanism of this paper provides a potential means of coordinating subnetworks acting on different time scales but maintaining fixed relationships between characteristic points of the cycles. Neural Computation 6, 69-84 (1994)


70

Thomas LoFaro et al.

1 Introduction

Most biological neurons show a wide variety of voltage and time-dependent conductances that shape their electrical properties. In particular many neurons have conductances that operate on multiple time scales, and that allow them to display plateau potentials and slow bursting pacemaker potentials. The extent to which neurons express these slow conductances is often controlled by neuromodulatory substances, or influenced directly by membrane potential (Harris-Warrick and Marder 1991; Kiehn and Harris-Warrick 1992; Marder 1991,1993; McCormick and Pape 1990). Many neurons that display plateau potentials show low threshold CaZ+ currents necessary for the sustained depolarization during the plateau (Angstadt and Calabrese 1991; Hounsgaard and Kiehn 1989). These same neurons often also show strong hyperpolarization activated inward currents ( ifl) that contribute to their recovery from inhibition (Angstadt and Calabrese 1989; Golowasch and Marder 1992; Golowasch et nl. 1992; Kiehn and Harris-Warrick 1992). The voltage dependence of ill is such that this current is very small when the neuron is depolarized, but may profoundly influence the neuron’s activity when the neuron is hyperpolarized. Therefore, we were particularly interested in exploring the role of ill in neurons that are phasically inhibited in pattern generating networks. This paper illustrates that if an oscillator is coupled by reciprocal inhibition to an excitable neuron with an if,-likecurrent, there can be stable patterns in which the excitable cell fires much less frequently than the oscillator. These N : 1 patterns are known as ”subharmonics” in mathematics and are sometimes called ”coupling ratios.” In this patterned output, for fast synapses, the characteristic point in the cycle at which the excitable cell begins to fire is relatively insensitive to parameter changes. This characteristic point is just after the end of inhibition from the oscillator to the excitable cell. (For a slower synapse the delay can be more variable.) Thus this mechanism is a candidate for maintaining timing relationships among cells or subnetworks that operate on different time scales. 2 Experimental Example

The pyloric rhythm of the crustacean stomatogastric ganglion displays alternating bursts of activity in the pyloric dilator (PD) and lateral pyloric (LP) motor neurons (Fig. 1A). The strict alternation of these functional antagonists is ensured by reciprocal inhibitory synaptic connections between the PD and LP neurons. However, if the LP neuron is hyperpolarized, either by current injection or by a strong, sustained inhibitory input, it produces a characteristic depolarizing voltage “sag” (Fig. lB), due to

Subharmonic Coordination in Networks

71

0 nA

PD

B LP

PD

C LP

PD 1 sec

Figure 1: (A) (left) A schematic of the experimental set up. The I’D and LP neuron mutually inhibit each other; each neuron’s membrane potential is being monitored intracellularly (V electrodes), and current can be injected into the LP neuron through a second electrode (i electrode). (right) The activity of the two neurons with 0 current injected into the LP neuron. The neurons exhibit 1 : 1 locking: (B) Negative current is injected into the LP neuron until the locking ratio is approximately 6 PD bursts per LP burst. (C) Negative current is injected into the LP neuron until the locking ratio is approximately 12 PD bursts per LP burst. the presence of a n i,, current (Golowasch and Marder 1992). Figure 1 B also shows that when the LP neuron is hyperpolarized by current injection, it rebounds every so often from inhibition to produce a sustained depolarization a n d high frequency firing of action potentials. In the presence of the inhibition of the PD neurons by the LP, this leads to a network output in which the number of P D bursts between each LP burst is a n

72


integer. This integer increases (Fig. 1C) with the amount of hyperpolarizing current until the LP neuron eventually completely ceases to burst (Hooper, unpublished data). This paper is not intended to model the above data but to explore how the presence of an i,, current can lead to subharmonics of the type shown and to investigate timing relationships of the model cells in the presence of fast and slow synapses.

3 Equations

We model the PD cell by the Morris-Lecar (1981)equations, a simple, twodimensional conductance-based system. These equations, originally formulated to describe electrical activity in barnacle muscle fiber, are sometimes used as a simple caricature of the envelope of bursting neurons without the spiking activity (Rinzel and Ermentrout 1989); they model situations in which the behavior of the envelope is determined primarily by calcium dynamics, which are explicitly included in the equations. We use nondimensional versions of these equations, as given in Rinzel and Ermentrout (1989). Parameters are chosen so that in the absence of coupling, the PD cell oscillates spontaneously [although in the real preparation, the oscillatory behavior of the PD neuron depends on that of another neuron, the anterior burster (AB) neuron]. The LP cell is also modeled with the Morris-Lecar equations but is given an additional current, based on ill. This current slowly activates when the LP neuron is hyperpolarized, and more rapidly inactivates when the neuron is depolarized. Its reversal potential is such that it is depolarizing when the current is active. This current thus responds to hyperpolarizing pulses (rhythmic inhibition) of the LP neuron with a delayed excitatory current that makes the LP neuron more likely to burst after the inhibition (postinhibitory rebound, Perkel and Mulloney 1974). In the absence of coupling to the PD cell, the LP cell has a stable equilibrium and a threshold for a large excursion before returning to rest. We denote the voltages of the PD and LP cells by DPD and u ~ r .The other variable in the Morris-Lecar equations is loosely modeled on the activation of an outward current. These variables for the PD and LP are labeled ~ P and D nLI'. The level of i,, is called s. The full equations are

(3.1)


73

(3.3) where vl = -0.01, v2 = 0.15, v3 = -0.12, v4 = 0.3, and u5 = 0.22. The constants for conductances and reversal potentials in 3.2 are g, = 2.0, vk = -0.7, ge = 0.5, up = -0.5, g,, = 1.1. These constants were assumed to be cell independent. The time constant TI = 3.3 was used to control the frequency of the PD oscillation. The LP cell was assumed to have a slow activation of the recovery variable, so we used &Lp = 0.01 and &D = 0.1. Finally, IPD= 0.35, and ILp was varied. The effect of ih has the form H(vLp. s) = gss[vs - vLp] with g, = 0.5 and v, = 0.3. The dynamics of i,, current are defined by the functions ks(v) and s,(v), which are given by kJv) 1 + e(L1-z17)/% 1 s,(v) E 1 + e(0-1’9)/vH (3.4) where v6 = 0.04, v7 = 0.05, v8 = 0.075, and v9 was varied. Since this process is slow 6 = 0.003. The inhibitory chemical synapses are modeled as

GPD(VLP,VPD) GLP(VLP. 4

(Ymm(vLP)[vsyn - VPD] (Yd[vsyn - ULP]

(3.5)

0.5 and vsyn = -0.7. Finally, in the last equation of 3.1 r2 = 10.0 except in some simulations in which we used a simplified version of the equations with r2 E 0, that is, d = W Z , ( U ~ D ) . The numerical simulations were done using the software PHASEPLANE (Ermentrout 1989) and dstool (Guckenheimer et al. 1991). The integration method employed was either a fourth order Runge-Kutta with small step size or the variable step size Gear’s method. (Y

=

4 Results

Two sets of simulations were done, the first using the simplified system assuming d = m,(v), that is, an instantaneous synapse from the I’Dto

74


the LP. The first simulation of the first set replicated some key features of the data in the motivating example in Figure 1. For this simulation, UY is set at 0.1. The parameter UY sets the position of the activation curve of 6,. At this position, i,, is active even at the uncoupled rest potential U L P x 0.3 of the cell. However, as shown by other simulations described below, this does not affect the qualitative behavior we are describing in this paper. Figures 2A,B,C give a typical picture of the voltages of LP and PD versus time at three levels of hyperpolarization, showing one LP burst for every one, four and seven cycles of the PD. In the absence of ill, there is no bursting in the LP cells (Fig. 2D). With a sufficient amount of tonic hyperpolarizing current, the LP cell is completely shut off (data not shown). Note that when an LP burst occurs, it does so at the same characteristic point within the PDs cycle: just on release of inhibition from the PD cell, regardless of the amount of injected current or the subharmonic locking it gives rise to. As discussed in Section 5, these results contrast strongly with the expected behavior of a pair of coupled phase oscillators whose uncoupled frequencies are nearly multiples of one another. We now give a heuristic explanation for the results, using geometrical phase plane analysis (Fig. 3) (Edelstein-Keshet 1988; Rinzel and Ermentrout 1989). (A more detailed analysis will be forthcoming; LoFaro 1993.) In the absence of the slow ill, the Morris-Lecar equations model the phenomenon of inhibitory rebound for parameter regimes in which the equations are excitable (Perkel and Mulloney 1974). That is, a pulse of inhibitory current, held for sufficiently long, moves the system to a new stationary state with a more hyperpolarized voltage; if the pulse is sufficiently strong, the trajectory from this new point in the uninhibited system jumps to the excited branch (Fig. 3A). Thus, the system is excited on release from inhibition. If the pulse is too weak, the system reverts quickly after release from inhibition to its previous stable steady state, without excitation (Fig. 38). In the parameter regime in which l i ' , ~ ~ is small, the threshold or boundary between these two possible responses is at the local minimum of the nullcline for the uninhibited system (see Figure 3A). Without if,, if the pulse of inhibitory current is sufficiently long, the system does not produce subharmonics. (Subharmonics and other patterned output can occur using another mechanism if the pulses are short; see Section 5.1.) The effect of the ill is to modulate the position of the threshold. During hyperpolarization, i,, is activated, opposing the hyperpolarizing current (causing the depolarizing "sag" back toward the original potential). Furthermore, until the LP bursts the activation of this current does not decrease or does so slowly. This enables the system to have a "memory" for hyperpolarizing pulses; the amount of excitation needed to pass the threshold decreases with each successive inhibitory pulse (Fig. 3C). Thus, an inhibitory pulse too small to cause inhibitory


V

o.6

75

V

t

A

B

0.61

LP

-0.6

-0.6

0.61

0.61

PD

0

-0.61 0

.

50

100

.

150

200

250

V

-0.61 0

50

100

150

200

250'

V

0.61

C

D

0.61

LP

JVVVIIVVVIIVIIVVIIUV

-0.6.

-0.6

0.6)

0.61

PD

0

-0.61

0

-0.61 50

100

150

200

250 I

0

50

100

150

200

2501

Figure 2: Numerical integrations of equations 3.1 with fast synaptic currents. In each part the upper trace is z ~ versus ~ p time. (A) 1 : 1 subharmonic; ILI>= 0.27, ug = 0.2. (8) 4 : 1 subharmonic; I1.p = 0.09, z)y = 0.2. ( C ) 7 : 1 subharmonic: I1.p = 0.08, z 9 = 0.2. (D) Small amplitude oscillations of LP due to the absence of the sag current; 41, = 0.09, z'g = 0.2, gs = 0.0.

rebound after one pulse can do so after some finite number. However, if the level of tonic inhibitory current is sufficiently large, then i,, builds slowly toward some saturation value over many cycles. For large enough


76

A vLpnullclines:

B

.. *

-

"u,

C


77

hyperpolarization, this saturated level is still insufficient to allow inhibitory rebound, and the LP shuts off. We note that this mechanism can work in a region in which ill is partially activated even at the resting potential of the uncoupled cell; in this case the inhibitory postsynaptic potentials (IPSPs) that provide the forcing d o not substantially change the rate of increase of ilf. Figures 4A,B show the range of voltages of the LP cell during the buildup of ih, and also during the burst of the LP for two different values of ILP.When the voltages in the high and low plateaus of the LP burst lie in the saturated regimes of the activation curve, as in Figure 4C, the IPSPs to the LP cell act mainly to prevent the firing of the LP; above a certain threshold necessary to keep the LP from firing during the on phase of the PD cell, an increase in the size of the IPSPs does not substantially change the output. Even in cases where the rate of increase of ih is changed by the IPSPs (Fig. 4D) the mechanism works as described above. The above heuristic explanation suggests that a similar effect to changing the level of hyperpolarization of the LP cell can be achieved by a shift of the activation curve of i l l . This is of interest because Kiehn and Harris-Warrick (1992) found that serotonin appears to shift the activation curve of ih in a crab STG neuron and similar results are suggested in P. interruptus cultured neurons (Turrigiano and Marder 1993). Thus, w e

p diagram, showing the mechanism for Figure 3: Facing: page. The VLJ - n ~ phase LP firing. (A) 1 : 1 firing without ih. The point A is the steady state before onset ~ (dashed of inhibition from PD.-Inhibition effectiiely lowers the V L nullcline curve) causing a sudden decrease in V L P and creating a "ghost" steady-state B to which all trajectories tend. On release from inhibition the nullcline reverts to its original position causing a sudden increase in ULP. If the steady-state B is below the threshold T and the inhibition is long enough to allow the trajectory to approach B, the firing ensues. (B) No firing with ill. The points A, B, and T are as in Figure 3A. In this figure, however, the "ghost" steady-state B is greater than the threshold T preventing the LP from firing on release from inhibition. (C) 2 : 1 firing with ill. At the end of an LP burst the Pb is released from inhibition and thus becomes depolarized. In turn, it inhibits the LP. During this initial inhibition v1.p and n L p drift along the dashed nullcline to the point denoted B. In addition, during this first inhibition the nullcline U L P = 0 drifts upward due to the slow increase in s (solid curve). Because the point B is above the threshold T, the LP trajectory tends during the PD interburst interval toward the point marked A, without a large excursion. During the second inhibition this mechanism repeats with the LP trajectory during the I'D burst tending to the point marked B. Again the buildup of i h causes the i'lAp = 0 nullcline to Now upon release from inhibition the LP bursts rise giving a new threshold since B is below T .

r.


78

-0.75 D

1

-0.75

0

0.75

Figure 4: i,, activation curves [s,(u)] versus LP depolarization and hyperpolarization. (Note: Because this current activates with hyperpolarization rather than depolarization, as is common for other currents, the activation curve has the opposite slope.) The left dark bar on the u axis represents the range of U L P during subthreshold oscillations and the right dark bar the range during depolarization; the numbers are taken from numerical integrations of equations 3.1. In the upper two graphs we kept ZJY fixed while changing only I I S . In both these graphs LP voltage during inhibition is on the upper branch of ,s while during excitation it is on the lower part of the transition. The identical qualitative aspects of these indicate that altering the amount of injected current to the LP does not significantly change the rate at which i,, increases or decreases. In figures A, C, and D, / ~ was p kept fixed while ug was reduced, shifting the if, activation curve leftward. Here a qualitative difference is apparent (especially in C and D) in the relationship between voltage ranges during subthreshold oscillations and the steep portion of the activation curve of if,. The shift of the activation curve significantly changes the average rate of increase of i,, during subthreshold oscillations. (A) 1 : 1 subharmonic; I L = ~ 0.27, z ~ g= 0.2; (B) 4 : 1 subharmonic; I L =~ 0.09, u g = 0.2; (C) 2 : 1 subharmonic; /Lp = 0.27, zlg = -0.2; (D) 2 : 1 subharmonic; IIap = 0.27, 719 = -0.4.

performed other simulations holding injected current fixed a n d shifting the activation curve s,(u) by varying parameter zly. As expected, if the curve is shifted in the depolarized direction, the effect of each pulse is enhanced, and the number of PD cycles per LP burst goes down; a shift in the hyperpolarizing direction produces the opposite effect. Figure 5 summarizes this work. For example, for I1.p = 0.1, subharmonics of three,


021784

0.15

.

0.1

.

0.05

.

0 .

3-

3

87654-4

i

2

3-3

5

4

3-1

087a

s

4

478

79

2

1-1

2

2

2

4.05 *

4.1

c

0.1

3-3 0.15

2

2

,

0.2

02s

Figure 5: Subharmonics of equations 3.1 with fast synaptic currents. For each of the four values of 719 shown the range of ILp exhibiting various subharmonics is plotted. For example, when u9 = 0.0 and ILP = 0.15 equations 3.1 display 3 : 1 subharmonics. five, six, and eight were observed. The simulations summarized in Figure 5 show that the activation curve can be shifted until ih is entirely off at the LP resting potential. The subharmonics thus obtained have high values of N unless the maximal conductance of i,, is increased, in which case low subharmonics can be obtained. We note that in the parameter regime in which ih is partially activated at rest, the reset of ih can create a postburst hyperpolarization of LP. This can affect the numerical relationship between the subharmonic N and the amount of injected current; it does not change the qualitative behavior. One difference between the motivating data shown in Figure 1 and the above simulations is that the biological LP neuron does not fire immediately on release from inhibition. This difference is to be expected, because the synapse from PD to LP is relatively slow (Eisen and Marder 1982; Hartline and Gassie 1979). The second set of simulations tested the effect of such a slow synapse by replacing the instantaneous function of voltage in the model synapse from PD to LP by a synapse with dynamics as given in Section 3. The subharmonic coordination continues to be displayed. Now, however, as in the data, there is a delay from the offset of inhibition of the LP cell to the firing of the LP cell. The delay increases with the amount of hyperpolarization. That delay can be seen in the I'D trace in Figure 6A and B; the I'D begins a part of its next cycle before being shut off by the LP burst.


80

A

V

B

V

0.61

LP -0.61

-0.61

0.61

0.61

PD

0

-0.61 0

50

100

150

200

250

-0.61 0

50

100

150

200

250

Figure 6: Numerical integration of equations 3.1 with slow synaptic currents. The traces are u ~ pand U P D versus time. (A) 1 : 1 subharmonic; ILP = 0.27, z77 = 0.2. (B) 4 : 1 subharmonic; ZLP = 0.15, u7 = 0.2. 5 Discussion

5.1 Other Mechanisms for Subharmonic Coordination and Related Work. Our simulations employ one inherently oscillatory model (the I'D) and one nonoscillatory, but excitable model (the LP) that possesses a hyperpolarization-activated, persistent inward current (i,,). The subharmonic coordination displayed in these simulations is reminiscent of the subharmonic coordination observed when two true oscillators with different inherent cycle frequencies are coupled through their phase differences. However, the two types of systems are very different. Though "phase" could be defined in various ways for the LP-PD system, the interactions depend on the rebound property and not the difference in phases, even if the LP cell is in a parameter regime where it can spontaneously oscillate. There are two kinds of differences between the two systems that are potentially relevant to regulatory properties. The first has to do with timing relationships between the elements as a parameter is varied. For a pair of oscillators whose interactions depend only on phases, there is usually a range of ratios of frequencies near each integer N in which stable N : 1 coordination is possible. As some parameter is changed, moving this ratio from one end of the possible range to the other, the subharmonic N remains fixed but the phase relationships between the oscillators in the locked solution change substantially. [For example, in


81

the equations 0: = LJ, + sin(0, - O,), i. j = 1.2, i # j , 1 : 1 coordination is possible if -L J < ~ ~1. For u1 fixed and L J ~varying between w1 - 1 and LJ~ +l, the difference 0, -02 in the locked solution varies between - x / 2 and +7r/2.] Such a large change in phase difference can be repeated within each of the regimes in which N : 1 coordination is stable (Ermentrout 1981). In particular there is no pair of points on the two cycles that occur at the same time, independent of parameters that change the relative frequency of the pair. However, in the case of instantaneous synapses illustrated in this paper, if the LP bursts at all it does so with the onset of its burst coming right after the offset of the PD burst. In Figure 2A,B,C one can see a slight delay in this timing. In the "singular limit" in which the time constant Q j ~ p+ 0, this delay goes to zero. Another difference between the two kinds of systems is their activity in the regimes near the transitions from one subharmonic to the next. For the coupled oscillator system, between the intervals of N : 1 coordination, there are parameter ranges in which other behavior is expected, including no locking or more complex frequency relations (Ermentrout 1981). These sort of complex activities are not seen in the model ill system; this system instead shifts from one subharmonic to the next in a stable, step-like pattern as the LP is further hyperpolarized, with possible bistability between N : 1 and ( N 1) : 1 behavior. This is proved under some limiting conditions in LoFaro (1993). Other papers dealing with similar dynamic phenomena in a different context are Levi (1990), Rinzel and Troy (1983). It should be emphasized that the rebound property alone is not sufficient to ensure the behavior described in the above paragraphs. The step-like changes in locking pattern occurs when the inhibitory pulses of the PD cell are long enough that the LP cell can approach sufficiently close to the critical point of the inhibited system (marked B in Fig. 3A-C) before being released; also important is the fast reset of ill in the depolarized regime, due to the magnitude of the function k,. In other parameter regimes, the behavior can be reminiscent of that of a forced oscillator when the forcing oscillator has significantly different frequency from the forced oscillator. Such systems display subharmonic oscillations along with more complicated behavior (Glass and Mackey 1979; Arnold 1989; Chialvo et al. 1990; Keener 1981). Indeed Wang has observed (Wang 1993) the characteristic "devil's staircase" well known in forced oscillators (Arnold 1989) in a system closely related to equations 3.1, but in a different parameter range. We have also observed such behavior in the absence of ill when the PD pulses are short. The effects of an il, current have been discussed in the context of half center oscillations in which two cells, neither of which is an oscillator, are reciprocally coupled to form an oscillatory circuit (Angstadt and Calabrese 1989; Wang and Rinzel 1992). If the inhibitory synapses between the cells are sufficiently slow, the reciprocal inhibition can give rise to

+


82

in-phase oscillations; otherwise they are antiphase. In these papers the locking ratio between the components is 1 : 1.

5.2 Biological Relevance. The mechanism described here was suggested by the electrophysiological recordings shown in Figure 1. However, it is important to reiterate that although stomatogastric ganglion neurons display i h (Golowasch and Marder 1992; Kiehn and HarrisWarrick 19921, much additional experimental work is needed to ascertain the role of this current in network function. Moreover, the kinds of recordings seen in Figure 1 could arise from other constellations of membrane currents and synaptic interactions in the STG. Despite the above caveat, the mechanism described here suggests that i,, could contribute to phenomena in which neurons change their activity pattern from the fast pyloric rhythm to the slow gastric rhythm (Weimann et al. 1991). One possible mechanism for these circuit "switches" is a decrease in synaptic input from the faster rhythm and an increase in an &like current that would provide slower time constant bursting. The rebound after many cycles displayed by cells with i,, is potentially relevant to other situations as well. As shown in Kopell and Le Masson (19931, it provides a possible mechanism for producing network oscillations from nonoscillatory cells connected together with a cortical-like architecture, in such a way that each cell fires only once in many cycles of the network oscillation. Such a network can be capable of modulating its field potential amplitude while keeping its frequency relatively fixed (Kopell and Le Masson 1993). Thus, the mechanisms of this paper may be relevant to complex brain functions. In another example (Kopell et a/. 19931, an ill current provides a possible mechanism for the rhythmic switching of each of the two leech heart tubes between peristaltic and synchronous pumping; this happens on a longer time scale than the period of the individual beats (Calabrese and Peterson 1983). Acknowledgments We wish to thank R. Harris-Warrick, F. Nagy, J. Rinzel, and X. J. Wang for useful conversations and helpful comments. T. L. and N. K. were supported in part by NIMH Grant MH47150 and E. M. and S. L. H. were supported in part by NIMH Grant MH46742. References Angstadt, J. D., and Calabrese, R. L. 1989. A hyperpolarization-activatedinward current in heart interneurons of the medicinal leech. 1. Neurosci. 9, 28462857.


83

Angstadt, J. D., and Calabrese, R. L. 1991. Calcium currents and graded synaptic transmission between heart interneurons of the leech. J. Neurosci. 11, 7 4 6 759. Arnold, V. I. 1989. Mathematical Metlzods of Classical Mechniiics. Graduate Texts in Math., 60. Springer-Verlag, Berlin. Calabrese, R., and Peterson, E. 1983. Neural control of heartbeat in the leech, Hivudo medicinalis. In SOC.for Exp. Biol. XXXVII, Neural Origin of Rhythmic Movements, A. Roberts and B. Roberts, eds., pp. 195-221. Cambridge University Press, Cambridge. Chialvo, D. R., Michaels, D., and Jalife, J. 1990. Supernormal excitability as a mechanism of chaotic dynamics of activation in cardiac Purkinje fibers. Circ. Res. 66, 525-545. Edelstein-Keshet, Leah 1988. Matheinatical Models in Biology. Random House, New Kork. Eisen, J. S., and Marder, E. 1982. Mechanisms underlying pattern generation in lobster stomatogastric ganglion as determined by selective inactivation of identified neurons. 111. Synaptic connections of electrically coupled pyloric neurons. 1. Neurophysiol. 48, 1392-1415. Ermentrout, G. B. 1981. n:m phase-locking of weakly coupled oscillators. J. Math. Bid. 12, 327-342. Ermentrout, G. B. 1989. PHASEPLANE, Version 3.0. Brooks/Cole. Glass, L., and Mackey, M. 1979. A simple model for phase locking of biological oscillators. J. Math. B i d . 7, 339-352. Golowasch, J., and Marder, E. 1992. Ionic currents of the lateral pyloric neuron of the stomatogastric ganglion of the crab. J. Neurophysiol. 67,318-331. Golowasch, J., Buchholtz, F., Epstein, I. R., and Marder, E. 1992. The contribution of individual ionic currents to the activity of a model stomatogastric ganglion neuron. J. Neuruphysiol. 67, 341-349. Guckenheimer, J., Myers, M. R., Wicklin, F. J., and Worfolk, P. A. 1991. dstool: A Dynamical System Toolkit with an Interactive Graphical Interface. Center for Applied Mathematics, Cornell University. Harris-Warrick, R. M., and Marder, E. 1991. Modulation of neural networks for behavior. Annu. Rev. Neurosci. 14, 39-57. Hartline, D. K., and Gassie, D. V. 1979. Pattern generation in the lobster (Punulirus) stomatogastric ganglion. I. Pyloric neuron kinetics and synaptic interactions. Bid. Cybernet. 33, 209-222. Hounsgaard, J., and Kiehn, 0. 1989. Serotonin-induced bistability of ru-motoneurones caused by a nifedipine-sensitive calcium plateau potential. J. Physi01. 414, 265-282. Keener, J. P. 1981. On cardiac arrhythmias: AV conduction block. J. Math. Bid. 12, 215-225. Kiehn, O., and Harris-Warrick, R. M. 1992. 5-HT modulation of hyperpolarization-activated inward current and calcium-dependent outward current in crustacean motor neuron. J. Neurophysiol. 68,496-508. Kopell, N., and Le Masson, G. 1993. Rhythmogenesis, amplitude modulation and multiplexing in a cortical architecture. Submitted. Kopell, N., Nadim, F., and Calabrese, R. 1993. In preparation.

84


Levi, M. 1990. A period-adding phenomenon. S I A M I. Appl. Math. 50, 943-955. LoFaro, T. 1993. Thesis: A period adding bifurcation in a family of maps describing a pair of coupled neurons. Boston University. Marder, E. 1991. Plateau in time. Current Biol. 1, 326-327. Marder, E. 1993. Modulating membrane properties of neurons: Role in information processing. In Exploring Brain Functions: Models in Neuroscience, Dahlem Conference, John Wiley, Chichester, pp. 27-42. McCormick, D. A., and Pape, H. C. 1990. Noradrenergic and serotonergic modulation of a hyperpolarization-activated cation current in thalmic relay neurons. 1.Physiol. 431, 319-342. Morris, H., and Lecar, C. 1981. Voltage oscillators in the barnacle giant muscle fiber. Biophys. I . 35, 193-213. Perkel, D. H., and Mulloney, B. 1974. Motor pattern production in reciprocally inhibitory neurons exhibiting postinhibitory rebound. Science 185, 181-183. Rinzel, J., and Ermentrout, G. B. 1989. Analysis of neutral excitability and oscillations. In Methods in Neuronal Modeling: From Synapses to Networks, C. Koch and I. Segev, eds., pp. 135-163. MIT Press, Cambridge, MA. Rinzel, J., and Troy, W. C. 1983. A one-variable map analysis of bursting in the Belousov-Zhabotinskii reaction. In Nonlinear Partial Differential Equations, 1982 Summer Research Conference, J. A. Smoller, ed., Durham, NH. Turrigiano, G. G., and Marder, E. 1993. Modulation of identified stomatogastric ganglion neurons in dissociated cell culture. 1. Neurophysiol. 69,1993-2002. Wang, X.-J. 1993. Multiple dynamical modes of thalamic relay neurons: rhythmic bursting and intermittent phase-locking. Neurosci., in press. Wang, X.-J., and Rinzel, J. 1992. Alternating and synchronous rhythms in reciprocally inhibitory model neurons. Neural Comp. 4, 84-97. Weimann, J. M., Meyrand, P., and Marder, E. 1991. Neurons that form multiple pattern generators: Identification and multiple activity patterns of gastric/pyloric neurons in the crab stomatogastric system. ]. Neurophysiol. 65, 111-1 22.

Received December 7, 1992; accepted May 13, 1993.

This article has been cited by: 2. Adam L. Taylor , Garrison W. Cottrell , William B. Kristan, Jr. . 2002. Analysis of Oscillations in a Reciprocally Inhibitory Network with Synaptic DepressionAnalysis of Oscillations in a Reciprocally Inhibitory Network with Synaptic Depression. Neural Computation 14:3, 561-581. [Abstract] [PDF] [PDF Plus] 3. S. Coombes, M Owen, G. Smith. 2001. Mode locking in a periodically forced integrate-and-fire-or-burst neuron model. Physical Review E 64:4. . [CrossRef] 4. S. Coombes, G. Lord. 1997. Intrinsic modulation of pulse-coupled integrate-and-fire neurons. Physical Review E 56:5, 5809-5818. [CrossRef] 5. David Terman, Euiwoo Lee. 1997. Partial Synchronization in a Network of Neural Oscillators. SIAM Journal on Applied Mathematics 57:1, 252. [CrossRef] 6. S. Coombes, S. Doole. 1996. Neuronal populations with reciprocal inhibition and rebound currents: Effects of synaptic and threshold noise. Physical Review E 54:4, 4054-4065. [CrossRef]

Communicated by Bruce McNaughton

Setting the Activity Level in Sparse Random Networks Ali A. Minai William B. Levy Depurtmeiit of Neurosurgery, Uiiiversity of Virginia, Charlottesoilk, VA 22908 U S A

We investigate the dynamics of a class of recurrent random networks with sparse, asymmetric excitatory connectivity and global shunting inhibition mediated by a single interneuron. Using probabilistic arguments and a hyperbolic tangent approximation to the gaussian, we develop a simple method for setting the average level of firing activity in these networks. We demonstrate through simulations that our technique works well and extends to networks with more complicated inhibitory schemes. We are interested primarily in the CA3 region of the mammalian hippocampus, and the random networks investigated here are seen as modeling the a priori dynamics of activity in this region. In the presence of external stimuli, a suitable synaptic modification rule could shape this dynamics to perform temporal information processing tasks such as sequence completion and prediction.

1 Introduction

Recurrent networks of neural-like threshold elements with sparse, asymmetric connectivity are of considerable interest from the computational and biological perspectives. From the perspective of neurobiology, sparse, recurrent networks are especially useful for modeling the CA3 region of the mammalian hippocampus. Due to its recurrent connectivity, CA3 is thought to play a central role in associative memory (Marr 1971; Levy 1989; McNaughton and Nadel 1989; Rolls 1989; Treves and Rolls 1992) and the processing of temporal information (Levy 1988, 1989). In both cases, CA3 is seen as a system capable of learning associations between patterns and using these associations for spatiotemporal pattern recognition. Synaptic counts (Amaral rt RI. 1990) indicate that CA3 is a sparsely connected recurrent network. Another characteristic of the system is that the primary pyramidal cells are inhibited by a much smaller population of interneurons (Buzsaki and Eidelberg 1982). The relative scarcity of Neural Cutnputatiun 6, 85-99 (1994)


86

Ali A. Minai and William B. Levy

inhibitory cells also implies that inhibition is broadly directed, with each interneuron inhibiting a large number of primary cells in its vicinity. Few neural network models take these two aspects of CA3 into account. Motivated to understand the role played by these characteristics in the network’s dynamical behavior, we have recently investigated a class of sparse, random networks with nonspecifically directed inhibition (Minai and Levy 1993a,b) and have found fixed point, cyclical, and effectively aperiodic behavior. We have also developed a simple model relating the level of network activity to parameters such as inhibition, firing threshold, and the strength of excitatory synapses. Several researchers have studied the dynamics of sparse, asymmetric networks within the framework of statistical mechanics (Derrida and Pomeau 1986; Sompolinsky and Kanter 1986; Derrida ef nl. 1987; Gutfreund and Mezard 1988; Kree and Zippelius 1991). One interesting conclusion to emerge from some studies is that, above some critical value, both sparseness (Kiirten, 1988) and asymmetry (Spitzner and Kinzel 1989a,b; Niitzel 1991) lead to an effectively aperiodic network dynamics, which may be called effectively aperiodic. In this paper, we present a simplification of our model based on an approximation to the standard error function. This simplification allows accurate prediction of network activity using a closed form equation. The level of activity in a recurrent network is of crucial importance for learning. A low but consistent level of activity enables the network to recode its stimuli as sparse patterns, thus decreasing interference between the representations of different stimuli and increasing capacity. Our model demonstrates how a CA3-like network without synaptic modification can control its level of activity using inhibition.

2 Network Specification and Firing Probability

Our network model is qualitatively similar to the associative memory model proposed by Marr (1971)and later investigated by others (GardnerMedwin 1976; McNaughton and Nadel 1989; Willshaw and Buckingham 1990; Gibson and Robinson 1992). A network consists of n binary (0/1) primary neurons, each with identical firing threshold 0. The network’s connectivity is generated through a Bernoulli process. Each neuron i has probability p of receiving a fixed excitatory connection of strength U J from each neuron j (including itself). The existence/nonexistence of such a connection is indicated by the 1 / 0 random variable ci, where P(c,, = 1) = p . Inhibition is provided by a single interneuron that takes input from all primary neurons and provides an identical shunting conductance proportional to its input to all primary neurons. Defining K as the inhibitory weight and m ( t )as the number of active neurons at time t,

Activity Level in Random Networks

87

the excitation yl and output zi of neuron i at time t are given by

(2.1) (2.2)

with yi(t) = 0 for all i if m ( t - 1) = 0, so that once the network becomes totally inactive, it cannot spontaneously reactivate itself. Substituting equation 2.2 in equation 2.1 and defining CP = HK/(l -H)w, the condition for firing is obtained as (2.3)

which means that, to fire at time t, neuron i must have at least ram(t - 1)1 active inputs, where [XIdenotes the smallest integer not less than x. Note that, due to the effect of averaging, the right-hand side of equation 2.3 is independent of i, and the inequality represents a universal firing condition. The firing condition also demonstrates that it is the composite, dimensionless parameter N that determines the dynamics of the network. In effect, N represents the relative strength of inhibition and excitation in the network, weighted appropriately by the threshold. If m(t - 1) = M , the average firing probability for a neuron i at the next time step t is

averagefiring probability

= p(M;n, p , (Y)

If M is sufficiently large, the binomial can be approximated by a gaussian, giving

p ( M ;? I .p . o)RZ - 1 - erf 2

(2.5)

where

A reasonable criterion for applying this approximation is M p > 5 and M ( 1 - p ) > 5. In sparse networks with p < 0.5, only the first condition is relevant, and the criterion is M > 5/p. Assuming that neurons fire independently, as they will tend to do in large and sparsely connected


88

networks (Minai and Levy 1993a), we obtain a stochastic return map relating m ( t ) to m ( t - 1):

m ( t ) = np[m(t- l ) ]+ O(fi)

(2.6)

Thus, the expected activity at time t is

( m ( t )I m(t - 1)) = np[m(t- l ) ]

(2.7) In the long term, the activity is attracted to 0 or to an O ( & ) region around f i , the point satisfying the fixed-point condition: tfi = n p ( m ) . Alternatively, one might look at the activity level r ( t ) E n - ’ m ( t ) rather than total activity m ( t ) to obtain the map

+

(2.8)

r ( t ) = p[nr(t- l ) ] O ( l / f i ) and expected activity level

(2.9) ( r ( t ) I v ( t - 1))= p[nr(t- l ) ] Here r ( t ) is an instance of what Amari (1974) calls the inacrostate of the system. The activity level fixed-point of the network is defined as r =

~ ’ mThe . qualitative behavior of the network depends in large measure on the value of ?, as discussed in our earlier studies (Minai and Levy 1993a,b). 3 Approximating the Firing Probability

While equations 2.4 and 2.5 can be used to obtain the activity map (and thus the activity level map) for a network with specified parameters, it is useful to look for a closed form, both for ease of calculation and for analytical manipulation. Such a closed form can be found using the approximation (see, e.g., Hertz et al. 1991) erf

(5)

z tanh ( E x )

(3.1)

Substituting equation 3.1 in equation 2.5, we obtain p(M;? I . p . 0 ) z

1

{

1 - tanh

[

2 [rrMl - M p

]}

JMp(l-1,

(3.2)

2 If M is large enough, [trMl z trM, which simplifies equation 3.2 to

T where

TE- 1 (Y

-p

(3.3)

py4

Equation 3.3 shows that the average firing probability is an increasing function of M for (I < p , a decreasing one for r t > p , and constant at 0.5 if 0 = p (see Figure 1).


I \

0

.'

\

89

a = 0.07

I

I

I

I

200

400

600

800

1 I0

rn(t-1)

Figure 1: The average firing probability at time t as a function of the activity at time t - 1 for a 1000 neuron network with p = 0.05. Each curve corresponds to a different value of (k, as indicated on the graph. Equation 3.3 was used for the calculation. The point where the diagonal crosses each curve is the predicted stable activity level for the corresponding value of N.

4 Setting Parameter Values to Obtain a Stable Activity Level

~

The most useful application of this model is in specifying the average level of activity in a network. Since the activity level stabilizes around 7 in the long term, we can use it as an estimate of the average activity level, (r), though, strictly speaking, there might be a slight discrepancy between the estimate and the actual value d u e to the asymmetric shape of the firing probability curve. As r ( t ) becomes more confined with increasing network size, this discrepancy becomes less and less significant.


90

To obtain a specific U by setting (r, we just need to find the o satisfying the activity fixed point equation (4.1)

r71 = n p ( f i t )

and substitute rn

( ~ ( rx) p +

=

nr. This gives

iTp(' 2n r

tanh-'(1

-

2V)

(4.2)

As long as V is not too close to 0 or 1, this is an adequate method for setting ( t , as shown by the results in Section 6. Note that the useful range of ( r is bounded as 0 5 (r 5 I; for values of ( I larger than 1, the firing condition (equation 2.3) shows that p(M) = 0 VM. When F is too low, fit is not large enough to allow the gaussian approximation of equation 2.5, and the model breaks down. However, as n grows, lower values of r come within the range of satisfactory approximation. Using the criterion Mp > 5 for the gaussian approximation, we conclude that equation 4.2 can be applied for F > 5 / n p . 5 Extension to the Multiple Interneuron Case So far in our model, we have assumed that inhibition is mediated by a single interneuron. In this simplification, we follow previous studies such as those by Gardner-Medwin (1976) and Gibson and Robinson (1992). However, as we argue in this section, the neuron model of equation 2.1 is also consistent with a population of interneurons, provided that these interneurons respond faster than the primary neurons and are statistically identical. Thus, let there be a set I of N interneurons, where each interneuron I receives an input synapse of weight u from each primary cell j with a fixed probability y and projects back to each j with probability X and weight ZI. Also, let E denote the set of n primary cells. The net excitation to each I is given by

(5.1) where c1, is a binary variable indicating the presence or absence of a connection from j to 1. Since interneurons are postulated to be linear (see the Appendix), the output of I is

Zdt) = C Y l ( t )

(5.2)

where C is a constant. Based on physiological evidence that hippocampal interneurons respond faster than pyramidal cells (Buzshki and Eidelberg 1982; McNaughton and Nadel 1989), we assume that I responds instantaneously to its input (unlike the primary cells that take one time unit to


91

respond). The inhibitory input to a primary cell i at time t is then given by (5.3)

If ?m(t - 1) is large enough, the distribution of Z , ( t ) for each I E I will approximate a gaussian with a mean value of uC?m(t - 1). By the central limit effect, then, q$( t ) will also be approximately normally distributed with mean uuCiXNin(t - 1). Thus, we can rewrite equation 2.1 as

where K = uuCrXN and / / [ u u. . C. 7 . A. m ( t - 1).N] is a random fluctuation that is O( and becomes increasingly insignificant as N m (t - 1) increases. Thus, equation 2.1 represents a reasonable approximation for equation 5.4 in large networks with a few thousand primary cells and a few hundred inhibitory neurons-especially if X is not too small. It should be noted that the multiple interneuron scheme described above is equivalent to having nonuniformly distributed random inhibitory connections between primary neurons, albeit with a shunting effect. Calculations from intracellular studies of inhibition in CA3 (Miles 1990) suggest that each pyramidal cell receives inhibitory synapses of widely varying strengths from 10 to 50 interneurons in its general neighborhood. The same study indicates that a specific inhibitory interneuron makes synapses of roughly equal strength with a large proportion of pyramidal cells in its neighborhood. Together, these factors suggest that the inhibition to neighboring pyramidal cells is highly correlated in amplitude and phase (Miles 1990). Thus, our model does capture part of the inhibitory structure in CA3, though more realistic models will probably be needed as the physiology becomes clearer.

JN.lcf-l,)

6 Results and Discussion

To show that the relationship between ( Y and U as expressed in equation 4.2 can be used to estimate the average activity level, we simulated a number of 300 and 1000 neuron networks and obtained empirical data to compare against the model. We obtained the average activity level, ( r ) , by running each network for 2000 steps and averaging its activity level over the last 1000 of these steps. We simulated seven different, randomly generated networks for each value of o and averaged the ( r ) s obtained in the seven cases. The results for 12 = 300 and PI = 1000 are shown in Figure 2a and b, respectively. It is clear from the graphs that r, as calculated from equation 4.2, is a good estimator for ( r ) when the activity level is not too high (> 0.8 or so) or too low (< 0.15 or so). One notable feature of the data is the large variance in the empirical average activity


92

081

062

' 0.4

~I

02

I

0

-/0

1---101

0 05

-~ 70 15

-i 02

(I

0

i 0.4

! !

'4

02.

Figure 2: The predicted and empirical activity level fixed points for different values of (t in (a) a 300 neuron network, and (b) a 1000 neuron network, with p = 0.05. The solid line shows the curve predicted by equation 4.2, while the bullets show the empirical values. The activity level is averaged over seven different networks for each (y value. All runs were started from the same initial condition, and the last 1000 steps of a 2000 step simulation were used in each case. It is clear that the model works very well when r is not too low or too high. The large error bars at low activity levels are due to the fact that some networks switched off while others converged to low activity cycles.

level at high N. This is due to the fact that some networks with these (k values switched off in the first 1000 steps while others settled down to relatively short limit cycles of the activity level predicted by equation 4.2. This trivial network behavior was discussed in our earlier study (Minai and Levy 1993a). Suffice it to say that this phenomenon is mediated by very low-activity states, and we expect it to become less significant in


93

larger networks where even low-activity states have a large number of active neurons. Figure 3 plots the empirically measured averaged activity level of a network with 1000 excitatory neurons and 50 inhibitory neurons for various values of C with fixed u, u,y, and A. The solid curve indicates the prediction generated using equation 4.2 with K = uuCyAN. The results at activity levels above 0.5 are not as good as in the single interneuron case, mainly because the variance in the inhibitory term is relatively high, but, comparing Figure 3 with Figure 2b, it is clear that the model of equation 4.2 works better in the multiple interneuron case when the activity level is low-presumably because the averaging in the inhibitory term makes simultaneous switch-off of all neurons less likely. Since the overall performance of the model should improve as n and N increase, there is reason to expect that equation 4.2 can be used to predict activity levels in large networks with multiple interneurons and very low activity.

7 Biological Considerations The results given above show that equation 4.2 is a good model for relating average activity to N . However, the network model is supposed to be a representation of biological networks such as the CA3, and it is important to put it in its full biological context. Above all, it is necessary to demonstrate that the model can be applied to networks of realistic size without running into problems. The most obvious potential problem, as implied by equation 4.2 and borne out by a comparison of Figure 2a and b, is that the dependence of V on cv tends toward a step function as n increases. In infinitely large networks, therefore, all neurons fire if (1 is less than p and none fire if it is greater than p , leading to an all-or-none activity situation. Even in large finite sized networks, however, obtaining a moderate activity level is problematic unless o is set very precisely. Since N is in arbitrary units, it is difficult to say exactly what degree of precision is prohibitive, but, as shown by equation 4.2, activity in large networks is easier to control at the lower end of the activity level spectrum. This is consistent with the physiological observation that activity levels in the rat CA3 are typically less than 1 % (Thompson and Best 1989). Since Amaral et al. (1990) estimate that the rat CA3 has 300,000 or so pyramidal cells and a connectivity of around 1.9%, we calculate the relationship between F and o for n = 300.000 and p = 0.02 as predicted by our model (Fig. 4). The activity levels shown are well above the value of 0.0008 needed for the gaussian approximation according to the criterion given earlier. It is apparent that U varies smoothly with IY in the range shown and (Y can, therefore, be used to control activity at typical CA3 levels. In the biological context, one must also try to account for the effects of synaptic modification. Whereas our model treats all excitatory synapses


94

1

0.8

0.6

-

r

0.4

0.2

0 0

I

I

0.05

0.1

c

15

a

Figure 3: The predicted and empirical activity level fixed points for different values of (Y in a network with 1000 primary neurons and 50 inhibitory interneurons. The connectivity parameters are p = 0.05, y = 0.1, and X = 0.5. The firing threshold is set to 0.5 and the connection weights are w = 1.0, u = 1.0, and u = 1.0. The values of w and 0 mean that (1 = K. The bullets indicate the empirically obtained average activities, while the solid line shows the curve predicted by equation 4.2 for a 1000 neuron network with a single interneuron and calculated using K = uuCyXN. The activity level is averaged over seven different networks for each value. All runs were started from the same initial condition, and the last 1000 steps of a 2000 step simulation were used in each case. The model works adequately for moderate activity values, and much better than in the single interneuron case for low activities. Its overall performance should improve when the number of neurons and interneurons is larger.


95

0.02-

0.015 -

~

r

0.005-

00.02

,

1

0.04

0.06

a

8

a

Figure 4: The relationship between parameter (Y and activity level fixed point r, as predicted by equation 4.2, for a network with 300,000 primary cells and a connectivity of p = 0.02. These values correspond roughly to those found in the rat CA3 region. The range of activity levels shown is also consistent with that seen in the rat CA3. It is clear that at these low activity levels, n is an effective means of controlling the average activity level of the network.

as identical, real synapses have varying strengths. Furthermore, due to the effects of synaptic modification, highly potentiated synapses on a primary cell would tend to be activated in correlated groups, not at random as in our model. This is both a cause and, through recurrence, an effect of correlated firing among the neurons themselves. The variation in synaptic strength and correlated input activity means that the excitation, wCjcjjzj(t- l ) , to a typical CA3 primary cell i will not necessarily have a unimodal gaussian distribution, as is the case in our model. This more

96


complex distribution of excitation to different cells will also alleviate the all-or-none activity problem described in the previous paragraph. Thus, it is best to see the random case treated in this paper as the starting point of the synaptic modification process, and the dynamics given by our model as the a priori or intrinsic dyiznrnics of a CA3-like network. Synaptic modification can then be considered a ”symmetry-breaking” process that gradually distorts the intrinsic dynamics into the spatiotemporal patterns implicit in the received environmental information. There have been some studies of synaptic modification in networks similar to ours, but only in the context of associative memory (Marr 1971; Gardner-Medwin 1976; Palm 1980; Gibson and Robinson 1992). Our main interest lies in the temporal aspects of network behavior and not just in the stability properties required by associative memory. Finally, we turn to another interesting aspect of our model: the assumption of continuous-valued (linear) inhibition. There is some evidence that the response of inhibitory interneurons in the hippocampus differs from that of the primary cells in more than just its speed. Interneurons have very low response thresholds and respond with multiple spikes whose number (and onset latency) is directly related to the intensity of the stimulus to the interneuron (Buzsaki and Eidelberg 1982). When integrated postsynaptically, these spike trains could represent an almost continuous-valued signal with a significant dynamic range. Of course, this does not imply that the interneuron’s response is lincnv in its stimulus as we, and others, have assumed. However, the analysis developed in this paper can be extended to some kinds of nonlinear inhibitory schemes, as described in the Appendix. 8 Conclusion

The mammalian hippocampus is a complex system and is probably involved in highly abstract information processing tasks such as recoding, data fusion, and prediction (Levy 1985, 1989; Rolls 1989; Rolls and Treves 1990; McNaughton and Nadel 1989). With its recurrent connectivity, it is natural to expect that the CA3 region plays an important role in any temporal processing done by the hippocampus (Levy 1989). In this paper, we have studied a recurrent network with CA3-like characteristics and have presented a model for relating its average activity level to parameters such as neuron firing threshold and the strength of inhibition. Using simulations, we have demonstrated that this model successfully relates network parameters to the activity level over a reasonable range, and that it is easily extended to situations with multiple inhibitory interneurons. Our model thus provides an understanding of the intrinsic dynamics of the untrained random network, and from this we can proceed to the problem of temporal learning and prediction through synaptic modification.


97

Appendix: Nonlinear Inhibition We could rewrite equation 5.2 more generally as Z,(t) = g [ m ( t - l)],where g(x) is an appropriate monotonically nondecreasing function (e.g., a sigmoid, as is often the case in artificial neural networks). Furthermore, since there is evidence of spontaneous, low-intensity firing in interneurons (Buzsaki and Eidelberg 1982), one could postulate an inhibition of the general form Kg[m(t- l)] K , where K is a small constant offset. The analysis given in the paper for linear inhibition transfers directly to the nonlinear case with offset. Thus, equation 2.1 becomes

+

Following the logic used in the linear case, and applying the hyperbolic tangent approximation to the error function, we get

where m(t - 1 ) = M and Lj = H K / ( ~- H)w. As before, we have taken [trg(M)] x trg(M). This, in turn, leads to a relationship between N and T, albeit a slightly more complicated one than in the linear case:

The extension of this more general case to multiple, statistically identical interneurons is straightforward, and leads essentially to equations A.2 and A.3 for large networks if g(x) is reasonably well-behaved.

Acknowledgments This research was supported by NIMH MH00622 and NIMH MH48161 to W.B.L., and by the Department of Neurosurgery, University of Virginia, Dr. John A. Jane, Chairman. The authors would like to thank Dawn Adelsberger-Mangan for her constructive comments. The paper has also benefited greatly from the suggestions of the reviewers.

References Amaral, D. G., Ishizuka, N., and Claiborne, 8. 1990. Neurons, numbers and the hippocampal networks. In Understanding the Brain through the Hippocarnpus: The Hippocampal Region as a Model for Studying Brain Striictiire and Function (Progress in Brain Research, Vol. 83), J. Storm-Mathisen, J. Zimmer, and 0. P. Ottersen, eds., pp. 1-11. Elsevier, Amsterdam.

98


Amari, S. 1974. A method of statistical neurodynamics. Kybcrnctik 14, 201-215. Buzsaki, G., and Eidelberg, E. 1982. Direct afferent excitation and long-term potentiation of hippocampal interneurons. /. Neuropkysiol. 48, 597-607. Derrida, B., and Pomeau, Y. 1986. Random networks of automata: A simple annealed approximation. Europhys. Lett. 1, 45-49. Derrida, B., Gardner, E., and Zippelius, A. 1987. An exactly solvable asymmetric neural network model. Europhys. Lett. 4, 167-173. Gardner-Medwin, A. R. 1976. The recall of events through the learning of associations between their parts. Proc. R. Soc. London B 194, 375402. Gibson, W. G., and Robinson, J. 1992. Statistical analysis of the dynamics of a sparse associative memory. Neural Networks 5, 645-661. Gutfreund, H., and Mezard, M. 1988. Processing of temporal sequences in neural networks. Phys. Rev. Lett. 61, 235-238. Gutfreund, H., Reger, J. D., and Young, A. P. 1988. The nature of attractors in an asymmetric spin glass with deterministic dynamics. 1.PlrJys.A: Moth. Gerr. 21, 2775-2797. Hertz, J., Krogh, A., and Palmer, R. G. 1991. Introduction to the Tlieor!/ of Neural Computation. Addison-Wesley, Redwood City, CA. Kree, R., and Zippelius, A. 1991. Asymmetrically diluted neural networks. In Models of Neural Networks, E. Domany, J. L. van Hemmen, and K. Schulten, eds., pp. 193-212. Springer-Verlag, New York. Kurten, K. E. 1988. Critical phenomena in model neural networks. Pliys. Lett. A 129, 157-160. Levy, W. B. 1988. A theory of the hippocampus based on reinforced synaptic modification in CAI. Soc. Neurosci. Abstr. 14, 833. Levy, W. B. 1989. A computational approach to hippocampal function. In C o n putationnlModels ofLearnirig in SirnpleNertral Systems. The PsychologyofLearriiri~ and Motivation, R. D. Hawkins and G. H. Bower, eds., Vol. 23, pp. 243-305. Academic Press, San Diego, CA. Marr, D. 1971. Simple memory: A theory for archicortex. Phil. Trans. R. Soc. London B 262, 23-81. McNaughton, B. L., and Nadel, L. 1989. Hebb-Marr networks and the neurobiological representation of action in space. In Neuroscience oiid Connectioiiist Theory, M. A. Gluck and D. Rumelhart, eds., pp. 1-63. Erlbaum, Hillsdale, NJ. Miles, R. 1990. Variation in strength of inhibitory synapses in the CA3 region of guinea-pig hippocampus in vitro. /. Physiol. 431, 659-676. Minai, A. A., and Levy, W. B. 1993a. The dynamics of sparse random networks. Biol. Cybernet. (in press). Minai, A. A., and Levy, W. B. 1993b. Predicting complex behavior in sparse asymmetric networks. In Advances in Neural Information Processing Systems 5, pp. 556-563. Morgan Kaufmann, San Mateo, CA. Niitzel, K. 1991. The length of attractors in asymmetric random neural networks with deterministic dynamics. /. Phys. A: Math. Gen. 24, L151-157. Rolls, E. T. 1989. Functions of neuronal networks in the hippocampus dnd neocortex in memory. In Neural Models of Plasticity, J. H. Byrne and W. 0. Berry, eds., pp. 240-265. Academic Press, New York.


99

Rolls, E. T., and Treves, A. 1990. The relative advantages of sparse versus distributed encoding for associative neuronal networks in the brain. Network 1, 407-421. Sompolinsky, H., and Kanter, I. 1986. Temporal association in asymmetric neural networks. Phys. Rev. Lett. 57, 2861-2864. Spitzner, P., and Kinzel, W. 1989a. Freezing transition in asymmetric random neural networks with deterministic dynamics. Z. Phys. B: Condensed Matter 77, 511-517. Spitzner, P., and Kinzel, W. 1989b. Hopfield network with directed bonds. Z. Phys. B: Condensed Matter 74, 539-545. Thompson, L. T., and Best, P. J. 1989. Place cells and silent cells in the hippocampus of freely-behaving rats. ].Neurosci. 9, 2382-2390. Treves, A., and Rolls, E. T. 1992. Computational constraints suggest the need for two distinct input systems to the hippocampal CA3. Hippocumpus 2, 189-200. Willshaw, D. J., and Buckingham, J. T. 1990. An assessment of Marr’s theory of the hippocampus as a temporary memory store. Phil. Trans. R. SOC.London B 329, 205-215. ~~

~

~

Received November 12, 1992; accepted May 26, 1993.

This article has been cited by: 2. Kenneth A. Norman, Ehren Newman, Greg Detre, Sean Polyn. 2006. How Inhibitory Oscillations Can Train Neural Networks and Punish CompetitorsHow Inhibitory Oscillations Can Train Neural Networks and Punish Competitors. Neural Computation 18:7, 1577-1610. [Abstract] [PDF] [PDF Plus] 3. Cengiz Günay, Anthony S. Maida. 2006. A stochastic population approach to the problem of stable recruitment hierarchies in spiking neural networks. Biological Cybernetics 94:1, 33-45. [CrossRef] 4. W. B Levy, A. Sanyal, X. Wu, P. Rodriguez, D.W. Sullivan. 2005. The formation of neural codes in the hippocampus: trace conditioning as a prototypical paradigm for studying the random recoding hypothesis. Biological Cybernetics 92:6, 409-426. [CrossRef] 5. Paul Rodriguez, William B. Levy. 2001. A model of hippocampal activity in trace conditioning: Where's the trace?. Behavioral Neuroscience 115:6, 1224-1238. [CrossRef] 6. Ali A. Minai. 1997. Covariance Learning of Correlated Patterns in Competitive NetworksCovariance Learning of Correlated Patterns in Competitive Networks. Neural Computation 9:3, 667-681. [Abstract] [PDF] [PDF Plus] 7. Xiangbao Wu, Robert A. Baxter, William B. Levy. 1996. Context codes and the effect of noisy learning on a simplified hippocampal CA3 model. Biological Cybernetics 74:2, 159-165. [CrossRef] 8. William B Levy. 1996. A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks. Hippocampus 6:6, 579-590. [CrossRef] 9. Michael E. Hasselmo, Bradley P. Wyble, Gene V. Wallenstein. 1996. Encoding and retrieval of episodic memories: Role of cholinergic and GABAergic modulation in the hippocampus. Hippocampus 6:6, 693-708. [CrossRef] 10. Eytan Ruppin , James A. Reggia . 1995. Patterns of Functional Damage in Neural Network Models of Associative MemoryPatterns of Functional Damage in Neural Network Models of Associative Memory. Neural Computation 7:5, 1105-1127. [Abstract] [PDF] [PDF Plus]

Communicated by Christof von der Malsburg

The Role of Constraints in Hebbian Learning Kenneth D. Miller * Diuision of Biology, Caltech 226-76, Pasadena, C A 92225 U S A David J. C. MacKayt Compictntioti atid Neural System, Caltech 139-74, Pnsndeiia C A 92125 USA

Models of unsupervised, correlation-based (Hebbian) synaptic plasticity are typically unstable: either all synapses grow until each reaches the maximum allowed strength, or all synapses decay to zero strength. A common method of avoiding these outcomes is to use a constraint that conserves or limits the total synaptic strength over a cell. We study the dynamic effects of such constraints. Two methods of enforcing a constraint are distinguished, multiplicative and subtractive. For otherwise linear learning rules, multiplicative enforcement of a constraint results in dynamics that converge to the principal eigenvector of the operator determining unconstrained synaptic development. Subtractive enforcement, in contrast, typically leads to a final state in which almost all synaptic strengths reach either the maximum or minimum allowed value. This final state is often dominated by weight configurations other than the principal eigenvector of the unconstrained operator. Multiplicative enforcement yields a "graded" receptive field in which most mutually correlated inputs are represented, whereas subtractive enforcement yields a receptive field that is "sharpened" to a subset of maximally correlated inputs. If two equivalent input populations (e.g., two eyes) innervate a common target, multiplicative enforcement prevents their segregation (ocular dominance segregation) when the two populations are weakly correlated; whereas subtractive enforcement allows segregation under these circumstances. These results may be used to understand constraints both over output cells and over input cells. A variety of rules that can implement constrained dynamics are discussed. Development in many neural systems appears to be guided by "Hebbian" or similar activity-dependent, correlation-based rules of synaptic modification (reviewed in Miller 1990a). Several lines of reasoning suggest 'Current address: Departments of Physiology and Otolaryngology, University of California, San Francisco, CA 94143-0444 USA. 'Current address: Radio Astronomy, Cavendish Laboratory, Madingley Road, Cambridge CB3 OHE, United Kingdom. Nrirrnl Cornpiitatioti 6, 100-126 (1994)


Constraints in Hebbian Learning

101

that constraints limiting available synaptic resources may play an important role in this development. Experimentally, such development often appears to be competitive. That is, the fate of one set of inputs depends not only on its own patterns of activity, but on the activity patterns of other, competing inputs. A classic example is given by the experiments of Wiesel and Hubel (1965) on the effects of monocular versus binocular visual deprivation in young animals (see also Guillery 1972). If neural activity is reduced in one eye, inputs responding to that eye lose most of their connections to the visual cortex, while the inputs responding to the normally active, opposite eye gain more than their normal share of connections. If activity is reduced simultaneously in both eyes for a similar period of time, normal development results: each eye’s inputs retain their normal cortical innervation. Such competition appears to yield a roughly constant final total strength of innervation regardless of the patterns of input activity, although the distribution of this innervation among the inputs depends on neural activities. Evidence of competition for a limited number of synaptic sites exists in many biological systems (e.g., Bourgeois et 01. 1989; Hayes and Meyer 1988a,b, 1989a,b; Murray et a / . 1982; Pallas and Finlay 1991). The existence of constraints limiting synaptic resources is also suggested on theoretical grounds. Development under simple correlationbased rules of synaptic modification typically leads to instability. Either all synapses grow to the maximum allowed value, or all synapses decay to zero strength. To achieve the results found biologically, a Hebbian rule must instead lead to the development of selectivity, so that some synaptic patterns grow in strength while others shrink. Von der Malsburg (1973) proposed the use of constraints conserving the total synaptic strength supported by each input or output cell to achieve selectivity; related proposals were also made by others (Perez et a / . 1975; Rochester et al. 1956; Rosenblatt 1961). A constraint that conserves total synaptic strength over a cell can be enforced through nonspecific decay of all synaptic strengths, provided the rate of this decay is set for the cell as a whole to cancel the total increase due to specific, Hebbian plasticity. Two simple types of decay can be considered. First, each synapse might decay at a rate proportional to its current strength; this is called multiplicative decay. Alternatively, each synapse might decay at a fixed rate, independent of its strength; this is called subtractive decay. The message of this paper is that the dynamic effects of a constraint depend significantly on whether it is enforced via multiplicative or subtractive decay. We have noted this briefly in previous work (MacKay and Miller 1990a,b; Miller 1990a; Miller ct nl. 1989). 1 Simple Examples of the Effects of Constraints A few simple examples will illustrate that strikingly different outcomes can result from the subtractive or multiplicative enforcement of a con-

102

Kenneth D. Miller and David J. C. MacKay

straint. The remainder of the paper will present a systematic analysis of these differences. Consider synaptic plasticity of a single postsynaptic cell. Let w be the vector of synaptic weights onto this cell; the ith component, w,, is the synaptic weight from the ith input. We assume synaptic weights are initially randomly distributed with mean wInlt,and are limited to remain between a maximum value w,, and minimum value wmin.We consider the effect of a constraint that conserves the total synaptic strength, C,w,, implemented either multiplicatively or subtractively. Consider first a simple equation for Hebbian synaptic plasticity, (d/dt)w = Cw, where C is a matrix describing correlations among input activities (derived for example in MacKay and Miller 1990a). Suppose this correlation is a gaussian function of the separation of two inputs. We assume first that wmln= 0. The final outcomes of development under this equation are shown in Figure 1A. With no constraint (column l), all synapses saturate at wmax,so all selectivity in the cell’s response is lost. Under a multiplicative constraint (column 2), synaptic strengths decrease gradually from center to periphery. The final synaptic pattern in this case is proportional to the principal eigenvector of C. Under a subtractive constraint (columns 3 and 4), a central core of synapses saturate at or near strength w,,,, while the remaining synapses saturate at wmln. If w,, is increased, or the total conserved synaptic strength wtot is decreased by decreasing winlt, the receptive field is sharpened (column 4). In contrast, under the multiplicative constraint or without constraints, the shape of the final receptive field is unaltered by such changes in w,, and wtot. This sharpening of the receptive field under subtractive constraints occurs because all synapses saturate, so the final number of nonzero synapses is approximately wtot/wmax.Such sharpening under subtractive constraints can create a precise match between two spatial maps, for example, maps of auditory and of visual space, despite spatially broad correlations between the two maps (Miller and MacKay 1992). If wmlnis decreased below zero, center-surround receptive fields can result under subtractive constraints (Fig. 1B). In contrast, the results under multiplicative constraints or unconstrained dynamics are unaltered by this change. This mechanism of developing center-surround receptive fields underlies the results of Linsker (19861, as explained in Section 2.5. Again, an increase in w,,, or decrease in wtot leads to sharpening of the positive part of the receptive field under subtractive constraints (column 4). Finally, consider ocular dominance segregation (Miller et al. 1989) (Fig. 1 0 . We suppose the output cell receives two equivalent sets of inputs: left-eye inputs and right-eye inputs. A gaussian correlation function C describes correlations within each eye as before, while between the two eyes there is zero correlation; and wmin = 0. Now results are much as in (A), but a new distinction emerges. Under subtractive constraints, ocular dominance segregation occurs: the output cell becomes


I

Right

;

I

,

103

-

I '

Figure 1: Outcomes of development without constraints and under multiplicative and subtractive constraints. (A,B) Outcome of a simple Hebbian development equation: unconstrained equation is (d/dt)w = Cw. Initial synaptic weights are shown at the top left. The correlation matrix C is a gaussian function of the separation between two synapses (shown at top right). (A) w,in = 0; (B) wmin = -2. ( C )Outcome of a similar equation but with two identical sets of inputs, representing left- and right-eye inputs. Within each eye, correlations are the same as in (A); between the eyes there is zero correlation. Unconstrained equations are (d/dt)wL = CwL;(d/df)wR = CwR.All results are from simulations of a two-dimensional receptive field consisting of a diameter-13 circle of inputs drawn from a 13 x 13 square grid. The resulting receptive fields were approximately circularly symmetric; the figures show a slice horizontally through the center of the field. All simulations used w,, = 8; all except (B) used w,in = 0. The left three columns show results for winit = 1. The right column [subtractive(2)]uses winlt = 0.5, which halves the conserved total synaptic strength wtot.

monocular, receiving input from only a single eye. Under multiplicative constraints, there is no ocular dominance segregation: the two eyes develop equal innervations to the output cell. Segregation under multiplicative constraints can occur only if there are anticorrelations between the two eyes, a s will be explained in Section 2.6. In summary, unconstrained Hebbian equations often lead all synapses to saturate at the maximal allowed value, destroying selectivity. Multiplicative constraints instead lead the inputs to develop graded strengths.


104

Subtractive constraints lead synapses to saturate at either the maximal or minimal allowed value, and can result in a sharpening to a few bestcorrelated inputs. They also can allow ocular dominance segregation to develop in circumstances where multiplicative constraints do not. These differences between subtractive and multiplicative constraints are easily understood, as we now show. 2 Multiplicative and Subtractive Constraints for a Single Output Cell

We begin with a general linear synaptic plasticity equation without decays, (d/dt)w(t)= Cw(t). We assume that the matrix C is symmetric: in Hebbian learning, C,, represents the correlation in activity between inputs i and j, so C,, = C,,.' Thus, C has a complete set of orthonormal eigenvectors e" with corresponding eigenvalues A" (that is, Ce" = A"e"). Typically most or all of the eigenvalues of C are positive; for example, if C is the covariance matrix of the input activities then all its eigenvalues are positive. We use indices i, j to refer to the synaptic basis, and n . b to refer to the eigenvector basis. The strength of the ith synapse is denoted by zu,. The weight vector w can also be written as a combination of the eigenvectors, w = 1,woe",where the components of w in the eigenvector basis are w,= w . e". We assume as before that the dynamics are linear u p to hard limits on the synaptic weights, wmln5 w,(t) 5 w,,,; we will not explicitly note these limits in subsequent equations. 2.1 Formulation of Multiplicative and Subtractive Constraints. By a multiplicative or subtractive constraint, respectively, we refer to a timevarying decay term y(t)w or c(t)n that moves w, after application of C, toward a constraint surface. We assume y and c are determined by the current weight vector w(t) and do not otherwise depend on t, so we write them as y(w) or c(w). Thus, the constrained equations are2

d -w(t) dt d -w(t) dt

=

Cw(t) - y(w)w(t)

(Multiplicative Constraint) (2.1)

=

Cw(t) - c(w)n

(Subtractive Constraint)

(2.2)

'We work in a representation in which each synapse is represented explicitly, and the density of synapses is implicit. Equivalently, one may use a representation in which the synaptic density or "arbor" function is explicit (MacKay and Miller 1990a, App. A). Then, although the equation governing synaptic development may appear nonsymmetric, it can be symmetrized by a coordinate transformation. Thus, the present analysis also applies in these representations, as further described in Miller and MacKay (1992, Ap B) ?To understand why the term -y(w)w(t) represents a multiplicative constraint, consider a multiplicatively constrained equation w(t + At) = /j(w) [w(t)+ Cw(t)At], where /j(w) achieves the constraint. This is identical to [w(t At) - w(l)]/At = Cw(t) - y(w)w(t At) where y ( w ) = [I - /j(w)]//j(w)At. For At 0 this becomes equation 2.1.

+

+

-


105

The vector n is a constant. Typically, all synapses have equal subtractive decay rate, so n = (1.1. . . . ,1)T in the synaptic basis. Multiplicative or subtractive constraints represent two methods of enforcing a constraint, that is, of maintaining the weight vector on some constraint surface. We now consider the type of constraint to be enforced. We will focus on two types. First, a constraint may conserve the total synaptic strength C ,w,, as in Section 1. We refer to this as a type 1 constraint, and to a multiplicative or subtractive constraint of this type as M1 or S1, respectively. These are frequently used in modeling studies (e.g., M1: Grajski and Merzenich 1990; von der Malsburg 1973, 1979; von der Malsburg and Willshaw 1976; Perez et al. 1975; Rochester et al. 1956; Whitelaw and Cowan 1981; Willshaw and von der Malsburg 1976, 1979; S1: Miller 1992; Miller et al. 1989).3 We define a type 1 constraint more generally as one that conserves the total weighted synaptic strength, 1, w,n, = w . n, where n is a constant vector. Typically, n = (1.1.. . . l)T. A type 1 constraint corresponds to a hyperplane constraint surface. For an S1 constraint, we choose the subtracted vector n in equation 2.2 to be the same as this constraint vector n. This means we consider only subtractive constraints that project perpendicularly onto the constraint surface. Then type 1 constraints can be achieved by choosing

M1: S1:

y(w) = n . C w / n .w F(W)

=

n.Cw/n.n

[with n .w(t = 0)# 01

(2.3) (2.4)

These choices yield n.(d/dt)w = (d/dt)(n.w ) = 0 under equations 2.1 or 2.2, respectively. Second, we consider a constraint that conserves the sum-squared synaptic strength, C, wf = w . w. This corresponds to a hypersphere constraint surface. We refer to this as a type 2 constraint (the numbers "1" and "2" refer to the exponent p in the constrained quantity C wr). This constraint, while not biologically motivated, is often used in theoretical studies (e.g., Kohonen 1989; Oja 1982). We will consider only multiplicative enforcement of this constraint? called M2. M2 can be achieved

'Many of these models used nonlinearities other than hypercube limits on synaptic weights. Our results nonetheless appear to correctly characterize the outcomes in these models. 4Subtractive enforcement, 52, does not work in the typical case in which the fixed points are unstable. The constraint fails where n is tangent to the constraint hypersphere (i.e., at points where w ' n = 0). Such points form a circumference around the hypersphere. The S2 dynamics flow away from the unstable fixed points, at opposite poles of the hypersphere, and flow into this circumference unless prevented by the bounds on synaptic weights.

106


Table 1: Abbreviations used." Type 1 constraint Conserves the total synaptic strength, C, zoI = w n Zero-sum vector A vector w with zero total synaptic strength: Elw,= w . n = 0 M1 Multiplicatively enforced type 1 constraint Subtractively enforced type 1 constraint, using s1 perpendicular projection onto the constraint surface Type 2 constraint Conserves the length of the weight vector, 1, ~f = w . w M2 Multiplicatively enforced type 2 constraint "The typical case n = ( 1 3 1 . . . . , l ) 'is used to describe Type 1 constraints.

by choosing

M2:

T(W)

=

W.CW/W.W

(2.5)

This yields 2 w . (d/dt)w = (d/dt)(w.w) = 0 under equation 2.1. The abbreviations introduced in this section are summarized in Table 1. 2.2 Projection Operators. Each form of constrained dynamics can be written (d/dt)w = PCw, where P is a projection operator that projects the unconstrained dynamics onto the constraint surface. For S1, the projection operator is P = 1 - ( n n ' / n . n), for M1, it is P = 1 - ( w n l / w .n ) , and for M2, it is P = 1 - (ww'/w. w). We can write these operators as P = 1 - (sc'/s. c), where s is the subtracted vector, c the coristrnint vector, and 1 the identity matrix (Fig. 2). The projection operator removes the c component of the unconstrained derivative Cw, through subtraction of a multiple of s. Thus, the subtracted vector s represents the method of constraint enforcement: s = w for multiplicative constraints, while s = n for subtractive constraints. The constraint vector c determines the constraint that is enforced: the dynamics remain on the constraint surface w . c = constant. Given a constraint surface, there are two "natural" methods of constraint enforcement: projection perpendicular to the surface (s = c), or projection toward the origin (s = w). For a type 1 constraint, these lead to different dynamics: S1 is perpendicular projection, while M1 is projection along w. For a type 2 constraint, these are identical: M2 is both perpendicular projection and projection along w. 2.3 Dynamic Effects of Multiplicative and Subtractive Constraints. In this section, we characterize the dynamics under M1, S1, and M2 constraints. In Section 2.3.1, we demonstrate that under '31, the dynamics


107

Figure 2: Projection onto the constraint surface. The projection operator is P = 1 - (scT/s . c). This acts on the unconstrained derivative Cw by removing its c component, projecting the dynamics onto the constraint surface c . PCw = c (d/dt)w = 0. This constraint surface is shown as the line perpendicular to c. The constraint is enforced through subtraction of a multiple of s: PCw = Cw-ps where /j = c . C w / c . s. For multiplicative constraints, s = w; for subtractive S1 constraints, s = c = n.

typically have no stable fixed point, and flow until all synapses are saturated; while under multiplicative constraints, the principal eigenvector eo of C is a stable fixed point of the dynamics, provided that it satisfies the constraint. In Section 2.3.2, we characterize the conditions under which multiplicatively constrained dynamics flow to the principal eigenvector fixed point. In Section 2.3.3, we characterize the outcome under S1 constraints in terms of the eigenvectors of C. We begin by illustrating in Figure 3 the typical dynamics under M1, S1, and M2 in the plane formed by the principal eigenvector of C, eo, and one other eigenvector with positive eigenvalue, e' . Figure 3 illustrates the main conclusions of this section, and may be taken as a visual aid for the remainder: 0

In Figure 3A we illustrate M1 and S1 dynamics in the case in which the principal eigenvector e" is close in direction to the constraint vector n (n is the vector perpendicular to the constraint surface). This is typical for Hebbian learning when there are only positive correlations and the total synaptic sum is conserved, as in the examples of Section 1. Positive correlations lead to a principal eigenvector in which all weights have a single sign; this is close in direction to the usual constraint vector (1.1... . l)T, which conserves total synaptic strength. In this case, growth of eo would violate the constraint. Multiplicative and subtractive constraints lead to very


108

different outcomes: multiplicative constraints lead to convergence to e", whereas subtractive constraints lead to unstable flow in a direction perpendicular to n. The outcome in this case was illustrated in Figures lA,B. 0

In Figure 38 we illustrate M1 and S1 dynamics in the case in which the principal eigenvector e" is parallel to the constraint surface: e" . n = 0. We call such vectors w, for which w . n = 0, zero-sum vectors. Growth of a zero-sum vector does not violate the type 1 constraint. For practical purposes, any vector that is approximately parallel to the constraint surface, so that it intersects the surface far

A

""t;."'

UNCON.

el

+

eo

\ e

eo'

eo

eo

C

M2

el

el

eo


109

outside the hypercube that limits synaptic weights, may be treated a s a zero-sum vector. The principal eigenvector is typically a zerosum vector in Hebbian learning when correlations among input activities oscillate in sign as a function of input separation (Miller 1990a). Such oscillations lead to a principal eigenvector in which weights oscillate in sign, and s u m approximately to zero; such a vector is approximately perpendicular to the constraint vector (1,1,.. . , l)T. In this case, growth of e0 does not violate the constraint. The type of constraint enforcement makes little difference: the weight vector typically flows to a saturated version of e". 0

Under M2 constraints (Fig. 3 0 , the principal eigenvector eo is always perpendicular to the constraint surface, and its growth would always violate the constraint. The dynamics converge to eo.

2.3.1 General Differences between Multiplicative and Subtractive Constraint Enforcement: Fixed Points and Stability. We now establish the essential difference between multiplicative and subtractive constraints. To d o so, w e examine the locations and stability of the fixed points that are in the interior of the hypercube of allowed synaptic weights ("interior fixed

Figure 3: Facing page. Dynamics under multiplicative and subtractive constraints. Dynamics in the plane formed by the principal eigenvector of C, e", and one other eigenvector with positive eigenvalue, el. (A) M1 and S1 constraints when e0 is close in direction to n. Diagonal lines indicate the constraint surface on which n . w is constant. Unconstrained: arrows show the unconstrained derivative Cw from the point w at the base of the arrow. M1: Solid arrows show the unconstrained flow; dashed arrows show the return path to the constraint surface (as in Fig. 2). Return path is in the direction w. Open circle indicates unstable fixed point, large filled circle indicates stable fixed point. The fixed points are the eigenvectors, where Cw cx w. S1: Return path is in the direction n. The fixed point is the point where Cw x n (indicated by perpendicular symbol), and is unstable. Second row: The resulting constrained flow along the constraint surface for M1 and S1. (B)M1 and S1 constraints when eo is perpendicular to n. The constraint surface does not intersect eo. M1 and S1 lead to similar outcomes: unstable growth occurs, predominantly in the eo direction, until the hypercube that limits synaptic weights is reached. The outcome is expected to be a saturated version of keo. Note that the unconstrained dynamics also flow predominantly in the heo direction and so should lead to a similar outcome. For convenience, we have chosen the constraint direction n = el. (C) M2 constraints. The return path is in the direction of w, as for M1. Thus, locally (for example, near the fixed points) the dynamics are like M1. On a large scale, the dynamics differ because of the difference in constraint surface. Left, unconstrained derivative and return path; right, constrained flow. Figures were drawn using eigenvalues: Xo/X' = 3; constraint vector in A: n . eo/n. e* = 1.5.


110

points"). A fixed point w"'is a point where the flow ( d / d t ) w = 0. For C symmetric, the constrained dynamics must either flow to a stable interior fixed point or else flow to the hypercube. We will show that the only stable fixed point of multiplicatively constrained dynamics is the intersection of the principal eigenvector e" of C with the constraint surface; and that if eo intersects the constraint surface, then the dynamics typically converge to eo,as was illustrated in Section 1. Under subtractive S1 constraints, there is generally no stable fixed point within the hypercube. S1 dynamics typically are only stabilized once all synapses (or all but one) reach saturation at w,,, or wn,in. The locations of the interior fixed points follow trivially from equations 2.1-2.2 and the fact that the dynamics remain on the constraint surface: 0

0

The fixed points under a multiplicatively enforced constraint are the intersections of the eigenvectors of C with the constraint surface, that is, the points w on the constraint surface at which Cw 0: w. The fixed points under a subtractively enforced constraint are the points w on the constraint surface at which Cw rx n.

The stability of a fixed point can be determined as shown in Figure 3A, by determining whether a point perturbed from the fixed point is taken farther away by the dynamics. Generalizing the reasoning illustrated there, it is easy to prove (Appendix):

Theorem 1. Under a multiplicatively enforced constraint, if the principal eigenvector of C is an interior fixed point it is stable. lnterior fixed points that are nonprincipal eigenvectors are unstable. Theorem 2. Under an S1 constraint, if C has at least two eigenvectors with positive eigenvalues, then any interior fixed point is unstable. A case of theorem 1 for M2 constraints was proven by Oja (1982). Theorem 2 shows that S1 dynamics are unstable when no synapse is saturated. If in addition the following condition holds, as occurs when C is a correlation matrix, then these dynamics remain unstable until all synapses have saturated (Appendix):

Theorem 3. Let i and j be indices in the synaptic basis. Suppose that for all i and j with i # j , Ci;> (C;,l. Then under an S1 constraint, either all synapses or all but one are saturated in a stable final condition. This condition is satisfied for Hebbian models, because Ci, represents the correlation in activities between input i and input j . The result is sharply different from that for multiplicative constraints: in that case, the principal eigenvector may be a stable fixed point with no synapse saturated. This theorem generalizes a result proven by Linsker (1986).


111

Theorem 3 explains the sharpening of the receptive field that occurs under an S1 constraint (Fig. 1). A practical implication is that an upper limit on synaptic strengths,,,,oz, is needed for stability under an S1 constraint (whereas no such limit is needed under a multiplicative constraint). If there is no upper synaptic limit, eventually one synapse will acquire all of the allowed synaptic strength, while all other synapses will become saturated at ~ i i ~ , ~ , , .

2.3.2 The Outcorne iirzifer Miiltiplicative Corzstraints. From the previous section, we conclude that a multiplicatively enforced constraint results in convergence to the principal eigenvector e0 of C provided (1) the principal eigenvector intersects the constraint surface within the hypercube of allowed weights, forming a stable fixed point; and (2) the initial weight vector is within the basin of attraction of this fixed point. We ignore the possible effect of the hypercube, and assess when these conditions are met. Under M2 constraints, both of these conditions are always satisfied (Fig. 3C), so M2 constraints always lead to convergence. Under M1 constraints, condition (1) is satisfied when en is nonzerosum (Fig. 3A). Then, for n = (1.1. . . . , 1)I, condition (2) is also satisfied in at least two typical cases: if both the initial weight vector and the principal eigenvector have no changes in sign, or if weights are initialized as small fluctuations about a nonzero mean5 Thus, M1 constraints typically converge to en when e" is nonzero-sum. 2.3.3 The Outcome iirzifcr SI Coiistroirzfs. S1 constraints lead to altered, linear dynamics. To see this, write the subtractively constrained equation as (d/dt)w(t) = PCw(t) with P = 1 - nn'; here, n = n/lnl. Write w as the sum w ( t ) = Pw(t) + zo,,n, where w,, = w . n is conserved. Then the dynamics can be written: d

-w(t) dt

=

+

PCPw(t) 7o,,Pcn

(2.6)

PCP is the operator C, restricted to the subspace of zero-sum vectors. w,,PCn is a constant vector. Thus, S1 constraints lead to linear dynamics driven by PCP rather than by C. These dynamics have been characterized in MacKay and

'Let Ae'l be the stable fixed point, and let wg be the initial weight vector on the constraint surface. Condition (2) is satisfied if wg . (/je") > 0. Suppose the constraint conserves w . ti = 7u,,, so that wll n = /je" . n = UI,,. Then if wo and e" are each singlesigned, they must have the same sign so wg . ( d e " ) > 0. If weights are initialized as small fluctuations about a nonzero mean, then wg Y 7u,,n, so wg (/jell) Y ws > 0.

112


Miller (1990a, Appendices B and E). To understand them, consider first the eigenvectors of PCP. These are of two types: 1. Any zero-sum eigenvector of C is also an eigenvector of PCP with identical eigenvalue. So zero-sum eigenvectors of C grow freely, at the same rate as they would in the absence of constraints.

2. Each nonzero-sum eigenvector of C is replaced by a corresponding zero-sum eigenvector of PCP with smaller eigenvalue+' for example, an all-positive, centrally peaked nonzero-sum eigenvector of C may be replaced by a center-surround (positive center and negative surround) zero-sum eigenvector of PCP. Eigenvalue order of the nonzero-sum eigenvectors is preserved under this correspondence.

Now consider the constant term w,,PCn. This term boosts the growth rate of the eigenvectors of PCP that compose it. These are the eigenvectors derived from the nonzero-sum eigenvectors of C. Thus, under S1 constraints, the dynamics may be dominated either by the principal zero-sum eigenvector of C, or by a zero-sum vector that replaces the principal eigenvector. Both vectors may be very different from the principal eigenvector of C. 2.3.4 Summary: The Outcome under M1, SZ and M2. Multiplicative and subtractive constraints lead to dramatically different outcomes in many cases. In particular, under a type 1 constraint, multiplicative constraints converge to a dominant nonzero-sum pattern, whereas subtractive constraints suppress such a pattern in favor of a zero-sum pattern. We may summarize as follows:

1. If the principal eigenvector eo of C is a nonzero-sum vector and intersects the constraint surface within the hypercube, as is typical for Hebbian learning when there are only positive correlations, then

a. M1 constraints lead to a stabilized version of eo; b. S1 constraints lead to a zero-sum vector that grows to complete saturation, superimposed on the constrained background (w . n)n. The dominant zero-sum vector may be either

i. A zero-sum vector derived from eo,as in Figure lA,B; or ii. The principal zero-sum eigenvector of C.

2. If the principal eigenvector e0 of C is a zero-sum vector, as is typical for Hebbian learning when correlations among input activities oscillate in sign, then a type 1 constraint has little effect; the un'There is one exception: the nonzero-sum eigenvector of C with smallest eigenvalue is replaced by n, which is an eigenvector of PCP with eigenvalue 0.


113

constrained dynamics or M1 or S1 constraints all lead to saturated versions of eo. 3. M2 constraints always lead to a stabilized version of the principal eigenvector of C , unless the hypercube limiting synaptic weights interferes with the dynamics. 2.4 What Is Maximized under Multiplicative and Subtractive Constraints? Under multiplicative constraints, the weight vector tends to a multiple of the principal eigenvector e0 of C. This is the direction in weight space that maximizes WTCWover all directions W. This maximizes the mutual correlations among the weights; so most mutually correlated inputs are expected to retain representation. Under S1 constraints, for C symmetric, the dynamics maximize E = ~ w T C P W+ w,wTPCn (Section 2.3.3). For w,,sufficiently small, the first term dominates, so the weight vector is dominated by the principal eigenvector e! of PCP. This is the direction in weight space that maximizes W'CW over all zero-sum directions W. When n = (1.1... . . l)T, e; is a vector in which some subset of maximally correlated weights are set to positive values, and remaining weights are set to negative values. In the while negative final weight structure, positive weights in e: tend to wnlJX, weights tend to wmin.The receptive field thus becomes sharpened to a subset of maximally correlated inputs.

2.5 Application to Simple Hebbian Learning, Including Linsker's Simulations. The results just derived explain the outcome for simple Hebbian learning with a positive correlation function (Fig. lA,B). M1 constraints lead to convergence to the principal eigenvector e", which is all-positive. S1 constraints instead lead to growth of a zero-sum vector; in Figure lA,B, this vector is a center-surround vector derived from e". The results of Section 2.3.3 explain some of the results found by Linsker (1986). He explored Hebbian dynamics under S1 constraints with a gaussian correlation function, using w,in = -w,,, and a spatially gaussian distribution of inputs. Then, as analyzed in MacKay and Miller (1990a,b), the leading eigenvector of C is an all-positive eigenvector, and the zero-sum vector derived from this is a center-surround vector. The leading zero-sum eigenvectors of C,and leading eigenvectors of the constrained operator PCP, are two vectors that are bilobed, half positive and half negative. The all-positive eigenvector dominates the unconstrained the bilobed vectors dominate the development. For small values of w,,, constrained development. For larger values of w,,,the contribution of the w,PCn term to the growth of the center-surround vector allows the center-surround vector to dominate the constrained development within the hypercube, despite its having a smaller eigenvalue than the bilobed vectors under PCP.

114

Kenneth D. Miller a n d David J. C. MacKay

2.6 Extension to Two Input Layers. We now consider the case in which two equivalent input layers innervate a common output cell (Fig. 1C). For example, in the visual system, inputs serving the left eye and right eye each project to the visual cortex, in an initially completely overlapping manner (Miller et al. 1989). Similarly, ON-center and OFFcenter cells make initially equivalent projections to visual cortex (Miller 1992). Let wl, w2, respectively, be the synaptic weight vector from each input projection. Define the sum, ws = w1 + w2, and the difference, WD = w1 - w2. Because of the symmetry between the two input layers, the eigenvectors of the unconstrained equation can be divided into sum eigenvectors, ws = e,S, wD = 0, with eigenvalues As; and difference eigenvectors, wD = e,D, ws = 0, with eigenvalues XA (Miller 19904. Now an additional critical distinction emerges between subtractive and multiplicative constraints. A type 1 constraint conserves the total synaptic strength C, ws.Patterns of wD have zero total synaptic strength (i.e., are zero-sum vectors). Therefore, wD grows freely under an S1 constraint, whereas under a multiplicative constraint growth of wD is suppressed unless a difference eigenvector is the principal eigenvector of the unconstrained development. In models of ocular dominance segregation, the principal eigenvector is typically determined as follows (Miller 1990a). Let, , :A and Xi,, be the largest sum and difference eigenvalues, respectively. If there are positive correlations between the activities of the two eyes, then ,,,A: > .,,A: If there are no between-eye correlations, then ,,,A: = Xi,. If these correlations are negative, then A!ax > AS,., Thus, under a multiplicative constraint, wD cannot grow, and ocular dominance segregation cannot occur, unless the two eyes are negatively correlated7 (Miller et al. 1989). Such anticorrelations could be produced by intralaminar inhibition within the LGN. However, it seems unlikely that ocular dominance segregation depends on anticorrelations, since ocular dominance develops in the presence of vision in some animals, and vision should partially correlate the two eyes. > 0. The dynamics unUnder an Sl constraint, wD will grow if der an S1 constraint may be dominated either by egax,or by the zero-sum vector that derives from e:, depending on which has the faster growth rate (Section 2.3.3). In practice, ocular dominance segregation develops under an S1 constraint even if there are positive between-eye correlations of moderate size relative to the within-eye correlations (unpublished observations). Thus, subtractive rather than multiplicative enforcement of constraints appears more appropriate for modeling Hebbian development in visual cortex. 71f between-eye correlations are zero, then under multiplicative constraints the ratio of the principal eigenvector components, w ~ ~ ~ /does w ~not, change ~ ~ , under time development, while all other components are suppressed. Typically this ratio is initially small, so ocular dominance segregation does not occur.


115

3 Constraints Given a Full Layer of Output Cells When modeling Hebbian learning in a full layer of output cells, two differences arise compared to the case of an isolated cell. First, constraints may be applied to the total innervation onto each output cell (M1: Grajski and Merzenich 1990; von der Malsburg 1973; von der Malsburg and Willshaw 1976; Willshaw and von der Malsburg 1976; S1: Miller 1992); or to the total innervation from each input cell (M1: von der Malsburg 1979; Willshaw and von der Malsburg 1979); or to both (M1: Whitelaw and Cowan 1981; S1: Miller et a/. 1989). Second, there is usually coupling between the weight changes on different output cells. For example, neighboring cells’ activities may interact through intralaminar synaptic connections, causing the evolution of their weights to be coupled; or modulatory factors may diffuse, directly affecting neighboring synapses. Both types of coupling may take the mathematical form of an output layer ”lateral interaction function” (Miller 1990b; Miller et nl. 1989). Formulation of constraints in the case of a full layer is discussed in Miller and MacKay (1992, Appendix C). We have not studied constraints over a full layer in detail. However, the following heuristics, based on the single cell studies of Section 2, appear to be compatible with the studies cited in the previous paragraph and with unpublished observations. We refer to the projection to a single output cell as a receptive field or RF, and the projection from a single input location as a projective field or PF. The eigenvectors are patterns of weights across the entire network, not just across individual RFs or PFs. In simple Hebbian models, the dominant, fastest-growing patterns can often be characterized as follows. First, in the absence of constraints, the RFs of the dominant patterns are primarily determined by a particular input correlation function, and the PFs of the dominant patterns are similarly determined by the output layer lateral interaction function (Miller 1990a). If the correlations are all positive, the RFs have a single sign; if correlations oscillate in sign with input separation, the RFs oscillate in sign with a similar wavelength. A single-signed RF can be regarded as one that oscillates with an infinite wavelength, so we may summarize: in the absence of constraints, the RFs of the dominant patterns vary between positive and negative values with a wavelength corresponding to the peak of the Fourier transform of the appropriate input correlation function. Similarly, the PFs of the dominant patterns vary between positive and negative values with a wavelength corresponding to the peak of the Fourier transform of the output layer lateral interaction function. Second, constraints on output cells appear only to affect the form of the individual RFs, while constraints on input cells only affect the form of the individual PFs. Consider the case of two layers that are topographically connected: each input cell initially makes synapses onto cells over a certain diameter (”arbor diameter”) in the output layer, and

116


adjacent input cells project adjacent arbors. Then output cells also receive connections over an arbor diameter from the input layer. Suppose that output or input cell constraints conserve total synaptic strength over the cell. Then an RF or PF that alternates in sign with a wavelength less than or equal to the arbor diameter is approximately zero-sum, that is, it has summed synaptic strength near 0. An RF or PF that alternates with longer wavelength is nonzero-sum. Subtractive constraints selectively suppress the growth of nonzero-sum patterns, whereas multiplicative constraints stabilize the growth of a dominant nonzero-sum pattern. Thus, we arrive at the following heuristic rules for the wavelength with which RFs or PFs alternate in sign (Fig. 4): 1. If the dominant pattern in the absence of constraints has (RF,PF) wavelength larger than an arbor diameter, then

a. Subtractive (output,input) constraints suppress this pattern in favor of a pattern with (RF,PF) wavelength of an arbor diameter; b. Multiplicative (output,input) constraints d o not alter the (RF, PF) wavelength of this dominant pattern, but only stabilize its amplitude. 2. If the dominant pattern in the absence of constraints has (RF,PF) wavelength smaller than an arbor diameter, then (output,input) constraints, whether enforced multiplicatively or subtractively, will have little effect. In all cases, saturation of all synapses is expected without constraints or under subtractive constraints, but not under multiplicative constraints. Several cautions must be emphasized about this approach. First, it predicts only the characteristic wavelength of weight alternation, and does not distinguish between different weight structures with similar wavelength. Second, the approach is heuristic: its validity must be checked in any particular case. In particular, the final weight pattern is expected to be one in which the dominant PF and RF patterns are ”knitted together” into a compatible overall pattern. If such a “knitting” is not possible, the heuristics will fail. This analysis can be applied to understand the effects of subtractive input constraints on ocular dominance segregation. Consider the development of the difference W D between projections from the two eyes (Section 2.6). An RF across which wD is all-positive or all-negative corresponds to a monocular receptive field. Subtractive output constraints have no effect on the development of wD: such constraints affect only the sum, not the difference, of the two projections. When RFs are monocular, an oscillation across PFs of wDcorresponds to the oscillation between ocular dominance columns (Fig. 4). Subtractive input constraints separately conserve the total strength from the left-eye input and from the right-eye

117


CORRELATION FUNCTION

LATERAL INTERACTION FUNCTION

RECEPTIVE FIELD

PROJECTIVE FIELD

Figure 4: The role of constraints on input and output cells: a heuristic approach. Top: Output cell receptive fields (RFs)expected to develop under unconstrained dynamics (U),or under M1 or S1 constraints on output cells. White regions in receptive fields indicate positive weights; dark regions indicate zero or negative weights, depending on whether w,in is zero or negative. Correlations between input activities are shown as a function of input separation. Without constraints or under M1, the weights vary in sign to match the oscillation of the correlation function. Under S1, the weights always alternate in sign, with wavelength no larger than an arbor diameter. Note that this approach does not distinguish between different weight structures with similar wavelength of alternation, such as the two lower RFs. Bottom: Input cell projective fields (PFs) are determined in the same manner as RFs, except that (1) the determining function is the output layer lateral interaction function; and (2) the determining constraints are those a n input cells. Here, solid lines indicate positive weights, dashed lines indicate zero or negative weights.

118


input at each position, and so conserve the total difference wD from each input position. Thus, these constraints ensure that there is an oscillation of wDacross PFs with wavelength no larger than an arbor diameter. Subtractive input constraints thus determine the width of a left-eye plus a right-eye ocular dominance column to be an arbor diameter when the unconstrained dynamics would lead to larger columns, but have little effect on ocular dominance segregation otherwise (Miller ct nl. 1989). 4 How Can Constraints Be Implemented?

4.1 Learning Rules That Converge to Constrained Dynamics. The formulations in equations 2.3-2.5 confine the dynamics to the constraint surface that contains the initial weight vector. Alternatively, constraints may be formulated so that the dynamics converge from an arbitrary initial weight vector to one particular constraint surface, and remain o n that constraint surface thereafter. In this case the dynamics are described by equations 2.3-2.5 after an initial transient in which the constraint surface is reached. Such a formulation of S1 constraints is obtained by setting c(w) = Ik2)(n.w-kl) for constants kl and k2 in equation 2.2. When lkzl is large, this term enforces the constraint n . w = kl (Linsker 1986) and is equivalent to an S1 constraint (MacKay and Miller 1990a, Appendix E). Multiplicative constraints can be similarly formulated. Dynamics that converge to a multiplicative constraint can also be obtained by substituting a constant k > 0 for the denominator of ~ ( win) equations 2.3 or 2.5. Let c be the constraint vector (for M1, c = n; for M2, c = w) and set y(w) = c.Cw/k. Then, if the initial condition and dynamics maintain c . Cw > 0, the dynamics will flow to the constraint surface c . w = k and remain stable to perturbations off it thereafter [as can be seen by examining c . (d/dt)w]. Oja (1982) studied such M2 constraints with k = 1 and proved convergence to the principal eigenvector. Finally, if the principal eigenvalue A" of C is positive, convergent multiplicative dynamics can also be formulated by using any y ( w ) in equation 2.1 that grows with IwI and takes values both smaller and larger than A" (B. Pearlmutter, unpublished manuscript). This leads to convergence to a multiple of the principal eigenvector, ijeo, satisfying the constraint i(i1e") = A". An example is y(w) = lwI2 (Yuille et al. 1989).

4.2 Use of Thresholds to Achieve Constraints. Consider a linear Hebbian rule: (4.1)

Here y is the activation of the output cell, x the vector of input activities, and xo and yo are threshold activity levels for Hebbian plasticity. Assume


119

a linear activation rule, y = w . x. We average equation 4.1 over input patterns, assuming that XH and yH are constant over input patterns. The resulting equation is

n

-w(t)

nt

= QW

+ A [ (y) - yo] [ (x)

~

(4.2)

xi,]

where Q is the input covariance matrix, Q = X ( ( x - ( x ) ) (x - (x))'), and (y) = w . (x). The second term is a decay term, and can enforce a constraint. If the elements of [(x) xo] are large and negative, the type 1 constraint (y) = yo, that is, w . (x) = yo, will be enforced. If XH is independent of w, this constraint is enforced subtractively; if furthermore all inputs have the same mean activity level and threshold, this is an S1 constraint, as discussed in Section 4.1 and as used in Linsker (1986). The presynaptic threshold xg can also enforce a constraint if its elements increase with those of w. For example, an M2 constraint that converges to w . w = 1 (Section 4.1), when applied to the unconstrained equation (d/Lit)w= yx, yields the rule proposed by Oja (1982): (d/cft)w = yx - w ( w . yx), or (d/dt)w = y ( x - wy). This is XH = wy.' Both of these mechanisms require that inputs activated at an average or below-average level lose synaptic strength when the postsynaptic cell is highly activated. This is not the case in at least one biological system, LTP in hippocampus (Gustafsson ef al. 1987). This difficulty is avoided by the choice xH = w, which yields the multiplicatively stabilized rule (d/dt)w = y ( x - w ) . This rule does not ensure a selective outcome: as noted in Kohonen (1989, Section 4.3.2), it converges either to the principal eigenvector of C = (xx'), or else to w = 0. However, with a nonlinear, competitive rule for y that ensures localized activation, this rule is that of the self-organizing feature map of Kohonen (1989) and does achieve selectivity. The postsynaptic threshold, yo, can enforce a constraint if it increases faster than linearly with the average postsynaptic activation, (y), and if the elements of [(x) - xf,] are positive. Bienenstock e t a l . (1982) proposed the rule yH = (y)'/~,,~, where ySetis a cell's "preset" desired activity level. They combined this with a nonlinear Hebbian rule, for example, y(y - yH) in place of (y - yH) in equation 4.1. With either Hebbian rule, this "sliding threshold" has the effect of adjusting synaptic strengths to achieve (y) % yH,or (!y) = w (x) % yset.Thus, it provides another method of achieving a type 1 constraint. Recent results both in the peripheral auditory system (Yang and Faber 1991) and in hippocampus (Huang et 01. 1992)are suggestive that increased neural activity may elevate a threshold for modification, but in a manner specific to those inputs whose activity is increased. This is consistent with an increase in the elements of xo corresponding to activated inputs, but not with an increase in yo, which would elevate thresholds for all inputs. ~

"Note: equation 4.2 is not valid for this case, because

XH

varies with input patterns.

120


Covariance models (Sejnowski 1977a,b) have been proposed to solve the problem that chance coincidences drive synapses to saturate at 7umdX. This problem was known to occur in Hebb models of the form (d/dt)w ix yx. In a covariance model, (d/dt)w cx (y - (y))(x - (x)). This is a Hebb model with linearly sliding thresholds: y~ = (y), XH = (x). In this case, the decay term in equation 4.2 is zero, so synaptic growth is driven by the unconstrained equation (d/dt)w = Qw. Thus, the problem of loss of selectivity under a Hebbian rule is not avoided by a linear covariance rule. 4.3 Biological Implementation of Constraints. Rules that conserve synaptic strength have been criticized as nonlocal (e.g., Bienenstock ~t al. 1982). Thus, it is important to note that multiplicative or subtractive constraints in their general form (equations 2.1-2.2) can be implemented locally if each of a cell’s synaptic weights undergoes decay, either at a fixed rate (subtractive decay) or at a rate proportional to its strength (multiplicative decay); and if the overall gain of this decay, 7(w) or F(w), is set for the cell as a whole, and increases with the cell’s total synaptic strength. Such a cellular increase in decay, implemented locally at each synapse, might be achieved in at least two ways. First, a cell might have a limited capacity to metabolically supply its synapses, so that greater total synaptic strength means less supply and thus faster decay for each synapse. Second, the overall rate of decay might increase with a cell’s average degree of activation, which in turn would increase with the total synaptic strength received by a cell. Increased activation could increase release of a molecule that degrades synapses, such as a protease, or decrease release of a molecule that supports synapses, such as a trophic, adhesion, or sprouting factor (evidence for such mechanisms is reviewed in Van Essen et al. 1990). Increased activation might also increase decay due to thresholds for synaptic modification, as just discussed.

5 Discussion

We have demonstrated that multiplicative and subtractive constraints can lead to fundamentally different outcomes in linear learning. Under multiplicative constraints, the weight vector tends to the principal eigenvector of the unconstrained time development operator. This is a “graded” receptive field in which most mutually correlated inputs are represented. Thus, when two equally active eyes compete, both retain equal innervation unless the two eyes are anticorrelated. Under subtractive constraints, the weight vector tends to a receptive field that is “sharpened” to a subset of maximally correlated inputs: the weights of these inputs reach the maximum allowed strength, while all other weights reach the minimum allowed strength. When two eyes compete, subtractive constraints can lead to domination by one eye (ocular dominance segregation) provided


121

only that correlations within one eye are stronger than those between the eyes. The instability of subtractive constraints depends on the unconstrained operator having at least two positive eigenvalues, which is typical for Hebbian learning. An interesting alternative is anti-Hebbian learning (Mitchison 1991): in this case, all unconstrained eigenvalues are reversed in sign from the Hebbian case, so typically no eigenvalue is positive. Our analysis applies to this case also: multiplicatively constrained dynamics flow to the principal eigenvector, which is the vector that would have the smallest eigenvalue under Hebbian dynamics (Mitchison 1991); while subtractively constrained dynamics flow to the fixed point, which is stable. Multiplicative and subtractive constraints represent two fundamentally different methods of controlling the size of the weight vector. Multiplication equally rescales all weight patterns, while subtraction directly acts on only a single weight pattern. Because this difference is general, many of the results we have found for the linear case may generalize to cases involving nonlinear rules. Biologically, there is as yet little evidence as to the mechanisms that lead activity-dependent plasticity to be competitive or to achieve selectivity. Among the two choices of subtractive and multiplicative constraints, subtractive seem to resemble biology more closely in systems where sharp receptive fields are achieved, and in visual cortex where ocular dominance columns are likely to develop without requiring anticorrelation between the eyes; while multiplicative constraints might resemble biology more closely in situations like adult cortical plasticity where continually moving and graded representations may occur (Kaas 1991). We do not advocate that one or the other of these is the biologically correct choice. Rather, we wish (1) to point out that different choices of competitive mechanism can yield different outcomes, so it is important for the modelers to know whether and how their results depend on these choices; and (2) to begin to distinguish and characterize different classes of such mechanisms, which might then be compared to biology.

Appendix: Proofs of Mathematical Results We study dynamics confined to a constraint surface and governed by a general multiplicative constraint (equation 2.1) or by an S1 subtractive constraint (equations 2.4). As in the text, we use indices n. b.. . . to refer to the eigenvector basis of C. We assume that C is symmetric and thus has a complete set of orthonormal eigenvectors e" with corresponding eigenvalues A". Write the constrained equation as (d/dt)w = f(w). To determine the stability of a fixed point w"'[where f(w) = 01, we linearize f(w) about the fixed point. Call this linearized operator D; in the eigenvector basis of C,

122


it is a matrix with elements D,,!, = i)fil(w)/i)zu~~w,w,l~. For an S1 constraint, f ( w ) = PCw is linear (here, P = [l - fin']),so D = PC. We define the constraint plane to be the hyperplane tangent to the constraint surface at the fixed point, and the constraint vector c to be the vector normal to the constraint plane. c is a left eigenvector of D.' The stability of a fixed point is determined by the eigenvalues of D (Hirsch and Smale 1974). If one eigenvalue is positive, the fixed point is unstable; if all eigenvalues are negative, the fixed point is stable. In assessing the outcome of the constrained dynamics, we are concerned only with stability of the fixed point to perturbations within the constraint surface. Thus, if all eigenvalues are negative except one zero eigenvalue corresponding to a direction outside the constraint surface, then the fixed point is stable.

Theorem 1 Proof. We consider a multiplicatively constrained equation, (rl/dt)w = Cw - ? ( w ) w . We assume that multiplicative confinement of the dynamics to the constraint surface means two things. First, D has one zero or negative eigenvalue corresponding to the enforcement of the constraint, with associated left eigenvector c. Therefore any right eigenvector of D with positive eigenvalue is parallel to the constraint plane. Second, the constraint plane is not parallel to the subtracted vector w that enforces the constraint. A fixed point is an eigenvector of C: w"' = zu,,e''for some n, ?(w'I')= A''. The linearized operator is D = C - X " l - zuIle"[V-,(w"')]', where V is the gradient operator defined by Vx(x) = I,, e"[i),p(w)/i)zu,,]~,_,.I n the eigenvector basis of C, D is a diagonal matrix with the addition of one row of off-diagonal elements; such a matrix has the same eigenvalues as the diagonal matrix alone [because the characteristic equation, det(D - X1) = 0, is unchanged by the additional row]. The diagonal part of D is the matrix C - X"1- -he"[e']]', where -h = 7 ( ~ , I [ C ) ~ ( ~ ) / D ~ , I ] I w - n , , , e , , . This is the operator C with eigenvalue X" reduced to --h and all other eigenvalues reduced by A". Note that e" is the right eigenvector of D with eigenvalue - h ; e" is not parallel to the constraint plane, so -tl 5 0. Now we determine whether D has positive eigenvalues. If e" is not the principal eigenvector of C, then D has a positive eigenvalue and the fixed point is unstable. If e" is the principal eigenvector of C, and it is nondegenerate (no other eigenvector has the same eigenvalue), then all eigenvalues of D except perhaps a zero corresponding to e" are negative; so the fixed point is stable. If C has N degenerate principal eigenvectors, and e" is one of them, then D has N - 1 zeros corresponding to perturbations within the degenerate subspace: the principal eigenvector fixed points are thus marginally stable (eigenvalue 0) to perturbations within 0 this subspace, and stable to other perturbations.

"Proof. For any Aw in the constraint plane, D A w must remain within the constraint plane; that is, c'DAw = 0 for all Aw satisfying c r a w= 0. Therefore, c ' D x c'.


123

Lemma 1. Under an S1 constraint, (d/dt)w = PCw with P = (1- nn'), if there is a vector v parallel to the constraint plane such that v'PCV > 0, then PC has a positive eigenvalue. Proof. v is parallel to the constraint plane, so v . n = 0 and Pv = v. Thus, from v'PCv > 0 we conclude that vTPCPv> 0. Since PCP is symmetric, this implies that PCP has a positive eigenvalue; call this eigenvalue Xo, with corresponding eigenvector eo. e0 is parallel to the constraint plane, that is, PeO = e0 (because eo = PCPeo/XO,and P2 = P). So PCeO = PCPeO = Xoeo. 0 Theorem 2 proof. We consider the S1 constrained equation (d/dt)w = PCw. Let v be a linear combination of the two eigenvectors of C with positive eigenvalues, such that v is parallel to the constraint plane: v . n = 0. Then V'PCV = V'CV > 0. So, by Lemma 1, PC must have a positive 0 eigenvalue. Theorem 3 proof. This is a generalization of a similar proof in Linsker (1986). Suppose PCwF"= "0" when synapses wy and w,"' are both not saturated. By "0," we mean that each component of the vector is either 0, or else of a sign that would take an already saturated synapse beyond its limiting value. Let U' be the unit vector with ith weight 1 and all other elements 0, and similarly for U'. Consider stability of the fixed point to a perturbation confined to the u'/ul plane. The action of C in this plane is given by the submatrix

The eigenvalues of Cu,ujare both real and positive when the conditions of the theorem are met. Let el and e2 be the two orthonormal eigenvectors of Cu,ij,with all synaptic components other than i and j set to zero. As in the proof of Theorem 2, let v be a linear combination of el and e2 that is parallel to the constraint plane, v . n = 0. Then vTPCv= vTCv> 0. So, 0 by Lemma 1, the fixed point is unstable.

Acknowledgments K. D. M. thanks C. Koch, Caltech, and M. P. Stryker, UCSF, for supporting this work, which was performed in their laboratories. K. D. M. was supported by a Del Webb fellowship and a Markey Foundation internal grant, both from Caltech Division of Biology, and by an N.E.I. Fellowship at UCSF. D. J. C. M. was supported by a Caltech Fellowship and a Studentship from SERC, UK. We thank Bartlett Me1 and Terry Sejnowski for helpful comments on the manuscript. This collaboration would have been impossible without the internet/NSFnet.

124


References Bienenstock, E. L., Cooper, L. N., and Munro, P. W. 1982. Theory for the development of neuron selectivity: Orientation specificity and binocular interaction in visual cortex. J. Neurosci. 2, 32-48. Bourgeois, J.-P., Jastreboff, P. J., and Rakic, P. 1989. Synaptogenesis in visual cortex of normal and preterm monkeys: Evidence for intrinsic regulation of synaptic overproduction. Proc. Natl. Acad. Sci. U.S.A. 86, 4297-4301. Grajski, K. A., and Merzenich, M. M. 1990. Hebb-type dynamics is sufficient to account for the inverse magnification rule in cortical somatotopy. Neural Comp. 2, 71-84. Guillery, R. W. 1972. Binocular competition in the control of geniculate cell Comp. Neurol. 144, 117-130. growth. I. Gustafsson, B., Wigstrom, H., Abraham, W. C., and Huang, Y.-Y. 1987. Longterm potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. J. Neurosci. 7, 774-780. Hayes, W. P., and Meyer, R. L. 1988a. Optic synapse number but not density is constrained during regeneration onto surgically halved tectum in goldfish: HRP-EM evidence that optic fibers compete for fixed numbers of postsynaptic sites on the tectum. j . Comp. Neurol. 274, 539-559. Hayes, W. P., and Meyer, R. L. 1988b. Retinotopically inappropriate synapses of subnormal density formed by misdirected optic fibers in goldfish tectum. Dev.Brain Res. 38, 304-312. Hayes, W. P., and Meyer, R. L. 1989a. Impulse blockade by intraocular tetrodoxin during optic regeneration in goldfish: HRP-EM evidence that the formation of normal numbers of optic synapses and the elimination of exuberant optic fibers is activity independent. J. Neurosci. 9, 1414-1423. Hayes, W. P., and Meyer, R. L. 1989b. Normal numbers of retinotectal synapses during the activity-sensitive period of optic regeneration in goldfish: HRPEM evidence implicating synapse rearrangement and collateral elimination during map refinement. J. Neurosci. 9, 1400-1413. Hirsch, M. W., and Smale, S. 1974. Differential Equations, Dynarnical Systems and Linear Algebra. Academic Press, New York. Huang, Y.Y., Colino, A., Selig, D. K., and Malenka, R. C. 1992. The influence of prior synaptic activity on the induction of long-term potentiation. Science 255, 730-733. Kaas, J. H. 1991. Plasticity of sensory and motor maps in adult mammals. A n n u . Rev. Neurosci. 14, 137-167. Kohonen, T. 1989. Self-Organization and Associative Memory, 3rd ed. SpringerVerlag, Berlin. Linsker, R. 1986. From basic network principles to neural architecture: Emergence of spatial-opponent cells. Proc. Natl. Acad. Sci. U.S.A. 83, 7508-7512. MacKay, D. J. C., and Miller, K. D. 1990a. Analysis of Linsker’s applications of Hebbian rules to linear networks. Network 1, 257-298. MacKay, D. J. C., and Miller, K. D. 1990b. Analysis of Linsker’s simulation of Hebbian rules. Neural Comp. 2, 173-187.


125

Miller, K. D. 1990a. Correlation-based models of neural development. In Neuroscience and Connectionist Theory, M. A. Gluck and D. E. Rumelhart, eds., pp. 267-353. Erlbaum, Hillsdale, NJ. Miller, K. D. 1990b. Derivation of linear Hebbian equations from a nonlinear Hebbian model of synaptic plasticity. Neural Comp. 2, 321-333. Miller, K. D. 1992. Development of orientation columns via competition between ON- and OFF-center inputs. NeuroReport 3, 73-76. Miller, K. D., Keller, J. B., and Stryker, M. I? 1989. Ocular dominance column development: Analysis and simulation. Science 245, 605-615. Miller, K. D., and MacKay, D. J. C. 1992. The role of constraints in Hebbian learning. Tech. Rep. Memo 19, Program in Computation and Neural Systems, Caltech, Pasadena, CA. Mitchison, G. 1991. Removing time variation with the anti-Hebbian differential synapse. Neural Comp. 3, 312-320. Murray, M., Sharma, S., and Edwards, M. A. 1982. Target regulation of synaptic number in the compressed retinotectal projection of goldfish. J. Comp. Neurol. 209, 374-385. Oja, E. 1982. A simplified neuron model as a principal component analyzer. J. Math. Biol. 15, 267-273. Pallas, S. L., and Finlay, B. L. 1991. Compensation for population-size mismatches in the hamster retinotectal system: Alterations in the organization of retinal projections. Vis. Neurosci. 6, 271-281. Perez, R., Glass, L., and Shlaer, R. 1975. Development of specificity in the cat visual cortex. J. Math. Biol. 1, 275-288. Rochester, N., Holland, J. H., Haibt, L. H., and Duda, W. L. 1956. Tests on a cell assembly theory of the action of the brain, using a large digital computer. IRE Trans. Info. Theory IT-2, 80-93. Rosenblatt, F. 1961. Principles ofhreurodynamics: Perceptrons and the Theory of Brain Mechanisms. Spartan Books, Washington, D.C. Sejnowski, T. J. 1977a. Statistical constraints on synaptic plasticity. J. Theor. Bid. 69, 385-389. Sejnowski, T. J. 197%. Storing covariance with nonlinearly interacting neurons. J. Math. Biol. 4, 303-321. Van Essen, D. C., Gordon, H., Soha, J. M., and Fraser, S. E. 1990. Synaptic dynamics at the neuromuscular junction: Mechanisms and models. J. Neurobiol. 21, 223-249. von der Malsburg, C. 1973. Self-organization of orientation selective cells in the striate cortex. Kybevnetik 14, 85-100. von der Malsburg, C. 1979. Development of ocularity domains and growth behavior of axon terminals. B i d . Cyber. 32, 49-62. von der Malsburg, C., and Willshaw, D. J. 1976. A mechanism for producing continuous neural mappings: Ocularity dominance stripes and ordered retino-tectal projections. Exp. Brain Res. Suppl. 1, 463-469. Whitelaw, V. A., and Cowan, J. D. 1981. Specificity and plasticity of retinotectal connections: A computational model. 7. Neurosci. 1, 1369-1387. Wiesel, T. N., and Hubel, D. H. 1965. Comparison of the effects of unilateral

126


and bilateral eye closure on cortical unit responses in kittens. 1.Neurophysiol. 28, 1029-1040. Willshaw, D. J., and von der Malsburg, C. 1976. How patterned neural connections can be set up by self-organization. Proc. R. SOC.Londoii B. 194, 431-445. Willshaw, D. J., and von der Malsburg, C. 1979. A marker induction mechanism for the establishment of ordered neural mappings: Its application to the retinotectal problem. Phil. Trans. R. SOC.Londoti B. 287, 203-243. Yang, X. D., and Faber, D. S. 1991. Initial synaptic efficacy influences induction and expression of long-term changes in transmission. Proc. Natl. Acad. Sci. U.S.A. 88, 4299-4303. Yuille, A. L., Kammen, D. M., and Cohen, D. S. 1989. Quadrature and the development of orientation selective cortical cells by Hebb rules. Biol. Cybmiet. 61, 183-194. ~

Received October 9, 1992; accepted May 13, 1993.

This article has been cited by: 2. Matthieu Gilson, Anthony N. Burkitt, David B. Grayden, Doreen A. Thomas, J. Leo Hemmen. 2010. Emergence of network structure due to spike-timing-dependent plasticity in recurrent neuronal networks V: self-organization schemes and weight dependence. Biological Cybernetics . [CrossRef] 3. Sean Byrnes, Anthony N. Burkitt, David B. Grayden, Hamish Meffin. 2010. Spiking Neuron Model for Temporal Sequence RecognitionSpiking Neuron Model for Temporal Sequence Recognition. Neural Computation 22:1, 61-93. [Abstract] [Full Text] [PDF] [PDF Plus] 4. Brian S. Blais, Harel Z. Shouval. 2009. Effect of correlated lateral geniculate nucleus firing rates on predictions for monocular eye closure versus monocular retinal inactivation. Physical Review E 80:6. . [CrossRef] 5. Shigeru Tanaka, Masanobu Miyashita. 2009. Constraint on the Number of Synaptic Inputs to a Visual Cortical Neuron Controls Receptive Field FormationConstraint on the Number of Synaptic Inputs to a Visual Cortical Neuron Controls Receptive Field Formation. Neural Computation 21:9, 2554-2580. [Abstract] [Full Text] [PDF] [PDF Plus] 6. Max Garagnani, Thomas Wennekers, Friedemann Pulvermüller. 2009. Recruitment and Consolidation of Cell Assemblies for Words by Way of Hebbian Learning and Competition in a Multi-Layer Neural Network. Cognitive Computation 1:2, 160-176. [CrossRef] 7. Niranjan Chakravarthy, Kostas Tsakalis, Shivkumar Sabesan, Leon Iasemidis. 2009. Homeostasis of Brain Dynamics in Epilepsy: A Feedback Control Systems Perspective of Seizures. Annals of Biomedical Engineering 37:3, 565-585. [CrossRef] 8. Niranjan Chakravarthy, Shivkumar Sabesan, Kostas Tsakalis, Leon Iasemidis. 2009. Controlling epileptic seizures in a neural mass model. Journal of Combinatorial Optimization 17:1, 98-116. [CrossRef] 9. David Hsu, Aonan Tang, Murielle Hsu, John Beggs. 2007. Simple spontaneously active Hebbian learning model: Homeostasis of activity and connectivity, and consequences for learning and epileptogenesis. Physical Review E 76:4. . [CrossRef] 10. Taro Toyoizumi, Jean-Pascal Pfister, Kazuyuki Aihara, Wulfram Gerstner. 2007. Optimality Model of Unsupervised Spike-Timing-Dependent Plasticity: Synaptic Memory and Weight DistributionOptimality Model of Unsupervised Spike-Timing-Dependent Plasticity: Synaptic Memory and Weight Distribution. Neural Computation 19:3, 639-671. [Abstract] [PDF] [PDF Plus]

11. Tomokazu Ohshiro, Michael Weliky. 2006. Simple fall-off pattern of correlated neural activity in the developing lateral geniculate nucleus. Nature Neuroscience 9:12, 1541-1548. [CrossRef] 12. H. Meffin, J. Besson, A. Burkitt, D. Grayden. 2006. Learning the structure of correlated synaptic subgroups using stable and competitive spike-timing-dependent plasticity. Physical Review E 73:4. . [CrossRef] 13. Mauro Ursino, Giuseppe-Emiliano Cara. 2005. Dependence of Visual Cell Properties on Intracortical Synapses Among Hypercolumns: Analysis by a Computer Model. Journal of Computational Neuroscience 19:3, 291-310. [CrossRef] 14. Paul C. Bressloff. 2005. Spontaneous symmetry breaking in self–organizing neural fields. Biological Cybernetics 93:4, 256-274. [CrossRef] 15. Simone Fiori . 2005. Nonlinear Complex-Valued Extensions of Hebbian Learning: An EssayNonlinear Complex-Valued Extensions of Hebbian Learning: An Essay. Neural Computation 17:4, 779-838. [Abstract] [PDF] [PDF Plus] 16. Anthony N. Burkitt , Hamish Meffin , David. B. Grayden . 2004. Spike-Timing-Dependent Plasticity: The Relationship to Rate-Based Learning for Models with Weight Dynamics Determined by a Stable Fixed PointSpike-Timing-Dependent Plasticity: The Relationship to Rate-Based Learning for Models with Weight Dynamics Determined by a Stable Fixed Point. Neural Computation 16:5, 885-940. [Abstract] [PDF] [PDF Plus] 17. Terry Elliott . 2003. An Analysis of Synaptic Normalization in a General Class of Hebbian ModelsAn Analysis of Synaptic Normalization in a General Class of Hebbian Models. Neural Computation 15:4, 937-963. [Abstract] [PDF] [PDF Plus] 18. T. Elliott , N. R. Shadbolt . 2002. Multiplicative Synaptic Normalization and a Nonlinear Hebb Rule Underlie a Neurotrophic Model of Competitive Synaptic PlasticityMultiplicative Synaptic Normalization and a Nonlinear Hebb Rule Underlie a Neurotrophic Model of Competitive Synaptic Plasticity. Neural Computation 14:6, 1311-1322. [Abstract] [PDF] [PDF Plus] 19. Richard Kempter , Wulfram Gerstner , J. Leo van Hemmen . 2001. Intrinsic Stabilization of Output Rates by Spike-Based Hebbian LearningIntrinsic Stabilization of Output Rates by Spike-Based Hebbian Learning. Neural Computation 13:12, 2709-2741. [Abstract] [PDF] [PDF Plus] 20. Randall C. O'Reilly . 2001. Generalization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian LearningGeneralization in Interactive Networks: The Benefits of Inhibitory Competition and Hebbian Learning. Neural Computation 13:6, 1199-1241. [Abstract] [PDF] [PDF Plus] 21. Gal Chechik , Isaac Meilijson , Eytan Ruppin . 2001. Effective Neuronal Learning with Ineffective Hebbian Learning RulesEffective Neuronal Learning with Ineffective Hebbian Learning Rules. Neural Computation 13:4, 817-840. [Abstract] [PDF] [PDF Plus]

22. James A. Bednar , Risto Miikkulainen . 2000. Tilt Aftereffects in a Self-Organizing Model of the Primary Visual CortexTilt Aftereffects in a Self-Organizing Model of the Primary Visual Cortex. Neural Computation 12:7, 1721-1740. [Abstract] [PDF] [PDF Plus] 23. Gal Chechik , Isaac Meilijson , Eytan Ruppin . 1999. Neuronal Regulation: A Mechanism for Synaptic Pruning During Brain MaturationNeuronal Regulation: A Mechanism for Synaptic Pruning During Brain Maturation. Neural Computation 11:8, 2061-2080. [Abstract] [PDF] [PDF Plus] 24. Richard Kempter, Wulfram Gerstner, J. van Hemmen. 1999. Hebbian learning and spiking neurons. Physical Review E 59:4, 4498-4514. [CrossRef] 25. Laurenz Wiskott, Terrence Sejnowski. 1998. Constrained Optimization for Neural Map Formation: A Unifying Framework for Weight Growth and NormalizationConstrained Optimization for Neural Map Formation: A Unifying Framework for Weight Growth and Normalization. Neural Computation 10:3, 671-716. [Abstract] [PDF] [PDF Plus] 26. Kenneth D. Miller. 1998. Equivalence of a Sprouting-and-Retraction Model and Correlation-Based Plasticity Models of Neural DevelopmentEquivalence of a Sprouting-and-Retraction Model and Correlation-Based Plasticity Models of Neural Development. Neural Computation 10:3, 529-547. [Abstract] [PDF] [PDF Plus] 27. T. Elliott, C. I. Howarth, N. R. Shadbolt. 1998. Axonal Processes and Neural Plasticity: A ReplyAxonal Processes and Neural Plasticity: A Reply. Neural Computation 10:3, 549-554. [Abstract] [PDF] [PDF Plus] 28. Dean V. Buonomano, Michael M. Merzenich. 1998. CORTICAL PLASTICITY: From Synapses to Maps. Annual Review of Neuroscience 21:1, 149-186. [CrossRef] 29. Jianfeng Feng , David Brown . 1998. Fixed-Point Attractor Analysis for a Class of NeurodynamicsFixed-Point Attractor Analysis for a Class of Neurodynamics. Neural Computation 10:1, 189-213. [Abstract] [PDF] [PDF Plus] 30. Radford M. Neal , Peter Dayan . 1997. Factor Analysis Using Delta-Rule Wake-Sleep LearningFactor Analysis Using Delta-Rule Wake-Sleep Learning. Neural Computation 9:8, 1781-1803. [Abstract] [PDF] [PDF Plus] 31. Christian Piepenbrock, Helge Ritter, Klaus Obermayer. 1997. The Joint Development of Orientation and Ocular Dominance: Role of ConstraintsThe Joint Development of Orientation and Ocular Dominance: Role of Constraints. Neural Computation 9:5, 959-970. [Abstract] [PDF] [PDF Plus] 32. David Willshaw, John Hallam, Sarah Gingell, Soo Leng Lau. 1997. Marr's Theory of the Neocortex as a Self-Organizing Neural NetworkMarr's Theory of the Neocortex as a Self-Organizing Neural Network. Neural Computation 9:4, 911-936. [Abstract] [PDF] [PDF Plus] 33. Jianfeng Feng, Hong Pan, Vwani P. Roychowdhury. 1996. On Neurodynamics with Limiter Function and Linsker's Developmental ModelOn Neurodynamics

with Limiter Function and Linsker's Developmental Model. Neural Computation 8:5, 1003-1019. [Abstract] [PDF] [PDF Plus] 34. Harel Shouval, Leon N. Cooper. 1996. Organization of receptive fields in networks with Hebbian learning: the connection between synaptic and phenomenological models. Biological Cybernetics 74:5, 439-447. [CrossRef] 35. Christopher W. Lee , Bruno A. Olshausen . 1996. A Nonlinear Hebbian Network that Learns to Detect Disparity in Random-Dot StereogramsA Nonlinear Hebbian Network that Learns to Detect Disparity in Random-Dot Stereograms. Neural Computation 8:3, 545-566. [Abstract] [PDF] [PDF Plus] 36. Colin Fyfe . 1995. Introducing Asymmetry into Interneuron LearningIntroducing Asymmetry into Interneuron Learning. Neural Computation 7:6, 1191-1205. [Abstract] [PDF] [PDF Plus] 37. Barak A. Pearlmutter . 1995. Time-Skew Hebb Rule in a Nonisopotential NeuronTime-Skew Hebb Rule in a Nonisopotential Neuron. Neural Computation 7:4, 706-712. [Abstract] [PDF] [PDF Plus] 38. Marco Idiart, Barry Berk, L. F. Abbott. 1995. Reduced Representation by Neural Networks with Restricted Receptive FieldsReduced Representation by Neural Networks with Restricted Receptive Fields. Neural Computation 7:3, 507-517. [Abstract] [PDF] [PDF Plus] 39. Yong Liu . 1994. Influence Function Analysis of PCA and BCM LearningInfluence Function Analysis of PCA and BCM Learning. Neural Computation 6:6, 1276-1288. [Abstract] [PDF] [PDF Plus] 40. L. F. Abbott. 1994. Decoding neuronal firing and modelling neural networks. Quarterly Reviews of Biophysics 27:03, 291. [CrossRef] 41. Joseph Sirosh, Risto Miikkulainen. 1994. Cooperative self-organization of afferent and lateral connections in cortical maps. Biological Cybernetics 71:1, 65-78. [CrossRef] 42. Geoffrey J. Goodhill , Harry G. Barrow . 1994. The Role of Weight Normalization in Competitive LearningThe Role of Weight Normalization in Competitive Learning. Neural Computation 6:2, 255-269. [Abstract] [PDF] [PDF Plus]

Communicated by Harry Barrow and David Field

Toward a Theory of the Striate Cortex Zhaoping Li Joseph J. Atick The Rockefeller University, 1230 York Avenue, New York, NY 10021, USA

We explore the hypothesis that linear cortical neurons are concerned with building a particular type of representation of the visual worldone that not only preserves the information and the efficiency achieved by the retina, but in addition preserves spatial relationships in the input-both in the plane of vision and in the depth dimension. Focusing on the linear cortical cells, we classify all transforms having these properties. They are given by representations of the scaling and translation group and turn out to be labeled by rational numbers '(p + q)/p' ( p , q integers). Any given ( p . 4 ) predicts a set of receptive fields that comes at different spatial locations and scales (sizes) with a bandwidth of log,[@ q ) / p ] octaves and, most interestingly, with a diversity of 'q' cell varieties. The bandwidth affects the trade-off between presewation of planar and depth relations and, we think, should be selected to match structures in natural scenes. For bandwidths between 1 and 2 octaves, which are the ones we feel provide the best matching, we find for each scale a minimum of two distinct cell types that reside next to each other and in phase quadrature, that is, differ by 90" in the phases of their receptive fields, as are found in the cortex, they resemble the "even-symmetric'' and "odd-symmetric" simple cells in special cases. An interesting consequence of the representations presented here is that the pattern of activation in the cells in response to a translation or scaling of an object remains the same but merely shifts its locus from one group of cells to another. This work also provides a new understanding of color coding changes from the retina to the cortex.

+

1 Introduction

What is the purpose of the signal processing performed by neurons in the visual pathway? Are there first principles that predict the computations of these neurons? Recently there has been some progress in answering these questions for neurons in the early stages of the visual pathway. In Atick and Redlich (1990, 1992) a quantitative theory, based on the principle of redundancy reduction, was proposed. It hypothesizes that the main goal of retinal transformations is to eliminate redundancy in input signals, particularly that due to pairwise correlations among Neural Computation 6,127-146 (1994) @ 1993 Massachusetts Institute of Technology

128

Zhaoping Li and Joseph J. Atick

pixels-second-order statistics.' The predictions of the theory agree well with experimental data on processing of retinal ganglion cells (Atick and Redlich 1992; Atick et al. 1992). Given the successes of this theory, it is natural to ask whether redundancy reduction is a computational strategy continued into the striate cortex, One possibility is that cortical neurons are concerned with eliminating higher-order redundancy, which is due to higher-order statistics. We think this is unlikely. To see why, we recall the facts that make redundancy reduction compelling when applied to the retina and see that these facts are not as relevant for the cortex. First, the retina has a clear bottleneck problem: the amount of visual data falling on the retina per second is enormous, of the order of tens of megabytes, while the retinal output has to fit into an optic nerve of a dynamic range significantly smaller than that of the input. Thus, the retina must compress the signal, and it can do so without significant loss of information by reducing redundancy. In contrast, after the signal is past the optic nerve, there is no identifiable bottleneck that requires continued redundancy reduction beyond the retina. Second, even if there were pressure to reduce data,2 eliminating higherorder statistics does not help. The reason is that higher-order statistics do not contribute significantly to the entropy of images, and hence no significant compression can be achieved by eliminating them (for reviews of information theory see Shannon and Weaver 1949; Atick 1992). The dominant redundancy comes from pairwise correlation^.^ There is another intrinsic difference between higher- and second-order statistics that suggests their different treatment by the visual pathway. Figure 1 shows image A and another image B that was obtained by randomizing the phases of the Fourier coefficients of A. B thus has the same second-order statistics as A but no higher-order ones. Contrary to A, B has no clear forms or structures (cf. Field 1989). This suggests that for defining forms and for discriminating between images, secondorder statistics are useless, while higher-order ones are essential. Actually, eliminating the former highlights the higher-order statistics that should be used to extract form signals from " n ~ i s e . " ~ 'Since retinal neurons receive noisy signals it is necessary to formulate the redundancy reduction hypothesis carefully taking noise into account. In Atick and Redlich (1990, 1992) a generalized notion of redundancy was defined, whose minimization leads to elimination of pairwise correlations and to noise smoothing. 2For example, there could be a computational bottleneck such as an attentional bottleneck occuring deep into the cortex-perhaps in the link between V4 and IT (Van Essen et nl. 1991). 3This fact is well known in the television industry (see, eg., Schreiber 1956). This is why practical compression schemes for television signals never take into account more than pairwise correlations, and even then, typically nearest neighbor correlations. This fact was also verified for several scanned natural images in our laboratory by N. Redlich and by Z. Li. 4Extractingsignal from noise can achieve by far more significant data reduction than trying to eliminate higher-order correlations.

129

Theory of the Striate Cortex

A

B

Figure 1: (A, B) Demonstration of the uselessness of second-order statistics for form definition and discrimination. Following Field (1989), image B is constructed by first Fourier transforming A, randomizing the phases of the coefficients and then taking the inverse Fourier transform. The two images thus have the same second-order statistics but B has no higher-order ones. All relevant object features disappeared from B.

So what is the cortex then trying to do? Ultimately, of course, the cortex is concerned with object and pattern recognition. One promising direction could be to use statistical regularities of images to discover matched filters that lead to better representations for pattern recognition. Research in this direction is currently under way. However, there is another important problem that a perceptual system has to face before the recognition task. This is the problem of segmentation, or equivalently, the problem of grouping features according to a hypothesis of which objects they belong to. It is a complex problem, which may turn out not to be solvable independently from the recognition problem. However, since objects are usually localized in space, we think an essential ingredient for its successful solution is a representation of the visual world where spatial relationships, both in the plane of vision and in the depth dimension, are preserved as much as possible. In this paper we hypothesize that the purpose of early cortical processing is to produce a representation that (1)preserves information, (2) is free of second-order statistics, and (3) preserves spatial relationships. The first two objectives are fully achieved by the retina so we merely require that they be maintained by cortical neurons. We think the third objective is attempted in the retina (e.g., retinotopic and scale invariant sampling);


130

however, it is completed only in the cortex where more computational and organizational resources are available. Here, we focus on the cortical transforms performed by the relatively linear cells; the first two requirements immediately limit the class of transforms that linear cells can perform on the retinal signals to the class of unitary matrices? U with U . Ut = 1. So the principle for deriving cortical cell kernels reduces to finding the U that best preserves spatial relationships. Actually, preserving planar and depth relationships simultaneously requires a trade-off between the two (see Section 2). This implies that there is a family of Us, one for every possible trade-off. Each U is labeled by the bandwidth of the resulting cell filters and forms a representation of the scaling and translation group (see Section 3). We show that the requirement of unitarity limits the allowed choices of bandwidths, and for each choice predicts the needed cell diversity. The bandwidth that should ultimately be selected is the one that best matches structures in natural scenes. For bandwidths around 1.6 octaves, which are the ones we feel are most relevant for natural scenes, the predicted cell kernels and cell diversity resemble those observed in the cortex. The resulting cell kernels also possess an interesting object constancy property: when an object in the visual field is translated in the plane or perpendicular to the plane of vision, the pattern of activation it evokes in the cells remains intrinsically the same but shifts its locus from one group of cells to another, leaving the same total number of cells activated. The importance of such representations for pattern recognition has been stressed repeatedly by many people before and recently by Olshausen et al. (1992). Furthermore, this work provides a new understanding of color coding change from the single opponency in the retina to the double opponency in the cortex. 2 Manifesting Spatial Relationships

In this section we examine the family of decorrelating maps and see how they differ in the degree with which they preserve spatial relationships. We start with the input, represented by the activities of photoreceptors in the retina, {S(z,)} where x, labels the spatial location of the nth photoreceptor in a two-dimensional (2D) grid. For simplicity, we take the grid to be uniform. To focus on the relevant issues without the notational complexity of 2D, we first examine the one-dimensional (1D) problem and then generalize the analysis to 2D in Section 4. The autocorrelator of the signals { S ( x , , ) } is

JLrn ( S ( x n ) S ( x m ) )

(2.1)

51n this paper we use the term "unitary" instead of "orthogonal"since we find it more convenient to use complex basis [e.g.,eifx instead of coslfx)]. Ut = U*T,where the asterisk denotes complex conjugate. For real matrices, unitary means orthogonal.


131

where brackets denote ensemble average. To eliminate this particular redundancy, one has to decorrelate the output and then apply the appropriate gain control to fit the signals into a limited dynamic range. This can be achieved by a linear transformation

where j = 1,.. . .N and the kernel K/,, is the product of two matrices. Using boldface to denote matrices: K=V.M

(2.3)

MI,, is the rotation to the principal components of R: (M . R .M’), = A,6,, where {A,} are the eigenvalues of R. While V is the gain control which is a diagonal matrix with elements V,, = l/A.Thus the output has the

property (O,O,) = ( K . R .KT),,= h,,

(2.4)

An important fact to note is that redefining K by K’ = U . K where U is a unitary matrix (U . Ut = 1) does not alter the decorrelation property (equation 2.4). (Actually U should be an orthogonal matrix for real 0,,but since we will for convenience use complex variables, unitary U is appropriate.) Therefore, there is a whole family of equally efficient representations parameterized by {U}. Any member is denoted by KU KU

=

U . (V . M)

U . K(p)

(2.5)

where K(p) = V . M is the transformation to the principal components. Without compromising efficiency, this nonuniqueness allows one to look for a specific U that leads to KU with other desirable properties such as manifest spatial relationships.6 To see this, let us exhibit the transformation Kc!’) more explicitly. For natural signals, the autocorrelator is translationally invariant, in the sense that R,,,, = R ( n -rn). One can then define the autocorrelator by its Fourier transform or its power spectrum, which in 2D is R ( f ) l/lfl*,where f is the 2D spatial frequency (Field 1987; Ruderman a i d Bialek 1993). For illustration purposes, we take in this section the analogous 1D “scale

-

should be noted that this nonuniqueness in receptive field properties is due to the fact that the principle used is decorrelation. If one insists on minimization of pixel entropy (which for gaussian signals is equivalent to decorrelation) this symmetry formally does not exist for ensembles of nongaussian signals. In other words some choice of U may be selected over others. However, for the ensemble of 40 images that we have considered, we found that the pixel entropy varied only by few percent for different Us. This is consistent with the idea that natural scenes are dominated by second-order statistics that do not select any particular U. In other systems it is possible that higher order statistics do select a special U, see, for example, Hopfield (1991). For another point of view see Linsker (1992).


132

-

invariant” spectrum, namely, R ( f ) l/f. In the 2D analysis of Section 4 we use the measured spectrum l/lfl’. For a translationally invariant autocorrelator, the transformation to principal components is a Fourier transform. This means, the principal components of natural scenes or the row vectors of the matrix M are sine waves of different frequencies N

where

] = ( 0 , 1 , 2. . . . . N - 1 )

f,

= {

2.13 ,

if j i s odd

--%i,

if j is even

While the gain control matrix V is V , transform then becomes

=

l/m

=

l/m. The total

(2.7)

This performs a Fourier transform and at the same time normalizes the output such that the power is equalized among frequency components ( O f ) = const. (i.e., output is whitened). One undesirable feature of the transformation K(p) is that it does not preserve spatial relationships in the plane. As an object is translated in the field of view, the locus of response { 0 , )will not simply translate. Also two objects separated in the input do not activate two separate groups of cells in the output. Typically all cells respond to a mixture of features of all objects in the visual field. Segmentation is thus not easily achievable in this representation. Mathematically, we say that the output ( 0 ; )preserves planar spatial relationships in the input if O;[S]= Oi-rii[S’]

xr=,

when

S’(X,,) = S(X~~+-,,,)

(2.8)

where O;[S]= K,,,S(x,). In other words, a translation in the input merely shifts the output from one group of cells to another. Implicitly, preserving planar spatial relationship also requires, and we will therefore enforce, that the cell receptive fields be local, so a spatially localized object evokes activities only in a local cell group, which shifts its location when the object moves and is separated from another cell group evoked by another spatially disjoint object in the image plane. Technically speaking, an (0,)that satisfies equation 2.8 is said to form a representation of the discrete ”translation group.”


133

Insisting on equation 2.8 picks u p a unique choice of U. In fact in this case U is given by (2.9)

which is just the inverse Fourier transform. The resulting transformation K(’) = U.V.Ut gives translationally invariant center-surround cell kernels

In two dimensions, taking into account optical properties of the eye medium and the noise, these kernels were shown to account well for properties of retinal ganglion cells (Atick and Redlich 1992). Although the representation defined by K(’) is ideal for preserving spatial relationships in the plane, it completely destroys spatial relations in scale or depth dimension. The change in the patterns of activation in {Oi} in response to a change in the object distance is very complicated. To preserve depth relations the output should form a representation of another group the so-called “scaling group.” This is because when an object recedes or approaches, the image it projects goes from S(x) to S( Ax) for some scale factor A. The requirement of object invariance under scaling dictates that

for some shift I depending on A. It is not difficult to see that Kc’), which satisfies equation 2.8 all the way down to the smallest possible translation, violates this condition. Actually, satisfying equations 2.8 and 2.10 for the smallest possible translation and scale changes simultaneously is not possible. A compromise between them has to be found. The problem of finding the kernels that lead to { 0,) with the best compromise between equations 2.8 and 2.10 is equivalent to the mathematical problem of constructing simultaneous representations of the translation and scaling group, which is what we do next.

3 Representations of Translation and Scaling Group

To satisfy equations 2.8 and 2.10 the cells must carry two different labels. One is a spatial position label ”n” and the other is a scale label “a.” The idea is that under translations of the input the output translates over the ”n” index, while under scaling by some scale factor X the output shifts


134

over the “a” index. Such cell groups can be obtained from 0 = U . K(p) using a U that is block diagonal:

U=

Each submatrix Uahas dimension Na and gives rise to No cells with outputs Onlocated at lattice points x“, = ( N / N a ) nfor n = 1,2,. . . Nn. Since the block matrices Ua act on K(p), which are the Fourier modes of the inputs, the resulting cells in any given block a filter the inputs through a limited and exclusive frequency band with frequencies f, for C,,,,Na’ 5 j < CntSaNa’. Since Na < N these cells sample more sparsely on the original visual field. Notice the cells from different blocks a are spatially mingled with each other, and their total number add up to N = C, Na.The hope is to have translation invariance within each block and scale invariance between blocks, that is, %

ql[S]

=

ql+,,[S’] for S ( x ) = S ( x + b x ) and hx for S ( x ) = S(Ax)

q [ S ] = q,+’[S’]

=

(N/Na)bn

(3.1) (3.2)

Each block ‘a’ thus represents a particular scale, the translation invariance within that scale can be achieved with a resolution bx 0; N / N a , inversely proportional to Na.Larger blocks or larger Na thus give better translation invariance, and the single block matrix U = Uo = Mt achieves this symmetry to the highest possible resolution. On the other hand, a higher resolution in scaling invariance calls for a smaller X > 1. As we will see below, ( A - 1) 0; Na/lfa, where f” is the smallest frequency sampled by the ath block. Hence a better scaling invariance requires smaller block sizes Na. A trade-off between better translation and scaling invariance reduces to choosing the scaling factor A, or the bandwidth depending on it. This will become clearer as we now follow the detailed construction of U. The unitarity condition now requires having Un(Un)t= 1 for each a, resulting in output cells uncorrelated within each scale and between scales. To construct Ua, one notices that the requirement of translation invariance is equivalent to having identical receptive fields, except for a spatial shift of the centers, within each scale a. It forces U;, 0; e’fic.For a general X, it turns out that the constraint Ua(Ua)t= 1 for a > 0 cannot be satisfied if one insists on only one cell or receptive field type within the scale. However, if one allows the existence of several say, ’q’, cell types


135

within the scale, Ua(Ua)t= 1 is again possible. In this case, each cell is identical to (or is the off-cell type of) the one that is 9 lattice spaces away in the same scale lattice (i.e., + The most general choice for real receptive fields is then L~JU&~~"~J+@ i f f) i > 0

ut, =

{

(3.3) 1-~(lfrl+(t)"n+@) if f i < 0 fie

where 0 is an arbitrary phase that can be thought of as zero for simplicity at the moment, and

for two relatively prime integers p and 9. This means the number of cell types in any given scaling block will be 9. The frequencies sampled by this cell group are f i = f ( 2 n / N ) j for f' < j 5 ?+'. Including both the positive and the negative frequencies, the total number of frequencies sampled, and, since U" is a square matrix, the total number of cells in this scale, is No = 2(j"+' -?). The constraint of unitarity for u > 0 leads to the equation 2/*+'

c uo

/=2/"+1

"I

c N"

,U+l

(

~

0

nI

),* =

1

-e~~~7[(2+wrp"1

(3.5)

+ c,c, = h1111'

/=/*+1

whose solution is

The condition

=

(p/9)7r then leads to the nontrivial consequence (3.7)

In a discrete system, the only acceptable solutions are those where 9/2p is an integer. For example, the choice of 9 = 2 and p = 1 leads to the scaling ?+' = 3? + 1. This is the most interesting solution as discussed below. Mathematically speaking, in the continuum limit a large class of 00 and N co such that solutions exists, since in that limit one takes? f" = (2n/N)j" remains finite, then we are simply lead to f"" = f"(9 + p ) / p for any 9 and p . Thus representations of the scaling and translation group are possible for all rational scaling factors X = (9 + p ) / p . The bandwidth, B,,,, of the corresponding cells is log,[(9 + p ) / p ] . Interesting consequences follow from the relationship between cell bandwidth and diversity: -+

Cell types

=9

-+

136


+

For example, a bandwidth of one octave or a scaling factor ( q p ) / p = 2 needs only one cell type in each scale, when 9 = p = 1. If it turns out to be necessary to have B,,, greater than 1 octave, then at least two classes of cells are needed to faithfully represent information in each scale, with q = 2 and p = 1 giving scaling factor of 3 or B,, close to 1.6 octaves. It is interesting to compare our solutions to the so called "wavelets" that, constructed in the mathematical literature, also form representations of the translation and scaling group. In the standard construction of Grossman and Morlet (1984) and Meyer (19851, the representations could be made orthonormal (i.e., unitary in the case of real matrices) only for limited choice of scaling factors given by 1 + l / m where m 2 1 is an integer. Such constructions need only one filter type in each scale and give scale factors no larger than 2 [equivalently the largest bandwidth is 1 octave--e.g., the well-known Haar basis wavelets (Daubechies 1988)l. This agrees with what we derived above for the special case of q = 1 where B,,, = log,(l + l / p ) . However, allowing q > 1 gives more bandwidth choices in our construction. For example, q = 2 gives Boct = log, (1 + 2/p), however, no larger than 1.6 octaves, and q = 3 gives log, (1 + 3/p), no larger than 2 octaves, etc. These results also agree with the recent theorem of Auscher (1992) who proved that multiscale representations can exist for scalings by any rational number k / l , provided k - I filter types are allowed in each scale. Our conclusion above yields exactly the same result by redefining k = p + q and I = q. We arrived at our conclusion independently through the explicit construction presented above.7 The connection between the number of cell types and the bandwidth that is possible to achieve is significant. We believe the bandwidth needed by cortical cells is determined by properties of natural images. Its value should be the best compromise between planar and depth resolution preservation for the distribution of structures in natural scenes. Actually, Field (1987, 1989) examined the issue of best bandwidth for filters that modeled cortical cells and found that bandwidths between 1 and 2 octaves best matched natural scene structures. Our results here show that cortical cells cannot achieve bandwidths more than one octave without having more than one cell type. Next we show what the predicted cell kernels look like. For generality, we give the expression for the kernels in the continuum limit for any scale factor X = (9 + p ) / p or equivalently with any allowed bandwidthalthough the ones we think are most relevant to the cortex are the discrete p = 1, q = 2 kernels. The cell kernels are given by {KO(,$ - x ) . u > 0 ) and { K " ( x ; - x)}. For any given a > 0, the kernels sample the frequency in the range f E cf",Xf") = cf".fR+'). For u = 0, K" samples only frequencies f E ( O , f ' ) , and Uo is given by U" = Mt in equation 2.9 with N replaced by

'We thank Ingrid Daubechies for pointing out the result of P. Auscher to us


137

Receptive fields

Sensitivity

A

B

C

D

Spatial distance

Spatial Frequency

Figure 2: "Even-symmetric" (A, C) and "odd-symmetric" (B, D) kernels predicted for the scale factor 3 (equivalently for BOct= 1.6 octaves) for two neighboring scales (top and bottom rows, respectively), together with their spectra (frequency sensitivities or selectivities).

N". Including both the positive and negative frequencies the predicted kernels are

-

-

x)

-

-m 9

+H

]

(3.8) (3.9)

For any given p and q the kernels for a > 0 come in q varieties. Even and odd varieties are immediately apparent when one sets q = 2, p = 1, and 0 = 0 [K"(q, - x) are even or odd functions of - x for even or odd n ] . In Figure 2 we exhibit the even and odd kernels in two adjacent scales and their spectra. The a = 0 kernels, where 0 = 0 is chosen, are similar to the center-surround retinal ganglion cells (however, they are larger in size), and hence we need not exhibit them here. In general, though, H can take any value, and the neighboring cells will simply differ by a 90" phase shift, or in quadrature, without necessarily having even or odd symmetry in their receptive field shapes.


138

From equation 3.8, it is easy to show that the kernels for a > 0 satisfy the following recursive relations:

P ( f ,- AX) -

(x

=

+ 4)] =

1

-K"'(f,+' - X) x K"(x& - x)

(3.10) (3.11)

To prove these one needs to use the following facts, fa+' = Xfa, N"+'= AN",and A$+' = 4:' = $. (Equation 3.11 also applies for KO.> The above relations imply that, except shifted in space, each cell has the same receptive field as its 9th neighbor within the same scale block, for example, when 9 = 2 in the example above, all the even (or odd) cells are identical. Furthermore, except for the lowest scale a = 0, the nth cell in all scales has the same receptive field except for a factor of X expansion in size and a X reduction in amplitude. Actually, since # .I$+',these cells are located at different spatial locations. Now it is straightforward to see that the translation invariance (equation 3.1) for 6n = 9 and scale invariance (equation 3.2) are the direct consequence of the translation and scaling relationships, 3.11 and 3.10, respectively, between the receptive fields. This is exactly our goal of object constancy. Notice that the scaling constancy would not have been possible if the whitening factor was not there in equation 3.8. These results can be extended to 2D where the whitening factor is l/T - = If1as we will see next.

e3

4

4 Extension to 2D and Color Vision: Oriented Filters and Color Oppo-

nent Cells

The extension to two dimensions of the above construction is not difficult but involves a new subtlety. In this case, the constraint of unitarity on the matrices U".a > 0 is hard to satisfy even if we allow for the phase factor q4 that leads ultimately to different classes of cells. This constraint is considered in more detail in the Appendix; here we only state the conclusions of that analysis. What one finds is that to ensure unitarity of U", one needs to allow for cell diversity of a different kind-cells in the ath scale need to be further broken down into different types or orientations, each sampling from a limited region of the frequency space in that scale. Three examples of acceptable unitary breakings are shown in Figure 3A, B, C. In A (B) filters are broken into two classes in any scale a > 0-in addition to the 9-cell diversity discussed in 1D. One filter type is a lowpass-bandpass in the x-y direction, and the other is a bandpass-lowpass in the x-y direction, which are denoted by " l b and "bl." In C there are three classes of filters, "lb," "bl," and finally a class of filters which are bandpass in both x and y, "bb." The " l b and "bl" filters are oriented while the " b b ones


139

I"+'

I"

I'

I'

.I

/"+I

B

I'

c

Figure 3: ( A X ) Proliferation of more cell types by the break-down of the frequency sampling region in 2D within a given scale a. Ignoring the negative frequencies, the frequencies f within the scale are inside the large solid box but outside the small dashed box. The solid lines within the large solid box further partition the sampling into subregions denoted by "bl," "lb," and "bb," which indicate bandpass-lowpass, lowpass-bandpass, and bandpass-bandpass, respectively, in x-y directions. (A, B) Asymmetric breakdown between x and y directions, the "lb cells are not equivalent to a 90" rotation of the "bl" cells. (C)Symmetric breakdown between x and y directions. The "bb" cells are significantly different from the others (see Fig. 4).

are not.8 Figure 4A and B shows the five cell types one encounters for the breaking in Figure 3B and the nine cell types for the breaking in Figure 3C, respectively, for a choice of scaling factor 3. Finally, the object constancy equations 3.1 and 3.2 still hold since equations 3.10 and 3.11 extend to 2D as

where x and g7,are 2D vectors, a n d zz and q are 2D indices. These relatronships are understood to hold between cells belonging to the same frequency sampling category ("lb," "bl," or " b b ) . The factor of 1 / A 2 comes because the whitening factor in 2D is l/m = IfI. 80ne notices that this extension to 2D requires a choice of orientations such as the x-y axes, breaking the rotational symmetry. Furthermore, it is natural to ask if the object constancy by translations and scalings should be extended to the object rotations in the image plane-requiring the cells be representationsof the rotation group. At this point, it is not clear whether the rotational invariance is necessary (noting that we usually tilt our heads to read a tilted book or fail to recognize a face upside down), and whether the rotational invariance can be incorporated simultanously with the translation and scaling ones without increasing the number of cells. We will leave this outside the paper.


140

A

B

Figure 4: (A, B) The predicted variety of cell receptive fields in 2D. The five cell types in (A) and the nine cell types in (B) arise from the frequency partitioning schemes in Figure 3B and C, respectively. The kernels in the lower-left corner of both images demonstrate the lowpass-lowpass filter K" in 2D and they are nonoriented. All others are bandpass in at least one direction. Those are actually significantly smaller but are expanded in size in this figure for demonstration. The " b b cells in the upper-right part of (B) come in four varieties (even-even, odd-odd, even-odd, and odd-even when H = 0 is taken for both x and y directions) and should exist in the cortex if the scheme in Figure 3C is favored. All kernels are constructed taking into account the optical MTF of the eye.

" :1

From equations 3.8 and 3.9, it is clear that the cortical kernels K"(x) 0; df[l/Jx(f)] coscfx + 4") differ from the retinal kernel

K(x) 0: ~ f m " x d f [ l / J R Vcoscfx ) ] + 9) only by the range of the frequency integration or selectivity. The cortical receptive fields are lowpass or bandpass versions of the retinal ones. One immediate consequence of this is that most cortical cells, especially the lowpass ones like those in the cytochrome oxidase blob cells, have larger receptive fields than the retinal ones. Second, when considering color vision, the power spectrums R,(f) and R,cf) for the luminance and chrominance channels, respectively, differ in their magnitudes. In reality when noises are considered, the receptive field filters are not simply which would have simply resulted in identical receptive field forms for luminance and chrominance except for their different strengths, but instead, the filter for luminance is more of a bandpass and the filter

l/m,


141

for chrominance a relatively lowpass. Since the retinal cells carry luminance and chrominance information simultanously by multiplexing the signals from both channels, the resulting retinal cells are of red-centergreen-surround (or green-center-red-surround) types (Atick et al. 1992). This is because at low spatial frequencies, the chrominance filter dominates, while at higher spatial frequencies, the luminance one dominates. As we argued above, the cortical cells simply lowpass or bandpass the signals from the retinal cells; thus the lowpass version will carry mostly the chrominance signals while the bandpass or highpass ones the luminance signals. This is indeed observed in the cortex (Livingstone and Hubel 1984; Ts'o and Gilbert 1988) where the large (lowpass) blob cells are more color selective, while the smaller (higher-pass) nonblob cells, which are also more orientation selective by our results above, are less color sensitive. Furthermore, since the luminance signals are negligible at low frequencies, when one only considers the linear cell properties, the color sensitive blob cells are double-opponent (e.g. red-excitatorygreen-inhibitory center and the red-inhibitory-green-excitatory surround) or color-opponent-center-only (type II), depending on the noise levels. This is apparent when one tries to spatially lowpass the signals from a group of single-opponent retinal cells (Fig. 5).

5 Discussion: Comparison with Other Work

The types of cells that we arrive at in constructing unitary representations of the translation and scaling group (see Figs. 2 and 4) are similar to simple cells in cat and monkey striate cortex. The analysis also predicts an interesting relationship between bandwidths of cells and their diversity as was discussed in Sections 3 and 4. One consequence of that relationship is that for cells to achieve a representation of the world with sampling bandwidth between 1 and 2 octaves there must be at least two cell types adjacent to each other and differ by 90" in their receptive field phases (Fig. 2). This bandwidth range is the range of measured bandwidths of simple cells (e.g., Kulikowski and Bishop 1981; Andrews and Pollen 1979) and also, we think, is best suited for matching structures in natural scenes (cf. Field 1987, 1989). This analysis thus explains the presence of phase quadrature (e.g., paired even-odd simple cells) observed in the cortex (Pollen and Ronner 1981): such cell diversity is needed to build a faithful multiscale representation of the visual world. The analysis also requires breaking orientation symmetry. Here we do not wish to advocate scaling symmetry as an explanation for the existence of oriented cells in the cortex. It may be that orientation symmetry is broken for a more fundamental reason and that scaling symmetry takes advantage of that. Either way, orientation symmetry breaking is an important ingredient in building these multiscale representations.


142

In the past, there has been a sizeable body of work on trying to model simple cells in terms of “Gabor” and “log Gabor” filters (Kulikowski et al. 1982; Daugman 1985, Field 1987, 1989). Such filters are qualitatively close to those derived here, and they describe some of the properties of simple cells well. Our work differs from previous work in many ways. The two most important differences are the following. First, the filters here are derived by unitary transforms on retinal filters that reduce

Luminance (Solid) & Chrominance (Dashed)

100.

1

Spatial Frequency (ddeg)

Ganglion red

Ganglion green

I

Blob green

I

I

I

Spatial Distance


143

redundancy in inputs by whitening. By selecting the unitary transformation that manifests spatial-scale relationships in signals, one arrives at a representation that exhibits object constancy-the output response to an input S(x) and its planar and depth translated version [i.e., S(x) + S[X(x + bx)]] are related by

Hence, a visual object moved in space simply shifts the outputs from one group of cells to another. Second, we find a direct linkage between cell bandwidth and diversity. Such linkage does not appear in previous works where orthonormality or unitarity was not required. More recently there has also been a lot of work on orthonormal multiscale representations of the scaling and the translation group, alternatively known as wavelets (Meyer 1985; Daubechies 1988; Mallat 1989). The relationship of our work to wavelets was discussed in Section 3. Here we should add that in this paper we provide explicit construction of these representations for any rational scaling factor. Furthermore, our filters satisfy K a ( X x ) = (l/Xd)Kaf'(x) where d is the dimension of the space (e.g., d = 1 or 2), while those in the wavelet construction satisfy Ka(Xx) = ( l / X d / 2 ) K a " ( x ) . This difference stems from the fact that our filters are the convolution of the whitening filter and the standardtype wavelet. The whitening filter-given by where R ( f ) is the scale-invariant power spectrum of natural scenes-is what ultimately leads to the object constancy property that is absent from the standardtype wavelets. The question at this stage is whether we could identify the pieces in our mathematical construction with classes of cells in the cortex. First, there is the class of lowpass cells u = 0, which have large receptive fields, and no orientation tuning (actually since their kernels have a whitening factor, they are not completely lowpass but an incomplete bandpassweak surround). We think a good candidate for these cells are the cells in the cytochrome oxidase blob areas in the cortex. When we add color

- l/m

Figure 5: Facing page, Change of color coding from retina to cortex. The top plot shows the visual contrast sensitivities to the luminance and chrominance signals. The bottom plot demonstrates the receptive field profiles (sensitivityto red or green cone inputs) of the color selective cells in the retina (or ganglion) and the cortex. The parameters used for the ganglion cells are the same as those in Atick et al. (1992). The blob cells are constructed by low ass filtering P the ganglion cell outputs with a filter frequency sensitivity of e-f ' ( 2 f i w ) where how = 1.5 c/deg. The strengths of the cell profiles are individually normalized for both the ganglion and the blob cells. The range of the spatial distance axes, or the size, of the blob cells is 3.7 times larger than that of ganglion cells. This means that each blob cell sums the outputs from (on the order of) at least about (3.7)* 16 local ganglion cells. N


144

to our analysis, this class will come out to be color opponent.’ These cells, a lowpass version of the single opponent retinal cells, turn out to be double opponent or color-opponent-center-only (see Fig. 5) from this mathematical construction, in agreement with observations. Second, the representation requires several orientation classes in every choice of higher scale; they are not as likely to be color selective and, within each orientation and scale, there are two types of cells-in phase quadrature (e.g., even and odd symmetric)-if the bandwidth of the cells is greater than one octave. These have kernels similar to simple cells. Also, in some choices of division of the two-dimensional frequency space into bands (see Fig. 41, one encounters cells that are very different from simple cells. These cells come from the bandpass region in both the x and y directions (the “bb” region in Fig. 3C) and as such possess relatively small receptive fields in space. It is amusing to note their resemblance to the type of cells that Van Essen discovered in V4 (private communication). It is important at this stage to look in detail for evidence that cortical neurons are building a multiscale, translationally invariant representation of the input along the lines described in this paper. However, in looking for those we must allow for the possibility that these representations are formed in an active process starting as early as the striate cortex, as was proposed recently by Olshausen et al. (1992). We also must keep in mind that to perform detailed comparison with real cortical filters, our filters have to be modified to take noise into account.

Appendix In this appendix we examine the condition of unitarity on the matrix Ufl. The matrix elements of Ua in the scale a > 0 are generalized from the 1D case simply as (taking H = 0)

n

j = Vx,jl/), f = and 4 = -I [(f,x/lh,l)4x,(f,,/lfi,I)4y]. A priori the ceils in Uasample from the frequency region inside the big solid box but outside the dashed box in Figure 3. The critical fact that makes the 2D case different from 1D is that there are = 4(j”+’)2 - 4W)’ cells in the ath class, while the total number then of cells is (N)’, where

= (nx,iiy), $ =

($lr,$lJ,

’It is easy to see why: since they are roughly lowpass-large receptive fields-they have higher signal-to-noise in space, and hence they can afford to have a low signalto-noise in color. While opponent cells in space have low signal-to-noise,they need to integrate in color to improve their signal-to-noise (see Atick et a / . 1992).


145

The unitarity requirement U"(Un)t = 1 (a > 0) can be shown to be equivalent to

where An, is any integer # 0. A similar condition in the y direction should also hold. To satisfy equation A.2 one can only hope that the cosine factor is zero for odd An, and the sine factor is zero for the rest. This is impossible in 2D although possible in 1D. To see this difference, note that in lD, N" = 2(j"+' -?) and the argument of the sine is An7r/2, which leads to vanishing sine for even An. One then makes cosine term zero for odd An by choosing 4 such that [(j"" +? ~ ) / N ' ] K + &= f r / 2 . This is exactly how equation 3.6 is reached. In 2D, N" = 2 d m , and hence the sine term is

+

sin[An,( dP+l- F ) / p + I +ja)7r/2] # 0 for even An,. Although we cannot prove that the negative result in 2D is not caused by the fact that we have a Euclidean grid, we think it not possible to construct the representation even when using a radially symmetric lattice. To ensure unitarity of U",we need to allow for cell diversity of a different kind-cells in ath scale need to be further broken down into different types or orientations, each type sampling from a limited region of the frequency space as shown, for example, in Figure 3.

Acknowledgments We would like to thank D. Field, C. Gilbert, and N. Redlich for useful discussions, and the Seaver Institute for its support.

References Andrews, B. W., and Pollen, D. A. 1979. Relationship between spatial frequency selectivity and receptive field profile of simple cells. 1. Physiol. (London) 287, 163-1 76. Atick, J. J. 1992. Could information theory provide an ecological theory of sensory processing? Network 3, 213-251. Atick, J. J., and Redlich, A. N. 1990. Towards a theory of early visual processing. Neural Comp. 2,308-320. Atick, J. J., and Redlich, A. N. 1992. What does the retina know about natural scenes? Neural Comp. 4, 196-210. Atick, J. J., Li, Z., and Redlich, A. N. 1992. Understanding retinal color coding from first principles. Neural Comp. 4, 559-572. Auscher, P. 1992. Wavelet bases for L 2 ( R ) with rational dilation factor. In Wavelets and their applications, M. B. Ruskai, ed., pp. 439-451. Jones and Bartlett, Boston.


146

Daubechies, I. 1988. Orthonormal bases of compactly supported waves. Commun. Pure Appl. Math. 41, 909-996. Daugman, J. G. 1985. Uncertainty relations for resolution in space, spatial frequency and orientation optimized by two-dimensional visual cortical filters. J. Opt. SOC.A m . A 2, 1160-1169. Field, D. J. 1987. Relations between the statistics of natural images and the response properties of cortical cells. I. Opt. SOC.A m . A 4, 2379-2394. Field, D. J. 1989. What the statistics of natural images tell us about visual coding. SPIE Vol. 1077, Human Vision, Visual Processing, and Digital Display, 269276. Grossmann, A., and Morlet, J. 1984. Decomposition of hardy functions into square integrable wavelets of constant shape. SlAM J. Math. 15, 17-34. Hopfield, J. J. 1991. Olfactory computation and object perception. Proc. Natl. Acad. Sci. U.S.A. 88, 6462-6466. Kulikowski, J. J., and Bishop, P. 1981. Linear analysis of the responses of simple cells in the cat visual cortex. Exp. Brain Res. 44, 386400. Kulikowski, J. J., Marcelja, S., and Bishop, P. 1982. Theory of spatial position and spatial frequency relations in the receptive fields of simple cells in the visual cortex. Biol. Cybern. 43, 187-198. Linsker, R. 1992. Private communication. See also, talk at NIPS 92. Livingstone M. S., and Hubel, D. H. 1984. Anatomy and physiology of a color system in the primate visual cortex. J. Neurosci. 4(1), 309-356. Mallat, S. 1989. A theory of multiresolution signal decomposition: The wavelet representation. I E E E Transact. Pattern Anal. Machine Intelligence 11, 674-693. Meyer, Y. 1985. Principe d’incertitude, bases hilbertiennes et algebres d’operateurs Sem. Bourbaki 662, 209-223. Olshausen, B., Anderson, C. H., and Van Essen, D. C. 1992. A neural model of visual attention and invariant pattern recognition. Caltech Report no. CNS MEMO 18, August. Pollen, D. A., and Ronner, S. F. 1981. Phase relationships between adjacent simple cells in the cat. Science 212, 1409-1411. Ruderman, D. L., and Bialek, W. 1993. Statistics of natural images: scaling in the woods. Private communication and to appear. Schreiber, W. F. 1956. The measurement of third order probability distributions of television signals. IRE Trans. Inform. Theory IT-2, 94-105. Shannon, C . E., and Weaver, W. 1949. The Mathematical Theory of Communication. University of Illinois Press, Urbana, IL. Ts’o, D. Y., and Gilbert, C. D. 1988. The organization of chromatic and spatial interactions in the primate striate cortex. I. Neurosci. 8(5), 1712-1727. Van Essen, D. C., Olshausen B., Anderson, C. H., and Gallant, J. L. 1991. Pattern recognition, attention, and information bottlenecks in the primate visual system. Conf. on Visual Information Processing: From Neurons to Chips ( S H E Proc. 2473). ~~

Received October 23, 1992; accepted April 20, 1993.

This article has been cited by: 2. Sheng Li, Si Wu. 2007. Robustness of neural codes and its implication on natural image processing. Cognitive Neurodynamics 1:3, 261-272. [CrossRef] 3. Odelia Schwartz, Anne Hsu, Peter Dayan. 2007. Space and time in visual context. Nature Reviews Neuroscience 8:7, 522-535. [CrossRef] 4. Yoshitatsu Matsuda, Kazunori Yamaguchi. 2007. Linear Multilayer ICA Generating Hierarchical Edge DetectorsLinear Multilayer ICA Generating Hierarchical Edge Detectors. Neural Computation 19:1, 218-230. [Abstract] [PDF] [PDF Plus] 5. Jeffrey Ng, Anil A. Bharath, Li Zhaoping. 2007. A Survey of Architecture and Function of the Primary Visual Cortex (V1). EURASIP Journal on Advances in Signal Processing 2007, 1-18. [CrossRef] 6. Odelia Schwartz, Terrence J. Sejnowski, Peter Dayan. 2006. Soft Mixer Assignment in a Hierarchical Generative Model of Natural Scene StatisticsSoft Mixer Assignment in a Hierarchical Generative Model of Natural Scene Statistics. Neural Computation 18:11, 2680-2718. [Abstract] [PDF] [PDF Plus] 7. Zhiyong Zhang, Willy Wriggers. 2006. Local feature analysis: A statistical theory for reproducible essential dynamics of large macromolecules. Proteins: Structure, Function, and Bioinformatics 64:2, 391-403. [CrossRef] 8. Thomas Wennekers , Nihat Ay . 2005. Finite State Automata Resulting from Temporal Information Maximization and a Temporal Learning RuleFinite State Automata Resulting from Temporal Information Maximization and a Temporal Learning Rule. Neural Computation 17:10, 2258-2290. [Abstract] [PDF] [PDF Plus] 9. Alexandre Pouget, Peter Dayan, Richard S. Zemel. 2003. INFERENCE AND COMPUTATION WITH POPULATION CODES. Annual Review of Neuroscience 26:1, 381-410. [CrossRef] 10. Yury Petrov, L. Zhaoping. 2003. Local correlations, information redundancy, and sufficient pixel depth in natural images. Journal of the Optical Society of America A 20:1, 56. [CrossRef] 11. Chris J. S. Webber . 2001. Predictions of the Spontaneous Symmetry-Breaking Theory for Visual Code Completeness and Spatial Scaling in Single-Cell Learning RulesPredictions of the Spontaneous Symmetry-Breaking Theory for Visual Code Completeness and Spatial Scaling in Single-Cell Learning Rules. Neural Computation 13:5, 1023-1043. [Abstract] [PDF] [PDF Plus] 12. Antonio Turiel, Néstor Parga, Daniel Ruderman, Thomas Cronin. 2000. Multiscaling and information content of natural color images. Physical Review E 62:1, 1138-1148. [CrossRef] 13. Alexander Dimitrov , Jack D. Cowan . 1998. Spatial Decorrelation in Orientation-Selective Cortical CellsSpatial Decorrelation in Orientation-Selective Cortical Cells. Neural Computation 10:7, 1779-1795. [Abstract] [PDF] [PDF Plus]

14. Tai Sing Lee. 1996. Image representation using 2D Gabor wavelets. IEEE Transactions on Pattern Analysis and Machine Intelligence 18:10, 959-971. [CrossRef] 15. Daniel Ruderman. 1994. The statistics of natural images. Network: Computation in Neural Systems 5:4, 517-548. [CrossRef]

Communicated by Yann Le Cun

Fast Exact Multiplication by the Hessian Barak A. Pearlmutter Siemens Corporate Research, 755 College Road East, Princeton, NJ 08540 USA

Just storing the Hessian H (the matrix of second derivatives a2E/aw,aw, of the error E with respect to each pair of weights) of a large neural network is difficult. Since a common use of a large matrix like H is to compute its product with various vectors, we derive a technique that directly calculates Hv, where v is an arbitrary vector. To calculate Hv, we first define a differential operator Rycf(w)} = (a/&)f ( w + W ) J ~ =note ~, that %{Vw} = Hv and %{w} = v, and then apply R{.} to the equations used to compute 0,. The result is an exact and numerically stable procedure for computing Hv, which takes about as much computation, and is about as local, as a gradient evaluation. We then apply the technique to a one pass gradient calculation algorithm (backpropagation), a relaxation gradient calculation algorithm (recurrent backpropagation), and two stochastic gradient calculation algorithms (Boltzmann machines and weight perturbation). Finally, we show that this technique can be used at the heart of many iterative techniques for computing various properties of H, obviating any need to calculate the full Hessian. 1 Introduction Efficiently extracting second-order information from large neural networks is an important problem, because properties of the Hessian appear frequently. For instance, in the analysis of the convergence of learning algorithms (Widrow et al. 1979; Le Cun et al. 1991; Pearlmutter 1992), in some techniques for predicting generalization rates in neural networks (MacKay 1991; Moody 19921, in techniques for enhancing generalization by weight elimination (Le Cun et al. 1990; Hassibi and Stork 19931, and in full second-urder optimization methods (Watrous 1987). There exist algorithms for calculating the full Hessian H (the matrix of second derivative terms a2E/awJw, of the error E with respect to the weights w1 of a backpropagation network (Bishop 1992; Buntine and Weigend 19941, or reasonable estimates thereof (MacKay 1991)-but even storing the full Hessian is impractical for large networks. There is also an algorithm for efficiently computing just the diagonal of the Hessian (Becker and Le Cun 1989; Le Cun et al. 1990). This is useful when the trace of the Hessian is needed, or when the diagonal approximation is being made-but there is no reason to believe that the diagonal approxNeural Computation 6, 147-160 (1994) @ 1993 Massachusetts Institute of Technology

148

Barak A. Pearlmutter

imation is good in general, and it is reasonable to suppose that, as the system grows, the diagonal elements of the Hessian become less and less dominant. Further, the inverse of the diagonal approximation of the Hessian is known to be a poor approximation to the diagonal of the inverse Hessian. Here we derive an efficient technique for calculating the product of an arbitrary vector v with the Hessian H. This allows information to be extracted from the Hessian without ever calculating or storing the Hessian itself. A common use for an estimate of the Hessian is to take its product with various vectors. This takes O(n2)time when there are n weights. The technique we derive here finds this product in O ( n )time and space,' and does not make any approximations. We first operate in a very general framework, to develop the basic technique. We then apply it to a series of more and more complicated systems, starting with a typical noniterative gradient calculation algorithm, in particular a backpropagation network, and proceeding to a deterministic relaxation system, and then to some stochastic systems, in particular a Boltzmann machine and a weight perturbation system. 2 The Relation between the Gradient and the Hessian

The basic technique is to note that the Hessian matrix appears in the expansion of the gradient about a point in weight space,

VW(w+Aw)= V w ( ~ ) + H A ~ + O ( l l A ~ 1 1 2 ) where w is a point in weight space, Aw is a perturbation of w, Owis the gradient, the vector of partial derivatives dE/dw;, and H is the Hessian, the matrix of second derivatives of E with respect to each pair of elements of w. This equation has been used to analyze the convergence properties of some variants of gradient descent (Widrow et al. 1979; Le Cun et al. 1991; Pearlmutter 1992), and to approximate the effect of deleting a weight from the network (Le Cun et al. 1990; Hassibi and Stork 1993). Here we instead use it by choosing A w = w , where v is a vector and r is a small number. We wish to compute Hv. Now we note that

+

H(w) = ~ H = v VW(w+ W ) - V ~ ( W ) O ( f ) or, dividing by r, (2.1) This equation provides a simple approximation algorithm for finding Hv for any system whose gradient can be efficiently computed, in time about 'Or O(pn) time when, as is typical for supervised neural networks, the full gradient is the sum of p gradients, each for one single exemplar.

Multiplication by the Hessian

149

that required to compute the gradient (assuming that the gradient at w has already been computed). Also, applying the technique requires minimal programming effort. This approximation was used to good effect in Le Cun et al. (1993) and in many numerical analysis optimization routines, which use it to gradually build up an approximation to the inverse Hessian. Unfortunately, this formula is susceptible to numeric and roundoff problems. The constant r must be small enough that the O(r) term is insignificant. But as r becomes small, large numbers are added to tiny ones in w + rv, causing a loss of precision of v. A similar loss of precision occurs in the subtraction of the original gradient from the perturbed one, because two nearly identical vectors are being subtracted to obtain the tiny difference between them. 3 The R{.}Technique

Fortunately, there is a way to make an algorithm which exactly computes Hv, rather than just approximating it, and simultaneously rid ourselves of these numeric difficulties. To do this, we first take the limit of equation (2.1) as r + 0. The left-hand side stays Hv, while the right-hand side matches the definition of a derivative, and thus

Hv = lim r-0

+

VW(W rv) - V r

W ( 4

=

8 -Vw(w+rv) dr

(3.1)

As we shall see, there is a simple transformation to convert an algorithm that computes the gradient of the system into one that computes this new quantity. The key to this transformation is to define the operator (3.2) so Hv = &{Vw(w)). (To avoid clutter we will usually write R{.} instead of %{.}.) We can then take all the equations of a procedure that calculates a gradient (e.g., the backpropagation procedure), and we can apply the %{.) operator to each equation. Because R{.}is a differential operator, it obeys the usual rules for differential operators, such as


150

Also note that

R{w}= v

(3.4)

These rules are sufficient to derive, from the equations normally used to compute the gradient, a new set of equations about a new set of Rvariables. These new equations make use of variables from the original gradient calculation on their right-hand sides. This can be thought of as an adjoint system to the gradient calculation, just as the gradient calculation of backpropagation can be thought of as an adjoint system to the forward calculation of the error measure. This new adjoint system computes the vector R{Vw}, which is precisely the vector Hv that we desire. 4 Application of the

R{.}Technique to Various Networks

Let us utilize this new technique for transforming the equations that compute the gradient into equations that compute Hv, the product of a vector v with the Hessian H. We will, rather mechanically, derive appropriate algorithms for some standard sorts of neural networks that typify three broad classes of gradient calculation algorithms. These examples are intended to be illustrative, as the technique applies equally well to most other gradient calculation procedures. Usually the error E is the sum of the errors for many patterns, E = & E,. Therefore Ow and H are sums over all the patterns, H = C, Hp, and Hv = C, H,v. As is usual, for clarity this outer sum over patterns is not shown except where necessary, and the gradient and Hv procedures are shown for only a single exemplar.

4.1 Simple Backpropagation Networks. Let us apply the above procedure to a simple backpropagation network, to derive the R{backprop} algorithm, a set of equations that can be used to efficiently calculate Hv for a backpropagation network. In press, I found that the R{backprop} algorithm was independently discovered a number of times. Werbos (1988, eq. 14) derived it as a backpropagation process to calculate Hv = V,(v . V,E), where V,E is also calculated by backpropagation. That derivation is dual to the one given here, in that the direction of the equations is reversed, the backwards pass of the V,E algorithm becoming a forward pass in the Hv algorithm, while here the direction of the equations is unchanged. Another derivation is given in Mdler (1993a). Also, the procedure is known to the automatic differentiation community (Christianson, 1992; Kim et al. 1985). For convenience, we will now change our notation for indexing the weights w. Let w be the weights, now doubly indexed by their source


151

and destination units’ indices, as in w,], the weight from unit i to unit j. Because v is of the same dimension as w, its elements will be similarly indexed. All sums over indices are limited to weights that exist in the network topology. As is usual, quantities that occur on the left sides of the equations are treated computationally as variables, and calculated in topological order, which is assumed to exist because the weights, regarded as a connection matrix, is zero-diagonal and can be put into triangular form (Werbos 1974). The forward computation of the network is2 (4.1)

y;

= c;(x;)

+ I;

where of(.)is the nonlinearity of the ith unit, x, is the total input to the ith unit, y, is the output of the ith unit, and I, is the external input (from outside the network) to the ith unit. Let the error measure be E = E(y), and its simple direct derivative with respect to y, be e, = dE/dy,. We assume that e, depends only on y,, and not on any yl for j # i. This is true of most common error measures, such as squared error or cross entropy (Hinton 1987).3 We can thus write el(yl)as a simple function. The backward pass is then (4.2)

Applying

R{.} to the above equations gives

2This compact form of the backpropagation equations, due to Fernando Pineda, unifies the special cases of input units, hidden units, and output units. In the case of a unit i with no incoming weights (i.e., an input unit), it simplifies to yI = nI(0)+ I,, allowing the value to be set entirely externally. For a hidden unit or output i, the term I, = 0. In the corresponding equations for the backward pass (4.2) only the output units have nonzero direct error terms el, and since such output units have no outgoing weights, the situation for an output unit i simplifies to i?E/i3yl = e,(y,). 31f this assumption is violated then in equation 4.4 the ei(y,)R{y,} term generalizes to (% /OY,)R{Y,

c,

1.


152

for the forward pass, and, for the backward pass, (4.4)

The vector whose elements are R{8 E / 3 w , } is just R{Ow}= Hv, the quantity we wish to compute. For sum squared error ei(y;)= y; - d ; where di is the desired output for unit i, so ei(yl) = 1. This simplifies (4.4) for simple output units to R{DE/8y;} = R{y,}.Note that, in the above equations, the topology of the neural network sometimes results in some R-variables being guaranteed to be zero when v is sparse-in particular when v = ( 0 . . . O 1 0 . . 0), which can be used to compute a single desired column of the Hessian. In this situation, some of the computation is also shared between various columns.

-

4.2 Recurrent Backpropagation Networks. The recurrent backpropagation algorithm (Almeida 1987; Pineda 1987)consists of a set of forward equations which relax to a solution for the gradient,

xi

=

xwjiyj

(4.5)

I

dt

0:

-yl

+ .i(Xi) + Ii

Adjoint equations for the calculation of Hv are obtained by applying the R{.} operator, yielding

R{xi)

1

C (wjlR{yi)+ vjiyj) j

dn{yi) dt

c(

-R{Yi)

+ a;(xl)R{xi}

(4.6)


153

These equations specify a relaxation process for computing Hv. Just as the relaxation equations for computing 0, are linear even though those for computing y and E are not, these new relaxation equations are linear. 4.3 Stochastic Boltzmann Machines. One might ask whether this technique can be used to derive a Hessian multiplication algorithm for a classic Boltzmann machine (Ackley et al. 1985), which is discrete and stochastic, unlike its continuous and deterministic cousin to which application of R{.} is simple. A classic Boltzmann machine operates stochastically, with its binary unit states s, taking on random values according to the probability

P(s, = 1) xi

= =

pl

c

a(x,/T)

=

(4.7)

W/IS\

/

At equilibrium, the probability of a state visible units) is related to its energy

Q

of all the units (not just the (4.8)

i<j

by P(a) = Z-' exp -E,/T, where the partition function is Z = C,,exp E,/T. The system's equilibrium statistics are sampled because, at equilibrium, (4.9) where pi, = (sis,), G is the asymmetric divergence, an information theoretic measure of the difference between the environmental distribution over the output units and that of the network, as used in Ackley et al. (1985), T is the temperature, and the + and - superscripts indicate the environmental distribution, + for waking and - for hallucinating. Applying the R{.}operator, we obtain

{},:

R

- =

(R{P$}- R(P,}) / T

(4.10)

We shall soon find it useful if we define

D,

=

R{E,}

=

CS:'S;U~,

(4.11)

I
1/2 and p < 11, two different behaviors appear depending on whether p is even or odd. If p is odd, after (p - 1)/2 steps the network relaxes to the fixed points p

3

=

p = 5 p = 7 p = 9

m, = m,, = m,= m,=

1/2(1,1,1) 1/8 (5,3,1,1,3) 1/32 (19,13,3,1,13,13) 1/128 (77,51,13.3.1,1,3,13,51)

They can be represented by 1 m T = __ 2P-2

m,

= 4qt1

(2.2)

+

(-I),+(?) 2p-2

,

r/

= 2. . . . * ,P

2

ml = m 2 + 2 m 3

mg+1

m, ml

(2.3)

3

(2.5)

= 0 = =

-

mg+1t,

1

(2.4)

y = I , ' . ' >P- + l mqt7 mt+(7-1) 2 If p is even, two-cycle solutions given by the formulas -

-

-

m2

+ 2m3

mg+'-?

,

y = I , ...,'

2

(2.6)

and - 1 -

m;+1 m,

=

ml

=

m2

+ 2m3

-

Wf+ltr -

(2.7)

are present (they appear because the dynamics is parallel). When p 2 10, the fixed points are reached after four iterations. The whole set of attractors is obtained by cyclic rotations of mL=

1 -

27

(77,51,13,3,1,0, . . . ,0.1,3,13,51)

(2.8)

and they are universal, that is, the nonzero components do not depend either on p or on a. Surprisingly, the overlap vector has exactly 9 components different from zero symmetrically disposed around the component associated with the stimulating pattern. These attractors are mutually correlated and the correlations depend only on the separation of the corresponding stimulating patterns in the

L. F. Cugliandolo

222

memorized sequence, d = N - B ( a > 13) and Cd = Cp-d. If p 2 n ( n is the number of nonzero components of m,, i.e., n = 9), C1 Cz C3

C4 Cs

= = = = =

Cp-l Cp-2 Cp-3 Cp-4 Cp-s

=

= = = =

1360/211 680/2"

N

252/Z1'

N

82/2" 23/211

N

2:

N

0.66 0.33 0.12 0.04 0.01

+2 +d

(2.9)

while more distant attractors are not significantly correlated. They are independent of p and a, if a E (1/2,1). Furthermore, if p 2 22 and 10 < d < p - 10 attractors are not correlated at all: Cd = 0. 3 Network of 0,l Neurons

The dynamics of the spike emission is

+

1 J,s,(t)

s,(t h t ) = 0

-

H

1

(3.1)

with I,, as in Griniasty et al. (1993) and P[r/,CL= 11 = f . The critical a, below which pure pattern states are stable, is (3.2)

The dependence of attractors on a and p , once the probability f and the threshold 0 are chosen, becomes more complicated. If a > a, and p is small enough, the overlap vectors corresponding to the new fixed points have all components different from zero. Increasing p (a fixed), a "critical" p,, is reached after which it has a relatively small number of components different from zero concentrated around the starting pattern and the rest of them exactly zero. p , increases in steps with a and inside each subinterval there is universality. The values of a dividing subintervals depend on f and H. Some examples are in order. The f l neural activity model is recovered taking f = 1/2 and H = 0. Then, acr = 1/2 and the whole interval (1/2,1) presents universality. p , = 10, n = 9, and attractors and correlations are given by equations 2.8-2.9. If, for instance, f = 1/10, a = 1/10, and 6' = 0: a > acr. The fixed points are

p = 3 p 2 4 Thus, p,,

=4

m,!L = (1- f ) 2 (1.1,1) rn; = (1-f)2(1.1.0. . . . , o , ~ )

and n

= 3.

Correlated Attractors

223

Increasing a so as to move to another subinterval, the fixed points change. Taking a = 3/10 they are

p = 3, p = 4, p = 5. p 2 6,

m; m' m! m:,

= (1- f ) 2 (1,1,1) = (1 - f ) 3 (1.1.1.1) = =

(1 - f ) 3 (1+ f , +~ f , f , f , ~ +j) (1- f ) 3 (1 + f , 1 + f , f . O . . . . , O . f , l + f )

pCr = 6 and n = 5. In the first example, if p 2 n

C1 C2 c3= ...

= C/,-l = = Cp-2 = = cp-3 =

+ 2 + d = 5 + d the correlations are

171/271 81/271 0

N N

0.63 0.30

while in the second example if p 2 n + 2 + d

C1 C2 C3 C4 c 5 = ...

= Cl,-l = C1'-2 = Cp-3 =

C1,-4

= q - 5

= = = = =

206119/306119 106119/306119 16119/306119 279/306119 0

=7 N N N

N

+ d, they are 0.673 0.347 0.053 0.002

In both examples the correlations are independent of p . Their values decay with increasing separation and they are exactly zero after a small distance. In general, if p > 2n + 3 and n + 1 < d < p - (n + 1) the correlations are independent of d and exactly zero. The attractors and correlations are similar to those given by the examples in the whole range of values of parameters such that inequality 3.2 holds. In conclusion, it has been shown that the model of Griniasty et al. (19931, both for f l and 1.0 neurons, has attractors with a finite similarity index (the components of the overlap) with a small number of nearest stimuli in the learning sequence (5 if a > 1/2 for the fl and a function of a and 8, at fixed f , varying in steps for the 1.0 neurons, respectively).

Acknowledgments We wish to thank D. J. Amit for stimulating and helpful discussions and J. Kurchan, E. Sivan, M. Tsodyks, and M. A. Virasoro for useful discussions.

References Griniasty, M., Tsodyks, M. V., and Amit, D. J. 1993. Conversion of temporal correlations between stimuli to spatial correlations between attractors. Neural Comp. 5, 1-17.

L. F. Cugliandolo

224

Miyashita, Y. 1988. Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature (London) 335, 817. Wolfram, S. 1990. Mathematica. Computer Program, Wolfram Research Inc. Cugliandolo, L. F., and Tsodyks, M. V. 1993. Capacity of networks with correlated attractors. I. Pkys. A, in press. ~

~

~

Received January 15, 1993; accepted July 27, 1993.

This article has been cited by: 1. F. Metz, W. Theumann. 2005. Pattern reconstruction and sequence processing in feed-forward layered neural networks near saturation. Physical Review E 72:2. . [CrossRef] 2. Masahiko Yoshioka, Masatoshi Shiino. 2000. Associative memory storing an extensive number of patterns based on a network of oscillators with distributed natural frequencies in the presence of external white noise. Physical Review E 61:5, 4732-4744. [CrossRef] 3. Masahiko Yoshioka, Masatoshi Shiino. 1998. Associative memory based on synchronized firing of spiking neurons with time-delayed interactions. Physical Review E 58:3, 3628-3639. [CrossRef] 4. Friedemann Pulvermüller, Hubert Preissl. 1995. Local or transcortical assemblies? Some evidence from cognitive neuroscience. Behavioral and Brain Sciences 18:04, 640. [CrossRef] 5. Morris W. Hirsch. 1995. Mathematics of Hebbian attractors. Behavioral and Brain Sciences 18:04, 633. [CrossRef] 6. J. J. Wright. 1995. How do local reverberations achieve global integration?. Behavioral and Brain Sciences 18:04, 644. [CrossRef] 7. Shimon Edelman. 1995. How representation works is more important than what representations are. Behavioral and Brain Sciences 18:04, 630. [CrossRef] 8. Elie Bienenstock, Stuart Geman. 1995. Where the adventure is. Behavioral and Brain Sciences 18:04, 627. [CrossRef] 9. Anders Lansner, Erik Fransén. 1995. Distributed cell assemblies and detailed cell models. Behavioral and Brain Sciences 18:04, 637. [CrossRef] 10. Ralph E. Hoffman. 1995. Additional tests of Amit's attractor neural networks. Behavioral and Brain Sciences 18:04, 634. [CrossRef] 11. David C. Krakauer, Alasdair I. Houston. 1995. An evolutionary perspective on Hebb's reverberatory representations. Behavioral and Brain Sciences 18:04, 636. [CrossRef] 12. Jean Petitot. 1995. The problems of cognitive dynamical models. Behavioral and Brain Sciences 18:04, 640. [CrossRef] 13. Frank der van Velde. 1995. Association and computation with cell assemblies. Behavioral and Brain Sciences 18:04, 643. [CrossRef] 14. Peter M. Milner. 1995. Attractors – don't get sucked in. Behavioral and Brain Sciences 18:04, 638. [CrossRef] 15. G. J. Dalenoort, P. H. de Vries. 1995. What's in a cell assembly?. Behavioral and Brain Sciences 18:04, 629. [CrossRef] 16. Joaquin M. Fuster. 1995. Not the module does memory make – but the network. Behavioral and Brain Sciences 18:04, 631. [CrossRef]

17. Daniel J. Amit. 1995. Empirical and theoretical active memory: The proper context. Behavioral and Brain Sciences 18:04, 645. [CrossRef] 18. Masahiko Morita. 1995. Another ANN model for the Miyashita experiments. Behavioral and Brain Sciences 18:04, 639. [CrossRef] 19. Walter J. Freeman. 1995. The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences 18:04, 631. [CrossRef] 20. Eric Chown. 1995. Reverberation reconsidered: On the path to cognitive theory. Behavioral and Brain Sciences 18:04, 628. [CrossRef] 21. Maartje E. J. Raijmakers, Peter C. M. Molenaar. 1995. How to decide whether a neural representation is a cognitive concept?. Behavioral and Brain Sciences 18:04, 641. [CrossRef] 22. Wolfgang Klimesch. 1995. The functional meaning of reverberations for sensoric and contextual encoding. Behavioral and Brain Sciences 18:04, 636. [CrossRef] 23. Michael Hucka, Mark Weaver, Stephen Kaplan. 1995. Hebb's accomplishments misunderstood. Behavioral and Brain Sciences 18:04, 635. [CrossRef] 24. Josef P. Rauschecker. 1995. Reverberations of Hebbian thinking. Behavioral and Brain Sciences 18:04, 642. [CrossRef] 25. Ehud Ahissar. 1995. Are single-cell data sufficient for testing neural network models?. Behavioral and Brain Sciences 18:04, 626. [CrossRef] 26. Daniel J. Amit. 1995. The Hebbian paradigm reintegrated: Local reverberations as internal representations. Behavioral and Brain Sciences 18:04, 617. [CrossRef] 27. L F Cugliandolo, M V Tsodyks. 1994. Journal of Physics A: Mathematical and General 27:3, 741-756. [CrossRef]

Communicated by Laurence Abbott

Learning of Phase Lags in Coupled Neural Oscillators Bard Ermentrout Department of Mathematics, University of Pittsburgh, Pittsburgh, PA 15260 USA Nancy Kopell Department of Mathematics, Boston University, Boston, M A 02215 U S A If an oscillating neural circuit is forced by another such circuit via a composite signal, the phase lag induced by the forcing can be changed by changing the relative strengths of components of the coupling. We consider such circuits, with the forced and forcing oscillators receiving signals with some given phase lag. We show how such signals can be transformed into an algorithm that yields connection strengths needed to produce that lag. The algorithm reduces the problem of producing a given phase lag to one of producing a kind of synchrony with a "teaching" signal; the algorithm can be interpreted as maximizing the correlation between voltages of a cell and the teaching signal. We apply these ideas to regulation of phase lags in chains of oscillators associated with undulatory locomotion. 1 Introduction Networks of oscillatory neurons are known to control the rhythmic motor patterns in vertebrate locomotion (Cohen et d . 1988). These networks often consist of large numbers of coupled subnetworks that are themselves capable of oscillating. By connecting such oscillatory units, it is possible to produce a variety of patterns in which each unit maintains a fixed phase relationship with other units in the network (Ermentrout and Kopelll993; Kiemell989; Schoner et al. 1990; Bay and Hemami 1987; Collins and Stewart 1992). The resulting pattern of phases determines the phase relations among the motor units that induce the locomotory activity. The various gaits of quadrupeds provide examples of such patterns, and have been theoretically explored by analyzing or simulating networks of four simple oscillators (Schoner et al. 1990; Bay and Hemami 1987; Collins and Stewart 1992). Other work on asynchronous behavior in networks of oscillators is in Tsang et al. (19911, Aronson et al. (1991), and Abbott and van Vreeswijk (1993). Neural Coniputation 6, 225-241 (1994) @ 1994 Massachusetts Institute of Technology

226

Bard Ermentrout and Nancy Kopell

The example that most strongly motivated the work of this paper is the undulatory swimming pattern of vertebrates such as lampreys and various species of fish. In such animals, the relevant network of neurons is believed to be a linear chain of subnetworks, each capable of autonomous oscillations (Cohen et al. 1982; Grillner 1981). In the lamprey, there is a traveling wave of one body length that is maintained over a broad range of swimming speeds, and for animals of many sizes. This traveling wave can be produced in vitru, using an isolated spinal cord for which activity is induced by the addition of an excitatory amino acid (D-glutamate); larger concentrations of the glutamate lead to higher frequencies of the local oscillators, but with phase lag between fixed points along the cord unchanged (Wallen and Williams 1984). One of the main questions for which a body of theory has been developed (Cohen et al. 1992) is what produces and regulates the phase lags between the successive oscillators in order that the wavelength is maintained at the body length. It has been shown that local coupling (nearest neighbor or multiple near neighbor) is sufficient to produce traveling waves with constant phase lags (Kopell and Ermentrout 1986; Kopell et al. 1990). Furthermore, such systems are capable of regulating the phase lags (and hence the wavelength) under changes of swimming speed (Wallen and Williams 1984; Kopell and Ermentrout 1986). However, there is no mechanism in these systems to ensure that the regulated wavelength will be one body length. To investigate how wavelengths equal to one body length might be produced, we studied chains of oscillators with long coupling fibers joining the ends of the chain to positions near the middle of the chain (Ermentrout and Kopell 1994). We showed that if the long-coupling fibers tended to produce antiphase relationships between the oscillators directly coupled, then the system with such long connections plus short coupling could produce a variety of wave-like patterns, including constant speed traveling waves with wavelength equal to the length of the chain (Ermentrout and Kopell 1994). This in itself was not an answer to the question of regulation in the adult animals, since it is known that short sections of the in vitru spinal cord can self-organize into traveling waves with essentially the same lag per unit length as larger sections of the in vitru cord. Such smaller segments may be considerably smaller than the half-body length connections required to produce the waves using the mechanism of Ermentrout and Kopell (1994). In this paper, we show how the long fibers might still play an important role in the production of the traveling waves with correct wavelength. We discussed in Ermentrout and Kopell (1994) how long fibers might be involved in the production of a variety of patterns associated with movement during very early development of vertebrates. Here we suggest how the patterns produced by such connections can be used as teaching signals to allow the local connections to be tuned to produce appropriate phase lags in the absence of the long connections (as, for

Learning of Phase Lags in Coupled Neural Oscillators

227

example, in sections of the A vifro cord). In such a circuit, after tuning, the phase lags can be expected to be the same even for sections of the cord significantly shorter than half a body length. The techniques used in this paper to achieve local connections sufficient by themselves to produce the waves make use of local circuits that are more complicated than a single cell. A central idea is to use a local circuit for the jth oscillator, in which the signal from the jth oscillator to the (i 1)st is a composite of signals coming from more than one cell in the jth oscillator. Then, by adjusting the relative strengths of the components of the signal, we show in Section 2 that a range of phase lags between the oscillators can be attained. We shall be concerned with methods for achieving appropriate phase lags for coupling in one direction only [the "dominant" direction; see Kopell and Ermentrout (198611. If the local coupling is not reciprocal, the lag to be "learned" depends only on the two oscillators directly involved and not the behavior of the rest of the network; thus it suffices to consider a pair of oscillators and coupling from one to the other. The work of Section 2 shows that for appropriate values of the connection strengths, and within some parameter ranges, the desired lags can be produced. Section 3 deals with the question of how the teaching signal, which specifies the desired lags, can be transformed into an algorithm that produces the appropriate connection strengths. The algorithm reduces the problem of producing an appropriate phase lag to one of producing a kind of synchrony between some component of the circuit and the teaching signal. Since the teaching signal need not have a wave form at all related to that of the voltage of a cell of the oscillator being taught, synchrony does not mean here total overlap in signal or that some thresholds for the teacher and for the oscillator being taught are reached simultaneously; our notion will be described in Section 3. We show that the algorithm is related to gradient ascent on the correlation of the voltages of the teaching signal and one of the cells of the oscillator. We give analysis and simulations to show that the algorithm works. In Section 4 we comment on the relationship of this paper to previous work.

+

2 Connection Strengths and Phase Lags

Consider a pair of limit cycle oscillators, the second forced by the first. If the unforced frequencies of the two are close, in general there is one-one locking with a phase lead or lag that varies with the forcing strength. Now assume that the forcing oscillator is composite, that is, that it has more than one component capable of sending a signal. Thus, within a cycle, the forced oscillator receives more than one signal. We can consider the strengths of these signals as independent. Each forcing signal alone, at a fixed strength, produces a characteristic lag. We show below that,

Bard Ermentrout a n d Nancy Kopell

228

under robust conditions, a predetermined phase lag can be obtained by suitably varying the relative strengths of the components of the signal. The learning algorithm we shall use in Section 3 works on composite oscillators, each component of which is described by a simple model of a neural oscillator. We first describe some behavior of composite oscillators in a simple phase model. We then present specific equations to be used in Section 3 and show that, at least in some parameter ranges, they behave like the general phase models. If one limit cycle forces another, and the forcing is not too strong, the averaging method can be used to reduce the full equations to ones involving the interactions of phases, one for each oscillator (Ermentrout and Kopell 1991). Let HI and 82 denote the phases of the two oscillators. To lowest order, the equation describing the forced system has the form

el,

= LlJ

0; = w + H ( B ,

(2.1)

-02)

where LJ is the uncoupled frequency of oscillators 1 and 2, and FT is a 2x-period function of its argument. It is well known that oscillators 1 and 2 phaselock with a phase difference 4 = HI - H2 satisfying H ( @ )= 0. The locking is stable if d H / d @ > 0. If there is more than one component to the forcing, and the forcing is weak, the effects of the two components are additive to lowest order, Thus 2.1 may be written as

el,

=

w

0;

=

LlJ

+ AH,(81

- 02)

+ BH,j(H, - 02)

(2.2)

where we think of A as the variable strength of a particular component of the forcing, while BH,j(Bl - 0 2 ) denotes all the other components of the forcing. As we show below, under fairly general conditions, the achievable phase lag between the forced and forcing oscillators includes some subset of the interval between the lags that would be produced by H , or H/, alone. A lag that can be produced by H,,(resp. H/J alone is a value (resp. Q,d of 4 for which H,(#)= 0 Iresp. H p ( 4 ) = 01; such a lag is stable providing that dH,,/dd(4,,) > 0 [resp. dH[j/dd(d/j)> 01. We assume that there are such lags (bn and &, and that they are not equal; for definiteness, we may assume that 4- > 4 p , which will not affect the conclusions below. 4'1, is one for which The subinterval in question, which we call dH,,/d$ > 0 on [$L.$,ll and dHp/d$ > 0 on [ap,Gf (see Fig. 1). (The functions in Figure 1 are qualitatively similar to those computed numerically from the more explicit equations to be given below. They show that, for those equations, there is such a subinterval, and that it is not all of In general, if there is such a subinterval, it is now easy tn see that any lag in that subinterval can be achieved stablyby choosing the appropriate A and B. Namely, let A = - B H , j ( $ ) / K 1 ( $ ) for B > 0. [@i%


229

Figure 1: A qualitative sketch of parts of the numerically computed coupling functions H , and Hfi associated to the full equations used in the simulations of Section 3.3. The attainable phase lags include those on the dark subinterval of the 6 axis. Then H ( 3 ) = 0, so 3 is a lag produced by 2.2. Note that A > 0, since the hypotheses on dHLl/d@ and dHO/d@ imply that Ha($) < 0 and Hi,($) > 0. Since the derivatives of each component are positive by hypothesis, this in turn implies that d H / d $ > 0 at 4 = 3,so the lag is stable. We now consider the equations that we shall use for the learning algorithm. For simplicity of exposition, we restrict ourselves to twodimensional equations such as those in Morris and Lecar (19811, though the ideas can be generalized. Each of the components of each of the two oscillators is then described by equations of the form

(2.3) Here, V is the voltage and n is a generalized recovery variable. The terms on the right-hand side of the first equation denote the ionic currents. Within a given oscillator, these components are coupled through the voltages excitatory and/or inhibitory synapses, that is, terms of the form z g(V)[VR- V], where V is the voltage of the component receiving the synaptic current, is the voltage of the presynaptic component, and z is the strength of the interactions. The components need not be the same or even similar. The essential requirement for each of the two “oscillators” is that, when its components are coupled, the system has a stable limit cycle.


230

Figure 2: Model architecture for learning a phase lag between a pair of oscillators, the first forcing the second. Each of the two oscillators has two components, an excitatory cell and an inhibitory cell. The connections between the components are fixed. In addition, there are synapses from both components of oscillator 1 to the excitatory cell of oscillator 2. Of these, the connection from 1, to €2 is held fixed, and the connection from €1 to €2 is varied in order to produce the desired lag. We assume that the effects of the forcing signals are additive to lowest order. (This is automatically true if the forcing is weak.) Thus, some or all of the component cells of the forced oscillator have a sum of synaptic currents added to the voltage equation, with the currents gated by components of the forcing oscillators. The coupling terms within a given oscillator are assumed to be stronger than the forcing between the oscillators. An explicit example is given by the architecture in Figure 2. In this architecture, each circuit has two cells, one excitatory (E) and one inhibitory (I). It is well known that two such cells can be connected to construct a local oscillating unit (Wilson and Cowan 1973). Let V,, and V,, denote the voltages of the excitatory and inhibitory cells of the jth circuit and rife. n,, the corresponding recovery variables. Then the equations for the ~ t circuit h are given by

Vie

=

F(Vp, n,e)

nie

=

G(V,e,n,e)

v\,

=

F(V,i, n ~ i + ) wege(V,e)[VNa - vp]

n;, =

w,,, q1)

f

wigi(vp)[vK - Vp]

(2.4)

Here the ge and the g, are the usual sigmoidal gating functions and w,, we are the coupling strengths of the units within an oscillator. We have


I

231

ze zg

0.125 Zi I

0.625 21

- 0.084

Figure 3: The numerically computed composite coupling function for three different values of z, and fixed zi. Note that changing the value of z, changes the zero of the composite coupling function.

chosen the reversal potential of the synapses to be VK and V N ~but , any choices suffice provided that the inhibitory synapse acts to hyperpolarize the cell and the excitatory synapse acts to depolarize the cell. We allow circuit 1 to force circuit 2 by adding to the VZeequation inputs from V1, and Vli. The equation for V2, is thus

If the forcing is not too large, the explicit equations above can be reduced to those of the form 2.2, where H c t ( 4 )comes from the excitatory coupling, Ho(4)comes from the inhibitory coupling, and A . B are z,, zi. Figure 3 shows the zeros of the composite function resulting from the averaging of 2.5 for various strengths of the excitatory coupling 2,. As 2, increases, the lag moves closer to d, = 0. The numerical computation of the functions of Figure 3 were done using PhasePlane and numerical procedures given in Ermentrout and Kopell(l991) with parameter values given in Section 3.3.

232


3 Learning Synchrony to Learn a Phase Lag 3.1 An Architecture for Learning Synchrony. In this section, we discuss an architecture that reduces the problem of learning a specified phase lag between two oscillators in a chain to the problem of learning connections that can produce an averaged version of synchrony between two signals. We start with some interconnected local circuit that is producing the traveling waves. We now suppose that another circuit is formed at each locus, capable of oscillation. (We have in mind that the first circuit may be very crude: anything capable of producing oscillations at an appropriate frequency. The second one, to be retained through adult life, may be more complicated and subject to functional constraints.) We shall refer to the former as the teacher circuits and the latter as the student circuits. For definiteness, we shall use the circuit example of Section 2 for the local student circuits. However, this is merely the minimal complexity needed to carry out the scheme; local circuits with more components can work as well. We envision the process of tuning the local connections between the local student circuits as starting at one end of the chain and proceeding one circuit at a time. The sequential changes in anatomical structures are consistent with other developmental changes that happen sequentially, starting at the head end (Bekoff 1985). Since the process is the same at each stage, we restrict ourselves to describing only the change in coupling between the first and second local circuits. We assume that the local teacher circuit produces the same signal at each site, but with a time lag of ( from site to site. This signal is oscillatory, with the same period P as that of the student circuits. There is no direct connection between the teaching circuit and the local circuits. They are assumed to be physically close enough to allow some unspecified process to have access to information from both, but the teacher does not interfere with the outcome of the coupling from circuit j to circuit j + l (see Fig. 4). Our problem is to change the weights of the connections from €1 and 11 to Ez so that the phase lag induced from circuit 1 to circuit 2 is exactly the phase lag ( / P of the teaching circuit. We assume that the connections are such that the lag ( / P lies between the lags induced by each of the connections alone. When the learning of the connections between student circuit 1 and student circuit 2 has been accomplished, the learning between student circuits 2 and 3 can begin. Thus, the learning progresses along the chain, one circuit at a time. As will be shown below, a lack of complete synchrony between teacher and student in circuit 1 leads to an error in the learned phase lag between circuit 1 and circuit 2. However, that error does not propagate down the chain. 3.2 Algorithms for Learning Synchrony. The essential idea is to change one of the connection strengths until there is "synchrony" be-


233

Figure 4: Schematic diagram of the teacher and student circuits. For each j, the teacher and student circuit are assumed to be physically close, so that some process may have access to information from both. The teaching circuit does not directly influence the student. tween the signals of teaching circuit 2 and Ez. Since these signals need not have even similar wave forms, we must first describe what we mean by synchrony. We will say that the signals are synchronous if

L'V;(t)T,(t)dt = 0

(3.1)

Here P is the common period of the local circuits, V, is the membrane potential of the E cell of the jth student circuit, and T, is the pulse-like signal from the jth teaching circuit (see Fig. 5A). The integral averages the current from the E cell and the voltage from the teacher circuit. This contrasts with some formulations of Hebbian rules in which what matters is the voltage of the two cells being compared (Amari 1977).

Remark Note that if the signals V,(t) and T,(t) happen to have the same wave form u p to a phase shift, then the integral vanishes if the phase shift is zero, that is, if there is exact synchrony. (This follows from the periodicity of V and T and the fundamental theorem of calculus.) Suppose instead that the teaching signal is more pulsatile (as in Fig. 5A). Then if that signal occurs when the postsynaptic potential is decreasing, as in Figure 5B, the integral is negative; if it occurs when the postsynaptic signal is increasing, the integral is positive.

234


Figure 5: (A) Voltage vs. time for signal from teacher circuit (dashed curve) and output of the E cell (solid curve) when these two signals are synchronous (in the sense of the text). (B) The voltage signal of the E cell (solid curve) lagging the teaching signal (dashed curve).

We now specify an algorithm to change z , so that the E2 cell will synchronize (in the above sense) with the teaching signal at teacher circuit 2. (Recall that z , is the strength of the connection from the E cell of circuit 1 to the E cell of circuit 2.) It follows from the fact that 4, < 4e and the monotonicity of Ht and H- that increasing z , increases the resulting value of 4 = dl - 02 and therefore shifts the curve V , , ( f ) to the right. In the explicit case in which we give the computations, the attainable values of 4 are all negative, that is, the forced oscillator leads the forcer.


235

In this case, increasing z, then decreases the phase lead of oscillator 2 over oscillator 1. We shall fix the other connection strength zi. The teaching signal at circuit 2 is the same as the signal at circuit 1, with a time lag of ( and thus a phase lag of €/P,so Tz(t)= T l ( t - I ) . The learning algorithm is

z:, = --EV;,(t)T1(f - ()

(3.2)

where t 0. That is, Vz,(t) = Vz,[t - 0, this changes z, in the same direction as 3.2, and has the same stable state. The advantage of 3.2 over 3.4 is that 3.2 is local in time, and could conceivably be performed by chemical processes monitoring the cross-membrane current of E2 and the voltage of the teaching signal. Algorithm 3.2 converges because the simple gradient ascent of 3.4 does. The algorithm is effective even if the frequencies of the student circuit differ somewhat from the phaselocked teacher circuit and from each other. In that case, different values of z , for each pair are needed to match the desired lag, but the process produces the same lag ( between each pair.

236


Remark: For weak coupling, the algorithm produces a phase lag that is independent of the frequency of the oscillating circuit. This is seen most easily from the phase equations to which the full equations reduce. Thus, the resulting system shares the property of the lamprey cord that the phase lags remain unchanged when the frequency is changed uniformly along the cord. 3.3 Numerics. The learning algorithm was simulated with the circuits as shown in Figure 4. The parameters in 2.1 specifying the components of one of the circuits are gL = 0.5, gK = 2, gNa = 1.33, VL = -0.5, 0.33cosh(V/0.2). The functions VK = -0.7, VNa = 1, lapp = 0.125, X(V) m x ( V ) and nr(V) are given by m=(V) = 0.5[1 tanh[(V 0.01)/0.15], n%(V) = 0.5[1+tanh(V/O.l)]. The synapses between the components of a single circuit give synaptic current to the E cell of the form w,g,(v,,)[V~ V] and a current to the I cell of the form we&(V2,) [VNa - V]. Here w,= 0.5, w,= 0.5, and gl(V) and g,(V) are given by g, = m,(V), gl(V)= n,(V). The synapses connecting the two circuits go from the I and E cells of circuit 1 to the E cell of circuit 2. The synaptic currents are as in 2.5, namely z,g,(V,,)[VK- V] + Z,&(Vl,)[VNa - V]. Here t , is fixed at 0.2, z, is a variable, and g, and g, are as above. Figure 5 shows the computed composite H functions for three different values of z,, showing how the relevant zero of H changes. The teaching signal is given by T ( t ) = af{l + tanh[s . sin(27rt/P) H I } . Here the strength is set at af = 0.1, the sharpness of the pulse is determined by s = 30, the period P = 8.15 is set to be the period of the student circuit oscillations, and the theshold is set at H = 1. The phase lags (/P to be learned were set at several values between -0.03 and -0.1 to show that a range of phase lags could be learned. The teaching algorithm was given in 3.2. The only parameter to be specified there is f, which was taken to be 0.02. For each of the values of the phase lag, the algorithm converged. We note that the averaging theorem, which we relied on for heuristic understanding, guarantees this method will work for "sufficiently small" values of all the coupling variables. The values used in the numerics may not be small enough to be within the range of this theorem. Nevertheless, as in many other circumstances involving asymptotics, the numerics (done with PhasePlane) show that the method appears to work quite well. To show that this method works for a chain of oscillators, we carried out the procedure for a chain of five composite oscillators. The coefficients z , from oscillator j to oscillator j 1 were allowed to vary one at a time according to the training equation 3.2; when each z, reached equilibrium, then the next was run. Figure 6A shows the values of z, vs. t for each of the four connections, with the phase lag to be learned set at -0.06. Figure 6B shows the output of the voltage vs. time for the five oscillators in the chain after the learning has been achieved and z, set to the equilibrium value of 3.2.

+

+

+


"je

237

B

.

5 4 3 2 1

t

Figure 6: (A) The functions z,(t) for the connections between adjacent oscillators in the chain during training. The numbers correspond to the source oscillator of the connection, for example, 1 represents the connection from oscillator 1 to oscillator 2. The first student oscillator receives a direct pulse from the teacher, creating different training conditions than for the other pairs. All weights start at z , = 0, and after the first pair, all converge to the same value. The time shown in the figure is t _< 2000, which corresponds to 245 cycles. The convergence occurs within 100 cycles. (B) The time course of V,,(t) for each of the 5 oscillators in the chain after training. The numbers labeling the curves give the value of j . Note that the phase lag between oscillators 1 and 2 differs from the other lags, which are all the same.

238


In these simulations, a small pulse was added in student circuit 1 to lock teacher 1 to student 1. [This extra pulse was given by the addition of the term 0.01 T(t)(l - Vie) to the equation for Vie.] For j > 1, there were no direct connections between student and teacher. The small pulse added creates a change in the signals from student 1 to student 2, creating a small error in the first phase lag. This error does not propagate, and the other lags are the one the network is intended to learn. Remark In the lamprey, which motivated this study, there is both tailto-head and head-to-tail coupling. Various sets of experiments (Williams et ul. 1990; Sigvardt et al. 1991) suggest via theory (Kopell et a!. 1991; Kopell and Ermentrout 1990) that the tail-to-head coupling is the "dominant" one that sets the phase lag. In applying this work to the lamprey, we consider the direction of the forcing to be that of the dominant coupling. Since the observed wave is head to toe, the oscillator forcing its rostra1 neighbor (i.e., the neighbor in the head direction) lugs this neighbor ((,h < 0) as in the simulations. Continuing with this interpretation, the sequential changes in coupling along the chain in the above simulation start at the tail end and proceed rostrally.

4 Discussion

There have been many papers addressing the question of how to train networks to learn certain quantities. Most of this work concerns algorithms for adjusting connection strengths so as to learn a family of patterns encoded by digital representations (Rumelhart and McClelland 1986). By contrast, a phase lag is an analogue quantity, and the methods presented for digital patterns d o not apply in an obvious way. Recently, there has been some interest in mechanisms for training networks to oscillate, apparently motivated by synchronous cortical activity during visual tasks. For example, Doya and Yoshizawa (1989) use backpropagation to alter the weights in a hidden layer of cells to generate oscillations with a particular wave form. Other authors (Pearlmutter 1989; Williams and Zipser 1989) have developed similar algorithms. Other work by Doya and Yoshizawa (1992) is more directly relevant to the present work. In the latter paper, the authors assume that the networks are oscillatory, as we do, and then attempt to connect them in such a way as to attain a given phase lag; thus, they are addressing the same problem as in this paper. The major difference between that paper and the current one is in the learning algorithm. In Doya and Yoshizawa (19921, the algorithm involves gradient descent on a function E ( t ) that measures the phase difference between the oscillators to be synchronized. A physical process that carries out that algorithm must be able to compute a weighted average of the output of cells and subtract that from the output of another


239

cell. By contrast, the learning rule proposed here requires only the calculation of a voltage and a membrane current, and really depends only on the sign of that product. Thus, it is easier to conceive of a physical process able to carry it out. Furthermore, the method could work equally well with any periodic signal produced by one of the cells of each student circuit; for example, a synaptic current can be used instead of a membrane current. This could produce a different phase between Ti and the ith student circuit, but would produce the same phase between student circuits when carried out in a chain. The mechanism proposed here has been implemented with membrane models based on gated currents and voltages. The mechanism could also be implemented with models in which the variables are firing rates. The essential idea is that the learning algorithm works on a variable associated with a cell in the circuit (such as firing rate of that cell) and the time derivative of the analogous variable in the teaching signal. Any physical process capable of determining the sign of the product of those two quantities could be used to produce the learning.

Acknowledgments We wish to thank T. LoFaro for his assistance in preparing the figures. Supported in part by NIMH Grant 47150 and NSF Grants DMS-9002028 (B.E.) and DMS-9200131 (N.K.).

References Abbott, L. F., and van Vreeswijk, C. 1993. Asynchronous states in networks of pulse-coupled oscillators. Neural Comp. 5, 823-848. Amari, S. I. 1977. Neural theory of association and concept formation. Biol. Cybern. 26, 175-185. Aronson, D. G., Golubitsky, M., and Mallet-Paret, J. 1991. Ponies on a menygo-round in large arrays of Josephson Junctions. Nonlinearity 4, 90S910. Bay, J. S., and Hemami, H. 1987. Modelling of a neural pattern generator with coupled nonlinear oscillators. ZEEE Trans. Biomed. Eng. 4, 297-306. Bekoff, A. 1985. Development of locomotion in vertebrates: A comparative perspective. In The Comparative Development of Adaptive Skills: Evolutiona y Implications, E. S. Gallin, ed., pp. 57-94. Erlbaum, Hillsdale, NJ. Cohen, A. H., Holmes, P. J., and Rand, R. H. 1982. The nature of the coupling between segmental oscillators of the lamprey spinal generator for locomotion: A mathematical model. I. Math. Biol. 13, 345-369. Cohen, A. H., Rossignol, S., and Grillner, S. 1988. Neural Control of Rhythmic Movements in Vertebrates. John Wiley, New York. Cohen, A. H., Ermentrout, G. B., Kiemel, T., Kopell, N., Sigvardt, K. A., and Williams, T. L. 1992. Modelling of intersegmental coordination in the lam-

240


prey central pattern generator for locomotion. Trends in Neurosci. 15, 434438. Collins, J. J., and Stewart, I. N. 1992. Symmetry-breaking bifurcation: A possible mechanism for 2:l frequency locking in animal locomotion. /. Math. Biol. 30, 827-838. Doya, K., and Yoshizawa, S. 1989. Adaptive neural oscillator using continuoustime back-propagation learning. Neural Networks 2, 375-386. Doya, K., and Yoshizawa, S. 1992. Adaptive synchronization of neural and physical oscillators. In Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lipmann, eds. Morgan Kaufmann, Sari Mateo, CA. Ermentrout, G. B., and Kopell, N. 1991. Multiple pulse interactions and averaging in systems of coupled neural oscillators. /. Math. Bid. 29, 195-217. Ermentrout, G. B., and Kopell, N. 1994. Inhibition produced patterning in chains of coupled nonlinear oscillators. S I A M J. Appl. Math. 54, in press. Grillner, S. 1981. Control of locomotion in bipeds, tetrapods and fish. In HandbookofPhysiology, Section 1: The Nervous System, 2, V. B. Brooks, ed., pp. 11791236. American Physiological Society, Bethesda, MD. Guckenheimer, J., and Holmes, I? J. 1983. Nonlinear Oscillntions, Dyrinniical Systems, and Bifurcation of Vector Fields. Springer-Verlag, New York. Kiemel, T. 1989. Three problems on coupled nonlinear oscillators. Ph.D. Thesis, Cornell University. Kopell, N., and Ermentrout, G. B. 1986. Symmetry and phaselocking in chains of coupled oscillators. Commun. Pure Appl. Math. 39, 623-660. Kopell, N., and Ermentrout, G. B. 1990. Phase transitions and other phenomena in chains of oscillators. SIAM J. Appl. Math. 50, 1014-1052. Kopell, N., Zhang, W., and Ermentrout, G. B. 1990. Multiple coupling in chains of oscillators. S I A M J. Math. Anal. 21, 935-953. Kopell, N., Ermentrout, G. B., and Williams, T. 1991. On chains of neural oscillators forced at one end. SIAM J. Appl. Math. 51, 1397-1417. Morris, C., and Lecar, H. 1981. Voltage oscillations in the barnacle giant muscle fiber. Biophys. J. 35, 193-213. Pearlmutter, B. A. 1989. Learning state space trajectories in recurrent neural networks. Neural Comp. 1,263-269. Rumelhart, D. E., and McClelland, J. L. 1986. Parallel Distributed Proc~ssing, Vols. 1 and 2. MIT Press, Cambridge, MA. Schoner, G., Young, W. Y., and Kelso, J. A. S. 1990. A synergetic theory of gaits and gait transitions. J. Theor. B i d . 142, 359-391. Sigvardt, K., Kopell, N., Ermentrout, G. B., and Remler, M. 1991. SOC.Neurosci. Abst. 17, 122. Tsang, K. Y., Mirollo, R. E., Strogatz, S. H., and Weisenfield, K. 1991. Dynamics of a globally coupled oscillator array. Physica D 48, 102-112. Wallen, P., and Williams, T. L. 1984. Fictive locomotion in the lamprey spinal cord in uitro compared with swimming in the intact and spinal animal. 1. Physiol. 347, 225-239. Williams, R. J., and Zipser, D. 1989. A learning algorithm for continually running fully recurrent neural networks. Neural Comp. 1, 270-280.


241

Williams, T., Sigvardt, K., Kopell, N., Ermentrout, G. B., and Remler, M. 1990. Forcing of coupled nonlinear oscillators: Studies of intersegmental coordination in the lamprey locomotor central pattern generator. I. Neurophysiol. 64,862-871. Wilson, H. R., and Cowan, J. 1973. A mathematical theory for the functional dynamics of cortical and thalamic tissue. Kybernetic 13,55-80. ___

~~

Received March 1,1993; accepted June 22,1993.

This article has been cited by: 2. Jae-Kwan Ryu, Nak Young Chong, Bum Jae You, Henrik I. Christensen. 2010. Locomotion of snake-like robots using adaptive neural oscillators. Intelligent Service Robotics 3:1, 1-10. [CrossRef] 3. Péter L. Várkonyi, Tim Kiemel, Kathleen Hoffman, Avis H. Cohen, Philip Holmes. 2008. On the derivation and tuning of phase oscillator models for lamprey central pattern generators. Journal of Computational Neuroscience 25:2, 245-261. [CrossRef] 4. Máté Lengyel, Jeehyun Kwag, Ole Paulsen, Peter Dayan. 2005. Matching storage and recall: hippocampal spike timing–dependent plasticity and phase response curves. Nature Neuroscience 8:12, 1677-1683. [CrossRef] 5. G.N. Borisyuk, R.M. Borisyuk, Yakov B. Kazanovich, Genrikh R. Ivanitskii. 2002. Models of neural dynamics in brain information processing the developments of 'the decade'. Uspekhi Fizicheskih Nauk 172:10, 1189. [CrossRef] 6. Toshio Aoyagi. 1995. Network of Neural Oscillators for Retrieving Phase Information. Physical Review Letters 74:20, 4075-4078. [CrossRef]

Communicated by Idan Segev

A Mechanism for Neuronal Gain Control by Descending Pathways Mark E. Nelson Department of Physiology and Biophysics and Beckman Institute for Advanced Science and Technology, University of lllinois, Urbana, 1L 61801 U S A

Many implementations of adaptive signal processing in the nervous system are likely to require a mechanism for gain control at the single neuron level. To properly adjust the gain of an individual neuron, it may be necessary to use information carried by neurons in other parts of the system. The ability to adjust the gain of neurons in one part of the brain, using control signals arising from another, has been observed in the electrosensory system of weakly electric fish, where descending pathways to a first-order sensory nucleus have been shown to influence the gain of its output neurons. Although the neural circuitry associated with this system is well studied, the exact nature of the gain control mechanism is not fully understood. In this paper, we propose a mechanism based on the regulation of total membrane conductance via synaptic activity on descending pathways. Using a simple neural model, we show how the activity levels of paired excitatory and inhibitory control pathways can regulate the gain and baseline excitation of a target neuron. 1 Introduction

Mechanisms for gain control at the single neuron level are likely to be involved in many implementations of adaptive signal processing in the nervous system. The need for some sort of adaptive gain control capability is most obvious at the sensory periphery, where adjustments to the gain of sensory neurons are often required to compensate for variations in external conditions, such as the average intensity of sensory stimuli. Adaptive gain control is also likely to be employed at higher levels of neural processing, where gain changes may be related to the functional state of the system. For both peripheral and higher-order neurons, the information necessary to make proper gain adjustments may be carried by neurons that are not part of the local circuitry associated with the target neuron. Thus it would be useful to have a means for adjusting the gain of ,neurons in one part of the nervous system, using control signals arising from another. Neural Computation 6, 242-254 (1994)


Mechanism for Neuronal Gain Control

243

Such a capability seems to exist in the electrosensory system of weakly electric fish, where descending pathways to a first-order sensory nucleus have been shown to influence the gain of its output neurons. Although the neural circuitry associated with this system is well studied, the exact nature of the gain control mechanism is not fully understood. In this paper, we propose a general neural mechanism that may underlie gain control in the electrosensory system, and which could potentially be used in many other systems, as a means for carrying out adaptive signal processing. In our model, gain control is achieved by regulating activity levels on paired excitatory and inhibitory control pathways that impinge on a target neuron. Increased synaptic activity on the control pathways gives rise to an increase in total membrane conductance of the target neuron, which lowers its input resistance. Consequently, input current is less effective in bringing about changes in membrane voltage and the effective gain of the neuron is reduced. Changing gain by increasing synaptic conductance levels would normally bring about a shift in the baseline level of excitation in the target neuron as well. In order to avoid this coupling between gain and baseline excitation, it is necessary to implement the conductance change using paired excitatory and inhibitory pathways. 2 Gain Control in the Electrosensory System

Certain species of freshwater tropical fish, known as weakly electric fish, detect and discriminate objects in their environment using self-generated electric fields (Bullock and Heiligenberg 1986). Unlike strongly electric fish, which use their electrogenic capabilities to stun prey, weakly electric fish produce fields that are generally too weak to be perceptible to other fish or to human touch. When a weakly electric fish discharges its electric organ, a small potential difference, on the order of a few millivolts, is established across its skin. Nearby objects in the water distort the fish’s field, giving rise to perturbations in the potential across the skin. Specialized electroreceptors embedded in the skin transform these small perturbations, which are typically less than 100 p V in amplitude, into changes in spike activity in primary afferent nerve fibers. In the genus Apteronotus, primary afferent fibers change their firing rate by about 1 spike/sec for a 1 p V change in potential across the skin (Bastian 1981a). Weakly electric fish often live in turbid water and tend to be nocturnal. These conditions, which hinder visual perception, do not adversely affect the electric sense. Using their electrosensory capabilities, weakly electric fish can navigate and capture prey in total darkness in much the same way that bats do using echolocation. A fundamental difference between bat echolocation and fish ”electrolocation” is that the propagation of the electric field emitted by the fish is essentially instantaneous when considered on the time scales that characterize nervous system function.

244

Mark E. Nelson

Thus rather than processing echo delays as bats do, electric fish extract information from instantaneous amplitude and phase modulations of their emitted signals. The electric sense must cope with a wide range of signal intensities because the magnitude of detectable electric field perturbations can vary over several orders of magnitude depending on the size, distance, and impedance of the object that gives rise to them (Bastian 1981a). In the electrosensory system there are no peripheral mechanisms to compensate for variations in signal intensity. Unlike the vertebrate visual system, which can regulate the intensity of light arriving at photoreceptors by adjusting pupil diameter, the electrosensory system has no equivalent means for directly regulating the overall electric field strength experienced by the electroreceptors,' and unlike the auditory system, there are no efferent projections to the sensory periphery to control the gain of the receptors themselves. The first opportunity for the electrosensory system to make adjustments in sensitivity occurs in a first-order sensory nucleus known as the electrosensory lateral line lobe (ELL). In the ELL, primary afferent axons from peripheral electroreceptors terminate on a class of pyramidal cells referred to as E-cells (Maler rt 01. 1981; Bastian 1981b), which represent a subset of the output neurons for the nucleus. Figure 1 shows a reconstructed E-cell (Bastian and Courtright 1991) that illustrates the general morphology of this class of neurons, including the basal dendrite with its terminal arborization that receives afferent input and the extensive apical dendrites that receive descending input from higher brain centers (Maler et al. 1981). Figure 2 shows a highly simplified diagram of the ELL circuitry, indicating the afferent and descending pathways, and their patterns of termination on E-cells. In view of our proposed gain control mechanism involving a conductance change, it is particularly noteworthy that descending inputs account for the majority of E-cell synapses, and can thus potentially make a significant contribution to the total synaptic conductance of the cell. In experiments in which the average electric field strength experienced by the fish is artificially increased or decreased, E-cell sensitivity is observed to change in a compensatory manner. For example, a 50'70 reduction in overall field strength would cause no significant change in the number of spikes generated by an E-cell in response to an object moving through its receptive field, whereas the number of spikes generated by primary afferent fibers that impinge on the E-cell would be greatly reduced. To maintain a constant output response, while receiving a reduced input signal, the effective gain of the E-cell must have somehow been increased. Interestingly, this apparent gain increase occurs without significantly altering the baseline level of spontaneous activity in the 'In principle, this could be achieved by regulating the strength of the fish's own electric discharge. However, these fish maintain a remarkably stable discharge amplitude and such a mechanism has never been observed.


245

1

a Icaldendrlte ($-nding input)

i f

I

isoma (inhibitory input)

1:

r

.....i ..... I

1

i .....i

basal dendrite (efferentinput)

Figure 1: Reconstructed E-cell from the electrosensory lateral line lobe (ELL) of the weakly electric fish Apteronotus leptorhynchus (brown ghost knife fish). This cell was reconstructed by Bastian and Courtright (1991) following intracellular injection of Lucifer yellow. E-cells receive excitatory primary afferent input on the terminal arbor of their basal dendrites, excitatory descending input on their extensive apical dendrites, and inhibitory input primarily on the soma and somatic dendrites. E-cells serve as output neurons of the ELL and send their axons to higher brain centers. Descending feedback pathways from these higher centers have been shown to play a role in controlling the gain of E-cell responses to afferent input (Bastian 1986a,b). Redrawn from Bastian and Courtright (1991). E-cell. Descending pathways are known to play an important role in this descending gain control capability, since the ability to make compensatory gain adjustments is abolished when descending inputs to the ELL are blocked (Bastian 1986a,b). Although the neural circuitry associated with this system is well studied, the mechanism underlying the gain control capability is not fully understood. 3 A Model of Descending Gain Control

In this paper, w e propose that the underlying mechanism could involve the regulation of total membrane conductance by activity on descending pathways. Figure 2 shows a schematic diagram of the relevant circuitry that forms the basis of our model. The target neuron receives afferent input on its basal dendrite and control inputs from two descending

Mark E. Nelson

246

(CONTROL)

0 excitatory

excitatory

inhibitory

*sanding

inhibitory

inhibitory i n m n

Figure 2: Neural circuitry for descending gain control based on the the circuitry of the electrosensory lateral line lobe (ELL). The target neuron (E-cell) receives primary afferent input on its basal dendrite, and descending control inputs on its apical dendrites and cell body. The control pathway is divided into an excitatory and an inhibitory component. One component makes excitatory synapses (open circles) directly on the apical dendrites of the target neuron, while the other acts through an inhibitory interneuron (shown in gray) to activate inhibitory synapses on the soma (filled circles). The gain and offset of the target neuron’s response to an input signal can be regulated by adjusting activity levels on the two descending control pathways. pathways on its apical dendrites and cell body. One descending pathway makes excitatory synaptic connections directly on the apical dendrite of the target neuron, while a second pathway exerts a net inhibitory effect by acting through an interneuron which makes inhibitory synapses on the cell body of the target neuron. The model circuitry shown in Figure 2 has the input and control pathways segregated onto different parts of the dendritic tree, as is the case for actual E-cells (Fig. 1). This spatial segregation has played a key role in the discovery and characterization of gain control in this system by allowing independent experimental manipulation of the input and control pathways (Bastian 1986a,b). However, in this paper, we will ignore the effects of the spatial distribution of synapses and will treat the target neuron as an electrotonically compact point neuron. This approximation turns out to be sufficient for describing the basic operation of the proposed gain control mechanism when the conductance changes associated with the descending pathways can be treated as slowly varying. In the actual system, the placement of monosynaptic excitatory inputs on the


247

Figure 3: Electrical equivalent circuit for the target neuron (E-cell) in Figure 2. The membrane capacitanceC, and leakage conductanceg],ak are intrinsic properties of the target neuron, while the excitatory and inhibitory conductances,g,, and ginh, are determined by activity levels on the descending pathways. The input signal Z(t) is modeled as a time-dependent current that can represent either the synaptic current arising from afferent input or an externally injected current. By adjusting g,, and ginh, activity levels on descending pathways can regulate the total membrane conductance and baseline level of excitation of the target neuron.

distal dendrites and bisynaptic inhibitory inputs on the soma may help maintain the relative timing between the two components of the control pathway, such that the system could better handle rapid gain changes associated with transient changes in descending activity. The gain control function of the neural circuitry in Figure 2 can be understood by considering the electrical equivalent circuit for the target neuron, as shown in Figure 3. For the purpose of understanding this model, it is sufficient to consider only the passive and synaptic conductances involved, and ignore the various types of voltage-dependent channels that are known to be present in ELL pyramidal cells (Mathieson and Maler 1988). The passive properties of the target neuron are described by a membrane capacitance C,, a leakage conductance &ak, and an associated reversal potential Eleak. The excitatory descending pathway directly activates excitatory synapses on the target neuron, giving rise to an excitatory synaptic conductance gexwith a reversal potential Eex. The inhibitory descending pathway acts by exciting a class of in-

Mark E. Nelson

248

hibitory interneurons, which in turn activates inhibitory synapses on the target neuron with inhibitory conductance g l n h and reversal potential Elnh. The excitatory and inhibitory conductances, g,, and g l n h are taken to represent the population conductances of all the individual excitatory and inhibitory synapses associated with the descending pathways. While individual synaptic events give rise to a time-dependent conductance change (which is often modeled by an o function), we consider the domain in which the activity levels on the descending pathways, the number of synapses involved, and the synaptic time constants are such that the summed effect can be well described by a single time-invavianf conductance value for each pathway. The input signal (the one under the influence of the gain control mechanism) is modeled in a general form as a time-dependent current Z(t). This current can represent either the synaptic current arising from activation of synapses in the primary afferent pathway, or it can represent direct current injection into the cell, such as might occur in an intracellular recording experiment. The output of the system is considered to be the time-varying membrane potential of the target neuron V ( f ) .For most biological neurons, there is a subsequent transformation in which changes in membrane potential give rise to changes in the rate at which action potentials are generated. However, for the purpose of understanding the gain control mechanism proposed here, it is sufficient to limit our consideration to the behavior of the underlying membrane potential in the absence of the spike generating processes. Thus, in this model, the gain of the system describes the magnitude of the transformation between input current and the output membrane voltage change. The behavior of the membrane potential V(t) for the circuit shown in Figure 3 is described by

dV

cm- dt

fgleak(v

- EIeak) + g e x ( V

-

E m )+ g i n h ( V

- Einh)

=I(t)

(3.1)

In the absence of an input signal (I = 0), the system will reach a steadystate (dV/dt = 0) membrane potential V,, given by Vss(I=O) =

gieakEleak f g e x E e x f 8inhEinh

gleak

-k g e x f

(3.2)

ginh

If we consider the input l ( t ) to give rise to fluctuations in membrane potential U (t ) about this steady state value

U ( t )= V ( t )- v,,

(3.3)

then 3.1 can be rewritten as (3.4)


249

Figure 4: Gain as a function of frequency for three different values of total membrane conductance gtot. At low frequencies, gain is inversely proportional to gtot. Note that the time constant T , which characterizes the low-pass cutr)ff frequency, also varies inversely with gtot. where gtotis the total membrane conductance gtot

= gleak -k gex

+ gin11

(3.5)

Equation 3.4 describes a first-order low-pass filter with a transfer function H ( s ) given by

H ( s )=

Rtot

7s +1

~

(3.6)

where s is the complex frequency (s = iw),Rtot is the total membrane resistance (Rtot = l/gtot),and T is the RC time constant ( T = RtotC,). The frequency dependence of the response gain JH(iu)l,as given by equation 3.6, is illustrated in Figure 4. In this figure, the gain has been normalized to the maximum system gain, JHlmax = l/gleak,which occurs at Y = 0 when g,, and gi,,~,are both zero. The normalized gain, in decibels, is given by 20log,,(~HJ/JHJ,,,). For frequency components of the input signal below the knee of the response curve (WT > l),the gain rolls off with increasing frequency and is independent

250

Mark E. Nelson

of Rtot, due to the fact that the impedance is dominated by the capacitive component in this domain. For frequency components of the input signal below the cutoff frequency, gain control can be accomplished by regulating the total membrane conductance. It is important to note that the value of the cutoff frequency is not constant, but is correlated with the magnitude of the response gain; both the membrane time constant T and the gain IN1 are directly proportional to R,,,. In many cases, the biologically relevant frequency components of the input signal will be well below the cutoff frequencies imposed by the gain control circuitry. For example, in the case of weakly electric fish with continuous wave-type electric organ discharge (EOD) signals, the primary afferent input signal carries information about amplitude modulations of the EOD signal due to objects (or other electric fish) in the vicinity. For typical object sizes and velocities, the relevant frequency components of the input signal are probably below about 10 Hz (Bastian 1981a,b) and can thus be considered low frequency with respect to the cutoffs imposed by the gain control circuitry. From the point of view of biological implementation, there is a potential concern that low-frequency signals are actually constructed from the summation of individual postsynaptic potentials (PSPs) that can have much higher frequency components and that may thus be significantly affected by the low-pass filtering mechanism. In the proposed model, increases in gain are accompanied by a reduction in the cutoff frequency. Hence there is a potential concern that attempts to increase the gain would be counteracted by decreases in amplitude of individual PSPs, rendering the overall gain control mechanism ineffective. However, because the system acts as a linear filter, it turns out that filtering the individual PSPs and then adding them together (filtering then summing) is equivalent to adding the individual PSPs together (to form a signal with only low-freqency components) and then passing that signal through the filter (summing then filtering). When a "fast" PSP is low-pass filtered, the peak amplitude is indeed attenuated, but the duration of the filtered PSP is prolonged, such that the overall contribution to a low-frequency signal is not diminished. Thus, to the extent that the neural processing associated with summing afferent PSPs together can be treated as linear, the gain control mechanism is not affected by the fact that low-frequency input signals are built u p from PSPs with higher frequency components. Note that the requirement of linearity only applies to neural processing associated with the summation of PSPs and not to subsequent processing, such as the generation of action potentials. In the case of E-cells in the ELL, it is interesting to note that the input region of the neuron in the terminal arbor of the basal dendrite is well separated from the spike generating region in the soma (Fig. 1). In this model, we propose that regulation of total membrane conductance occurs via activity on descending pathways that activate excitatory and inhibitory synaptic conductances. For this proposed mechanism to be effective, descending synaptic conductances must make a significant


251

contribution to the total membrane conductance of the target neuron. Whether this condition actually holds for ELL pyramidal cells has not yet been experimentally tested. However, it is not an unreasonable assumption to make, considering recent reports that synaptic background activity can have a significant influence on the total membrane conductance of cortical pyramidal cells (Bernander et al. 1991) and cerebellar Purkinje cells (Rapp et al. 1992). 4 Control of Baseline Excitation

If the only functional goal of the circuitry shown in Figure 2 was to regulate total membrane conductance, then synaptic activity on a single descending pathway would be sufficient and there would be no need for paired excitatory and inhibitory pathways. However, attempting to control response gain using a single pathway results in a coupling between the gain and the baseline level of excitation of the target neuron, which may be undesirable. For example, if one tried to decrease response gain using a single inhibitory control pathway, then as the inhibitory conductance was increased to lower the gain, the steady-state membrane potential of the target neuron would simultaneously be pulled toward the inhibitory reversal potential. If we would like to be able to change the sensitivity of a neuron’s response without changing its baseline level of excitation, as has been observed in E-cells in the ELL, then we need a mechanism for decoupling the gain of the response from the steady-state membrane potential. The second control pathway in Figure 2 provides the extra degree of freedom necessary to achieve this goal. To change the gain of a neuron without changing its baseline level of excitation, the excitatory and inhibitory conductances must be adjusted so as to achieve the desired total membrane conductance gtot,as given by equation 3.2, while maintaining a constant steady-state membrane voltage V,,, as given by equation 3.5. Solving equations 3.2 and 3.5 simultaneously for gexand ginhr we find gex

=

(4.1)

ginh

=

(4.2)

For example, consider a case where the reversal potentials are beak = -70 mV, Eex = 0 mV, and Einh = -90 mV. Assume-we want to find Values of the steady-state conductances, gexand ginh that would result in a total membrane conductance that is twice the leakage conductance ke., gtot = 2&ak), and would produce a steady-state depolarization of 10 mV (i.e., V,, = -60 mV). Using 4.1 and 4.2 we find the required synaptic conductance levels are gex= &leak and ginh = &leak.

252

Mark E. Nelson

5 Discussion

The ability to regulate a target neuron’s gain using descending control signals would provide the nervous system with a powerful means for implementing adaptive signal processing algorithms in sensory processing pathways as well as other parts of the brain. The simple gain control mechanism proposed here, involving the regulation of total membrane conductance, may find widespread use in the nervous system. Determining whether this is the case, of course, requires experimental verification. Even in the electrosensory system, which provided the inspiration for this model, definitive experimental tests of the proposed mechanism have yet to be carried out. Fortunately the model is capable of generating straightforward, experimentally testable predictions. The main prediction of the model is that gain changes will be correlated with changes in input resistance of the target neuron and with changes in RC time constant, as illustrated in Figure 4. If such correlations were observed experimentally, more subtle aspects of the model could be tested, such as the prediction that both the excitatory and inhibitory pathways should contribute to the conductance change. To make the discussion more specific, we will describe how one might go about testing this model in the ELL of the weakly electric fish Apteronotus leptorhynchus. In this system it is possible to make in v i m intracellular recordings from intact animals (e.g., Bastian and Courtright 19911, thus making it possible to measure input resistance directly while manipulating the system so as to bring about gain changes. One experimental procedure for testing the gain control model would involve delivering hyperpolarizing current step pulses into the soma of an E-cell via an intracellular electrode to determine its input resistance and membrane time constant. Input resistance would be determined by measuring the steady-state ratio of the change in membrane potential to injected current, while the time constant could be estimated from the time course of the exponential onset of the voltage change.2 The input resistance and time constant would first be determined under “normal” conditions, and then under conditions of increased or decreased gain. Bastian (1986a) has demonstrated that gain changes can be induced by artificially altering the nominal strength of the electric organ discharge (EOD) signal by adding or subtracting a scaled version of the fish’s own EOD signal back into the experimental tank. Thus, if the gain of the system were increased using this technique, the model predicts that a hyperpolarizing test pulse of fixed amplitude should give rise to a larger change in membrane voltage (i.e., increased input resistance) with a slower onset

’Previous in nitro studies (Mathieson and Maler 1988) have demonstrated that hyperpolarizing current steps do not seem to activate any large nonlinear conductances, thus validating this approach to measuring membrane time constant.


253

(i.e., increased RC time constant) relative to the "normal" case. Note that the gain of the system can be monitored independently from the input resistance by delivering peripheral stimuli to the electroreceptors and observing the response of the E-cell. Thus the proposed model could be rejected by the above experimental procedure if the system gain were shown to increase, while the observed input resistance and time constant remained unchanged. This could occur, for example, if the actual gain control mechanism were implemented at the level of the afferent synapses onto the E-cell, such that an increased gain was associated with increased synaptic current, whereas our model predicts that the synaptic current remains the same, but the responsiveness of the E-cell changes. We have mentioned that the model circuitry of Figure 2 was inspired by the circuitry of the ELL. For those familiar with this circuitry, it is interesting to speculate on the identity of the interneuron in the inhibitory control pathway. In the gymnotid ELL, there are at least six identified classes of inhibitory interneurons. For the proposed gain control mechanism, we are interested in identifying those that receive descending input and that make inhibitory synapses onto pyramidal cells. Four of the six classes meet these criteria: granule cell type 2 (G2), polymorphic, stellate, and ventral molecular layer neurons. While all four classes may participate to some extent in the gain control mechanism, one would predict that G2 (as suggested by Shumway and Maler 1989) and polymorphic cells make the dominant contribution, based.on cell number and synapse location. The morphology of G2 and polymorphic neurons differs somewhat from that shown in Figure 2. In addition to the apical dendrite, which is shown in the figure, these neurons also have a basal dendrite that receives primary afferent input. G2 and polymorphic neurons are excited by primary afferent input and thus provide additional inhibition to pyramidal cells when afferent activity levels increase. This can be viewed as providing a feedforward component to the inhibitory conductance change associated with the proposed gain control mechanism. In this paper, we have confined our analysis to the effects of tonic changes in descending activity, which has allowed us to treat the control conductances as time-invariant quantities. While this may be a reasonable approximation for certain experimental situations, it is unlikely to be a good representation of the actual patterns of control activity that occur under natural conditions. This is particularly true in the electrosensory system, where the descending pathways are known to form part of a feedback loop that includes the ELL output neurons. In fact, there is already experimental evidence demonstrating that in addition to gain control, descending pathways influence the spatial and temporal filtering properties of ELL output neurons (Bastian 1986a,b; Shumway and Maler 1989). Thus the simple model presented here is only a first step toward understanding the full range of effects that descending pathways can have on the signal processing capabilities of single neurons.

254

Mark E. Nelson

Acknowledgments Thanks to J. Bastian and L. Maler for many enlightening discussions on descending gain control in the ELL, and to J. Payne for his detailed comments on this manuscript. This work was supported by NIMH 1R29-MH49242.

References Bastian, J. 1981a. Electrolocation I: How the electroreceptors of Apteroriotus ulbifrons code for moving objects and other electrical stimuli. J. Comp. Physiol. 144, 465-479. Bastian, J. 1981b. Electrolocation 11: The effects of moving objects and other electrical stimuli on the activities of two categories of posterior lateral line lobe cells in Apteronotus albifrons. 1. Comp. Physiol. 144, 481-494. Bastian, J. 1986a. Gain control in the electrosensory system mediated by descending inputs to the electrosensory lateral line lobe. 1. Neurosci. 6, 553562. Bastian, J. 1986b. Gain control in the electrosensory system: A role for the descending projections to the electrosensory lateral line lobe. 1. Comp.Physiol. 158,505-515. Bastian, J., and Courtright, J. 1991. Morphological correlates of pyramidal cell adaptation rate in the electrosensory lateral line lobe of weakly electric fish. J. Cornp. Physiol. 168, 393-407. Bernander, O., Douglas, R. J., Martin, K. A. C., and Koch, C. 1991. Synaptic background activity influences spatiotemporal integration in single pyramidal cells. Proc. Nut/. Acad. Sci. U.S.A. 88, 11569-11573. Bullock, T. H., and Heiligenberg, W., eds. 1986. Electroreception. Wiley, New York. Knudsen, E. I. 1975. Spatial aspects of the electric fields generated by weakly electric fish. J. Comp. Physiol. 99, 103-118. Maler, L., Sas, E., and Rogers, J. 1981. The cytology of the posterior lateral line lobe of high frequency weakly electric fish (Gymnotidei): Dendritic differentiation and synaptic specificity in a simple cortex. J. Comp. Neurol. 195,87-140. Mathieson, W. B., and Maler, L. 1988. Morphological and electrophysiological properties of a novel in vitro preparation: The electrosensory lateral line lobe brain slice. J. Cornp. Physiol. 163, 489-506. Rapp, M., Yarom, Y., and Segev, I. 1992. The impact of parallel fiber background activity on the cable properties of cerebellar Purkinje cells. Neural Comp. 4, 518-533. Shumway, C. A., and Maler, L. M. 1989. GABAergic inhibition shapes temporal and spatial response properties of pyramidal cells in the electrosensory lateral line lobe of gymnotiform fish. 1. Cornp. Physiol. 164, 391-407. Received February 8, 1993; accepted July 9, 1993.

This article has been cited by: 2. R. Angus Silver. 2010. Neuronal arithmetic. Nature Reviews Neuroscience 11:7, 474-489. [CrossRef] 3. Connie Sutherland, Brent Doiron, André Longtin. 2009. Feedback-induced gain control in stochastic spiking networks. Biological Cybernetics 100:6, 475-489. [CrossRef] 4. Jeffrey M. Beck, Alexandre Pouget. 2007. Exact Inferences in a Neural Implementation of a Hidden Markov ModelExact Inferences in a Neural Implementation of a Hidden Markov Model. Neural Computation 19:5, 1344-1361. [Abstract] [PDF] [PDF Plus] 5. R. Jacob Vogelstein, Udayan Mallik, Joshua T. Vogelstein, Gert Cauwenberghs. 2007. Dynamically Reconfigurable Silicon Array of Spiking Neurons With Conductance-Based Synapses. IEEE Transactions on Neural Networks 18:1, 253-265. [CrossRef] 6. R. Eckhorn, A.M. Gail, A. Bruns, A. Gabriel, B. Al-Shaikhli, M. Saam. 2004. Different Types of Signal Coupling in the Visual Cortex Related to Neural Mechanisms of Associative Processing and Perception. IEEE Transactions on Neural Networks 15:5, 1039-1052. [CrossRef] 7. Carlo R. Laing , André Longtin . 2003. Dynamics of Deterministic and Stochastic Paired Excitatory—Inhibitory Delayed FeedbackDynamics of Deterministic and Stochastic Paired Excitatory—Inhibitory Delayed Feedback. Neural Computation 15:12, 2779-2822. [Abstract] [PDF] [PDF Plus] 8. Alexandre Pouget, Peter Dayan, Richard S. Zemel. 2003. INFERENCE AND COMPUTATION WITH POPULATION CODES. Annual Review of Neuroscience 26:1, 381-410. [CrossRef] 9. Robert J. Calin-Jageman, Thomas M. Fischer. 2003. Temporal and spatial aspects of an environmental stimulus influence the dynamics of behavioral regulation of the Aplysia siphon-withdrawal response. Behavioral Neuroscience 117:3, 555-565. [CrossRef] 10. Brent Doiron , André Longtin , Neil Berman , Leonard Maler . 2001. Subtractive and Divisive Inhibition: Effect of Voltage-Dependent Inhibitory Conductances and NoiseSubtractive and Divisive Inhibition: Effect of Voltage-Dependent Inhibitory Conductances and Noise. Neural Computation 13:1, 227-248. [Abstract] [PDF] [PDF Plus] 11. Thomas M. Fischer, Jean W. Yuan, Thomas J. Carew. 2000. Dynamic regulation of the siphon withdrawal reflex of Aplysia californica in response to changes in the ambient tactile environment. Behavioral Neuroscience 114:6, 1209-1222. [CrossRef] 12. Gary R. Holt, Christof Koch. 1997. Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates*Shunting Inhibition Does Not Have a Divisive Effect on Firing Rates*. Neural Computation 9:5, 1001-1013. [Abstract] [PDF] [PDF Plus]

Communicated by Todd Leen

The Role of Weight Normalization in Competitive Learning Geoffrey J. Goodhill University of Edinburgh, Centre for Cognitive Science, 2 Buccleiich Place, Edinburgh EH8 9LW, United Kingdom

Harry G. Barrow University of Sussex, School of Cognitive and Computing Sciences, Falmer, Brighton BN1 9QH, United Kingdom

The effect of different kinds of weight normalization on the outcome of a simple competitive learning rule is analyzed. It is shown that there are important differences in the representation formed depending on whether the constraint is enforced by dividing each weight by the same amount ("divisive enforcement") or subtracting a fixed amount from each weight ("subtractive enforcement"). For the divisive cases weight vectors spread out over the space so as to evenly represent "typical" inputs, whereas for the subtractive cases the weight vectors tend to the axes of the space, so as to represent "extreme" inputs. The consequences of these differences are examined. 1 Introduction

Competitive learning (Rumelhart and Zipser 1986) has been shown to produce interesting solutions to many unsupervised learning problems [see, e.g., Becker (1991); Hertz et al. (199111. However, an issue that has not been greatly discussed is the effect of the type of weight normalization used. In common with other learning procedures that employ a simple Hebbian-type rule, it is necessary in competitive learning to introduce some form of constraint on the weights to prevent them from growing without bounds. This is often done by specifying that the sum [e.g., von der Malsburg (197311 or the sum-of-squares [e.g., Barrow (198711 of the weights for each unit should be maintained at a constant value. Weight adaptation in competitive learning is usually performed only for the "winning" unit w, which we take to be the unit whose weight vector has the largest inner product with the input pattern x. Adaptation usually consists of taking a linear combination of the current weight vector and the input vector. The two most common rules are

w'

=w

+tX

Neural Computation 6, 255-269 (1994)

(1.11 @ 1994 Massachusetts Institute of Technology

Geoffrey J. Goodhill and Harry G. Barrow

256

and WI =

w + ( ( X - w)

(1.2)

Consider the general case WI

= nw

+

FX

(1.3)

where a = 1 for rule 1.1 and a = 1 - for rule 1.2. For a particular normalization constraint, e.g. llwll = L, there are various ways in which that constraint may be enforced. The two main approaches are

w

= wl/rr

(1.4)

w

= w’ - ijc

(1.5)

and

where c is a fixed vector, and (1 and /;’ are calculated to enforce the constraint. For instance, if the constraint is llwll = L then (L = Ilw’ll/L. The simplest case for c is c, = 1 Vi. We refer to rule 1.4 as ”divisive” enforcement, since each weight is divided by the same amount so as to enforce the constraint, and rule 1.5 as ”subtractive” enforcement, since here an amount is subtrncted from each weight so as to enforce the constraint. It should be noted that the qualitative behavior of each rule does not depend on the value of a. It is straightforward to show that any case in which R # 1 is equivalent to a case in which n = 1 and the parameters t and L have different values. In this paper, therefore, we will consider only the case a = 1. The effect of these two types of enforcement on a model for ocular dominance segregation, where development is driven by the timeaveraged correlation matrix of the inputs, was mentioned by Miller (1990, footnote 24). Divisive and subtractive enforcements have been thoroughly analyzed for the case of general linear learning rules in Miller and MacKay (1993, 1994). They show that in this case divisive enforcement causes the weight pattern to tend to the principal eigenvector of the synaptic development operator, whereas subtractive enforcement causes almost all weights to reach either their minimum or maximum values. Competitive learning however involves choosing a winner, and thus does not succumb to the analysis employed by Miller and MacKay (1993, 1994), since account needs to be taken of the changing subset of inputs for which each output unit wins. In this paper we analyze a special case of competitive learning that, although simple, highlights the differences between divisive and subtractive enforcement. We also consider both normalization constraints C,w,= constant and C,4 = constant, and thus compare four cases in all. The analysis focuses on the case of two units (Lee,two weight vectors) evolving in the positive quadrant of a two-dimensional space under the influence of normalized input vectors uniformly distributed in direction.

Weight Normalization in Competitive Learning

257

Table 1: Notation for Calculation of Weight Vectors.

Parameter Description W X

hW

w hw

0

4 U

L d €

Weight vector Input vector Change in weight vector Angle of weight vector to right axis Change in angle of weight vector to right axis Angle of input pattern vector to right axis Angle of enforcement vector to right axis Angle of normal to constraint surface to right axis Magnitude of the normalization constraint llxll (constant) Learning rate

Later it is suggested how the conclusions can be extended to various more complex situations. It is shown that, for uniformly distributed inputs, divisive enforcement leads to weight vectors becoming evenly distributed through the space, while subtractive enforcement leads to weight vectors tending to the axes of the space. 2 Analysis

The analysis proceeds in the following stages: (1) Calculate the weight change for the winning unit in response to an input pattern. (2) Calculate the average rate of change of a weight vector, by averaging over all patterns for which that unit wins. (3) Calculate the phase plane dynamics, in particular the stable states. 2.1 Weight Changes. The change in direction of the weight vector for the winning unit is derived by considering the geometric effect of updating weights and then enforcing the normalization constraint. A formula for the change in the weight in the general case is derived, and then instantiated to each of the four cases under consideration. For convenience the axes are referred to as "left" (y axis) and "right" ( x axis). Figure 1 shows the effect of updating a weight vector w with angle w to the right axis, and then enforcing a normalization constraint. Notation is summarized in Table 1. A small fraction of x is added to w, and then the constraint is enforced by projecting back to the normalization surface (the surface in which all normalized vectors lie) at angle 4, thus defining the new weight. For the squared constraint case, the surface is a circle

258


Figure 1: (a) The general case of updating a weight vector w by adding a small fraction of the input vector x and then projecting at angle 4 back to the normalization surface. (b) The change in angle w, hw, produced by the weight update.

centered on the origin with radius L. For the linear constraint case, the surface is a line normal to the vector (1, l), which cuts the right axis at (L.0). When E is very small, we may consider the normalization surface to be a plane, even in the squared constraint case. For this case the normalization surface is normal to the weight vector, a tangent of the circle. For divisive enforcement, the projection direction is back along w’, directly toward the origin. For subtractive enforcement, the projection direction is back along a fixed vector c, typically (1.1).


259

Table 2: Value of 6w for winning unit. Constraint Enforcement

Equivalences

nw

Referring to Figure la, consider hw = f x - i-lc. Resolving horizontally and vertically and then eliminating ijllcll yields sin(8 - 4)

IlWl =

- f I ~ X ~ I C O S (fJ

(2.1)

4)

Now referring to Figure lb, consider the change in angle w,hw: llw11hw = -11SWII cos(0 - w )

which in conjunction with equation 2.1 gives

hw

=

sin(0 - 4)cos(O - w ) llwll c o s ( r - 4)

fIIXII ~

For the squared constraint case llwll case

llwll

=

L ficos((r

-

= L,

(2.2) whereas in the linear constraint

d)

For divisive enforcement d = w,whereas for subtractive enforcement 4 is constant. From now on we assume llxll = d, a constant. Table 2 shows the instantiation of equation 2.2 in the four particular cases studied below. An important difference between divisive and subtractive enforcement is immediately apparent: for divisive enforcement the sign of the change is dependent on sign(9 - w),while for subtractive enforcement it is dependent on sign(8 - 4). (Note that cos(w - (I,), cos(z - 4) and cos(: - w)are always positive for w,4 E [O. 51.) Thus in the divisive case a weight vector only moves toward (say) the right axis if the input pattern is more inclined to the right axis than the weight is already, whereas in the subtractive case the vector moves toward the right axis whenever the input pattern is inclined farther to the right axis than the constraint vector.


260

2.2 Averaged Weight Changes. The case of two competing weight vectors w1 and w2 with angles w1 and w2, respectively, to the right axis is now considered. It is assumed that w1 < w2: this is simply a matter of the labeling of the weights. The problem is to calculate the motion of each weight vector in response to the input patterns for which it wins, taking account of the fact that this set changes with time. This is done by assuming that the learning rate f is small enough so that the weight vectors move infinitesimally in the time it takes to present all the input patterns. Pattern order is then not important, and it is possible to average over the entire set of inputs in calculating the rates of change. Consider the evolution of w,, i = 1.2. In the continuous time limit, from equation 2.2 we have

Using the assumption that f is small, an average is now taken over all the patterns for which wi wins the competition. In two dimensions this is straightforward. For instance consider w1: in the squared constraint cases w1 wins for all 0 < (wl+ w2)/2. In the linear constraint cases the weight vectors have variable length, and the condition for w1 to win for input H is now IlWlll cos(H - d l )

>

IIw211 cos(0 - iJ2)

where

This yields the condition H < for wI to win for input H . That is, in the linear cases the unit that wins is the unit closest to the axis to which the input is closest, and the weights evolve effectively independently of each other. (Note that we have only assumed w1 < w2, not dl < 7r/4.) First equation 2.2 is integrated for general limits 01 and Hz, and then the particular values of 01 and 02 for each of the cases are substituted. We have (2.3)

where the angle brackets denote averaging over the specified range of H , and P( 0) is the probability of input 0. The outcome under any continuous distribution can be determined by appropriate choice of P(H). Here we just consider the simplest case of the uniform distribution P(H) = p , a constant. With some trigonometrical manipulation it follows that

2tdp cos(a - W i )

(;if)= -

((willC O S ( ~-

41)

(

O1 02) sin sin 4 - -

(y)

(2.4)


261

2.3 Stable States.

2.3.1 Linear Constraints. Substituting the limits derived above for linear constraints into equation 2.4 yields for the divisive enforcement case

where for conciseness we have defined C = 2 ~ d p / L .To determine the behavior of the system the conditions for which (&I) and (&) are positive, negative, and zero are examined. It is clear that w1 moves toward the right axis for w1 > ~ / 8 w,2 moves towards the left axis for w2 < 3 ~ 1 8 , and the stable state is ?l

q=-.

8

w2=-

371 8

Each weight captures half the patterns, and comes to rest balanced by inputs on either side of it. Weights do not saturate at the axes. This behavior can be clearly visualized in the phase plane portrait (Fig. 2a). For the subtractive enforcement case

For (GI) < 0, that is w1 heading for the right axis, it is required that 4 > x / 8 . Similarly for w2 to be heading for the left axis it is required that d, < 3x18. Thus the weights saturate, one at each axis, if x / 8 < d, < 3 ~ 1 8 . They both saturate at the left axis for d, < 7 r / & and both at the right axis . plane portraits for some illustrative values of 4 are for 4 > 3 ~ / 8 Phase shown in Figure 2b-d. 2.3.2 Squared Constraints. Instantiating equation 2.4 in the divisive enforcement case yields

(G2)

=

Csin

4

1


262

-

a

0 2

0 2

n/4

0

b

rd2

C

d

Figure 2: Phase plane portraits of the dynamics for linear constraint cases. . Subtractive en(a) Divisive enforcement: weights tend to ( i ~ / 8 , 3 ~ / 8 )(b,c,d) forcement for = a/4,+ = ~ / 6and , d = ~ 1 1 6respectively. , For T / 8 < + < 37r/8 weights saturate one at each axis, otherwise both saturate at the same axis.

+

+

For (GI) < 0 we require 3wl > w2, for (ij2) > 0 we require 3w2 < w1 a n d the stable state is the same as in the linear constraint, divisive

K,

enforcement case: K LJ1=-,

8

3K w2=8

The phase plane portrait is shown in Figure 3a.


a

263

b 0 2

0 7

XI4

d

C

Figure 3: Phase plane portraits of the dynamics for squared constraint cases. (a) Divisive enforcement: weights tend to ( s / 8 , 3 ~ / 8 ) . (b,c,d) Subtractive enforcement for @ = s/4, @ = s/6, and @ = x/16 respectively. For 4 = ir/4 weights saturate at different axes. As @ moves from ~ / 4 there , is an increasing region of the (w1, w2)plane for which the final outcome is saturation at the same axis. In the subtractive enforcement case we have (&I)

=

-c cos($1

-

(W2) =

c

1 C O S ( 4 - w2)

For (Wl)< 0 we require

4

w1

>

w1)

+ w24

w1

+ w 2 + ") sin ( 4

wf :1 -

")


264

Table 3: Convergence Properties of the Four Cases.

Constraints Divisive

Weights stable at Weights saturate at different axes for

Linear

LJl

=;.

w * = 3.

i

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

No title

Recommend Documents