Neuro-informatics and Neural Modelling (Handbook of Biological Physics)

Neuro-Informatics and Neural Modelling HANDBOOK OF BIOLOGICAL PHYSICS Series Editor: A.J. Hoff Volume 1" A: B: ...

Author: F. Moss | S. Gielen

46 downloads 864 Views 51MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

Neuro-Informatics and Neural Modelling

HANDBOOK

OF

BIOLOGICAL

PHYSICS

Series Editor: A.J. Hoff

Volume 1" A: B:

Structure and Dynamics of Membranes From Cells to Visicles Generic and Specific Interactions

Volume 2:

Transport Processes in Eukaryotic and Prokaryotic Organisms

Volume 3:

Molecular Mechanisms in Visual Transduction

Volume 4:

Neural-Informatics and Neural Modelling

Neuro-Informatics and Neural Mo delling Editors: E Moss

Center for Neurodynamics, University of Missouri at St. Louis St. Louis, MO 63121, USA S. Gielen

Department of Medical Physics and Biophysics, University of Nijmegen 6525 EZ Nijmegen, The Netherlands

2001 ELSEVIER Amsterdam

- London

- New York-

Oxford

- Paris - Shannon

- Tokyo

ELSEVIER SCIENCE B.V. Sara Burgerhartstraat 25 P.O. Box 211, 1000 AE Amsterdam, The Netherlands

9 2001 Elsevier Science B.V. All rights reserved. This w o r k is protected u n d e r c o p y r i g h t by E l s e v i e r Science, and the f o l l o w i n g terms and c o n d i t i o n s a p p l y to its use: Photocopying Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish to make photocopies for non-profit educational classroom use. Permissions may be sought directly from Elsevier Science Global Rights Department, PO Box 800, Oxford OX5 1DX, UK; phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: [email protected]. You may also contact Global Rights directly through Elsevier's home page (http://www.elsevier.nl), by selecting "Obtaining Permissions'. In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA; phone: (978) 7508400, fax: (978) 7504744, and in the UK through the Copyright Licensing Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London WIP 0LP, UK: phone: (+44) 207 631 5555: fax: (+44) 207 631 5500. Other countries may have a local reprographic rights agency for payments. Derivative Works Tables of contents may be reproduced for internal circulation, but permission of Elsevier Science is required for external resale or distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and translations. Electronic Storage or Usage Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or part of a chapter. Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher. Address permissions requests to: Elsevier Global Rights Department, at the mail, fax and e-mail addresses noted above. Notice No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made.

First edition 2001 Library of Congress Cataloging in Publication Data A catalog record from the Library of Congress has been applied for.

ISSN: 1383-8121 ISBN: 0 444 50284 X Q The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper). Printed in The Netherlands.

General Preface

Biological Physics encompasses the study of the processes of life with physical concepts and methods based on the laws of nature, which are assumed to be equally valid for living and dead matter. A multidisciplinary approach brings together elements from b i o l o g y - knowledge of the problem that is attacked - and from the physical sciences- the techniques and the methodology for solving the problem. In principle, Biological Physics covers the physics of all of biology, including medicine, and therefore its range is extremely broad. There clearly is a need to bring order to the growing complexity of research in Biological Physics. The Handbook of Biological Physics answers this need with a series of interconnected monographs, each devoted to a certain subfield that is covered in depth and with great attention to the clarity of presentation. The Handbook is structured such that interrelations between fields and subfields are made transparent. Evaluations are presented of the extent to which the application of physical concepts and methodologies (often with considerable effort in terms of personal and material input) have advanced our understanding of the biological process under examination, and areas in which a concentrated effort might solve a long-standing problem, are identified. Individual volumes of the Handbook are devoted to an entire "system" unless the field is very active or extended (as e.g. for membranes or vision research), in which case the system is broken down into two or more subsystems. The guiding principle in planning the individual volumes is that of going from simple, welldefined concepts and model systems on a molecular and (supra)cellular level, to the highly complex structures and functional mechanisms of living beings. Each volume contains an introduction defining the (sub)field and the contribution of each of the following chapters. Chapters generally end with an overview of areas that need further attention, and provide an outlook into future developments. The first volume of the Handbook, Structure and Dynamics of Membranes, deals with the morphology of biomembranes and with different aspects of lipid and lipid-protein model membranes (Part A), and with membrane adhesion, membrane fusion and the interaction of biomembranes with polymer networks such as the cytoskeleton (Part B). The second volume, Transport Processes in Eukaryotic and Prokaryotic Organisms, continues the discussion of biomembranes as barriers be-

vi

General Preface

tween the inside of the cell and the outside world, or between distinct compartments of the cellular inner space, across which a multitude of transport processes occur. The third volume, Molecular Mechanisms of Visual Transduction, extends the scope of the previous volumes to perhaps the most intensely studied signal transduction process, visual transduction. The molecular mechanisms of phototransduction in vertebrates and invertebrates are treated in depth. The structure and comparative molecular biology of numerous visual pigments are discussed. The primary photoreactions in rhodopsin and in the microvillar and ciliary photoreceptors of invertebrates are examined and compared. Finally, the visual processes in insect photoreceptors are modelled in detail. The present volume, Neuro-Informatics and Neural Modelling, is perhaps the most ambitious of the Handbook volumes so far. It addresses the next step on the road from sensory and transduction phenomena to perception and consciousness, viz. neural information transmission, and the storage and decoding of this information by neural networks. The volume is divided into two sections, one on the dynamics associated with non-linear processes in a complex neural system, and one on structure and properties of single neurons and neural networks. Information coding and decoding in a stochastic environment is treated in detail. The dynamics of learning, data storage and retrieval are analyzed, and learning rules designed. It is shown that non-trivial brain anatomy emerges in a natural way from the combination of stimulus properties and learning rules. In the other section, it is shown that noise and fluctuations are natural to all biological systems, and often enhance rather than corrupt neural signal transmission and processing. Chaos is introduced in neural systems and thoroughly analyzed. Synchronization on various levels, such as single ion channels in cell membranes or populations of spiking neurons, is discussed and the tantalizing idea put forward that synchronized coherent structures in the brain may play a role in information transmission. AII chapters have been written following a tutorial approach, providing a sound conceptual and theoretical framework. Each chapter focuses on a particular research topic; together they provide senior and junior researchers a challenging overview of the field, which, we hope, will inspire exciting new research in the years to come. Planned volumes

The "bottom-up" approach adopted for individual volumes of the Handbook, is also the guideline for the entire series. Having started with two volumes treating the molecular and supramolecular structure of the cell, Volume 3 is the first of several volumes on cellular and supracellular systems. The present volume, No. 4, on neuroinformatics and neural modelling provides the bridge between sensory cellular input, perception and cognitive processes. The next two planned volumes are on Molecular Motors as chemo-mechanical transduction devices, and on Biological Electron Transport processes. Further planned volumes are: * V i s i o n - perception, pattern recognition, imaging 9 The vestibular system

General Preface

vii

Hearing 9 The cardio-vascular system, fluid dynamics and chaos 9 Electro-reception and magnetic field effects Further volumes will be added as the need arises. We hope that the present volume of the Handbook will find an equally warm welcome in the Biological Physics community as the first three volumes, and that those who read these volumes will communicate their criticisms and suggestions for the future development of the project. 9

Leiden, Spring 2001 Arnold J. Hoff Editor

This Page Intentionally Left Blank

Preface to Volume 4 Neuro-Informatics and Neural Modelling

How do sensory neurons transmit information about environmental stimuli to the central nervous system? How do networks of neurons in the CNS decode that information, thus leading to perception and consciousness? These questions are among the oldest in neuroscience. Quite recently, new approaches to exploration of these questions have arisen, often from interdisciplinary approaches combining traditional computational neuroscience with dynamical systems theory, including nonlinear dynamics and stochastic processes. In this volume, we present in two sections a selection of contributions about these topics from a collection of wellknown authors. One section highlights some insights that have recently developed out of the nonlinear systems approach. The second section focuses on computational aspects from single neurons to networks with a major emphasis on the latter. All authors were instructed to prepare papers following a tutorial approach providing a conceptual and theoretical framework, which can serve as a standard for researchers in this field for at least the next ten years. Following the general tutorial introduction, each chapter presents a developing focus on particular research topics toward the end, confronting conceptual and theoretical predictions with recent experimental data. The result is a collection of outstanding tutorial papers with highlights from contemporary research. In the first section (edited by Frank Moss), we look at some unique contemporary phenomena arising from dynamical systems approaches, which have sparked recent attention and interest in neuroscience. Here we encounter Stochastic Resonance and Synchronization and learn how a random process, or "noise" as it is often called, can enhance rather than corrupt the detection and/or transmission and processing of information in neural systems. And it is shown that noise and fluctuations are ubiquitous in biological settings at all levels from the subcellular to the organismal level. Next, chaos in neural systems is introduced and questions concerning the meaning and information content of the unstable periodic orbits characteristic of chaotic systems are raised. Tutorials and research on how these orbits are detected, counted and classified topologically in biological systems are presented. Following this, we look at how these orbits are controlled in cardiac p r e p a r a t i o n s - applications that one day might be developed into useful therapies

x

Preface to Volume 4

for the treatment of heart diseases. We then look more closely at synchronization, a general process that is widely found in biological systems, from the subcellular level to the level of populations of neurons in the brain. The role of noise and fluctuations on the quality of synchronization among populations of oscillators represented by single ion channels in cell membranes or populations of spiking neurons giving rise to the data of magnetoencephalographic recordings from the brain are discussed. Finally we come to self-organized criticality- arising from complex systems that have no single time or length scale - and ask if the noisy but coherent structures resulting from such systems can carry or transmit useful information. For example, can the synchronized propagation of noisy coherent structures in the brain transmit useful information? And could the quantitative characterization of the statistics of such structures be developed into a useful diagnostic of brain disease? In the second section (edited by Stan Gielen), we are led to look closely at how single neurons function. These, in turn, show us how well-known properties of brain architecture and functions, like perception, cognition and motor control, can be understood as emergent properties of single neuron behavior and learning in a population of neurons. The first section starts with an extensive description of biological (spiking) neurons and a formal description of dynamical behavior of neurons. This part provides a general framework for the information processing by single neurons in various parts of the nervous system. The aim is to provide a formal representation, which captures the complexity of the biological neuron, yet allows an analytical approach to understand the relation between structure and function of a neuron. The stochastic behavior of neurons necessitates a probabilistic approach to characterize the input-output relation. This brings us to the second part of this section, where we focus on the statistical mechanics as a tool to characterize the behavior of populations of stochastic neurons. After a general broad introduction, we address extensively the storage capacity, convergence, and the stability of attractor states, which are the result of learning processes in neuronal networks with various types of connectivity (full connectivity, sparse connectivity, symmetric and asymmetric connections, connectivity decreasing with distance between neurons). Next, this analysis is extended by studying the dynamics of learning, data storage and retrieval, and the evolution of the states of a neural network. This requires a simultaneous analysis at two time scales: a fast time scale of milliseconds (evolution of the neuronal states) and a slow time scale (seconds to minutes) corresponding to the rate of synaptic plasticity. This is followed by a thorough analysis of learning and how learning rules should be designed in order to guarantee optimal convergence to the desired network performance. The next question to be addressed deals with the efficiency of information storage and transmission by an ensemble of neurons. How is information coded in recruitment, firing rate and synchrony of firing of neurons? How can we interpret the neuronal activity in a population of neurons in terms of sensory stimuli or motor behavior? An answer to this question is absolutely necessary if one wants to gain some understanding of brain function from measured neuronal activity (either single-unit or multi-unit). The overview presented here provides a framework for all future work on this topic. The last part of this section is devoted to the self-organizing properties of neuronal networks with regard

Preface to Volume 4

xi

to connectivity. It explains how the well-structured (for example topographical) organization that is frequently observed in brain structures, emerges in a natural way from the combined effect of stimulus properties and learning rules, leading to non-trivial brain anatomy. Furthermore, it explains how the different architectures in various parts of the brain form the basis for different sensory-motor functions. We hope that students and researchers alike will be challenged and stimulated by the selections collected in this volume. In tutorial fashion the authors do answer many questions. But many more remain unanswered or only partially addressed. We hope the articles in this volume will inspire the readers to consider some of the topics treated and the questions raised for their own research. We promise that such enterprises will be challenging and fruitful. Stan Gielen and Frank Moss July, 2000


Contents of Volume 4

General Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

v

Preface to Volume 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

ix

Contents o f Volume 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xiii

Contributors to Volume 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

xvii

SECTION 1: STATISTICAL AND NONLINEAR DYNAMICS IN NEUROSCIENCE

Stochastic Resonance, Noise and Information in Biophysical Systems 1. 2.

K.A. Richardson and J.J. Collins Electrical Stimulation o f the Somatosensory System . . . . . . . . . . . . L. Schimansky-Geier, V.S. Anishchenko and A. Neiman Phase Synchronization." From Periodic to Chaotic and Noisy . . . . . . .

3.

1

23

P. Arhem and H. Liljenstr6m Fluctuations in Neural Systems: From Subcellular to Network L e v e l s . .

83

Chaos and the Detection of Unstable Periodic Orbits in Biological Systems .

,

K. Dolan, M.L. Spano and F. Moss Detecting Unstable Periodic Orbits in Biological Systems . . . . . . . . .

131

R. Gilmore and X. Pei The Topology and Organization o f Unstable Periodic Orbits in H o d g k i n - H u x l e y Models o f Receptors with Subthreshold Oscillations .

155

Chaos Control in Cardiac and Other Applications .

D.J. Christini, K. Hall, J.J. Collins and L. Glass Controlling Cardiac Arrhythmias: The Relevance o f Nonlinear Dynamics . . . . . . . . . . . . . . . . . . . .

Xlll

205


xiv

o

D.J. Gauthier, S. Bahar and G.M. Hall Controlling the Dynamics o f Cardiac Muscle Using Small Electrical Stimuli . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

229

Synchronization .

,

J.A. White and J.S. Haas Intrinsic Noise from Voltage-Gated Ion Channels: Effects on Dynamics and Reliability in Intrinsically Oscillatory Neurons . . . . . . . . . . . . . . . . . . . . . . . .

257

M. Rosenblum, A. Pikovsky, C. Sch~fer, P.A. Tass and J. Kurths Phase Synchronization: From Theory to Data Analysis . . . . . . . . . .

279

Self Organized Criticality in Biophysical Applications 10.

P. Jung, A.H. Cornell-Bell, M. Dreher, A. deGrauw, R. Strawsburg and V. Trinkaus-Randall Statistical Analysis and Modeling o f Calcium Waves in Healthy and Pathological Astrocyte Syncytia . . . . . . . . . . . . . . . .

323

SECTION 2: BIOLOGICAL PHYSICS OF N E U R O N S AND NEURAL NETWORKS

Biophysical Models for Biological Neurons 11.

12.

C. Meunier and I. Segev Neurones as Physical Objects." Structure, Dynamics and Function . . . .

353

W. Gerstner A Framework for Spiking Neuron Models: The Spike Response Model . . . . . . . . . . . . . . . . . . . . . . . . . . .

469

Introduction to Neural Networks 13. 14. 15. 16.

H.J. Kappen An Introduction to Stochastic Neural Networks . . . . . . . . . . . . . . .

517

A.C.C. Coolen Statistical Mechanics o f Recurrent Neural Networks I - Statics . . . . .

553

A.C.C. Coolen Statistical Mechanics o f Recurrent Neural Networks H - D y n a m i c s . . .

619

J.A. Flanagan Topologically Ordered Neural Networks . . . . . . . . . . . . . . . . . . .

685


xv

Learning in Neural Networks 17. 18.

K. Fukumizu Geometry of Neural Networks: Natural Gradient for Learning . . . . . .

731

J.L. van Hemmen Theory of Synaptic Plasticity . . . . . . . . . . . . . . . . . . . . . . . . . .

771

Information Coding in Neural Networks 19. 20.

21.

A. Treves Information Coding in Higher Sensory and Memory Areas . . . . . . . .

825

C.C.A.M. Gielen Population Coding: Efficiency and Interpretation of Neuronal Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

853

D. Golomb, D. Hansel and G. Mato Mechanisms of Synchrony of Neural Activity in Large Networks . . . .

887

Self-Organisation in Cortex 22.

23.

U. Ernst, M. Tsodyks and K. Pawelzik Emergence of Feature Selectivity from Lateral Interactions in the Visual Cortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . M. Lappe Information Transfer Between Sensory and Motor Networks . . . . . . .

969 1001

Epilogue to Volume 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1043

Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1045


Contributors to Volume 4

V.S. Anishchenko, Nonlinear Dynamics Laboratory, Department of Physics, Saratov State University, Saratov 410026, Russian Federation P. Jlrhem, Agora for Biosystems and Department of Neuroscience, Karolinska Institutet, SE- 171 77 Stockholm, Sweden S. Bahar, Center for Neurodynamics, University of Missouri- St. Louis, 8001 Natural Bridge Rd., St. Louis, MO 63121, USA D.J. Christini, Division of Cardiology, Department of Medicine, Cornell University Medical College, NY 10021, USA J.J. Collins, Department of Biomedical Engineering and Center for BioDynamics, Boston Univesity, Boston, MA 10021, USA A.C.C. Coolen, Department of Mathematics, King's College London, Strand, London WC2R 2LS, UK A.H. Cornell-Bell, Viatech Imaging/Cognetix, 58 Main Street Ivoryton, CT 06442, USA Ao deGrauw, Division of Neurology Children's Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229-3039, USA M. Dreher, Viatech Imaging/Cognetix, 58 Main Street Ivoryton, CT 06442, USA K. Dolan, Center for Neurodynamics, University of Missouri at St. Louis, 8001 Natural Bridge Rd., St. Louis, MO 63121, USA U. Ernst, Institute for Theoretical Physics, University of Bremen, Kufsteiner Str., D-28334 Bremen, Germany J.A. Flanagan, Neural Network Research Center, Helsinki University of Technology, P.O. Box 5400, Fin-02015 HUT, Finland K. Fukumizu, The Institute of Statistical Mathematics, 4-6-7 Minami-Azabu, Minato-ku, Tokyo 106-8569, Japan D.J. Gauthier, Department of Physics, Department of Biomedical Engineering, and Center for Nonlinear and Complex Systems, Duke University, Box 90305, Durham, NC 27708, USA

xvii

xviii


W. Gerstner, Center for Neuro-mimetic Systems, Computer Science Department, EPFL-DI, Swiss Federal Institute of Technology, CH-1015 Lausanne EPFL, Switzerland C.C.A.M. Gielen, Department of Medical Physics and Biophysics, University of Nijmegen, Geert Grooteplein Noord 21, NL 6525 EZ Nijmegen, The Netherlands R~ Gilmore, Department of Physics, Drexel University, Philadelphia, PA 19104,

USA L. Glass, Departments of Physics and Physiology, McGill University, Montreal, Que., Canada D. Golomb, Zlotowski Center for Neuroscience and Department of Physiology Faculty of Health Sciences, Ben Gurion University of the Negev, Be'er-Sheva 84105, Israel G.M. Hall, The Corporate Executive Board, 2000 Pennsylvania Avenue, N.W., Suite 6000, Washington, DC 20006, USA K. Hall, Departments of Physics and Physiology, McGill University, Montreal, Que., Canada D. Hansel, Laboratoire de Neurophysique et de Physiologie du Syst6me Moteur EP 1848 CNRS, Universit6 Ren6 Descartes, 45 rue des Saints P6res, 75270 Paris Cedex 06, France J.S. Hass, Department of Biomedical Engineering, Center for Biodynamics, 44 Cummington Street, Boston, MA 02215, USA P. Jung, Department of Physics and Astronomy and Program for Neuroscience, Ohio University, Athens, OH 45701, USA H.J. Kappen, SNN University of Nijmegen, Geert Grooteplein Noord 21, 6525 EZ Nijmegen, The Netherlands J. Kurths, Department of Physics, University of Potsdam, Am Neuen Palais 10, D- 14415 Potsdam, Germany M. Lappe, Computational and Cognitive Neuroscience Laboratory, Department of Zoology and Neurobiology, Ruhr-University, 44780 Bochum, Germany H. Liljenstr6m, Agora for Biosystems and Department of Biometry and Informatics, SLU, SE-750 07 Uppsala, Sweden G. Mato, Comisi6n Nacional; de Energia At6mica and CONICET, Centro At6mico Bariloche and Instituto Balseiro (CNEA) and UNC), 8400 San Carlos de Bariloche, R.N., Argentina C. Meunier, Laboratoire de Neurophysique et physiologie du Syst6me moteur (EP 1848 CNRS), Universit6 Ren6 Descartes, 75270 Paris cedex 06, France F. Moss, Center for Neurodynamics, University of Missouri at St. Louis, 8001 Natural Bridge Rd., St. Louis, MO 63121, USA


xix

A. Neiman, Center for Neurodynamics, University of Missouri at St. Louis, St. Louis, MO 63121, USA K. Pawelzik, Institute for Theoretical Physics, University of Bremen, Kufsteiner Str., D-28334 Bremen, Germany X. Pei, Center for Neurodynamics, University of Missouri, St. Louis, MO 63121, USA A. Pikovsky, Department of Physics, University of Potsdam, Am Neuen Palais 10, D- 14415 Potsdam, Germany K.A. Richardson, Center for Biodynamics and Department of Biomedical Engineering, Boston University, 44 Cummington Street, Boston, MA 02215, USA M. Rosenblum, Department of Physics, University of Potsdam, Am Neuen Palais 10, D- 14415 Potsdam, Germany L Segev, Department of Neurobiology, Institute of Life Sciences, Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel C. Schi~fer, Centre for Nonlinear Dynamics, Department of Physiology, McGill University, 3655 Drummond Street, Montreal, Que. Canada H3G 1Y6 L. Schimansky-Geier, Institut ftir Physik, Humboldt-Universitat zu Berlin, Invalidenstr 110, D- 10115 Berlin, Germany M.L. Spano, NSWC, Carderock Laboratory, 9500 MacArthur Blvd., Code 681, W. Bethesda, MD 20817, USA R. Strawsburg, Division of Neurology, Children's Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229-3039, USA P.A. Tass, Institute of Medicine (MEG), Research Centre Julich, D-52425 Julich, Germany A. Treves, SISSA, Cognitive Neuroscience, Trieste, Italy V. Trinkaus-Randall, Department of Opthalmology, Boston University, School of Medicine, 80 E Concord Str., Boston, MA 02118, USA J.L. Van Hemmen, Physik Department der TU Mtinchen, D-85747, Garching bei Mtinchen, Germany J.A. White, Department of Biomedical Engineering, Boston University, 44 Cummington Street, Boston University, Boston, MA 02215, USA


CHAPTER 1

Electrical Stimulation of the Somatosensory System K.A. R I C H A R D S O N

and J.J. C O L L I N S

Center for Biodynamics and Department of Biomedical Engineering, Boston University, 44 Cummington Street, Boston, MA 02215, USA

9 2001 Elsevier Science B.V. All rights reserved

Handbook of Biological Physics Volume 4, edited by F. Moss and S. Gielen

Contents

1.

Introduction

2.

Somatosensory anatomy ..........................................

3.

4.

.................................................

2.1.

Skin s t r u c t u r e

2.2.

R e c e p t o r types

3 3

.............................................

3

............................................

4

Electrical s t i m u l a t i o n o f the s o m a t o s e n s o r y system . . . . . . . . . . . . . . . . . . . . . . . . . .

6

3.1.

Electrical e x c i t a t i o n of p e r i p h e r a l nerve fibers . . . . . . . . . . . . . . . . . . . . . . . . . .

6

3.2.

C o n v e r g e n c e o f electrical a n d m e c h a n i c a l s t i m u l a t i o n

C o m p u t a t i o n a l studies

.....................

...........................................

7 8

4.1.

M o d e l i n g basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4.2.

Our computational model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4.3.

Point current source excitation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

4.4.

C i r c u l a r electrode e x c i t a t i o n

15

4.5.

Electrical noise e x c i t a t i o n

Appendix A References

....................................

......................................

.................................................

.....................................................

8 10

18 19 21

1. Introduction According to the theory of stochastic resonance (SR), the ability of certain nonlinear systems to detect weak signals can be enhanced with the addition of noise. SR has been demonstrated in a wide variety of biological systems [1-9], including the human somatosensory system [10-12]. Previous studies dealing with SR in biological systems have largely employed noise that is of the same modality as the stimulus. For example, we have used random vibrations to enhance the ability of human subjects to detect weak mechanical cutaneous stimuli [10,11]. However, if a noise-based technique is to be used to enhance tactile sensation in humans, it is possible that the noise will be introduced electrically via stimulation systems, such as glove electrodes and sock electrodes. Accordingly, in a follow-on psychophysical study with healthy young subjects, we examined the effects of electrical input noise on the detectability of weak mechanical cutaneous stimuli [12]. We found that the ability of an individual to detect a subthreshold mechanical cutaneous stimulus can be significantly enhanced by introducing a particular level of electrical noise. This work has laid the foundation for the development of noise-based sensory prosthetics for individuals with elevated sensory thresholds, such as older adults, stroke patients, and diabetic patients with peripheral neuropathy. To optimize the development of such devices, it is important to have an understanding of the effects of electrical stimulation on the somatosensory system. This chapter will explore this topic in detail. In Section 2, we briefly describe the anatomy of the somatosensory system, as it relates to electrical stimulation. In Section 3, we review some of the key neurophysiological and psychophysical studies that have involved somatosensory stimulation. In Section 4, we discuss our computational studies that explore the effects of electrical stimulation on cutaneous mechanoreception.

2. Somatosensory anatomy 2.1. Skin structure

In human glabrous skin, there are two major tissue layers: the dermis and the epidermis, as seen in Fig. 1 [13]. The outer layer, the epidermis, is constructed mostly of stratified epithelium cells. The dermis, which lies below the epidermis, is made of dense connective tissue that contains nervous and vascular networks [14]. The dermis structure follows the undulations of the epidermis and forms papillae, which are finger-like projections up into the epidermis. Nerve endings are found in both the dermis and epidermis layers; the location of each receptor termination depends on the receptor type. Free nerve endings travel through the connective

4

K.A. Richardson and J.J. Collins

,~ 9

Hairy skin-~-.--~-Glabrous skin~

,

|

|

J

|

|

i

i

|

Epidermal- ~,~..~,...~"~;,"?~.~.:~ri.:.? dermal b~der Free nerve ~,"~. ~~, ~ ..,, Mndi~ngner,s~ ' " corpuscle ~ Hair f o l l i c l e ~ ;~,~.~2

:.:~

..: . ~4ii;:,.~)~i.~!" "

~i,

9

~""~

i.:I

[ii~J ,, Dermis i.".I~ii~ .

Pacinian

' .,,~.~~~.~!'

""

.

.

.

.

.

.

~"

ending Fig. 1. Skin anatomy: a diagram of the skin found at the human fingertip. Shown are the two major layers of the skin, along with the different receptors found in each layer. Adapted from Ref. [13]. tissue of the upper portion of the dermal papillae and enter the epidermis. Other fibers terminate in various depths of the dermis. Fibers that terminate in the upper part of the dermis do so in the upper part of the dermal papillae [15].

2.2. Receptor types There are four basic types of mechanoreceptors found in human glabrous skin. These receptors are grouped according to their adaptation properties. The Pacinian corpuscles (PCs) and the Meissner corpuscles adapt rapidly to sustained stimuli and encode information about transient stimuli, such as vibration or flutter. The Merkel cells and Ruffini's endings adapt slowly to sustained stimuli and encode information about spatial discrimination and the presence of a signal [16]. Each of these receptor types is innervated by large diameter myelinated fibers. The transduction of mechanical stimuli is thought to occur through stretch sensitive gates at the nerve terminus [17]. Information about this process has been extracted largely from work on PCs. Mechanical stimuli induce a stretch on the membrane of the receptor terminal. This causes the stretch-gated ion channels in the membrane to open, which results in a local depolarization of the cell. This local potential is graded so that a more intense stimulation causes a larger depolarization over a larger area of the receptor membrane. Local potentials set up on the membrane can then summate both temporally and spatially. Once the potential reaches a certain value, the receptor generates an action potential and the mechanical signal is transduced into an electrical signal. This signal is then centrally processed to obtain information about the original mechanical stimulus. It is interesting to note that typically one afferent firing can cause a psychophysical response [16,18].

Electrical stimulation of the somatosensory system

5

The receptors, while similar in transduction elements (i.e., stretch-gated ion channels), are anatomically distinguishable by the terminal structure that contains their transduction elements and by the location of their termination (see Fig. 1). The PCs are large structures found deep in the dermal layer. The axon of each PC terminates inside an encapsulating bulb. The myelinated fiber enters the bulb and ends in an unmyelinated nerve terminal called a neurite. The bulb itself is made of many layers called lamellae, which are responsible for rapid adaptation to stimuli. When a mechanical stimulus is introduced to the receptor, the fluid between the layers redistributes and the neurite no longer experiences a load. The stretch-gated channels are located on the tip of this neurite; the site of action potential generation is further down the neuron, just before myelin begins to ensheath the axon but not before it leaves the capsule. Thus, the site of transduction is within the capsule. The PCs are the most widely studied mechanoreceptor type and thus much is known about how their structure relates to action potential initiation [17]. Although the remaining three receptors- the Meissner corpuscles, Ruffini's endings, and Merkel cells- are not as widely studied as the PCs, general information about their location and terminal structures are available. The Meissner corpuscles are found in the dermal papillae. In young subjects, each papillae contains at least one corpuscle; in older subjects, the distribution of receptors is sparser [15]. The nerve endings of these receptors are encapsulated in a connective tissue sheath, but several neurons may innervate a given capsule. Once the myelinated fiber enters the capsule, it loses its myelin sheath, branches and then tangles around other fibers, axon terminals and connective tissue. The Ruffini's endings are also encapsulated receptors found in the dermal layer. The connective tissue compartment of each Ruffini's ending encloses the axon and branching terminal nerve fibers. Finally, the Merkel cells are found in the epidermis. The lower part of the cell is in direct contact with a nerve terminus that contains many vesicles. It is thought that the Merkel cells function as a transducer, communicating via a chemical transmitter to the nerve terminal directly below it [ 14,15,19]. There are also other types of fibers, in addition to these mechanoreceptors, innervating the glabrous skin of the hand and foot. These include smaller fibers, both myelinated and unmyelinated, that are involved in nociception and thermoreception. The nociceptors are specialized to different types of injurious stimuli. For instance, there are mechanical, thermal and chemical nociceptors. These are highthreshold units that are found throughout the dermis [20]. In general, the foot and hand (the major areas of interest for noise-based prosthetics) contain a wide variety of receptor types with different stimulus specificities. These receptors are found in various tissue layers in the skin and have different terminal structures. Such features may affect the ability of electrical stimulation to excite these endings. For example, the capsules that encase some of the endings may act as insulation. Alternatively, a superficial receptor may be excited by proximity alone. These issues need to be taken into account when considering the ability of electrical stimuli to excite various receptors and fiber types.

6


3. Electrical stimulation of the somatosensory system

3.1. Electrical excitation of peripheral nerve fibers Sensory effects from electrical stimulation have been reported since the 1700s, and systematic experiments on this topic have been conducted since at least the early 1900s [21 ]. Of note, there have been many investigations that have studied the effects of electrical stimuli on somesthetic sensation. Several of these studies have shown that the ability to excite certain receptors is related to the anatomical structure and location of a given receptor and the configuration of the stimulating electrode. Garnsworthy et al. [22], for instance, conducted a series of animal experiments and found that most unmyelinated fibers, at threshold, can be activated with low-current, high-voltage stimulation from needle electrodes placed on the surface of the skin. The stimulation selectively excites unmyelinated C fibers (which are of small diameter) rather than the large diameter myelinated fibers. This is probably due to the fact that C fibers terminate more superficially and are recruited first, based on proximity to the localized stimulating electrode. Low-threshold C fibers are not present in humans, so it is presumed that the poly-modal C fibers and possibly the A8 (smallest diameter of the largest fibers) nociceptive fibers would be activated in humans during electrical stimulation. These two types of human receptors have been shown to have similar properties. Pfeiffer [21], in a 1968 review article, noted that according to the studies performed by Von Frey, a small electrode causing a localized current density is best for stimulating individual receptors. This type of electrode configuration typically results in stinging pain, a temperature sensation, or a vibrating pressure sensation, depending on the type of receptor that is stimulated. When a large area is stimulated, the sensation is that of hammering or buzzing, and it is felt to be deep in the skin and non-painful, until higher intensities. The notion of gating is commonly invoked to explain the flow of different types of information in the somatosensory system. It has been hypothesized that there is a pain gate that modulates the input of nociceptive information at the level of the spinal cord, and that tactile input inhibits information transmission from nociceptors. Apkarian et al. [23] examined the reciprocal form of control; they found that painful heat can reduce tactile sensitivity (i.e., increase tactile sensory thresholds), suggesting the presence of a touch gate. The increased thresholds were not due to shifts in attention, as auditory thresholds were unaffected by the heat pain. Kauppila et al. [24] also investigated the possibility of a touch gate. They found that pain causes an increase in the mechanoreceptive fields, which results in a loss of two-point discrimination. If there is a touch gate and electrical stimulation excites nociceptors, then there may be an inhibitory effect, depending on the extent of the excitation of the nociceptors. These receptors generally have high thresholds, but they are superficial and may be excited by proximity alone [22]. Many aspects of these studies are relevant to the development of noise-based sensory prosthetics. For instance, according to the work of Garnsworthy et al. [22], Von Frey [21] and Adrian [21], the location of nerve excitation (receptor or nerve


7

track) is dependent on the configuration of the stimulating electrode. Localized current (e.g., from a needle electrode) excites the receptor endings that favor excitation of superficial receptors; a wider electrode configuration favors excitation of nerve tracks. Since it is possible that there is inhibition of mechanoreception with nociceptive excitation, it is important, in the context of noise-based sensory prosthetics, to ensure that the employed electrical stimulation does not excite the superficial nociceptors.

3.2. Convergence of electrical and mechanical stimulation Vernon [25] and B6k6sy [26], in the early 1900s, independently examined the effects of electrical stimulation on mechanoreception. Vernon specifically considered the effects of a subthreshold periodic electrical stimulus on vibrotactile detection thresholds. In Vernon's experiments, the vibrotactile stimulus and electrical stimulus were of the same frequency. He found that if the electrical and mechanical stimuli were in phase, then the detection threshold was lower than that for the mechanical vibration alone; if the two stimuli were out of phase, then the detection threshold was not significantly different from that for the mechanical vibration alone. B~k6sy, after performing similar experiments, suggested that the interaction of the two modalities (i.e., electrical stimuli and mechanical stimuli) may be through the nerve tracks and not at the end organs. He also speculated that since the two types of stimulation travel at different speeds through the affected area (the electrical stimulus moving faster than the mechanical stimulus), there may be a spatially varying phase delay between the two signals. B6k~sy also found that if the electrical stimulus and mechanical stimulus are presented at the same sensation magnitude, then the sensation area for the mechanical stimulus is larger. In addition, contrary to Vernon's results, B6k~sy found that the electrical stimulus could almost fully cancel out the mechanical sensation. One of the basic ingredients necessary for SR-type effects is that of information summation, i.e., the noise signal needs to add in some way (not necessarily directly) to the signal of interest. For noise-based sensory prosthetics, this implies that the electrical stimulation needs to enter the system close to the area where the transduction of the mechanical stimulus occurs. Thus, stimulation along the nerve track may not lead to functional enhancement unless the detection task requires central processing. Typically, though, for low threshold receptors such as the PC, a single afferent firing can generate a psychophysical response. An action potential in a sensory neuron occurs when summation of local potentials in the nerve endings exceeds the receptor's threshold. Electrical stimulation may only enhance detection in areas of local potential summation because a subthreshold stimulus may not conduct any information along the nerve track. A limited number of studies on PCs have examined the interactions between local potentials arising from mechanical stimuli and antidromic electrical stimulation [27,28]. The objective of these studies was to identify the site of action potential generation. It was found that in a decapsulated PC, an electrical signal sent antidromically (i.e., toward the receptor) can summate with subthreshold activity from a

8

K.A. Richardson and J.J. Coffins

mechanical stimulus [27]. It was also found that an antidromically initiated action potential can cause depression of the nerve terminal, leading to an increase in the receptor's threshold to mechanical stimuli [28]. These studies indicate that depolarization of a receptor's terminal parts arising from electrical stimulation can interact with end-organ activity. It is clear from previous studies that the cutaneous mechanoreceptive system is tuned to derive certain information from mechanical stimuli. It is not understood, however, exactly how this system responds to electrical stimulation. It is not completely known what types of receptors are excited, where they are excited, and what kind of information is introduced into the system with electrical stimulation. Vernon and B6k6sy generally found that the interaction between electrical stimulation and mechanical stimulation is not simply linear summation. Gray [27] and Hunt et al. [28] indicate that there is indeed local summation of potentials when signals from both modalities meet near the nerve terminal. These studies indicate further work is needed to understand the effects of electrical stimuli on mechanoreceptors. We have initiated computational studies to begin to address this important issue. In the next section, we discuss the modeling background and some of our preliminary results.

4. Computational studies 4.1. Modeling basics 4.1.1. Membrane dynamics Information in the nervous system propagates as a change in membrane voltage down an axon. The axon membrane can be described by a simple electrical circuit composed of capacitors and resistors, as shown in Fig. 2. The stimulating current either flows through the ion channels, modeled as conductances, or it charges the capacitance of the membrane. By solving Kirchoff's law, the following expression for the time-varying change in membrane voltage can be derived: d V/dt = (--lio n "-[- Iapp)/C m. The membrane can be modeled either as a passive system with a

lapp

T! ',on G~ Vr

Cm

1 Fig. 2. Circuit model of neural membrane. This model shows the circuit elements that represent the membrane components. The conductance (Gin) represents the ion channels in the membrane and the capacitance (Cm) is an intrinsic property of the membrane.


9

constant conductance or as an active system with a nonlinear conductance. In the axon of excitable neurons, the ion channels are voltage-gated and are therefore active. When there is a change in the membrane potential, the gates open or close, allowing or disallowing the flow of ions, particular to the channel. In the 1950s, Hodgkin and Huxley provided the first mathematical description of this membrane behavior that accounted for nonlinear conductances [29]. The parameters used in the model for each of the ion channels were derived from fits to experimental data. 4.1.2. E x t e r n a l excitation

Recent computational studies on external excitation of sensory neural fibers have used a modified version of the Hodgkin-Huxley fiber model - the FrankenhaeuserHuxley (FH) m o d e l - to describe the excitable membrane of nerve fibers. Frankenhaeuser and Huxley [30] modeled the membrane as having four major ion channels. In this model, the initial depolarization is due to the influx of sodium ions; potassium ions are responsible for the repolarization of the neuron; a delayed depolarizing current from nonspecific ions is largely mediated by sodium concentrations; and lastly, there is a small repolarizing leakage current that is a linear function of membrane voltage. McNeal [31], in an attempt to model external neuronal stimulation, proposed a model wherein ions could pass through the membrane at specific areas (called nodes) between the insulating myelinated sections. The external stimulation, in this case, was an electric field set up by a current point source located a certain distance above one of the nodes (Fig. 3). McNeal examined subthreshold activity just prior to action potential initiation so that he only modeled the node directly under the electrode with F H dynamics. All other nodes were modeled as passive membranes, as subthreshold activity does not cause large enough depolarizations at these nodes to affect their conductances. Reilly [32] extended this system to include F H conductances at each of the nodes so that properties of the external stimulation could be chosen arbitrarily. He then derived a mathematical equation for the time-varying

lapp

-3

f m ~ ' ~ . i j ,,,~ ~ ! , - 2 ~

~

nodes'

-1

0

,tlF~mm~ ,~:~ ~ .

1

~.~.:~ .

2

~:~-i~~ . . i , ,

3

~...7~I~~1

myelin

Fig. 3. Fiber model used by McNeal [31] and Reilly [32]. The external stimulation is introduced above node 0. Each node is separated by insulating myelin sheaths. In the McNeal model, only node 0 is modeled as active membrane; in the Reilly model, all nodes are modeled with FH nonlinearities.

10


change in membrane voltage by applying Kirchoff's law to the model. The exact form of the resulting equations is presented in Section 4.2.

4.1.3. Activating function Rattay [33-35], using the same model as Reilly, noticed that the effects of the external stimulation are built into the resulting equations for myelinated fibers as the second difference function f = V~.,_l- 2V~., + V~.,+l (see Eq. (7) in Section 4.2.2). This so-called activating function is a good approximation of how well a stimulus can excite the myelinated fiber. The function f depends on the field set up by the electrical stimulation and the membrane properties of the target fiber. The shape of the activating function for a myelinated fiber with a point current source directly above one node is shown in Fig. 4.

4.2. Our computational model As noted above, the voltage difference across the membrane is related to current flow in the system. Current can flow through ion channels, charge the capacitance of the membrane, and when considered in an extended system such as a fiber, flow axially along the axon. The model used for the present studies consists of a modified version of the spatially extended nonlinear node (SENN) model first developed by Reilly [32]. For these studies, we have added a specialized receptor ending to the model. A diagram of the fiber of interest is given in Fig. 5. The receptor (node 0) is modeled as an equipotential sphere with a passive membrane; all other nodes are modeled with FH-type dynamics and are considered to be cylindrical in shape.

4.2.1. Receptor dynamics Typically, mechanoreceptors are thought to have stretch-gated ion channels in their terminal membrane. A mechanical stimulus creates voltage changes across the membrane as the channels open and close in response to the stimulus. These gates (a) 200

(b) ~oo

150

5O

/

100

> E 50 o

~, o g -5o *

-50 -100

" ~ -100 -150

-;-2-'1

6

node

i

2

;

4

-200

- 4 ;_ - 2i

_11

;i

;4

node

Fig. 4. Activating function. The spatial variation of the activating function (f) is shown. (a) Excitation with negative polarity. The solid-line trace (with symbol o) shows lap p ~--- --0.1 mA, and the dotted-line trace (with symbol A) denotes lap p -- --0.05 mA. (b) Activating function for positive polarity stimulation. The solid-line trace (with symbol o) shows lap p - - 0 . 1 mA, and the dotted-line trace (with symbol A) denotes lap p "-- 0.05 mA.


M

c2

11

Ve,1

e , 0

xo axon

c', " ~ V r

V

r

Vi,0

Ve,2

G a

,

yeli

Cm'~~V;

Ga

V',2

Myelin

Fig. 5. Neuron model. The end of the mechanoreceptor (~,0) is modeled as a passive, equipotential sphere with membrane parameters C~ c and G~c. The myelin sheaths between each node are considered electrically insulating. All nodes n r 0 (l~,,) are modeled as excitable cylindrical-shaped membranes with parameters Cax~ and G~ ~ The value of G~ ~ however, is a nonlinear function of time and membrane voltage. Ve,, is the value of the external voltage field induced by the electrical stimulation. The parameter Ga is the axial conductance. operate relatively independently of the receptor voltage and thus the membrane is considered to be electrically passive. The circuit describing this node is shown in Fig. 6. Note that the receptor is the terminal end of the neuron and axial current only flows down the axon (I+, to the right in the diagram). The equation for the sum of the currents through the model is:

/C +/ionmY§

(1)

Since the receptor membrane is electrically passive, the current/ion is modeled with a linear conductance, G~ c. Both the expressions for the ionic current and the axial current are derived from Ohm's law, V = IR, where R is the reciprocal of conductance. The current across the capacitor obeys the constitutive relation for capacitance, I = CdV/dt. The resulting expression for the equivalence of currents is - C ~ c d V o / d t - G~CVo = Ga(~,o - ~,1),

(2)

where V0 is the voltage across the membrane of node 0, C~ c the membrane capacitance, Ga the axial conductance, I~,0 and Vi,1 the internal voltages at nodes 0 and n, respectively, and G~ c is the conductance across the membrane. By noting that ~,, = Ve,, + V~ for all n, the final expression becomes dVo/dt = (1/C~C)[-Ga(-Vo + V~ - Ve,o+ Ve,Z) - G~cvs].

(3)

The value of Ve,, is governed by the external excitation and is related to the shape of the electrode as well as the electrode distance (from the point of interest). The neural transduction of mechanical stimuli is a complicated process involving the mechanical properties of the receptor and specialized ion channels on the

12

K.A. Richardson and J.J. Coll&s

Me,0

T c

~j~l r C ~--4

i t------o

Vi,0 Ga Vi,1 Fig. 6. Circuit diagram for the receptor. A voltage difference across the membrane causes current to flow through the passive conductance, ( T r e c and allows the membrane capacitance, C reC to be charged. These two currents sum to become the axial current, I+, which flows down the axon to node 1 (1~.1). "~" IYl

11

'

'

membrane. We will not include the transduction process in our modeling in order to simplify the generation of local receptor potentials set up by mechanical stimuli; instead, we will simply model the mechanical stimulus as a discrete pulse of a local voltage potential, spatially confined to the receptor node. For the present computational studies, the mechanical waveform will be a square voltage pulse. Eq. (3) then becomes d V o / d t - (1/Cr_mC)[-Ga(-V o + Vl - Ve.o +

Ve.1)

G~cv0 + mstim].

-

(4)

4.2.2. Fiber d y n a m i c s

The end of the receptor is modeled as being the site of mechanical transduction, and therefore it does not behave as active membrane. Local potential information from node 0 leaks to the first node of the axon; it is here that the model allows for the initiation of an action potential. At this node, and all other nodes n, where n -r 0, the membrane is active and contains a number of nonlinear conductances. The detailed circuit diagram for these nodes is shown in Fig. 7. The expression for the current flow across the membrane, which is derived from solving KirchotFs law, is IC ++++lion - I _

+l+.

(5)

With the circuit elements for each current at node n, the expression becomes C ax~ d V , / d t - / i o n

--~ITI

--

G a ( ~ n - ~n-1) + G a ( ~ . - V,,+I) ~

~

~

~

"

(6)


13

V e,n

T

G,

IL---* V l,on Vi,n_ 1 G a Vi,n

G. Vi,n+l

Fig. 7. Circuit diagram for all active nodes. Current induced by the charged membrane capacitance, Ic, adds to the current traveling through the four ion channels. This summed current then can travel either up the axon to node n - 1 or leak down the axon to node n + 1 through the axial conductance Ga.

By again using ld,. = Ve,n + Vn, we obtain d G / d t = (1/cax~

1 - 2 V~ + G+I + Ve,~-I - 2 Ve,, + ge,n+l) - l i o n ] ,

(7)

where lion = gdl ( iNa q- iK -}- ip -}- iL).

(8)

The sodium current, iNa, is responsible for the initial depolarization of the membrane. The potassium current, iK, is responsible for the hyperpolarization of the membrane. The nonspecific current, ip, is a delayed depolarizing current carried mostly by sodium ions, and the leakage current, iL is another hyperpolarizing current. The first three channels are modeled with nonlinear gating functions; the last channel, responsible for the leakage current, is modeled with a passive membrane. The F H expressions for the gated ion channels and the model parameters used in our simulations are included in Appendix A. A total of nine nodes, including the receptor (node 0), were solved in the simulations. The last node, node 8, was held fixed at d V / d t = 0 for the end boundary condition. 4.3. Po&t current source excitation The expression for the activating function is given in Eq. (7) as f = Ve,n_ 1 2Ve,. + Ve,.+l. The term Ve,. represents the field induced by a current source at node n. Node 0 is considered to be the origin of the fiber axis. For a point source, the voltage field set up by the electrical stimulation is given by the expression

14


Ve.,, =

Pelapp

(9)

4 ~ ~Z2' _.[_(Xe _ IlL) 2 '

where Pe is the resistivity of the volume conductor surrounding the fiber, lapp the current source, z the distance between the electrode and the fiber, xe the distance along the fiber, n the number of the node of interest, and L is the internodal distance. The electrical excitation must create a depolarization as close as possible to the receptor site in order that such changes can add constructively with the depolarizations set up by the mechanical stimulus, i.e., regions of depolarization in the activating function (see Fig. 4) must be near the depolarizations resulting from the mechanical stimulus. One of the first modeling predictions that we can make is that electrical stimulation with different polarities results in different system responses, depending on the location of the stimulation. In this study, the mechanical stimulus was modeled as a depolarizing square voltage pulse (duration: 1 ms), introduced at the receptor (node 0). We also included a pulsed current signal (of 1 ms duration) that was assumed to be applied simultaneously by a stimulating electrode. The electrode was held at a perpendicular distance of 0.1 cm above the nerve fiber (z = 0.1 cm). The distance along the fiber (xe) was allowed to vary from - 0 . 6 (behind the receptor, on the opposite side as the axon), to 0.6 cm (over the axon, at node 3). (The location Xe - 0 cm is directly above the receptor). Note that in our simulations, the internodal distance is taken to be 0.2 cm. The amplitude of the electrical stimulus was allowed to vary from -0.1 to 0.1 mA. The minimum amplitude of the mechanical depolarization necessary to generate an action potential (the threshold) was found as a function of the electrode position and the amplitude of the electrical stimulus. The results are shown in Fig. 8. We found that when the polarity of the electrical stimulus is negative, there is an enhancement effect (i.e., the receptor threshold is reduced) only when the electrode is located over the axon (see Fig. 8). The threshold is decreased in this case because the center depolarization caused by the negative stimulation (see Fig. 4a) can add to the depolarization resulting from the mechanical stimulus. If, however, the electrode is shifted to the other side of the receptor (not over the axon), the threshold is increased. This change in threshold is due to the outer hyperpolarizing lobes of the electrical stimulation being located next to the receptor, which suppresses the depolarization resulting from the mechanical impulse. The hyperpolarizing and depolarizing regions of the electrical stimulation are not of equal strengths, and thus the changes in threshold due to each of these regions are not symmetric. We found the opposite effect when the polarity of the electrical stimulus is positive. Specifically, we found that there is an enhancement effect (i.e., the receptor threshold is decreased) when the electrode lies away from the fiber, and there is a detrimental effect (i.e., the receptor threshold is increased) when the electrode is placed over the axon (see Fig. 8). The enhancement effect is present when the outer lobes of excitation, which are depolarizing for positive excitation (see Fig. 4b), are located near the receptor. The increases in threshold, then, are seen when the hyperpolarizing region, directly below the electrode, is located near the receptor and


15

'...

0.8

'....

1,, ~ " 0.5

vE

0.6

; %/,,

0.4

o

0.2

o c.. (a - 0 . 5 (D (. I

"

".

~

-0.1

..

.f,:-

_5> a/(o~l 9A0), where

Phase synchronization: from periodic to chaotic and noisy

43

A0 is the amplitude of the unperturbed Van der Pol oscillator. This condition can be established in the region I of the bifurcation diagram Fig. 10. It is possible to show that in this case the amplitude changes much faster than the phase (to do so we compared coefficients at the linear terms, which defines relaxational time scales). That is why we can substitute qb = const to the first equation of system (19) and use the unperturbed amplitude A0 in the equation for the phase d dt~)- A

~t A-~COS~.

(21)

This equation is one of the canonical in the theory of phase synchronization [59]. It can be re-written in a potential form: ~ =-dU(q~)/dq~ with the potential U ( ~ ) - - A q ~ + ~0sin ~. Therefore, the dynamics of the phase difference q~ can be viewed as the motion of an overdamped particle in the tilted potential U(~) (see Fig. 12). The detuning parameter A determines the slope of the potential and ~t/Ao gives the height of the potential barriers. For A < p/A the minima of the potential dpk =arccos(A.Ao/~t)+2nk exist and correspond to synchronization as the instantaneous phase difference remains constant in time. The instantaneous frequency is constant and matches the driving frequency in the regime of synchronization. Otherwise it changes in time and we have to calculate the mean frequency as (co)- lim~_~ 1/T fo co(t)dt. The dependence of the mean frequency versus detuning is shown in Fig. 13 by the solid line. As clearly seen from this figure, the mean frequency coincides with the external frequency col in a finite range of A. The plateau in Fig. 13 corresponds to the synchronization region. Outside this region, co(t) differs from the external frequency and two frequency oscillations occur. If we increase detuning further then higher-order regimes of synchronization can occur. To study these regimes we introduce the ratio of the driving frequency to the mean frequency of the oscillator, 0 = col/(co) which is also called winding number. This ratio tells how many periods of external force are within one period of the

0 -4n

Fig. 12.

-2g

Schematic potential profile U(qb) in the case of phase locking.

L. Schimansky-Geier et al.

44

0.15

3 ,'7

0.05

j,,?---

/.(""

-0.05

-0.15

-~

~" "

/

1

o oS

D=O.O

- - - D=0.07 -- -- D = 0.02

9

olo oi~ '0'.2 A

.

0.3 . . .0.4 .

Fig. 13. Dependence of the difference between the mean frequency of oscillations in system (18) and the frequency of the external signal vs. detuning parameter for different values of the noise intensity. oscillator. Up to now we study the regime of 1:1 synchronization when 0 - 1. As we already know, this situation corresponds to the existence of the resonance stable limit cycle on a two-dimensional torus. However, with the increase of the driving frequency o~, 0 also increases and can take both rational and irrational values. The structure of the phase trajectories on the torus will undergo bifurcations. Irrational numbers of 0 belong to ergodic motion on the torus. In this case the phase trajectories cover the whole surface of the torus. Rational values of 0 conform to resonant limit cycles lying on the torus surface. Such resonant motion on the torus is unambiguously related to synchronization with locked frequency relations corresponding to the winding number. Some regions of high-order synchronization for different winding numbers are qualitatively presented in Fig. 14a. These regions are called "Arnold tongues". The rational values m:n of the winding number are indicated by the numbers in the plot. As seen additionally from the figure, the tongues are topologically equivalent to the synchronization region at the basic tone 1:1. The phenomenon of synchronization, whose mathematical image is represented by a resonant torus with winding number 0 = m : n, can be described using the circle map. The Poincar6 section along the small torus circle gives rise to a one-dimensional map of the circle to itself. It has the form ~,+, = qb, + f ( ~ , ) ,

f(d~,) =_ f(d?, + 2 rtk).

(22)

Each iteration of the map corresponds to one turn of a phase trajectory along the large torus circle and in general case leads to a shift of a representative point on the circle on a certain angle ~. If a finite number of points is fixed on the circle as

Phase synchronization." from periodic to chaotic and noisy

(a)

'

0.5

b) ~.o

r

5:1

45

0.5

[

--T--

0.9

.

0.4

0.8

0.4

0.3-

0.7 0.6

0.3 i

0.2. 6

0.5 0,4

0.2 !

0.3 0.2

0.0 0.5

Fig. 14.

i

,

1.0

t

,

1.5

i

,

....

2.0

2.5

3.0

I

3,5

,

L

4.(

,,

i

4.5

4

0.1

I

0.0

,,

_

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

(a) Typical resonance regions for indicated values of the winding number; (b) dependence of the winding number vs. detuning parameter.

n --+ ~ , then we can observe the image of a resonant torus. If the number of points is infinite and they cover densely the circle, then we deal with the image of an ergodic torus in the form of invariant circle. The circle map is governed by the following difference equation: K X~+l - xk + ~ - ~ sin(2 ~ xk),

mod 1.

(23)

For K = 0 the parameter 6 represents the winding number, which characterizes the ratio of two frequencies of uncoupled oscillators. If 0 < K < 1, the map (23) may have a period n cycle (n = 1,2,...) even in cases when ~ is an irrational number. In this case we deal with the effect of synchronization. To illustrate this, let us calculate the winding number 0, 0 = lim k~

Xk

-- Xo

k

'

(24)

as a function of the parameter 6 (in addition one needs to exclude the operation mod 1). The results are shown in Fig. 14b and indicate the presence of plateaus, which correspond to the synchronization regions with different rational winding numbers 0 = m :n. The graph in Fig. 14b also demonstrates the property o f selfsimilarity. The self-similarity manifests itself in the fact that between any two plateaus with winding numbers 01 = r :s and 02 = p : q there always exists one more region of synchronization with winding number 0 = (r + p ) / ( s + q). For this reason, the dependence 0(~i) in Fig. 14b is called "devil's stair". On the parameter plane (K, 8) synchronization regions, inside which the winding number 0 = m : n is rational, forms Arnold tongues.


46

3.6. Mutual synchronization: two coupled Van der Pol oscillators So far we were concerned with forced synchronization when the driving influence on the oscillator is unidirectional without a feedback to the force. However, let us imagine the periodic force is originated by a second Van der Pol oscillator and both generators interact but have different natural frequencies m0~ and co02. The interaction is symmetric and we assume that one oscillator is driven additively b y the second one proportionally to the difference of their coordinates and with the strength 7- From the physical point of view it may be realized by a spring with constant 7 which tries to synchronize the motion of the two oscillators. The particular equations are: X, -- 8(1 -- Xl2)XI -+- (021 x ' - - ')t(X 2 -- X 1),

3~2 - c(1 - x2)~2 + m~2x2 -- 7(x, - x2)

(25)

and starting from here we call 7 the coupling parameter. The question which should be sketched here is: Is it possible to observe the effect of synchronization in this case and what are its peculiarities? The answer can be given qualitatively by means of an analysis of the bifurcations in system (25). The structure of the bifurcation diagram for system (25) on the parameter plane is pictured in Fig. 15. As seen from the figure, the bifurcation diagram for the case of the two coupled oscillators is topologically equivalent to the situation in Fig. 14a. From the viewpoint of bifurcation analysis the case of mutual synchronization is completely equivalent to the earlier studied case of forced synchronization.

/

0.5

rp 2.'1

~1.1

0.4 7 0.3

/, //

0.2

// //

0.1

0"%.5

Fig. 15.

,

1.0

1.5 ~

,

2.0 0 ' p l

215

t

,/ ,/ ,

3.0 I

,

35 i

Resonance regions for system (25) on the parameter plane "detuning- coupling". The parameters are p -- m0z/m01, ~ = 2.0.


47

Concluding this section we point out that the analysis of types of synchronization allows us to formulate some fundamental properties and criteria of synchronization. The fact that we first consider the case of forced synchronization is not of principal importance. It is important that the external force is periodic. This follows, in particular, from the qualitative consideration of the dynamics of two symmetrically coupled generators. It is known that oscillations in real generators are periodic but not harmonic. The basic indication of both forced and mutual synchronizations is the appearance of the oscillatory regime with a constant and rational winding number 0 = m : n which holds in some finite region of the system's parameter space. This region is called the synchronization region and is characterized by the effects of phase and frequency locking. The frequency locking means a rational ratio of two initially independent frequencies o l / o 2 = m :n everywhere in the synchronization region. The phase locking means that the instantaneous phase difference is constant in the synchronization region (~ = 0, qbst = const). 4. Synchronization in the presence of noise

The above-considered problems do not take into account the presence of random perturbations. Noise is inevitably present in any real system in the form of natural (or internal) fluctuations caused by the presence of dissipation as well as in the form of random perturbations of the environment. The introduction of a phase in a noisy oscillating system requires a probabilistic approach. With the transformation of the the previous section

x(t) =- A(t) cos @(t),

k(t) = v(t) = -A(t)ol sin ~(t)

(26)

the instantaneous amplitude and phase convert into stochastic variables since x(t) and k(t) are stochastic. With noise taken into account, the amplitude and phase dynamics are described by stochastic differential equations including a noise term ~(t). In the physical literature this type of equation is called a Langevin equation first being developed for Brownian motion. To extract information from stochastic dynamics we have to calculate the moments of A(t), O(t) and o ( t ) = 4)(t) or consider the transition probability density P(A, a#, tlAo, ~o, to) which is sufficient for the Markovian approximations. It gives the conditional probability to observe the amplitude A and the phase 9 at time t if started at time to with A0 and ~0, respectively. In noisy systems the phase ~(t) as well as the difference with respect to external driving ~ ( t ) = O ( t ) - Ost performs motion similar to a Brownian particle in the potential U(qb) (see Fig. 12). The stochastic process qb(t) can be decomposed into two parts: a deterministic part given by its mean value or the mean value of the instantaneous frequency, and a fluctuating part characterized, for example, by the diffusion coefficient around its mean value. Synchronization as a fixed relation between two phases is always interrupted by randomly occurring abrupt changes of the phase difference, also known as phase slips. Therefore, in noisy oscillating systems the notion of synchronization must be mathematically expressed by relations and


48

conditions between the moments of the fluctuating phase or its corresponding probability density. The noise influence on the periodically driven Van der Pol generator was first studied in details by Stratonovich [20]. He considered different types of noise and found conditions of synchronization perturbed by noise. We will restrict ourselves in basic results for weak additive Gaussian white noise.

4.1. Langevin equation description The stochastic force ~(t) is added to the deterministic differential equation of a periodically driven Van der Pol oscillator as 5~- s(l -x2)k +o~2x - a cos(o~,t + qb0) + x/2D~(t).

(27)

For simplicity we argue that the noise is a part of the external driving. We assume ~(t) as Gaussian white noise with zero mean and the new parameter, D, is the noise intensity. Following [20] with the ansatz (26) we can obtain reduced equations for the stochastic amplitude and phase difference: ~ / - -~- 1 -

- g s i n , + 2A~o~ + ~~l(t)'o~, (28)

where ~1,2 are statistically independent Gaussian noise sources: I~i(t)~j(t + ~ ) ) 8i,j8(~) and {~i(t))= 0, where i,j = 1,2. Again let us consider the most interesting situation corresponding to the region I in Fig. 10. With small noise, D 0 H[exp(int)] = - i exp(int),

H[exp(-int)] = i exp int,

(47)

y(t) = A sin(rot). (48)

which again determines the Fourier transform of the kernel of the assumed convolution Kn = - i sign(~).

(49)

In the result one retrieves the Hilbert transform in the shape of a convolution (41) with the kernel (44). As a linear transformation H[x] obeys several useful properties [64]. Every Hilbert transform of a linear superposition of two signals is the superposition of the separate Hilbert transforms. If the time of the signal is shifted by some amount it shifts also the argument of the Hilbert transform. The Hilbert transform of a Hilbert transform gives the negative original signal. Even functions give odd Hilbert transforms and vice versa. The original signal and the Hilbert transform are orthogonal. The full energy of the original signal, the integral of xZ(t) over all times, equals the energy of the transformed one. Hilbert transform can be also performed for stochastic variables. In the case of a stochastic signal x(t) the convergence of this integral should be understood in the

54


mean square sense [63]. The transformed signal is correlated in the same manner as the original signal. But both are anticorrelated with a correlation function being the Hilbert transform of the autocorrelation function of the original signal. The stochastic instantaneous amplitude A(t) and phase ~(t) are thus defined by Eq. (38). Parameters for an effective synchronization can be found from the study of the mean frequency

(0,)

-

rl i~mT

l fr

m(t) dt

(50)

and the diffusion coefficient (35) of the instantaneous phase difference ~ ( t ) 9 (t) - m s t with respect to an inputting periodic signal with ms. The concept of analytical signal found applications in the theory of nonlinear oscillations [64] as an unique technique to separate motions with different time scales. To demonstrate the wide ranging property of the Hilbert transform, let us consider amplitude-modulated signal in the form x(t) = 2q(t)cos(mft), where q(t) = cos(rest) is slowly modulated amplitude with of > COs.The Hilbert transform of x(t) is H{2q(t) cos(oft)] - H{(cos((mf + ms)t) + cos((mf - ms)t))]

= sin((mf + ms)t) + sin((mf- ms)t) = 2q(t)sin(m#).

(51)

Hence, only the fast part of the signal was transformed. A similar expression yields for a high frequency sine-carrier mode. Let A(t) be an amplitude and qb(t) a phase which are not harmonic but slowly varying compared with a carrying frequency mc and the Fourier spectrum is concentrated in a band [So[ < mc/2. How are the instantaneous amplitude and phase of the signal x(t) = A ( t ) c o s ( m e t + ~(t)) defined from the analytical signal representation? One may expand the effective amplitudes of the signal x(t) - A(t) cos 4)(t) cos(met) + A(t) sin ~(t) sin(met) in Fourier modes all having frequencies smaller than me. Hence, every mode obeys the derived property. In result, the Hilbert transform of x(t) y(t) = H[A(t)cos(met + 4)(0)] - A(t) sin(met + 4)(0)

(52)

and the slowly varying parts remain unaffected. 5.2. Periodically driven chaotic systems

In this section we demonstrate the phase of synchronization of chaos on the example of periodically driven oscillator with inertial nonlinearity [58a]: 2 = mx + y - xz + B sin(ml t), p = -x, = - g z + g(x + Ixl)x/2,

(53)

where m, g are parameters of the system. This oscillator is representative of the systems with saddle-focus separatrix loop [65] and a regime of dynamical chaos can be realized in the autonomous case (53) for m = 1.1, g = 0.3. This regime is

55

Phase synchronization."from periodic to chaotic and noisy 0.20 0.15

s~

~- = B=O.04 ~-- -::, B - 0.02

S

s/

0.10 s,r 0.05 0.00 -0.05 -0.10 e

-0.15 -0.20 -0, 5

- 0 10

- 0 05

0.00

0.05

0.10

Fig. 17. The dependence of the difference between the mean frequency of chaotic oscillations and the external signal frequency vs. detuning parameter for different values of the amplitude of periodic force. Other parameters are m - 1 . 1 , 9 - 0.3.

characterized by a b r o a d b a n d power spectrum, however it also contains sharp peaks at a basic frequency m2, its harmonics n0)2 and sub-harmonics co2/2n. We take the external frequency as ml = m2 - A and consider some characteristics of chaotic oscillations for different values of amplitude B and detuning A. The results of calculation of the region of synchronization and the phase difference using the analytic signal concept are presented in Figs. 17 and 19. As seen from these figures, chaotic synchronization occurs for A - -0.02121 and breaks down with the increase of the detuning parameter. Using the definition of mean frequency (50), we calculate a dependence of the difference (0)2) --0)1 VS. A. The mean frequency

(a)

o

,

.

,

9

,

,

,

,

,

,

,

(b)

,

~2 -10

i/,

-20

Jt -50

l

-40

-60

0,0

. . . . . . . . . . . . . . . . . . . 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8

-100

2.0

-150

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

Fig. 18. Power spectra of chaotic oscillations and the external signal inside (a) and outside (b) the synchronization region. Other parameters are: m = 1.1, 9 = 0.3, B = 0.02.


56

200 150 A=0.03171 100

~-

A = -0.02121

0 J -50

~=-0.02421

-100

" ~

-150 -200

Fig. 19.

o

so/~o

'

loo~

'

l~xx,o

2000o0

The instantaneous phase difference for different values of the detuning parameter. Other parameters are m - 1.1, 9 = 0.3, B - 0.02.

locking of chaotic oscillations takes place in a finite range of detuning parameter values. The synchronization region width is determined by the external force amplitude B (see Fig. 17). The effect of frequency locking can be also illustrated in terms of power spectrum. In Fig. 18 we show the power spectra of chaotic oscillations and of external signal for A = 0.01 (a) and A = 0.03 (b). As clearly seen, the basic frequency co2 is tuned up as a result of synchronization and coincides with the external frequency, co2 = col. As we already mentioned, the entrainment of the basic frequency is accompanied by the mean frequency locking and also phase locking. A peculiarity of system (53) is the presence of the basic frequency in the power spectrum which almost coincides with the mean frequency (co2 - (co)). A similar picture is observed in all cases when a chaotic attractor in the phase space of a synchronized system is born in accordance with the Shilnikov's theorem (for example, the case of synchronization of the R6ssler system). Recently the problem of bifurcation mechanisms of transition to phase synchronous chaos has been addressed [66-68] in terms of unstable periodic orbits. In particular, it has been shown [67] that for chaotic systems with broad spectrum of intrinsic time scales perfect synchronization cannot achieved due to existence of periodic orbit with very long periods.

5.3. Synchronization of stochastic systems with continuous output From the previous consideration of the Van der Pol oscillator one may conclude that noise plays a negative role shrinking the regions of synchronization. The period of the limit cycle defines the mean characteristic time of the oscillator. Noise causes deviations from the mean value of this time which becomes more and more widely spread and, in consequence, noise deteriorates synchronization in the Van der Pol oscillator.


57

Oppositely, there are systems whose characteristic time only appears in the presence of noise and in which noise plays a positive role. One example, the noisy Schmitt trigger, we presented already in Section 2. Generally these are dynamical systems with a single or several thresholds which can be surmounted in the presence of noise only. Candidates of such behavior are excitable, bi- and multistable systems. After surmounting(s) and, if the motion is bounded and ergodic, trajectories return to their original value or the neighborhood of it. The characteristic period of such processes is determined by noise. Let us consider the motion of a particle in a potential U(x) dU(x)

m2+ v 2 + ~

= O,

(54)

where m is the mass of a particle and v is the friction coefficient (Stokes friction). For the linear potential U(x) - mo32x2/2i yields with 7 - vim the above-studied linear damped oscillator of Section 3.1. In this section we consider the motion of a particle in a double-well potential U(x) - - a x 2 / 2 + bx4/4. Under the assumption of large friction the inertia of the particle can be neglected (m2 1). However, with the increase of noise the dependence of the mean frequency on a becomes very weak. That is why with the increase of noise from a very low level, the mismatch between characteristic frequencies of elements in the coupled array decreases providing better conditions for mutual synchronization. On the other hand, the noise-induced oscillations becomes more coherent. These effects will tend to facilitate synchronization among elements in the network. Large noise, alternatively, destroys again the coherence of local stochastic oscillations (the frequency and phase fluctuations grow rapidly) and also leads to destruction of spatial coherent structures. The optimal noise intensity at which synchronization is most pronounced depends on the range of the distribution of activation parameters a(n): with the increase of the range of disorder the optimal noise intensity shifts towards smaller values.

6. Synchronization in stochastic point processes 6.1. Phases for discrete events In this section we will deal with synchronization in connection with stochastic point processes. We will interpret changes of the phase in a stochastic system as events at random times t~ which occurrence is due to a distribution function or a dynamical process. For example, the above-considered Schmitt trigger as well as the stochastic bistable system are well described by a point process: in this case the random times tk are moments at which switches from one state to another occur. Thus, we map a continuous process x(t) to a point process tk, where tk are moments of time at which the trajectory of the system crosses some secant plane. The time between two successive crossings is T ( t ) = t k + l - tk, tk < t < t k + l . In this case the instantaneous phase of x(t) is defined as (I)(t)lin - - r C ~t--+ -t~

tk+ l -- tk

rtk,

tk < t < tk+l,

(67)


67

which is a piecewise linear function of time. The mean frequency reads

1 (co)- lim - - ) s

(68)

m-+ocM k=l tk+l -- tk

A second definition of a phase neglects the linear interpolation between two subsequent events taking (I)(t)discr -- g k ( t ) -- ~ ~

O(t - tk)

(69)

k

with k(t) being the sequence of increasing integers and tk is again subject to some dynamics or distribution. 0(x) stands for the Heavyside function. The mean frequency of this definition takes the average of a sequence of delta pulses (o~)(t) = r t ( ~ 8 ( t - k

tk)).

(70)

For the overdamped bistable oscillator k(t) defines a dichotomic process

x(t) = exp[i (I)discr(t)],

(71)

which represents a time sequence of successive states + 1 and - 1 . The sequence of the states may be periodic or random, respectively, we speak about dichotomic periodic and random sequences. Periodic processes additionally require averaging over the initial state to become a stationary process. If the probability that k subsequent events of x(t) have taken place during the time interval [0, t] is given by

P k ( t ) - (Tt)k exp(-Tt) k!

(72)

then x(t) is the Markovian dichotomic process exponentially correlated in time with correlation time tc = 1/3'. Switches occur due to a Poissonian distribution with the mean switching time T = 1/7. Let us compare the definition of the phase (67) and (69) with the analytical signal concept. For this purpose we calculate the Hilbert transform of a dichotomic process x(t) (even k stands for a transition - 1 ~ + 1)

1 t - tk __2~--'ln t - t2k ' ~kt , - 1 ) k In t L tk-y(t) -- --~~-'~' 7"C@ t -- t2k+l

(73)

Assume that at even k's a transition - 1 ~ +1 takes place. For t < tk, therefore,

x(t) = - 1 and the Hilbert transform y(t) decreases monotonously and reaches - o c at t = tk. At this moment x switches to + 1 and y(t) starts to grow becoming +oc at t = tk+l. Thus an arrow following x and y in the phase space fulfills a circle of 2rt during two subsequent transitions and returning to the initial state x = - 1 .

68


Adding one ~ during every transition in definition (39) the instantaneous phase from the analytical signal reads

O'"i,ert

§ arctan;l;I

Fig. 27 shows the dichotomic signal, its Hilbert transform, the instantaneous amplitude and phase. As seen, the instantaneous phase monotonously increases in agreement with the former introduced phases of stochastic point processes. Its mean slope defines the mean frequency of the signal. Hence, we find that the phase concept in stochastic point processes well approximates the findings from the analytical signal. This is presented in Fig. 28 where the three different phases are compared. Both, the piecewise linear and the piecewise constant definition fit well (I)Hilber t f r o m the analytical signal. For simplicity we will later on use the piecewise constant phase.

6.2. Synchronization of noisy bistable systems by stochastic signals Although the majority of SR of studies consider periodic input signals, it was shown that SR can also be observed with nonperiodic signals [42,76]. For such complex signals the effect was called aperiodic stochastic resonance (ASR) [42] and it was shown that ASR can be described in the framework of conventional theory of SR [77]. In this section we show that noisy systems which do not have any deterministic natural frequency can be synchronized by a stochastic stationary driving signal represented by a dichotomic Markovian process. This new type of synchronization [36] will be shown on behalf of a simple but generic kinetic model. A bistable system is driven by two noises: the first is broad-

1 X

-1

~ 1

L ~

Ill L

l

10

20

Y o

A 1

r

10 5 0

-30

-20

-10

0

30

t Fig. 27. Dichotomic signal, its Hilbert transform, the instantaneous amplitude and phase according to (74). Note that y and A reach infinitely large values if the dichotomic input switches.

Phase synchronization."from periodic to chaotic and noisy

69 ..........

i .-|

...............

(~)Hilbert

10

f..~

(~)linear ~dL~crctc

-

i

-

-30

|

,

I

9

-10

i

10

-

30

Fig. 28. Comparison of the three different definitions of the instantaneous phase. Both the linear interpolation as well as the definition by piecewise constant segments well approximate the phase from the analytical signal. band Gaussian noise which represents internal (or thermal) noise, while the second, dichotomic noise, stands for the input signal. The bistable system driven by dichotomic noise is mapped on a simplified 4-state Markovian model where the output is approximated by dichotomic process also. Therefore, within this model we can define phases according to stochastic point processes as in the previous section.

6.2.1. The simplified 4-state Markovian model Let a stochastic bistable system possess two symmetric stable states a(t) = + l which we call the output. The dynamics of transitions between them is characterized by the rate rk (58). Suppose now that Markovian dichotomic process (71) with values d ( t ) = +1 drives the bistable dynamics as an additive input signal. Again we assume that transitions 1 ---+ - 1 occur with the rate ~, which is assumed to be smaller than the intrawell relaxation rate y Q. In this adiabatic approach the rate (58) is modified in dependence on the state of the output ~(t) and the input d(t). Transitions between two states of a are determined by the rates W~_~(d(t)) - rk exp - ~Qcy(t)d(t)J

(75)

70


Hence, with cy(t)d(t) = + l the original rate decomposes into two new rates al - so exp -

au+o) D

'

a 2 - ao e x p ( - -

AU D- Q ) "

(76)

It allows the distinction of the following rate separations: al _< a2

-100 -150 0

100

....

I 200

I 400

t 600

,

.

i 200

t

I

400

600

.

I 800

.

I 1000 msec

.

I 1200

I 1400

l 1600

~ 1800

I

I

I

I

1400

1600

1800

2000

-

50

~>

o

-5o -100 -150

.. 0

.

I 800

.

t

1000 "msec"

I

1200

2000

Fig. 5. Real (top) and simulated (bottom) EEG, showing the complex dynamics of cortical structures. The upper trace is from rat olfactory cortex (data courtesy of Leslie Kay), whereas the bottom trace is from a simulation with the current model of the olfactory cortex. The x-axis shows milliseconds, and the y-axis is in microvolts. From Ref. [74]. circuitry dynamics, and regardless if it is useful to the system or not, it should be important to modulate this dynamics by means of regulatory or control mechanisms of some kind. Many factors influence the dynamical state of brain structures, for example the excitability of neurons and the synaptic strengths in the connections between them. A number of chemical agents, such as neuropeptides, acetylcholine (ACh) and serotonin (5-HT), can change the excitability of a large number of neurons simultaneously, or the synaptic transmission between them (see e.g. Ref. [92]). Such changes normally also result in changes in network dynamics. Other means of regulating the network dynamics, include various feedback mechanisms [93] and fast, nonsynaptic effects (gap junction and/or electromagnetic fields) that could cause synchronization over large cortical areas [94,95].

5.2. Computational approaches Computational methods have since long been used in neuroscience, most successfully for the description of APs, by the work of Hodgkin and Huxley in the early 1950s [27]. Also when approaching the problem of interactions between different neural levels, perhaps in particular for fluctuations at the network level, computa-

Fluctuations in neural systems

107

tional models can prove useful, and sometimes be the sole method of investigation. The main problem for the modeler is to find an appropriate level of description, or level of detail. An appropriate level is one that is sufficient in capturing any particular feature, process or function of the real system, but that does not include more details than necessary for the problem under investigation. In short, the model used should be "as simple as possible, but no simpler". A great number of neural network models have been developed with the aim of capturing some feature(s) of biological neural networks. Such attempts include the so-called multilayer perceptron, the neo-cognitron, and self-organizing feature maps. Many of these models have also been used, more or less successfully, for some kind of pattern recognition. We will not further discuss this type of network models, but refer to any textbook on artificial neural networks (see for example Ref. [96]). A good account of different computational approaches in neuroscience and brain theory is given in Refs. [97,98]. Today, different models cover a range of functions and systems, from early sensory processing and spinal motor control to perception and associative memory, as well as many intermediate processing stages. Many of these models are based on recurrent, attractor neural networks, the most well known of which is the Hopfield net [99,100]. The function of such an associative, or content-addressable memory, is based on the ability to retrieve a pattern stored in memory, in response to the presentation of an incomplete or noisy version of that pattern. The Hopfield net is based on an analogy with the spin-glass model in statistical mechanics, with a large number of identical two state elements, corresponding to electronic spin up and spin down of atoms in a magnetic crystal. In the original Hopfield net, the basic processing unit used is the formal neuron of McCulloch and Pitts [101]. A Hamiltonian, or energy function, for the system gives a multidimensional "energy landscape", which determines the system dynamics. In the most simple case, this landscape is more or less fixed with "valleys" and "ridges" that are statically determined by the network connections, and where the valleys correspond to fixed point attractor memory states. As mentioned above, the dynamics of a biological neural network is not based on point attractor dynamics, but is rich and complex, and seems more associated with limit cycle and chaotic attractor dynamics. Perhaps a more realistic picture than that of a static landscape would be that of a roaring sea, which constantly is changing, and where the memory states rather would correspond to the rolling waves. Alternative models with a more realistic brain dynamics have been developed and investigated, for example for describing the olfactory system, as will be described below (see Ref. [84]). In contrast to the study of neural systems at a microscopic or mesoscopic level, the study of the "macroscopic" dynamics of a biological neural network, it may not be necessary to use very detailed network elements. Instead, more importance could be given to the network structure. For example, in many cases it may be sufficient to model the network elements as single compartment units, with a continuous inputoutput relation, where the output corresponds to the mean firing frequency of a large population of neurons. However, in some cases, such as when the temporal

108

P. ~lrhem and H. Liljenstr6m

relation in the neuronal firing is considered important, spiking network elements, perhaps also with several compartments, would be needed. A good overview of neural computation at the single neuron level is given in Ref. [102].

5.2.1. Simulation results: a model example We have used a cortical neural network model [55,72], resembling the three-layered structure of the olfactory cortex and the hippocampus, for the study of fluctuations at different neuronal levels. The model has one layer of excitatory units, corresponding to pyramidal cells, and two layers of inhibitory units, corresponding to two different kinds of interneurons, slow feedforward and fast feedback, respectively. Network units are mostly modeled with a non-spiking, continuous input-output relation, but many of the model properties and simulation results discussed below have also been reproduced with spiking network units of integrate-and-fire type. The basic model is described in more detail in Appendix B. Simulations with our three-layered cortical model display a range of dynamics found in the olfactory cortex and the hippocampus. In particular, oscillations at different frequencies and spatio-temporal activity waves, but also more complex behavior that can be controlled by various means. Here, we will primarily focus on simulations of noisy and nonregular, chaotic behavior, but we will start by giving a brief description of the oscillatory dynamics of the model. The model accurately reproduces response patterns associated with a continuous random input signal and with a shock pulse applied to the cortex [72]. In the latter case, waves of activity move across the model cortex consistent with corresponding global dynamic behavior of the functioning cortex. A strong pulse gives a biphasic response with a single fast wave moving across the surface, whereas a weak pulse results in an oscillatory response, showing up as a series of waves with diminishing amplitude. For a constant random input, the network is able to oscillate with two separate frequencies simultaneously, around 5 Hz (theta rhythm) and around 40 Hz (gamma rhythm), purely as a result of its intrinsic network properties. Under certain conditions, the system can also display chaotic-like behavior, similar to that seen in EEG traces [3,90] (see Fig. 5, and discussion below). In associative memory tasks, the network initially displays a chaotic-like dynamics, which can converge to a near limit cycle attractor, representing a stored memory (of an activity pattern). Such a case is shown in Fig. 6a. In Fig. 6b two attractors are shown, corresponding to two different memory states. All of these phenomena depend critically upon the network structure, in particular on the feedforward and feedback inhibitory loops, and the long-range excitatory connections, modeled with distance-dependent time delays. Details concerning neuron structure or spiking activity seem not necessary for this type of dynamic behavior. Instead, a balance between inhibition and excitation, in terms of connection strength and timing of events, is necessary for coherent frequency and phase of the oscillating neural units. (There are no inhibitory-inhibitory connections in this model, since there is no clear evidence for their existence in the real system. Yet, in a test simulation such disinhibitory effects resulted in decreased frequencies in the network oscillations. The oscillations ceased completely for large connection strengths between feedback inhibitory units.)

109

Fluctuations in neural systems

a

7.. 6\

........ .... . .

5-.

9 9 ,, 9 . . , . . . -

.

. ,.

..

:..

. .... ........

...

9

9 5, .. .. :

., . . : : , . . . . . . : ~ .

4-. 3.. 2.. 1.. 0.. -1~

9. . .

... ..........

..

. ..

. .. ..:...

.

6

..

5

~

,

.

~ ' ~

4

~ ~ ~ ~ 3

2 2

"~

j 3) strange attractors. This means that the reduction in dimension 5 ~ 3 described above is essential for the topological analysis which will be carried out in the following sections.

6. Classification of strange attractors by integers A powerful theory has recently been developed to classify low-dimensional strange attractors [2]. Strange attractors of three-dimensional chaotic dynamical systems can be classified by their topological properties. These properties are summarized, in turn, by sets of integers. We will describe how to extract these integers from chaotic data. This classification theory extends to N-dimensional dynamical systems provided their strange attractors have Lyapunov exponents which obey ~1 > ~2 = 0 > )~3 > ' ' "

and

-)~3 > )L1.

(15)

The topology and organization of unstable periodic orbits

167

Such strongly contracting dynamical systems have a Lyapunov dimension, dL, which is less than three by the Kaplan-Yorke conjecture [10,11] dL--2+~~2m + 1). At such self-intersections the uniqueness theorem of the theory of ODEs is generally violated. According to this theorem, to reconstruct the dynamics of a three-dimensional system it is sufficient to embed it in a seven-dimensional space. However, it is not necessary, and our experience [2] has shown that it almost always suffices to embed data generated by a three-dimensional dynamical system in a three-dimensional space. We always attempt such an embedding first, and go to higher-dimensional embeddings only when low-dimensional embeddings fail.

176

R. Gilmore and X. Pei

A second embedding procedure is the differential embedding [2,19], where a vector time series y(t) is created from a scalar time series x(t) by x(t) ~ (yl (t),y2(t) = Yl (t),y3(t) = ,v2(t))

(22)

yl (t) - x(t).

Since the data are discretely sampled (x(t) ---+x(ti) = x(i)), the time derivatives are estimated by the usual difference formulas. This embedding is useful for two reasons. First, it already has the desired dynamical system form. Since dyl/dt = y2 and d y z / d t = y3, it is sufficient to determine an equation of motion only for y3. Second, in an embedding using three differentially related variables, it is a simple matter to compute linking numbers of periodic orbits. In Fig. 11 we show a projection of a phase space constructed with the differential embedding (22). The y~ axis is horizontal, the y2 axis is vertical, and the y3 axis is out of the page. Now consider two segments which cross in the upper half plane y2 > 0. The slope of either is given by dy2 = d y z / d t = y 3 . slope - dyl dyl/dt y2

(23)

It is then clear that y3 = (slope)x y2. Thus, the larger the slope, the closer the observer. As a result, all crossings in the upper half plane are left handed with crossing number - 1 , and those in the lower half plane are right handed with crossing number +1. This makes computing linking numbers particularly convenient in this representation of the data (see below). The differential-integral embedding is closely related to (22). The difference is that in this embedding yz(t) - x ( t ) , so that yl(t) - f t ( x ( ~ ) - 2)d~ and y 3 ( t ) - 5c(t). This has the same two virtues as the differential embedding. It is different in the following two ways. As a general rule of thumb, each differential or integral operation on experimental data decreases the signal to noise ratio by an order of mag-

Y2 ~lope= dY' dy, _ dy, l dt _ y~

ay,/at y, Y3 = slope* Y2

Yl Y3 Fig. 11.

Crossing information. In a differential embedding, all crossings in the upper half plane are negative, those in the lower half plane are positive.


177

nitude. Except for very clean data, in the differential embedding the coordinate

y3 - dZx/dt 2 may have an unacceptable signal to noise ratio, since SIN is reduced by two orders of magnitude. Unacceptable means, in this case, that we cannot compute linking numbers. For the differential-integral embedding, both Yl (the integral of x) and y3 (the differential of x) have SIN reduced by only one order of magnitude, which is often acceptable. However, since yl integrates the data (subtracting out the data average), it is susceptible to long-term secular trends. These can produce nonstationarity in the embedded data which are harmless in all other types of embeddings. When this occurs, the nonstationarity must be addressed by appropriate filters [2]. Another method for constructing vector from scalar data involves the singular value decomposition (SVD) [2,20]. This use of the SVD has been discussed in the literature and used effectively in many applications.

7.1. Topological organization of periodic orbits Once a selection of periodic orbits has been extracted from the data and an embedding in R 3 has been adopted, it becomes possible to compute the topological invariants of these orbits. The topological invariants which are always useful are the Linking Numbers of pairs of orbits and the local torsions of individual orbits. If the attractor is contained in a solid torus D 2 • T I (O 2 is the two-dimensional disk and T 1 is the circle) then the relative rotation rates [25] are even more powerful topological invariants than LNs. In a differential embedding, linking numbers of orbit pairs are easily computed by counting the number of crossings in the lower half and upper half planes, subtracting the second from the first, and dividing by 2.

7.2. Identify a branched manifold The next step is to use the topological information gained in the previous step to guess an appropriate branched manifold. For complicated branched manifolds, such as that for the Fig. 8 knot, this is not easy. For branched manifolds which can be embedded in the solid torus D 2 • T 1 the task is simpler. In this case, each branch carries a period one orbit. The topological structure of the branched manifold is determined by: 9 Computing the linking numbers of all the period one orbits with each other. The off-diagonal matrix element T(i,j) is twice this integer. 9 Computing the local torsion of each period one orbit. This gives the diagonal matrix elements T(i, i). 9 Computing the linking numbers of some of the period two orbits with the period one orbits. This gives the array information J(i). In case some of the period one and/or period two orbits are not available, higher period orbits can be used to fill in the missing information. This procedure is not entirely straightforward. The reason is as follows. The periodic orbits in phase space are simple closed curves. The periodic orbits on

178


branched manifolds are labeled by the branches which are traversed. Each periodic orbit is labeled by a sequence of symbols. The problem is to identify a symbol sequence (on the branched manifold) with a closed curve (in phase space). In other words, we need a 1-1 mapping between orbits and symbols. When the return map is very 'thin', as is the case shown in Fig. 4 for the modified Hodgkin-Huxley equations, creating this 1-1 map (a symbolic dynamics) is straightforward. In fact, we did this without even mentioning any problems in Section 5. However, for systems which are not strongly dissipative, creating consistent partitions on appropriate Poincar~ sections in phase space is a longstanding problem [26]. Fortunately, Plumecoq and Lefranc have now proposed a useful solution to the partition problem [26]. 7.3. Validate the branched manifold Once a branched manifold has been proposed (tentatively identified), it is possible to compute the LNs and relative rotation rates of all the periodic orbits which the branched manifold supports. These must be compared with the corresponding topological invariants for all the remaining periodic orbits extracted from the data, which were not used in the first place to identify the branched manifold. If the two sets of topological invariants (one from the orbits extracted from data, one from the corresponding orbits on the branched manifold) agree, then we have added confidence that the initial identification of the branched manifold was correct. If there is not complete disagreement, then either the branched manifold was identified incorrectly, or the partition needs to be modified. The problems of creating a symbolic dynamics (creating a partition and a 1-1 mapping between symbol sequences and periodic orbits in phase space) and identifying the correct branched manifold are global problems. They must be solved simultaneously. There must be complete agreement between the topological invariants of all orbits extracted from data and their corresponding symbol sequences on a branched manifold. This internal self-consistency check (rejection criterion) is absent from both the metric and dynamical approaches to the analysis of chaotic data. Strictly speaking, the topological analysis program stops here. However, there is always the desire to do better: to construct an appropriate model to describe data which have been analyzed. We describe here the next two steps which can be taken in this effort. 7.4. Construct a flow model A dynamical system model has the form dy/dt = F(y), y E R n. To model data, the functions F,.(y) are usually expanded as a linear superposition of some set of basis functions r

dyi

dt = ZAi'~c~a(Y)" o~

(24)


179

This is a general linear model, so standard methods (least squares, maximum liklihood) can be used to estimate the expansion coefficients A;,~. Standard methods (Z2 test) can also be used to test whether this model is any good. For the present purposes, y c R 3. For a differential embedding two of the functions ~ are already known: F1 ( y ) = Y2, Fz(y)= Y3. Only the third function must be modeled. Thus the differential embedding has the added utility that it reduces by a factor of three the effort which is required to develop a model of the dynamics [2]. Once a model has been created, the qualitative validity of the model can be tested. This is done by subjecting its output to a Topological Analysis. If the branched manifolds determined from the data and from the model are not equivalent, the model is not a good representation of the data, and must be rejected. On the other hand, if the branched manifolds are the same, the model cannot be rejected.

7.5. Validate the model A model of a physical process may pass the qualitative test just described, and still not be a very good representation of the dynamics. It would be useful to have some goodness of fit criterion for nonlinear models, analogous to the ;~2 goodness of fit test for linear models. At the present time there is a very useful goodness of fit criterion for nonlinear models. Unfortunately, it lacks a quantitative underpinning. It is hoped that this quantitative underpinning will be supplied during the next decade. The idea behind this goodness of fit test was proposed by Fujisaka and Yamada [27] and independently by Brown et al. [28]. It goes back to an observation by Huyghens made 300 years ago. Huyghens observed that two pendulum clocks on opposite walls gained/lost time at slightly different rates. When they were placed on the same wall close enough they would synchronize their timekeeping. The synchronization effect provides the basis for a nonlinear goodness of fit test. The idea is as follows. Assume that a real physical system satisfies the dynamical system equation • = F(x), and a model for this process is 3~= G(y), x c Rn,y c R ~, where y is supposed to describe x. Then in general, no matter how good the model is, sensitivity to initial conditions and sensitivity to control parameter values will guarantee that the distance between x(t) and y(t) will eventually become large. A perturbation term can be added to the model equation which reduces yi when it gets larger than xi and increases yi when it gets too small. A linear perturbation with this property has the form -)~i(yi - x i ) . The appropriately modified dynamical system becomes

dy~ dt - = ai(y) - ~ i ( Y i - xi).

(25)

If the model is 'good', a small value of )~--()~1,9~2,... ,9~) will cause the model output to follow the data. We then say that the data entrain the model output. The

180


better the model, the smaller the value of k which causes entrainment. The entrainment test has been used effectively to test the validity of some models [2]. Unfortunately, the entrainment test for nonlinear systems has not yet been made quantitative, as has the Z2 test for linear systems.

8. Topological analysis of the modified Hodgkin-Huxley equations The analysis of strange attractors generated by the modified Hodgkin-Huxley equations follows the procedure described in the previous section. The first step is the determination of unstable periodic orbits. This has already been done. It is facilitated using an appropriate first return map. For temperature regions in which the output is periodic (e.g., T ~ 20~ the first return map consists of a single point. For regions in which a strange attractor exists, the first return map has the form shown in Fig. 4. From this map it is possible to locate initial conditions for unstable period one orbits which exist in the strange attractor. The second iterate of the map has been used to locate period two orbits. The pth return allows location of period p orbits. The second step in the topological analysis program is the construction of a useful embedding. One already exists (cf. Fig. 3): it is the projection into the threedimensional inertial manifold (yl, y4, ys). However, we also constructed differential and integral embeddings based on single variables. The strange attractor shown in Fig. 3 in the (yl,y4,ys) embedding is replotted in the differential embedding (Fig. 12a) and integral-differential embedding (Fig. 12b) obtained from the coordinate y4. These embeddings are useful since they simplify the computation of linking numbers. We computed the branched manifold for this system in each of these embeddings, and found it to be embedding independent. The third step is the computation of the topological invariants of the unstable periodic orbits extracted from the strange attractor. The local torsion of the period one orbits was computed by displacing the initial condition slightly from that for the closed orbit, and computing the number of crossings of the closed orbit with its perturbation (which was not closed). The local torsion of the orbit nr is ~ x 2n while that for the orbit nf is ~ x (2n - 1). The relation between this sequence of orbits and local torsions is systematic:

Fig. 12.

Strange attractor of Fig. 3 replotted: (a) in the differential embedding; (b) in the integral-differential embedding.


6f 11

5r 10

5f 9

4r 8

4f 7

3r 6

3f 5

2r 4

2f 3

181

lr 2"

(26)

The local torsions are measured in units of r~ in the series above. The linking numbers of adjacent period one orbits with local torsions nrc and (n + 1)rt is n/2 or (n + 1)/2, whichever is integer. Two branched manifolds are compatible with the information obtained from the period one orbits. We know this because an identical mechanism has already been studied in the driven Duffing equation [29] and in the YAG laser [30]. Both branched manifolds are simple extensions of the branched manifold which describes the Smale horseshoe mechanism, which is illustrated in Fig. 6. Both of these branched manifolds roll up: one from outside to inside; the other from inside to outside. These two scrolling mechanisms are shown in Fig. 13. These figures summarize how neighborhoods in phase space are deformed under the flow. The algebraic description for each of these two branched manifolds is presented below each of the branched manifolds. It can be seen from these figures that the topological organization of adjacent branches is identical in both the inward winding scroll and the outward winding scroll. Therefore, it is impossible to distinguish between the two branched manifolds on the basis of orbits extracted from a strange attractor that contains only two distinct, adjacent, unstable period one orbits. In order to distinguish between the two, we must study a strange attractor that possesses at least three inequivalent unstable period one orbits. Such strange attractors exist only at lower temperators. For this reason, we concentrated on the strange attractor which exists at 12~ The return map shows intersections with six contiguous branches, but we were able to find only three unstable period one orbits in the strange attractor. These were 5f, 4r, 4f. The other three branches (6f, 5r, 3r) are extremely unstable. That the orbits located belonged to contiguous branches simplified the calculations somewhat. The linking numbers for all three pairs of period one orbits in both branched manifolds are 8/2=4. This can be seen from the template matrices, since T(4f, 4 r ) = T(4f, 5f) = T(4r, 5f) - 8 in both matrices. The two branched manifolds could only be distinguished by locating unstable period two orbits and computing their LNs. In fact, the orbits AB and BC (A = 4f, B - 4r, C = 5f) have identical linking numbers with the period one orbits in both branched manifolds. It is only the period two orbit AC which has different linking numbers with the period one orbits in the two branched manifolds. The linking numbers in the two cases are

LN(A, AC) LN(B, AC) LN(C, AC)

Outside to inside

Inside to outside

7 8 8

8 8 9

(27)

These LNs were computed using a general purpose code designed to compute linking numbers and relative rotation rates for periodic orbits [31]. The inputs to the code consist of the algebraic description of a branched manifold and a list of orbits by their symbolic dynamics. The output consists of a table of their LNs or a table of their relative rotation rates.

182


0

1

2

3

0

+N-0

0

0

0

0

1

-N+0

0

1

2

2

Branch Array

4 5 ,... 0 O 2

2

6

7

8

9

0

1

2

3

4

5

6

7

8

9

0

0

0

0

0

0

0

0

2

2

4

4

6

6

8

8

2

2

2

2

1

-1

0

1

2

2

4

4

6

6

8

8

2

+1

2

2

2

2

4

4

6

6

8

8

~.~Bran c h = : ~ a Y

2

+N-I

0

2

2

2

2

2

2

2

2

2

3

-N+ 1

0

2

2

3

4

4

4

4

4

4

3

-2

2

2

2

3

4

4

6

6

8

8

4

+N-2

0

2

2

4

4

4

4

4

4

4

4

+2

4

4

4

4

4

4

6

6

8

8 8

5

-N+2

0

2

2

4

4

5

6

6

6

6

5

-3

4

4

4

4

4

5

6

6

8

6

+N-3

0

2

2

4

4

6

6

6

6

6

6

+3

6

6

6

6

6

6

6

6

8

8

7

-N+3

0

2

2

4

4

6

6

7

8

8

7

-4

6

6

6

6

6

6

6

7

8

8

8

+N-4

0

2

2

4

4

6

6

8

8

8

8

+4

8

8

8

8

8

8

8

8

8

8

9

-N+4

0

2

2

4

4

6

6

8

8

9

9

-5

8

8

8

8

8

8

8

8

8

9

Fig. 13.

(a) Inwards and (b) outward winding scroll templates.

These linking numbers were also computed visually, as illustrated in Fig. 14. The stretching and squeezing sections of the two branched manifolds are shown, with the outside to inside scroll on the left. The three period one orbits are shown propagating through the middle of the three branches. The period 2 orbit AC is shown propagating through the two outside branches. The linking number of the period two orbit (half the signed number of crossings) with each of the period one orbits is shown beneath each of the branches. For the outside to inside scroll these three integers are ( - 1 , 0 , 0), while for the inside to outside scroll the three integers are (0, 0, + 1). Each must be added to the LN in the return part of the map. This consists of 8 half-twists. These entangle each period one orbit with the period two orbit with a linking number of 8. The results of this computation are summarized in Eq. (27). We therefore located the period 2 orbit AC - (4f, 5f) in the second return map at T = 12~ This orbit is shown in two embeddings in Figs. 15 and 16. The LNs of this orbit with the three period one orbits were computed. This computation showed clearly that the branched manifold which describes the strange attractor generated by the modified Hodgkin-Huxley equations is the outside to inside scroll template.

183


(a)

(b)

_

1

~1 ~ - 1 1 -1 +8 +7

0 +8 +8

0 +8 +8

0 +8 +8

0 +8 +8

+1 +8 +9

Fig. 14. Distinguishing between (a) outside to inside and (b) inside to outside scroll templates. Three branches (4f, 4r, 5f) of the two scroll templates are shown. The three period one orbits are shown as vertical lines through the middle of each branch. The period two orbit (4f, 5f) is shown going through the outer edge of the two exterior branches. Linking numbers of this period two orbit with the three period one orbits are half the sum of the signed crossings shown, plus half the sum of the additional 16 crossings in the return flow, which has 8 half twists. B

Fig. 15.

Periodic orbits in the differential embedding. (a) 4f; (b) 4r; (c) 5f; (d) (4f, 5f).

Table 2 provides the L N s of all orbits up to period three which can be found in the three branches 4f, 4r, 5f of this branched manifold. 9. Jelly rolls

The b r a n c h e d manifold which describes the strange attractor generated by the modified H o d g k i n - H u x l e y equations has been observed previously in both the pe-

184

Fig. 16.


(a)

(b)

(c)

(d)

Periodic orbits in the integral-differential embedding. (a) 4f; (b) 4r; (c) 5f; (d)

(4f, 5f). riodically driven Duffing oscillator [29] and in experimental data generated by a YAG laser [30]. It has been affectionately named the 'jelly roll' (Duffing) and the 'gateau roul6' (YAG laser). The three systems which exhibit this jelly roll behavior are all slightly different. The YAG laser is a nonautonomous dynamical system, driven by external forcing with fixed periodicity. The Duffing oscillator is also a nonautonomous dynamical system, driven by external forcing with fixed periodicity. However, this stretching and squeezing mechanism operates in an identical way over two half-cycles, so that the branched manifold for the Duffing oscillator is actually the second iterate of the jelly roll. In both systems, at any given forcing frequency, all coexisting unstable period one orbits have the same period. By contrast, the modified Hodgkin-Huxley equations form an autonomous dynamical system. Coexisting unstable period one orbits have somewhat different periods. This can be seen from the original bifurcation diagram (Fig. 1). The time duration of a period one orbit is the sum of its interspike time intervals. This sum increases nonmonotonically as T decreases, with peaks at intermittency, that is, when orbits of type nr are present. The jelly roll template will be used to provide a very simple, intuitive, and appealing description of the dynamics of receptors with subthreshold oscillations. The description involves two useful ratios. We first provide this description for the YAG laser. We then describe the small modifications needed to carry over the description to receptors with subthreshold oscillations. As a first step, we unroll the scroll shown in Fig. 13a. The result is the distorted rectangle shown in Fig. 17. The flow is from left to right. At the beginning of a period (t -- 0) a set of initial conditions exists along the vertical edge at the left. The vertical edge at the right (t =- P) marks the end of a period. Fiducial marks measure the right-hand edge in units (i.e., re) of the left-hand edge. Each unit carries an

The topology and organEation of unstable periodic orbits

185

Table 2 Linking numbers for all orbits up to period three that occur on the three-branch template: A = 4f, B = 4r, C = 5f Orbits

A

A B

C AB AC BC AAB AAC ABB ABC ACB ACC BBC BCC

B

4 4

4 7 7 8 11 11 11 11 11 11 12 12

4 8 8 8 12 12 12 12 12 12 12 12

C

A

A

B

A

A

A

A

A

A

B

B

B

C

C

A B

A C

B B

B C

C B

C C

B C

C C

11 12 12 22 22 24

11 12 12 22 22 24 33

11 12 12 22 23 24 33 34

11 12 13 23 23 25 34 35 35

11 12 12 22 23 24 33 34 34 35

11 12 13 23 23 25 34 35 35 36 35

12 12 13 24 24 26 36 36 36 37 36 37

12 12 13 24 24 26 36 36 36 38 36 38 39

4

7

7

8

4

8

8

8

8

8 15

9 16 16

8 8 9 12 12 12 13 12 13 13 13

15 16 22 22 22 23 22 23 24 24

16 22 22 23 23 23 23 24 24

24 24 24 25 24 25 26 26

33 33 34 33 34 36 36

34 35 34 35 36 36

35 34 35 36 36

35 36 37 38

35 36 36

37 38

39

i n t e g e r w h i c h reflects the t o r s i o n w h e n this s t r u c t u r e is r o l l e d b a c k u p to the o r i g i n a l scrolled s t r u c t u r e . As t i m e evolves, the set o f initial c o n d i t i o n s ( l e f t - h a n d edge) m o v e s to the right, stretches (sensitivity to initial c o n d i t i o n s , positive L y a p u n o v e x p o n e n t ) , a n d drifts u p w a r d ( i n c r e a s i n g t o r s i o n ) . W h e n the set o f initial c o n d i t i o n s arrives at the righth a n d edge, it is s p r e a d o v e r several c o n t i g u o u s s e g m e n t s . T h e r a t i o o f its l e n g t h at the r i g h t - h a n d edge to its o r i g i n a l l e n g t h is R = e ~', w h e r e ~1 is t h e p o s i t i v e L y a p u n o v e x p o n e n t (R -- s t r e t c h ratio). T h e r a t e o f u p w a r d drift is the r a t i o o f the t w o time scales o f the laser. T h e r e is a n intrinsic o s c i l l a t i o n time t, a n d the e x t e r n a l l y i m p o s e d drive p e r i o d , P. T h e i m a g e o f the l e f t - h a n d edge e x t e n d s a l o n g the righth a n d e d g e f r o m a b o u t ( P / t ) to a b o u t ( P / t ) + R. 7 6 5 4

3 2

Fig. 17. Intuitive description of scroll dynamics for the Y A G laser. The scroll shown in Fig. 13 is unrolled. A set of initial conditions flows from the left edge to the right, drifting upward and expanding. The right end is then rolled back up and the flow is reinjected back into the left edge. For the strange attractor generated by the modified Hodgkin-Huxley equations, the wavy line indicates the duration of a period one orbit.

186


The two ratios which characterize the dynamics are the time scale ratio P/'c and the expansion ratio R = e ~ . 1. The longer the period, P, the more scrolling (torsion) occurs. 2. The larger the stretching, R, the more branches are involved in the strange attractor. Only slight modifications are required to port this intuitive description from the nonautonomous Y A G laser to the autonomous receptor with subthreshold oscillations. In the latter case the period depends on the orbit. We have indicated this by a wavy solid line in Fig. 17. The role of P, the period of the external drive in the Y A G laser, is replaced by 1/T (T -temperature) in the modified Hodgkin-Huxley equations. We now cut away all the branches which are not visited in this deformed rectangle, and rescroll the remaining branches. The resulting structure has the form shown in Fig. 18a. What happens next can best be illustrated using a cut rubber band, half a pair of suspenders, or a stretchy belt (all of which are useless!). Imagine taking one of these deformable structures, stretching it by pulling it along its long axis, and then twisting it about its long axis several times. What results has the form shown in Fig. 18a. If the tension is now relaxed, the structure 'untwists', as shown in Fig. 18b. Mathematicians would describe this deformation as the conversion of twist for writhe. Indeed, there is a well-known conservation relation among the three quantities Link, Twist, and Writhe: Link = Twist + Writhe.

(28)

Mathematically, this is a remarkable relation, since neither term on the right is a topological quantity. They are both geometric, and when computed, may be real

c

Fig. 18. Illustration of Eq. (28). (a) A rubber band is twisted about its stretched length. (b) When the tension is relaxed, it deforms, exchanging twist for writhe. (c) When the two ends are reconnected, the shape of the flow generated by the modified Hodgkin-Huxley equations is apparent.


187

rather than integer-valued. However, their sum is a topological quantity and always an integer [32]. In fact, one does not even have to go so far as using rubber bands or belts to visualize the transformation of twist to writhe. Anyone who has used a telephone (not cordless) has experienced this. This also occurs in DNA. We make a geometrical model of the flow in the reduced phase space of the modified Hodgkin-Huxley model as follows. We relax the twist out of the branched manifold almost entirely. This converts, for example, 4 full twists on the branch 4r into 4 loops (writhes) without twist. The resulting structure mimics very well the flow in the phase space. Passage through the maximum of each loop corresponds to a spike in a burst. This is indicated in Fig. 18c. It is not difficult to make predictions about what happens when control parameters are changed. The flow is pushed to contiguous branches. In the algebraic description of the branched manifold, the topological matrix T(i,j) remains unchanged, as does the joining information contained in the array J(i). The only part of the algebraic description which changes with control parameters is the Markov transition matrix. For flows involving branches 4f, 4r, 5f this matrix is Label lr 2f 2r 3f 3r 4f 4r 5f 5r 6f

Torsion 2 3 4 5 6 7 8 9 10 ll

2 -0 0 0 0 0 0 0 0 0 _0

3 0 0 0 0 0 0 0 0 0 0

4

5 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 0 0 0 0 0

6

7 0 0 0 0 0 0 0 0 0 0

0 0 0 0 0 1 1 1 0 0

8 0 0 0 0 0 1 1 1 0 0

9 0 0 0 0 0 1 1 1 0 0

10

11

0 0 0 0 0 0 0 0 0 0

00 0 0 0 0 0 0 0 0_

(29)

As control parameters are changed (e.g., the ambient temperature), the block of l's on the diagonal moves up or down the diagonal, possibly contracting to a 2 • 2 submatrix, perhaps expanding to a 4 • 4 matrix. The direction in which the allowed transition block moves depends on the ratio P/~ in the Y A G laser, or its analog in the nerve cell. The size of the block depends on the stretching factor or Lyapunov exponent, and is n • n, where n = [R] + 1 or [R] + 2, JR] is the integer part of R, and R = e~ .

10. Flows without equations Branched manifolds provide more than a means of classifying strange attractors. They even provide more than a very good representation of the flow in strongly dissipative strange attractors of dimension 2 + 8, 8 ~ 0. They provide a means for


188

accurately representing the flow in strange attractors which are far from the dissipative limit (e not small). The basic idea is that the branched manifold provides a 'backbone' or 'skeleton' for the flow. If one 'blows up' by expanding in the transverse direction, this is the same as expanding against the contracting direction. This replaces two-dimensional splitting and joining charts by their three-dimensional counterparts (cf., Fig. 8), and replaces two-dimensional branches by three-dimensional 'flow tubes' [2]. The result is a flow in R 3 which has the topological organization of the initial strange attractor, but for which the flow is now generated without the benefit of dynamical system equations of motion. How does this work for the modified Hodgkin-Huxley equations? Flow modeling of this type has already been done for the Duffing oscillator [29]. We simply take the results of that study and apply them to the current problem, with appropriate modifications. The flow takes place in a topological torus D 2 x T ~, which has been deformed by the conversion of twist to writhe. Two views of a structure of this type ('writhing torus'), with a writhe of 2, are shown in Fig. 19 [33]. We first imagine the flow to take place within a solid cylinder. Then we map the cylinder into the writhing torus, identifying the two ends of the cylinder. This produces a strange attractor with the correct topological structure, provided suitable care is taken. We will describe the necessary 'suitable care' below. The flow in the cylinder is modeled in two phases. The two processes described in Fig. 17 provide the skeleton for this model. However, those described in Fig. 17 are noninvertible, while the model we provide below is for an invertible map, and subsequently, for a flow. The first phase, which models stretching, occurs from s = 0 to s = 1 (s measures distance along the axis of the cylinder of length 2). In this phase, a set of initial

1.2

1.2i

0.8

O.8

0.4

0.4

YO

Y 01

-0.4

-0.4i

-0.8

-0.81 -1.2 T2

Fig. 19.

10

8

6

~

4 t

,

2

'

0

r 8 -2

-4

Two projections of a 'writhing torus' with writhe = 2 [33]. This figure was kindly supplied by J. Palencia.

189


conditions in a strip at s = 0 is stretched out and rotated around the axis of the cylinder. The second phase, which models squeezing, occurs from s = 1 to s = 2. In this phase, a set of initial conditions, the circle at s = 1, is deformed into the interior of an open strip in the interval 0 ~ Dlmin, and DImin is the minimum diastolic interval the tissue can sustain [20,21]. Combining Eqs. (1) and (2) results in a mathematical mapping relating the future APD in terms of its past value in a form given by

N 9PP-

APDn+I =

f(N

9P P -

APDn).

(3)

For some tissue preparations, including bullfrog myocardium [32], a restitution relation of the form f(DIn) - APDmax - A e -DI"/r,

(4)

gives good agreement with experimental observations, where APDmax, A, and z are tissue-dependent constants. For long PP, the behavior of the paced muscle settles down to a regular periodic behavior after an initial transient. In this case, DI >> DImin and every stimulus produces an action potential whose shape and duration are identical so that N = 1 (this is often referred to as 1"1 or period-1 behavior). The steady-state action potential duration, known as the fixed point of the map, is denoted by APD* and is given by the solution of the transcendental relation APD* - f ( N

9P P -

APD*).

(5)

It can also be determined graphically by finding the intersection of the restitution (4) and pacing (2) relations, as shown in Fig. 8. Note that the pacing relation line shown in the figure shifts to the left as PP decreases so that APD* decreases as the pacing becomes faster. The stability of the fixed point is governed by the value of Dlmin and the slope of the restitution relation evaluated at the fixed point Of] ix = 9

= - A - e- (PP-APD*) APD=APD* ~"

(6)

where ~t is known as the Floquet multiplier. When the actual value of the APD is close to the fixed point, the temporal evolution of the action potential duration is given approximately by 8APD~+l = ~SAPD~,

(7)

where 8 A P D , - A P D ~ - APD*. By inspection of Eq. (7), any slight perturbation that moves the system away from the fixed point will change by a factor Ibt] on each iterate of the map. Therefore, the fixed point is unstable (the perturbation grows)

242

D.J. Gauthier et al.

800

.

, restitution relation

600 E ~ " 400

, APD*

< pacing relation

200

o

0

,

|

200

400

~

...... J

!

600

800

1000

DI (ms)

Fig. 8. A mathematical model of the restitution relation (solid line), and the pacing relation (dashed line) describing the constraint between PP, APD, and DI. The intersection of the two curves gives APD*. whenever 1~[ > 1. In addition, the fixed point will become unstable whenever P P - APD* < Dlmin. As PP decreases (thereby decreasing APD* and increasing IPl), the fixed point becomes unstable and the tissue response switches over to a new pattern whose form depends on the precise values of the model parameters. Such a transition in the response pattern as a system parameter is swept (PP in this case) is known as a bifurcation and is one typical characteristic a nonlinear dynamical system. For tissue where DImin is large and characterized by a shallow restitution relation, the 1:1 behavior first becomes unstable when P P - APD* < Dlmin and is replaced by a pattern in which an action potential is induced for every other stimulus (N = 2). That is, the pacing is so rapid that the stimulus occurs during the period when the tissue is refractory and hence cannot respond with a new action potential. An action potential is only induced on the next stimulus after the tissue has recovered. This is known as a subcritical bifurcation and the response pattern is often referred to as 2:1. Experimental observations of this behavior are described in Section 5. In a situation where Dlmin is small ( P P - APD* > Dlmin for all PP considered) and the tissue is characterized by a steep restitution relation, the 1:1 response pattern becomes unstable when I~ < - 1 and is replaced by a pattern in which an action potential is induced by every stimulus but the duration alternates between two different values. This pattern is referred to as alternans, period-2 behavior, or 2:2 behavior by various authors. Alternans is sometimes observed in human electrocardiograms recorded from electrodes placed on the body surface, and can be correlated with an increased risk of a future episode of sudden cardiac death. Hence, it is important to understand the mechanisms giving rise to alternans and methods for controlling or suppressing this dynamical state. The transition to alternans as PP is varied and can be visualized using a bifurcation diagram, as shown in Fig. 9, for tissue parameters APDmax "- 680 ms, A =

Controlling the dynamics of cardiac muscle using small electrical stimuli

243

400 ms, z = 190 ms, and DImin -- 80 ms. To generate the diagram, map (3) is iterated to determine its asymptotic behavior for each value of PP and the resulting APD for several iterates of the map are plotted. The value of PP is then changed and the procedure repeated. For long PP, only one value of APD appears, indicating a stable 1:1 pattern. For decreasing PP, the APD shortens, consistent with normal cardiac tissue restitution properties. At PP = 631 ms, an abrupt transition from a 1:1 to a 2:2 pattern occurs where it is seen that the APD alternates between short and long values and l-t < - 1 . This type of transition is called a forward or supercritical bifurcation [33]. Even though the 1:1 pattern is unstable for PP < 631 ms, there still exists a solution to Eq. (5) for APD*. The value of this unstable fixed point as a function of PP is shown as the dashed line in Fig. 9. A properly designed controller can stabilize this state (the 1:1 pattern), thereby suppressing alternans. In general, feedback control of dynamical systems involves measuring the state of the system (characterized by APD in our model), generating an appropriate feedback signal, and adjusting an actuator that modifies an accessible system parameter (PP in our model) by an amount ~,. The size of the feedback signal can be small and little energy is expended by the controller when the system is stabilized to the desired state because the scheme stabilizes a state that already exists in the system but is unstable in the absence of control. To illustrate suppression of alternans using our simple mathematical model, we consider closed-loop proportional feedback. Proportional control seeks to minimize the difference between the current state of the system and a reference state, which we take as APD*. The algorithm uses an error signal given by

(8)

~, = y(APD. - APD*),

800

Alternans (2:2) Stable 1"1

600

E

a

v

~

~

-

r

400 Unstable 1:

l

slope

.

< ~-~ v

Control On

I

253

Control Off I

0.4

0.2 0.0 -0.2

(b)

0.2

~.

0.1

n "e

0.0

i

-0.1

-0.2 .

0

.

.

.

m

5

,

,

10

15

20

time (s) Fig. 13. Temporal evolution of the (a) monophasic action potential signal (MAP) and (b) the error signal e, (quantified as a fraction of the average period PP*) during a control experiment.

experimental noise than PPF methods, have potential advantages for the control of complex behavior in biological systems.

8. Outlook Our preliminary experiments on controlling alternans in the bullfrog tissue is a first step toward realizing a method for controlling complex whole-heart dynamics. We are currently investigating the use of variants of this method to control the dynamics of a fibrillating sheep atrium in conjunction with a high-density mapping system to record the effects of the control perturbations on the entire atrium. This basic research program will provide new insights into the fundamental mechanisms that regulate the dynamics of small pieces of cardiac muscle subjected to small, appropriately timed electrical stimuli. The studies will help to elucidate the important factors that determine the success or failure of nonlinear dynamics control techniques in stabilizing cardiac arrythmias. The knowledge gained from these studies may improve current clinical methods of cardioversion and will increase the number of treatment options for cardiac patients.

Acknowledgements We gratefully acknowledge invaluable collaborations with Ellen Dixon-Tulloch, Wanda Krassowska, Robert Oliver, and Patrick Wolf of the Duke Experimental Electrophysiology Laboratory, Henry Greenside and Joshua Socolar of the Duke

254

D.J. Gauthier et al.

Physics Department, and David Schaeffer of the Duke Mathematics Department, a n d f i n a n c i a l s u p p o r t o f t h e W h i t a k e r F o u n d a t i o n a n d t h e N a t i o n a l Science Foundation.

References 1. 2. 3. 4. 5. 6. 7.

8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 29. 30. 31. 32.

Shinbrot, T., Grebogi, C., Ott, E. and Yorke, J.A. (1993) Nature 363, 411. Ott, E. and Spano, M. (1995) Phys. Today 48, 34. Garfinkel, A., Spano, M.L., Ditto, W.L. and Weiss, J.N. (1992) Science 257, 1230. Hall, K., Christini, D.J., Tremblay, M., Collins, J.J., Glass, U and Billette, J. (1997) Phys. Rev. Lett. 78, 4518. Wiggers, C.J. (1940) Am. Heart J. 20, 399. Zipes, D.P., Fischer, J., King, R.M., Nicoll, A. and Jolly, W.W. (1975) Am. J. Cardiol. 36, 37. Ideker, R.E., Chen, P-S., Shibata, N., Colavita, P.G. and Wharton, J.M. (1987) in: Non Pharmacological Therapy of Tachyarrhythmias, eds G. Breithardt, M. Borggrefe, and D. Zipes, pp. 449-464. Futura, Mount Kisco. For a recent review, see Winfree, A.T. (1998) Chaos 8, 1 and the accompanying articles on the special. Focus Issue on Fibrillation in Normal Ventricular Mycoardium. Gray, R.A., Pertsov, A.M. and Jalife, J. (1998) Nature (London) 392, 75. Witkowski, F.X., Penkoske, L.J., Giles, P.A., Spano, W.R., Ditto, M.U and Winfree, A.T. (1998) Nature (London) 392, 78. Cross, M.C. and Hohenberg, P.C. (1994) Science 263, 146. Rappel, W.-J., Fenton, F. and Karma, A. (1999) Phys. Rev. Lett. 83, 456. Ditto, W.L., Spano, M.L., In, V., Neff, J., Meadows, B., Langberg, J.J., Bolmann, A. and McTeague, K. (2000) Int. J. Bifurcations Chaos 10, 593. Oliver, R.A., Hall, G.M., Bahar, S., Krassowska, W., Wolf, P.D., Krassowska, W., Dixon-Tulloch, E.G. and Gauthier, D. (2000) J. Cardiovasc. Electrophysiol. 11, 797. Katz, A.M. (1992) Physiology of the Heart, 2nd Edn, Chapter 2. Raven Press, New York. Plonsey, R., Barr, R.C. (1991) Bioelectricity: A Quantitative Approach. Plenum Press, New York. Luo, C.-H. and Rudy, Y. (1991) Circ. Res. 68, 1501. Rasmusson, R.L., Clark, J.W., Giles, W.R., Robinson, K., Clark, R.B., Shibata, E.F. and Campbell, D.L. (1990) Am. J. Physiol. (Heart Circ. Physiol. 28) 259, H370. Rasmusson, R.L., Clark, J.W., Giles, W.R., Robinson, K., Clark, R.B. and Campbell, D.L. (1990) Am. J. Physiol. (Heart Circ. Physiol. 28) 259, H352-H369. Guevara, M., Ward, G., Shrier, A. and Glass, L. (1984) in: Computers in Cardiology, IEEE Comp. Soc., p. 167. Glass, L. and Mackey, M.C. (1988) From Clocks to Chaos: The Rhythms of Life. Princeton University Press, Princeton, NJ. Hall, G.M., Bahar, S., Gauthier, D.J. (1999) Phys. Rev. Lett. 82, 2995. Henriquez, C.S. (1993) Crit. Revs. Biomed. Eng. 21, 1. Gallagher, J.J., Kasell, J.H., Cox, J.L., Smith, W.M., Ideker, R.E. and Smith, W.M. (1982) Am. J. Cardiol. 49, 221. Witkowski, F.X. and Corr, P.B. (1984) Am. J. Physiol. 247, H661. Ideker, R.E., Smith, W.M., Wolf, P.D., Danieley, N.D. and Bartram, F.R. (1987) PACE 10, 281. Shenasa, M., Borggrefe, M. and Breithardt, G. eds (1993) Cardiac Mapping, Futura Publishing Company, Inc., Mount Kisco, New York. Hall, G.M. and Gauthier, D.J. (2000) Submitted for publication. Christini, D.J., Hall, K., Collins, J.J. and Glass, L. (2000) in: Handbook of Biological Physics: Neuro-Informatics, Neural Modeling, Vol. 4, eds, F. Moss and S. Gielen. Elsevier, New York. Ott, E., Grebogi, C. and Yorke, J.A. (1990) Phys. Rev. Lett. 64, 1196. Sukow, D.W., Bleich, M.E., Gauthier, D.J. and Socolar, J.E.S. (1997) Chaos 7, 560. Nolasco, J.B. and Dahlen, R.W. (1968) J. Appl. Physiol. 25, 191.

Controlling the dynamics of cardiac muscle using small electrical stimuli

255

33. Berge, P., Pomeau, Y., Vidal, C. (1984) Order within Chaos: Towards a Deterministic Approach to Turbulence, pp. 40-42. Wiley, New York. 34. Romeiras, F.J., Grebogi, C., Ott, E. and Dayawansa, W.P. (1992) Physica D 58, 165. 35. Gluckman, B.J., Spano, M.L., Yang, W., In, V. and Ditto, W.L. (1997) Phys. Rev. E 55, 4935. 36. Chen, P-S., Wolf, P.D., Dixon, E.G., Danieley, N.D., Frazier, D.W., Smith, W.M. and Ideker, R.E. (1988) Circ. Res. 62, 1191. 37. Kirchhof, C., Chorro, F., Scheffer, G.J., Brugada, J., Konings, K., Zetelaki, Z. and Allessie, M. (1993) Circulation 88, 736. 38. KenKnight, B.H., Bayly, P.V., Gerstle, R.J., Rollins, D.L., Wolf, P.D., Smith, W.M. and Ideker, R.E. (1995) Circ. Res. 77, 849. 39. Watanabe, M. and Gilmour, R.F. Jr. (1996) J. Math. Biol. 35, 73. 40. Christini, D.J. and Collins, J.J. (1996) Phys. Rev. E (Rapid Communition) 53, R49. 41. Schiff, S.J., Jerger, K., Duong, D.H., Chang, T., Spano, M.L. and Ditto, W.L. (1994) Nature (London) 370, 615. 42. Pierson, D. and Moss, F. (1995) Phys. Rev. Lett. 75, 2124. 43. Christini, D.J. and Collins, J.J. (1995) Phys. Rev. Lett. 75, 2782. 44. Gauthier, D.J. and Socolar, J.E.S. (1997) Phys. Rev. Lett. 79, 4938. 45. Glass, L. and Josephson, M.F. (1995) Phys. Rev. Lett. 75, 2059. 46. Biktashev, V.N. and Holden, A.V. (1995) Proc. R. Soc. Lond. B 261, 211. 47. Aranson, I., Levine, H. and Tsimring, L. (1995) Phys. Rev. Lett. 72, 2561. 48. Karma, A. (1994) Chaos 4, 461. 49. Clay, J.R., Brochu, R.M. and Shrier, A. (1990) Biophys. J. 58, 609. 50. Kowtha, V.C., Kunysz, A., Clay, J.R., Glass, L. and Shrier, A. (1994) Prog. Biophys. Molec. Biol. 61, 255. 51. Delmar, M., Glass, L., Michaels, D.C. and Jalife, J. (1989) Circ. Res. 65, 775. 52. Roth, B.J. (1997) Cardiovasc. Electrophysiol. 8, 768. 53. Lin, S.-F., Roth, B.J., Eclat, D.S. and Wikswo, J.P. Jr. (1996) circulation 94, 1-714. 54. Roth, B.J. and Krassowska, W. (1998) Chaos 8, 204. 55. Holmes, P. (1990) in: Mathematical Approaches to Cardiac Arrhythmias, Ann. N. Y. Acad. Sci., Vol. 591, p. 301. ed. J. Jalife. The New York Academy of Sciences, New York. 56. Chialvo, D.R. (1990) in: Mathematical Approaches to Cardiac Arrhythmias, Ann. N. Y. Acad. Sci., Vol. 591, p. 351. ed. J. Jalife. The New York Academy of Sciences, New York. 57. Chialvo, D.R., Michaels, D.C., Jalife, J. (1990) Circ. Res. 66, 525. 58. Hall, G.M., Bahar, S. and Gauthier, D.J. (2000) Submitted for publication. 59. Mines, G.R. (1913) J. Physiol (London) 46, 349. 60. Yehia, A.R., Jeandupeux, D., Alonso, F. and Guevara, M.R. (1999) Chaos 9, 916. 61. Socolar, J.E.S. and Gauthier, D.J. (1998) Phys. Rev. E 75, 6589.


CHAPTER 8

Intrinsic Noise from Voltage-Gated Ion Channels" Effects on Dynamics and Reliability in Intrinsically Oscillatory Neurons J.A. W H I T E * and J.S. HAAS Department of Biomedical Engineering, Center for BioDynamics, 44 Cummington Street, Boston, MA 02215, USA *Department of Biomedical Engineering, Boston University, 44 Cummington Street, Boston University, Boston, MA 02215 USA



257

Contents 1.

Introduction

.................................................

2.

Background

.................................................

3.

4.

259 259

2.1.

Sources of electrical noise in the n e r v o u s system

2.2.

Effects o f electrical noise in the p o s t s y n a p t i c n e u r o n . . . . . . . . . . . . . . . . . . . . . .

Case study: Stellate n e u r o n s o f the e n t o r h i n a l cortex

........................ ........................

259 264 265

3.1.

Basic electrophysiological p r o p e r t i e s

...............................

3.2.

Q u a n t i f y i n g biological noise sources

...............................

266 268

3.3.

M o d e l i n g the effects o f c h a n n e l noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

269

Summary and conclusions .........................................

276

Acknowledgements

277

References

.............................................

.....................................................

258

277

1. Introduction Bioelectrical events underlie many of the processes necessary for life, including communication among nerve cells; the peristaltic wave of activity that moves food through the digestive system; secretion of insulin by the pancreas; and contraction of cardiac and skeletal muscle. All of these events are generated by ion channelsprotein complexes, inserted in the cell membrane, that act as molecular "gates" of electrical current [1-4]. The opening and closing of these gates can be controlled by one or more variables, including transmembrane voltage or the concentration of a chemical (e.g., a neurotransmitter or hormone) inside or outside the cell. A fundamental property of ion channels and other bioelectrical elements is that they are stochastic: one cannot specify the exact behavior of an ion channel, but only statistical descriptors of its behavior (e.g., the probability that the channel is open; the moments of the single-channel conductance). Similarly, the events underlying synaptic communication between nerve cells are stochastic at many levels. These stochastic phenomena add an element of noise to electrical responses in excitable cells. In this chapter, we begin by describing the major sources of biological noise in neurons (nerve cells), and how each can be characterized experimentally. We review some of the consequences of biological noise, and present a detailed case study of how noise interacts with oscillatory dynamics in a particular population of neurons in the mammalian brain (stellate neurons of the medial entorhinal cortex). We argue that intrinsic noise from voltage-gated Na + channels is sufficient to alter the electrical dynamics of stellate cells in a number of specific ways.

2. Background 2.1. Sources of electrical noise in the nervous system

Neuronal noise sources fall into many categories, and can be organized differently according to one's perspective. Here, we categorize them with loose correspondence to the neurophysiologist's world view, in which noise sources are grouped by physiological underpinnings rather than some other attribute (e.g., frequency content). 2.1.1. Voltage-gated ion channels Voltage-dependent conductances underlie action potentials and other intrinsic electrical activity in nerve and muscle cells. These conductances are generated by the concerted action of 102-106 voltage-gated ion channels, each of which appears to "gate" (open and close) probabilistically. The source of the apparent random behavior is believed to be thermal excitation of a molecule with multiple stable states. Typically, the gating of voltage-dependent ion channels is modeled as a Markov

259

260

J.A. White and J.S. Haas

process with a finite number of states (S;) and state-transition rates (a~) that depend instantaneously on membrane potential [1,2,4,5], although attractive alternatives to the Markov model exist [6-8]. Eq. (1) shows a hypothetical Markov model with four states" ~12 Sl~

~23 $2.,.

~21

~34 83,,

~32

(1)

84 ~43

The Markov process formalism is memoryless (i.e., its transitional probabilities are independent of its history), making it rather simple to derive much about the statistics of the modeled channel. For example, with fixed membrane potential, the autocorrelation function for single-channel conductance is a multi-exponential function [1,2,4,5]. A population of n such channels, assumed independent and each open with probability p, has simple binomial statistics under steady-state conditions. The noisiness of this group of channels can be quantified by the coefficient of variation (CV, the ratio of standard deviation to mean) of membrane current or conductance. For a binomial distribution, C V = {(1 -p)/(np)} 1/2 [4]. Thus, under assumptions of stationarity, the noisiness of a given population of voltage-gated channels is proportional to n -1/2. Use of Markov models to describe voltage-gated ion channels can best be illustrated through an example. Figure 1A shows a schematic representation of the classical voltage-activated K + channel from the Hodgkin-Huxley model. In this representation, the conductance state of the channel is determined by the states of four "gates". The gates open and close randomly, with independent and identical distributions. The channel conductance is zero, unless all four gates are open, as in the rightmost state. If each gate opens with rate constant an and closes with rate constant 13,, the channel as a whole is described by the 5-state rate scheme [1,4]" 4~n nO ~

3~n

2~n

~n

nl ~ n2 ~ n3 ,< _ n 4 [3, 2[~, 313,, 4[~,

(2)

In this scheme, the state nJ has j open gates. Integer coefficients of rate constants between two states are determined by the number of possible paths between the states. Responses of populations of n such 5-state channels to step changes in membrane potential are shown in Fig. lB. The steady-state autocorrelation of conductance in a single channel of this form is shown in Fig. 1C (solid line). The Markov description is mathematically elegant, but some voltage-gated channels seem to violate its underlying assumption of memorylessness; instead, transition rate constants can be proportional to t -c, where t is the amount of time already spent in a given state and c is a constant [6,7]. At the single-channel level, this fractal behavior in rate constants prolongs the "tails" of the autocorrelation function (Fig. 1C). At the population level, fractal rate constants make estimates of variance grow systematically with the time window of analysis. Fractal behavior in ion channel state transition rates gives individual channels memory, greatly complicating the task of representing a population of channels in

Intrinsic noise from voltage-gated Ion channels

261

(A)

(B)

0

~ open

0

~ closed

(c)

n=l

120 ;i Ii

100 (/3 cl.

n=lO

,-.., +

n= 100

o'~ I.l.J

n : 1000

6

' 1'0

' 2'0

40

.........

p .

.

.

.

5

p=11

......................................

-~0 - ~ 0

L_______ ' 3'0

60

Markov

/ ~ ~ ~ ~ " ]

deterministic//""-

i. I

80

' 4'0

-~0

~

~'0

;o

'

~'o

(ms)

' s'o

Time (ms)

Fig. 1. Stochastic models of ion channels. (A) A pictorial representation of a voltage-gated channel with four gating particles. The channel conducts only when all four gating particles are open. Typically, individual gating particles are modeled as independent, 2-state Markov processes with identically distributions. Under these assumptions, only the number of open gates, rather than the state of each gate, must be tracked in a simulation. (B) The behavior of a population of n Hodgkin-Huxley [60] K + channels in response to a voltage-clamp step from a holding potential o f - 7 0 mV to a clamp potential of 0 inV. Stimulus onset at t - 10 ms; offset at t = 40 ms. Each plot shows the conductance from a single run, scaled by the maximal possible conductance for comparative purposes. Single channels (top) have only two conductance states. With increasing n, the ratio of standard deviation to mean conductance drops proportionally to 1/~[n. The deterministic curve represents the behavior of the system in the limit as n ~ oc. (C) The autocorrelation of the single-channel conductance under steady-state conditions ( V = 0 mV). The Markov channel description (solid line) has a multi-exponential autocorrelation function with correlation time < 20 ms. In pseudo-fractal channel descriptions (dashed line, dotted line), multi-exponential autocorrelation functions approximate power law relationships. As p (the number of states in the pseudo-fractal channel model) increases, the approximation remains valid for longer time intervals [8]. a computational model. These difficulties can be circumvented by creating pseudofractal representations of ion channels [8]. In these representations, m e m o r y in rate constants is captured over some user-defined bandwidth by a rate scheme with multiplicatively related rate constants. For the H o d g k i n - H u x l e y K + channel, the


262

appropriate pseudo-fractal representation of a model with memory in the closed state is: A/K p- l nl

( B/KP -1

B/Kp-2 n2

( B/KP -2

A/K ""

~ B/K

A np-1 (

np

(3)

B

where A = {4(~n + ~n)4~n}l{(O~n -t- ~ n ) 4 -- ~n4}, B = 413n, and K is a constant [81. The pseudo-fractal model accounts for memory by allowing closed channels to wander several steps away from the open state (np). In the limit as the number of states p oe, the expected value of single-channel conductance approaches that of the Markov model (Eq. (2)), with fractally distributed closed-time intervals. The dashed and dotted lines in Fig. 1C show the effects on the single-channel conductance autocorrelation function for two values of p. The parameters of models like those in Eqs. (1)-(3) are determined routinely from physiological data. The procedure typically involves holding transmembrane voltage V constant at a series of physiologically relevant values (-100 mV < V < 50 mV), and measuring the resultant current that passes through a single channel or a presumed homogeneous population of channels. Parameters are determined by matching measured and modeled response attributes (e.g., spectral densities of transmembrane current).

2.1.2. Synaptic mechanisms The primary form of inter-cellular communication in most neural and neuromuscular systems, chemical synaptic transmission involves at least three important random processes. First, the number of quanta (packets) of neurotransmitter released in response to a presynaptic action potential is a discretely distributed random variable. Second, single quanta of neurotransmitter can be released without occurrence of a presynaptic action potential. Third, neurotransmitter-gated ion-channels, like their voltage-gated counterparts, gate stochastically. 2.1.2.1. Quantal release. The discovery and characterization of quantal release of neurotransmitter [9] stand as one of the great discoveries in the history of neuroscience. Typically, the number of released quanta is described by a binomial distribution, where in this case n is the number of sites of synaptic release (or of synaptic receptors, whichever is in short supply) and p is the probability of release at each site [2]. The binomial distribution implies that synaptic conductances in postsynaptic cells, which are proportional to the number of quanta released presynaptically, have mean value 3'np and variance 3'2np(l- p), where 3' is the magnitude of the conductance change induced by each quantum of neurotransmitter. The Poisson distribution is also used commonly to model synaptic release; in this model the mean and variance of induced synaptic events are equal [2]. Parameter estimation based on either the binomial or the Poisson model is based on several implicit assumptions, including those of statistical independence of release sites and uniform quantal size. In parameter estimation experiments, the researcher electrically stimulates the presynaptic nerve fiber(s) and uses intracellular


263

recording techniques to measure the postsynaptic response. Evoked postsynaptic responses can be measured as changes in membrane potential, or as changes in membrane current. With a sufficiently high signal to noise ratio, it is possible to reconstruct the discretely valued distribution function describing the probability of release of k quanta of neurotransmitter (k = 0, 1, 2,...). With these data in hand, it is relatively straightforward to determine the applicability of the binomial or Poisson models, and to estimate best-fit parameter values [2]. 2.1.2.2. Spontaneous release. A second source of synaptic noise comes from the fact that single quanta of neurotransmitter can be released from presynaptic cells at seemingly random times. The time intervals between spontaneous release events are often assumed to be exponentially distributed with a constant mean rate of spontaneous release [2], although the strict validity of this assumption is doubtful [10]. Spontaneous release events can be measured and characterized from intracellular recordings, typically in the presence of pharmacological agents that block presynaptic action potentials. Depending on the biological preparation, measured spontaneous release events (often called "mini" postsynaptic potentials in the neurobiological literature) can reflect spontaneous release from one presynaptic axon, or the superposition of spontaneous release events from a large number of presynaptic cells. 2.1.2.3. Stochastic gating of neurotransmitter-gated ion channels. The gating behavior of neurotransmitter-responsive (or ligand-gated) ion channels is described classically by the same Markov-process constructs used to describe voltage-gated channels (Eq. (1)), with the difference that some state-transition rates are functions of the concentration of ligand (neurotransmitter) in the immediate vicinity of the receptor. Parameter identification procedures for ligand-gated channels typically involve recording responses to step changes of ligand concentration in the extracellular solution. 2.1.2.4. Other potentially important synaptic noise sources. Chemical synaptic transmission is a complex process. Other mechanisms that may add to the noisy character of synaptic communication include the diffusion of a relatively small number of molecules of neurotransmitter within the synaptic cleft, and the likely stochastic nature of the chemical reactions responsible for degradation or re-uptake of neurotransmitter molecules. The success of researchers in detecting quantal synaptic events from experimental data suggests that the contributions of these additional noise sources are not as large as those of quantal synaptic release.

2.1.3. Randomly distributed interspike &terval stat&tics in presynaptic neurons A third factor that contributes significantly to electrical noise levels in neurons is that of apparent randomness in the time sequence of action potentials in presynaptic neurons. Patterns of activity in presynaptic neurons are most often modeled as homogeneous or inhomogeneous Poisson processes, which offer many numerical

264


and analytical conveniences and are often reasonably accurate descriptions of neuronal firing patterns (but see [11]). Estimating the probability density functions describing presynaptic interspike interval statistics from experimental data is a difficult problem. Recordings in reduced preparations like the brain slice show abnormally low presynaptic firing rates that do not reflect rates seen in vivo (i.e., in the living animal), and intracellular recordings are notoriously difficult to make in vivo. Even in the few cases that such data have been collected, detection of presynaptic interspike interval statistics is prevented, because these events are so numerous and are masked by the slow kinetics of synaptic conductance changes and the synaptic noise sources listed above. In practice, many researchers [12-15] lump synaptic noise sources and interspike interval statistics together into one noisy current or conductance source that can, in principle, be characterized from intracellular measurements in vivo. 2.1.4. Other sources Several additional sources may contribute significant electrical noise in neurons. First, the cell membrane, like any element with electrical impedance, has associated with it thermal or Johnson noise. Typically, Johnson noise is believed to have small amplitude and wide bandwidth in neurons [1]. Second, electrical transients from activity in nearby neurons may be coupled capacitatively to the neuron in question, leading to a phenomenon called ephaptic noise. Noise from nearby neurons can be coupled resistively, through inter-cellular channels called gap junctions, as well [16]. Third, variations in intra- and extracellular ionic concentrations [17] may affect processes that depend on either membrane potential (e.g., ion channels) or chemical gradients (e.g., pumps), in a phenomenon sometimes called metabolic noise. Unlike the other noise sources listed here, metabolic noise depending on bulk concentrations would be expected to have a slow time scale. However, consideration of microdomains and other spatial gradients in concentration within the cell may lead to fast components of metabolic noise. 2.2. Effects of electrical no&e in the postsynaptic neuron

The ramifications of biological noise sources have been studied using a number of measures and methods. Here, we discuss briefly three interrelated measures that reflect the presence of biological noise: reliability, stochastic resonance, and noisedriven spontaneous activity. 2.2.1. Reliability and threshold To a first approximation, neurons can be thought of as devices that fire an all-ornothing action potential with a distinct threshold. It stands to reason that electrical noise from voltage-gated channels, for example, could make the neuronal threshold less distinct: cells might be expected to generate action potentials in response to brief current pulses with some probability that is a sigmoid function of stimulus amplitude. "Soft" thresholds of this kind have been seen in experimental data for over


265

60 years [18-20], and have been attributed quantitatively to intrinsic noise from voltage-gated ion channels [21-23]. More recent work has focused on neuronal reliability to repeated presentations of broad-band stimuli [12,24-27]. "Reliability" in this context is typically quantified as the average cross-correlation of successive responses to an identical, broad-band stimulus. This approach has the advantage that the frequency content of presented stimuli is more realistic, but the potential disadvantage that it can be difficult to interpret responses to mixed sub- and suprathreshold stimulus sequences [28]. The underlying train of thought behind these experiments is that a cell with important effects of intrinsic noise should be unreliable in response to repeated presentations of broad-band stimuli with levels of fluctuation mimicking those seen in vivo; in contrast, high reliability indicates that intrinsic noise may be unimportant, and that variable responses seen in vivo are caused by different synaptic signals received on successive presentations of the stimulus. In recorded and simulated responses, reliability is maximal for stimuli with large fluctuations and frequency content matching the preferred firing frequency of the cell [12,24-27,29], although exceptions to this rule may exist [30]. 2.2.2. Stochastic resonance The sigmoid response probabilities mentioned above point to the result that intrinsic and other noise sources can enhance neuronal detection of a small stimulus. In this form of stochastic resonance, the neuron itself, and/or the probabilistic input signal it receives, provide the noise that enhances representations of small inputs. In considering the output of a network of neurons, receiving a common stimulus, it is interesting to note that the majority of the noise sources mentioned above would naturally operate in a neuronal population as independent noise sources, with consequent advantages in signal representation [31]. 2.2.3. Spontaneous firing and interspike interval stat&tics Intrinsic noise from voltage-gated ion channels has been shown to be sufficient to cause spontaneous firing in otherwise quiet neuronal models [3,21,22,32-36]. In the classical Hodgkin-Huxley model, in which repetitive action potentials arise via a subcritical Hopf bifurcation, interspike interval statistics of spontaneous activity are well described by a Poisson process with dead time [8,33]. Modification of ion channel kinetics to have long-term correlations in state occupancy gives rise to related longterm trends in the ordering of interspike intervals [8]. Bifurcation structure also has dramatic effects on interspike interval statistics in noisy models of single neurons [37]. Models that give rise to periodic firing via saddle-node bifurcations have long tails in their interspike interval distributions; models that give rise to periodic firing via Hopf bifurcations have more constrained distributions of interspike intervals. 3. Case study: stellate neurons of the entorhinal cortex

The hippocampal region, consisting of the hippocampus proper and the associated entorhinal, perirhinal, and parahippocampal cortices, plays a crucial role in memory

266


[38-40], and is a site of critical neuropathologies in Alzheimer's disease [41] and temporal lobe epilepsy [42]. During periods of attentiveness or intentional movement, the rat hippocampal EEG is dominated by a 4-12 Hz rhythm called the theta rhythm [43,44], which is the product of synchronous oscillatory activity in several interconnected brain regions, including the hippocampus and entorhinal cortex [43]. Two lines of evidence link the theta rhythm to the memory-related functions of the hippocampal region. First, disabling theta activity leads to severe memory impairment [40,45]. Second, theta-patterned activity is effective in inducing long-term changes in synaptic efficacy thought to be linked to memory formation in the hippocampus [46,47]. Two complementary mechanisms contribute to the hippocampal theta rhythm. First, the hippocampus is "paced" by theta-coherent input from external brain structures, in particular a nearby structure called the medial septum [45]. Second, the hippocampal formation includes intrinsic mechanisms that seem to aid in developing coherence in the 4-12 Hz band, including specialized synaptic kinetics [48] and specialized single-cell interspike interval dynamics [49,50]. In this case study, we will focus on stellate cells (SCs) of the medial entorhinal cortex (MEC). These neurons, which deliver much of the information from the neocortex to hippocampus, exhibit intrinsic oscillations in membrane potential at theta frequencies, even without synaptic input [50,51]. The mechanisms underlying these intrinsic oscillations are inherently noisy, with interesting consequences for spiking dynamics and information representation in the hippocampal formation.

3.1. Basic electrophysiological properties SCs of the MEC have unmistakable and unusual intrinsic electrophysiological properties [50,51]. Fig. 2A shows responses of an MEC stellate cell at rest (bottom trace), and at two levels of applied DC current. For intermediate current levels of DC current (middle trace), SCs generate ~8 Hz, subthreshold oscillations, with occasional action potentials locked to the depolarizing phase of the underlying oscillations. At higher levels (top trace), the frequency of the oscillations and probability of generating an action potential per oscillatory cycle both increase. For suprathreshold current levels, action potentials do not occur in a memoryless fashion. Instead, action potentials tend to "cluster" on adjacent cycles of the slow oscillations (Fig. 5A). The mechanisms underlying subthreshold oscillations and phase-locked spikes in SCs are relatively well understood (Fig. 2B). Spikes are generated by kinetically typical Na + and K + conductances 9Nal and 9Kl [52,53]. Subthreshold oscillations are independent of synaptic input. Their depolarizing phase is caused by a persistent (noninactivating) Na + current (gNa2). Their hyperpolarizing phase is caused by a slow hyperpolarizing conductance (9s) that may include contributions from activation of a slow K + current and deactivation of the slow inwardly rectifying cation current Ih [32,54-57]. The basic properties of the oscillations (e.g., amplitude, frequency) do not seem to depend critically on the precise identity of the slow hyperpolarizing current [57].


267

(A)

0.6nA

0.3nA

0.0 nA

=;--. . . . . . . . - ~ 20 mV 250 ms externally applied inDut

(B)

--q app

inside +

C _

V _T

_

outside responsible for action potentials

~ responsible for slow oscillations

synaptic input

Fig. 2. Subthreshold oscillations and rhythmic spiking in stellate cells (SCs) of the medial entorhinal cortex (MEC). (A) Electrophysiologically recorded responses of a stellate cell to DC current, applied through the recording electrode, at three levels. At lapp - 0.3 nA, the cell exhibits noisy subthreshold oscillations and the occasional action potential (truncated, the raw action potential is ~100 mV in amplitude). At lapp = 0.6 nA, oscillatory frequency and the probability of spiking increase. (B) A minimal, biophysically based model that accounts for subthreshold oscillations and rhythmic spikes. In this circuit diagram, C represents cellular capacitance per unit area; 9N~ and 9~ the voltage-gated Na + and K + conductances responsible for action potentials; gNa2 the persistent (noninactivating) Na + conductance responsible for the rising phase of the subthreshold oscillations; 95 the slow conductance responsible for the falling phase of the subthreshold oscillations; 9e the ohmic "leak" conductance; and 9~yn represents synaptic input. Each conductance is in series with a voltage source representing its reversal potential (i.e., the potential at which current through each conductance changes sign). The current s o u r c e lap p represents current applied via the recording electrode.

268


3.2. Quantify&g biological no&e sources

The electrical behavior of MEC SCs is noticeably noisy in the subthreshold voltage range (Fig. 2A). Because the persistent Na + conductance gNa2 is activated in this voltage range and is mediated by a relatively small number of channels, it seemed to us a likely dominant contributor to this membrane electrical noise. We tested this hypothesis by measuring the noise contributions of the persistent Na + conductance, and comparing these contributions to those of all other sources [32]. Fig. 3A shows

(A) 50

TTX A Q..

-50

Control -100 -150 I

200

'

I

'

300

I

I

500

400

Time (ms)

(B)

1.0-

> (J

0.5

iiiiiili 0.0 I

-75

'

I

-50

'

I

-25

'

I

0

Vm ( m Y )

Fig. 3. Noise from persbtent channels & SCs. (A) Sample responses under steady-state voltage-clamp in control conditions (lower trace), and in the presence of tetrodotoxin (TTX, 1 pm), which blocks Na + channels selectively. In TTX, the mean current is reduced to zero, and the variance is reduced over threefold (from 144 to 42 pA2); the latter result implies that Na + channels are the major electrical noise source in these neurons under these experimental conditions (isolated neurons from the MEC; see [52]). The data shown were collected starting 200 ms after switching membrane potential to the clamp potential o f - 4 0 mV. Data of this kind can be used to estimate the rate constants, open-channel probabilities, and number of channels in an assumed independent and identically distributed population of stochastic ion channels [4,32]. (B): CVN~, the coefficient of variation of the persistent Na + current, plotted vs. steady-state membrane potential. In the subthreshold range (-60 to -50 mV), CVNa > 0.25, implying that the persistent Na + current is significantly noisy. Data (mean + SEM) are lumped from 4 neurons.


269

sample data of this kind. The lower trace in Fig. 3A shows steady-state membrane current measured while membrane potential is held at -40 mV. The upper trace shows the equivalent measurement made in the presence of tetrodotoxin (TTX), a highly selective blocker of Na + channels. Application of TTX reduces the mean membrane current under these conditions from -75 to 0 pA, and the variance in membrane current from 144 to 42 pA 2. This reduction in variance indicates that the persistent, TTX-sensitive Na + current is the major source of membrane current noise in these cells under these experimental conditions (which do not account for noise from synaptic sources). Fig. 3B shows CVNa, the coefficient of variation (ratio of standard deviation to mean) of the persistent Na + current from 4 MEC SCs (mean i SEM). As predicted by binomial models, CVNa is relatively high for membrane potentials in the subthreshold range.

3.3. Model#zg the effects of channel noise 3.3.1. Effects on excitability and bifurcation structure Using data like those in Fig. 3, we constructed a model of MEC SCs (Fig. 2B). This model includes standard nonlinear differential equation-based descriptions of the voltage-gated conductances gNal, gK, and gs. The persistent Na + conductance gNa2, on the other hand, is modeled as a collection of independent, probabilistically gating channels [32]. Fig. 4 shows schematic bifurcation diagrams for deterministic (A) and stochastic (B) SC models, in response to DC current. Above each qualitative region in the bifurcation diagram is a schematic of the time-domain response of the model in that region. Deterministic models have a stable fixed point at rest. With increasing applied current (lapp), the stable fixed point becomes more positive, then loses its stability via a subcritical Hopf bifurcation. A stable limit cycle appears, representing rhythmic firing of action potentials at theta frequencies. Around the bifurcation point, there is a small region of bistability. Subthreshold oscillations are rarely seen in deterministic models in practice [32], because parameters must be tuned very precisely for them to occur. As indicated by the schematic bifurcation diagram in Fig. 4B, intrinsic noise from Na + channels changes the behavior of the model SC considerably. As applied current is increased, subthreshold oscillations arise (Fig. 4B, leftmost bifurcation). These oscillations resemble those that would arise via a supercritical Hopf bifurcation, but in fact they reflect the behavior of a noise-driven system spiraling around a stable critical point with associated complex eigenvalues. At a second critical value of lapp, the model is able to fire with nonzero probability (second bifurcation). With further increases in applied current, the probability of firing in the stochastic SC model changes gradually from near zero to near one spike per cycle, giving the noisy model a "soft," rather than "hard," threshold. The effect of noise is to increase the cell's dynamic range, by increasing sensitivity for small stimuli and decreasing sensitivity for larger stimuli. This effect is robust over a large parameter space [32]. The parameters of the stochastic (but not deterministic) model SC can be


270

(B) Stochastic

(A) Deterministic

oLL \ooo~176176

ooOO oo

vT

~oooo

....

f

f

lapp

lapp

Fixed points Stable

Oscillations ......

Unstable

Fig. 4. Intrinsic noise alters bifurcation behavior in SC models. Schematic bifurcation diagrams (main panels) and time-domain traces (insets) for deterministic (A) and stochastic (B) models of SCs are shown V - membrane potential; lap p -- DC applied current. (A) Deterministic models switch from a stable fixed point to rhythmic spiking. (B): With increasing lapp, stochastic models shift from a stable fixed point to subthreshold oscillations to rhythmic firing with some probability of firing an action potential on a given oscillatory cycle. This probability varies seemingly continuously from near zero to near one, giving the stochastic model a "soft" threshold.

tuned to match experimental data quite well, in terms of spectral density of subthreshold oscillations and spiking probabilities as functions of DC current level [32]. 3.3.2. Interspike interval statistics

Experimental responses of SCs to DC current show a notable phenomenon called "spike clustering" (Fig. 5A, inset), in which action potentials tend to occur on adjacent cycles of the underlying slow oscillations [51]. The main panel of Fig. 5A shows an interspike interval histogram derived from such experimental data (kindly supplied by Angel Alonso). Spike clustering is evident in the high probability of intervals corresponding to one period of the underlying oscillations in experimental data (bars) compared with the expected probability in a memoryless model of the same mean firing probability per cycle (solid line). The stochastic SC model can account to some degree for the spike clustering seen in experimental data. Fig. 5B shows simulated responses at the same mean firing rate (or, equivalently, the same mean probability of firing per slow oscillatory cycle).

271

Intrinsic noisefrom voltage-gated Ion channels

( A ) Experimental Data

(B) simulations

0,6-

0.6-

0.5-

0.5-

0.4-

0.4-

0.3-

0.3

0,2-

0.2

0.1-

0.1

0.0-

0.05

Number of Periods

Number of Periods

(C) pre-eventEnvelopes

Simulations

I

-500

2'o

(D) Markov Chain Model of Spiking

. . . . .

'

I

-250

'

]

0

Time before event (ms)

Fig. 5. Spike clustering in experimental data and stochastic simulations. (A) and (B) Bars show estimated interspike interval distributions derived from experimental data (A) and stochastic simulations (B). Time has been normalized by the period of the underlying subthreshold oscillation (~8 Hz), derived from the spectral density of extended subthreshold epochs of the data. Solid lines: expected interspike interval distributions for memoryless processes with the same mean probabilities of firing per cycle (0.2 for both the experimental data and simulations). Dashed lines: best-fit models with spiking probabilities that are conditional on the occurrence of an action potential on the previous cycle. Insets: sample timedomain data. Horizontal scale bar: 250 ms. Vertical scale bar: 10 mV. (C) Envelopes of subthreshold oscillations preceding action potentials (solid lines; mean + SEM) or action potential failures (dashed lines; mean + SEM), derived from experimental (top) and simulated (bottom) data. Selected data were purely subthreshold epochs >1 s, referenced to the time of occurrence of a spike or peak of a subthreshold oscillation at time = 0. Selected data were processed by subtracting out the mean value, taking the absolute value, and low-pass filtering. Both experimental and simulated results show significant growth in the oscillatory envelope for 1-2 oscillatory periods before action potentials. Initiation of envelope growth (arrows) begins earlier for experimental results than for simulated results. (D) A Markov chain model of conditional spiking probabilities, a represents the probability of spiking on the next cycle after a spike; 13represents the probability of spiking on the cycle after not spiking. Best-fit versions of interspike interval distributions based on this model are plotted as dashed lines in A-B.

272


Again, the probability of having an interspike interval of 1 cycle is elevated (cf. the bars and solid line), but not to the degree seen in experimental data. Although the stochastic MEC model can account partially for spike clustering, details of interspike interval distributions are different in Fig. 5A,B. Simulation results are well fit by a two-state Markov chain model of spiking per oscillatory cycle (Fig. 5D, dashed line in Fig. 5B). This result implies that the memory underlying spike clustering in stochastic simulations can be expressed accurately in terms of a simple conditional probability based on the occurrence (or lack thereof) of an action potential in the previous cycle. Two results indicate that memory in experimental data is somewhat more complex than in stochastic simulations. First, the two-state Markov model (Fig. 5D) does not do as good a job in fitting experimental interspike interval distributions (cf. bars and dashed line in Fig. 5A). In particular, the Markov model necessarily has a monotonically decreasing distribution, P{ISI--j} =(1 - a)(1 - 13)J-213forj > 2. The ISI distribution in experimental data is significantly nonmonotonic. Second, the preevent envelopes of membrane potential (Fig. 5C) preceding action potentials (solid lines; mean + standard deviation) look different in experimental data than in simulations. Specifically, the envelopes, obtained by filtering raw signals to obtain a signal representing the magnitude of the subthreshold oscillations over time, seem to "break away" from the internal control (dashed lines; mean + standard deviation for envelopes that do not precede spikes) earlier in the experimental data than in simulation results (cf. locations of arrows marking the approximate "break-away" point). These results imply that subthreshold oscillations wax significantly for some time before spiking in both experimental data and stochastic simulations, but that this form of self-organization occurs over a longer time scale in experimental data. The multiplicative nature of channel noise, caused by the voltage-dependence of the transition rate constants, contributes to the spike clustering seen in stochastic simulations at low spike rates. This result is seen in Fig. 6, in which we have plotted the estimated probability of occurrence of a spike cluster of length (M) vs. M at two spike rates. Simulations with multiplicative (voltage-dependent) noise show memory that is consistent with the 2-state Markov model of Fig. 5D at both low (Fig. 6A) and moderate (Fig. 6B) spike rates. Simulations with additive current noise, on the other hand, show no memory at low spike rates (Fig. 6A). Interestingly, current noise simulations show significant spike clustering at higher spike rates (Fig. 6B), indicating that the voltage-dependence of channel noise is a contributing, but not necessary, factor for spike clustering. 3.3.3. Stochastic resonance

In Fig. 4, we showed schematic bifurcation diagrams indicating that intrinsic noise contributed by persistent Na + channels is sufficient to "soften" the threshold for firing in response to DC stimuli. A similar effect is seen with 8-Hz, half-wave rectified sinusoidal input, mimicking the excitatory input SCs receive from their neighbors during the theta rhythm. In Fig. 7, we have plotted response probabilities (per cycle) vs. synaptic weight for deterministic and stochastic models of SCs. The deterministic model (solid line) shows three robust firing probabilities (p = 0, 0.5,

273


(A)

C l u s t e r l e n g t h distributions: low r a t e

- -o- - Multiplicative noise - -m- - Additive noise - - -2-state Markov model Memoryless 9 model

10 ~ -z

~,

~

e...~ II t-

~-)

1 0 -1

I~

10 .2

' , .w,,~

. . . .

10 .3

b

I

I

I

I

1

2

3

4

M (B)

C l u s t e r l e n g t h distributions: high r a t e

10 o

1 0 "1

1 0 .2

1 0 .3

I

I

I

2

I

I

3

4

Fig. 6. Multiplicative noise alters cluster length distributions at low, but not high, spike rates. Plots of the probability of having a "spike cluster" (group of action potentials on adjacent slow oscillatory cycles) of length M vs. M, at low (0.05) and moderate (0.2) mean probabilities of firing per cycle. Results are shown for stochastic simulations with multiplicative (voltagegated) noise in the persistent Na + conductance (open circles); simulations with additive current noise, scaled to give the same mean firing rate (closed squares); a best-fit 2-state Markov chain model (Fig. 5D), which allows spiking probability to depend on the occurrence (or lack thereof) of a spike in the previous oscillatory cycle (dashed line); and a memoryless model, which has the same spiking probability (0.05 in A, 0.2 in B) every cycle (solid line).

274

J.A. White and J.S. Haas 1.0-

0.8

L_

Q.

0.6

./

U3 tO

ca. 0.4 t_

o ~Q.

0.2 ~ o ~ ~ ~

0.0 '

'

'

''I

'

'

10 .3

'

'

'

'

''I

10 z gsyn

( mS/cm2 )

Fig. 7. Intrinsic noise smoothes the dependence of spiking probability on magnitude of conductance input. Responses to half-wave rectified, sinusoidal conductance input at 8 Hz. The probability of an action potential per cycle of input is plotted vs. synaptic conductance magnitude 9syn- The reversal potential associated with the synapse was 0 mV. Solid line: response of the deterministic SC model. Dotted line: response of the stochastic model. Adapted from [32]. and 1), with somewhat complex transitions between these states. In contrast, the stochastic model exhibits firing probabilities that vary smoothly and largely monotonically between zero and one. The threshold for nonzero spiking probabilities is two to fivefold lower in the stochastic model, exhibiting a form of stochastic resonance in which the neuron itself provides a noise source that boosts sensitivity. 3.3.4. Effects on reliability As discussed in Section 2, the presence of significant intrinsic noise would be expected to lower the reliability (repeatability) of responses to repeated presentations of timevarying stimuli. Figure 8 shows results from electrophysiological experiments de-

Opposite: Fig. 8. Measurement of reliability in SCs. (A) and (B) Membrane potential responses (lower 10 traces of each panel; action potentials clipped at + 10 mV) to repeated presentations of fluctuating current stimulus (top traces of each panel). Vertical scale = 100 mV/division for voltage traces. The current traces in panels A and B have the same mean value (70 pA) and level of fluctuation ( ~ , the standard deviation, = 100 pA), but different frequency content. Each were generated by a 2-pole, low-pass filter with cutoff frequency ft. The filter was designed to preserve the overall level of fluctuations (cri). (C) Pooled reliability results from 8 putative SCs, identified electrophysiologically by their tendency to generate subthreshold oscillations and fire rhythmically. All cells fired at approximately the same rate (1-3 Hz) for the level of DC current used. Data were collected at two values of fc for each cell. Points are mean + standard deviation.

275

Intr&sic no&e from voltage-gated Ion channels

(A)

(B)

f = 53 Hz

I

0

'

I

'

200

I

'

400

I

'

I

600

'

800

/

fc =

I

'

'

0

1000

8 Hz

I

200

'

I

'

I

400

'

600

I

800

'

/

1000

Time after stimulus onset (ms)

(C)

Summaryresults 0.7-

0.6

--"--f~ = 53 HZ - - o - - f = 8 Hz

T

n

/'T

0.5

~, ._ ._

.m_ "~ n,

L

0.4

0.3

0.2

0.1

0.0

'

2'0

'

40

'

6;

'

80

I

I

I

q

100

120

140

160

~, (pA)

Fig. 8.

Caption opposite.

'

276


signed to test this hypothesis. Fig. 8A shows l0 responses (lower l0 traces) of an MEC SC to a repeated, broad-band current stimulus (top trace; current stimuli were generated as pseudo-random, Gaussian white noise of a given variance, then filtered with cutoff frequency fc). This cell responds reliably (9/10 times) at t ~ 80 ms after the onset of the stimulus, but subsequent action potential responses are far less reliable. Fig. 8B shows responses of the same cell to repeated presentations of a stimulus of equal variance but with most of its energy concentrated between 0 and 8 Hz. The SC responds to this stimulus more frequently and more reliably. Fig. 8C summarizes results based on recordings from 8 MEC SCs in brain slices. Plotted is reliability (calculated using the method of Hunter et al. [27]) vs. cyi, the standard deviation in current (or, more accurately, the RMS value of the zero-meaned current waveform). We draw two main conclusions from these results. First, intrinsic noise from voltage-gated channels seems to render SCs unreliable at moderate levels of signal fluctuation (cyl < 140 pA), implying that these neurons may be more difficult to entrain than, for example, neocortical pyramidal cells [12,25]. Second, SCs are sensitive to changes in the frequency content of the timevarying signal. In particular, they fire more reliably (and, in many cases, at higher rates) when the fluctuating stimulus contains more energy within the theta (4-12 Hz) frequency band. Thus SCs may be "tuned" to fire preferentially reliably to input bandwidths associated with a particular behavioral state.

4. Summary and conclusions The biophysical mechanisms underlying electrical excitability and communication in the nervous system make electrical noise ubiquitous and seemingly inescapable. Here, we have reviewed the basic characteristics and expected consequences of several biological noise sources. We have presented a case study in which we argue that the probabilistic gating of tetrodotoxin-sensitive Na + channels may alter neurophysiological function in stellate neurons of the medial entorhinal cortex, a memory-related structure in the mammalian brain. Key effects of intrinsic noise from Na + channels include enhancement of excitability and sensitivity to small stimuli; alteration of cellular bifurcation behavior and "softening" of the neural threshold; enhancement of robustness of electrical behaviors in the face of perturbations of model parameters; reduction of neuronal reliability in response to broad-band stimuli; and enhanced reliability with inputs that have significant energy content within the theta (4-12 Hz) band. The example we highlight here focuses on one intrinsic noise source in a particular neuronal population, but the phenomena associated with noisy, subthreshold oscillations and phase-locked action potentials are likely to be widespread and capable of revealing general principles of single-neuron computation [58,59,61,62]. The consequences of biological noise at the level of neuronal networks seem almost certain to be a fruitful area of research that has only begun to be explored.


277

Acknowledgements W e t h a n k A. A l o n s o for p r o v i d i n g the e l e c t r o p h y s i o l o g i c a l d a t a for Fig. 5, a n d A . D . D o r v a l for t h o u g h t f u l c o m m e n t s o n a p r e l i m i n a r y v e r s i o n o f this c h a p t e r . This w o r k was s u p p o r t e d b y g r a n t s f r o m T h e W h i t a k e r F o u n d a t i o n a n d the N a t i o n a l I n s t i t u t e s o f H e a l t h (NS34425).

References 1. Hille, B. (1992) Ionic Channels of Excitable Membranes, 2nd Edn. Sinauer Associates, Inc., Sunderland, MA. 2. Johnston, D. and Wu, S.M.-S. (1995) Foundations of Cellular Neurophysiology. MIT Press, Cambridge, MA. 3. Koch, C. (1999) Biophysics of Computation. Information Processing in Single Neurons. Computational Neuroscience, ed M. Stryker, Oxford University Press, New York. 4. Weiss, T. (1996) Cellular Biophysics. Vol. 2: Electrical Properties. MIT Press, Cambridge, MA. 5. Sakmann, B. and Neher, E. (1995) Single-Channel Recording, 2nd Edn. Plenum Press, New York. 6. Liebovitch, L.S. and Toth, T.I. (1990) Ann. NYAS 591, 375-391. 7. Liebovitch, L.S. and Todorov, A.T. (1996) Crit. Rev. Neurosci. 10, 169-187. 8. Lowen, S.B., Liebovitch, L. and White, J.A. (1999) Phys. Rev. E 59, 5970-5980. 9. del Castillo, J. and Katz, B. (1954) J. Physiol. (Lond.) 124, 560-573. 10. Lowen, S.B., Cash, S.S., Poo, M.-m. and Teich, M.C. (1997) J. Neurosci. 17, 5666-5677. 11. Lowen, S.B. and Teich, M.C. (1996) J. Acoust. Soc. America 99, 3585-3591. 12. Nowak, L.G., Sanchez-Vives, M.V. and McCormick, D.A. (1997) Cereb. Cortex 7, 487-501. 13. Mainen, Z.F., Joerges, J., Huguenard, J.R. and Sejnowski, T.J. (1995) Neuron 15(6), 1427-1439. 14. Stevens, C.F. and Zador, A.M. (1998) Nature Neurosci. 1, 210-217. 15. Zador, A. (1998) J. Neurophysiol. 79, 1230-1238. 16. Draguhn, A., Traub, R.D., Schmitz, D. and Jefferys, J.G. (1998) Nature 394(6689), 189-192. 17. O'Rourke, B., Ramza, B.M. and Marban, E. (1994) Science 265, 962-966. 18. Pecher, C. (1939) Arch. Int. Physiol. Biochem. 49, 129-152. 19. Verveen, A.A. and Derksen, H.E. (1968) Proc. IEEE 56, 906-916. 20. Verveen, A.A. and DeFelice, L.J. (1974) Prog. Biophys. Mol. Biol. 28, 189-265. 21. Lecar, H. and Nossal, R. (1971) Biophys. J. 11, 1068-1084. 22. Lecar, H. and Nossal, R. (1971) Biophys. J. 11, 1048-1067. 23. Sigworth, F.J. (1980) J. Physiol. (Lond.) 307, 97-129. 24. Bryant, H.L. and Segundo, J.P. (1976) J. Physiol. (Lond.) 260, 279-314. 25. Mainen, Z.F. and Sejnowski, T.J. (1995) Science 268, 1503-1506. 26. Schneidman, E., Freedman, B., and Segev, I. (1998) Neural Comput. 10(7), 1679-1703. 27. Hunter, J.D., Milton, J.G., Thomas, P.J. and Cowan, J.D. (1998) J. Neurophysiol. 80, 1427-1438. 28. Pei, X., Wilkins, L. and Moss, F. (1996) Phys. Rev. Lett. 77, 4679-4682. 29. Jensen, R.V. (1998) Phys. Rev. E 58, R6907-R6910. 30. Gutkin, B. and Ermentrout, G.B. (1999) CNS 99: Proceedings of the Eighth Annual Computational Neuroscience Meeting, Vol. 8, p. 1. 31. Collins, J.J., Chow, C.C. and Imhoff, T.T. (1995) Nature 376, 236-238. 32. White, J.A., Klink, R., Alonso, A. and Kay, A.R. (1998) J. Neurophysiol. 80, 262-269. 33. Chow, C.C. and White, J.A. (1996) Biophys. J. 71, 3013-3021. 34. DeFelice, L.J. and Isaac, A. (1992) J. Stat. Phys. 70, 339-354. 35. DeFelice, L.J. and Goolsby, W.N. (1996) in: Fluctuations and Order, eds M. Millionas, M. Rotenberg, M. Gharib, W. Heller, K. Lindenberg, H.D. Arbarbanel and J.D. Simon, pp. 331-342. Springer, Berlin. 36. Strassberg, A.F. and DeFelice, L.J. (1993) Neural Comput. 5, 843-855.

278


37. Gutnick, B. and Ermentrout, G.B. (1998) Neural Comput. 10(5), 1047-1065. 38. Cohen, N.J. and Eichenbaum, H. (1993) Memory, Amnesia, and the Hippocampal System. MIT Press, Cambridge. 39. Eichenbaum, H. (1997) Science 277, 330-332. 40. Squire, L.R. and Zola-Morgan, S. (1991) Science 253(5026), 1380-1386. 41. Van Hoesen, G.W. and Hyman, B.T. (1990) Prog. Brain Res. 83, 445-457. 42. Wieser, H.G. (1983) Electroclinical Features of the Psychomotor Seizure. Butterworths, London. 43. Bland, B.H. and Colom, L.V. (1993) Prog. Neurobiol. 41(2), 157-208. 44. O'Keefe, J. (1993) Curr. Opin. Neurobiol. 3, 917-924. 45. Stewart, M. and Fox, S.E. (1990) Trends Neurosci. 13, 163-168. 46. Christie, B.R. and Abraham, W.C. (1992) Neuron 9(1), 79-84. 47. Larson, J. and Lynch, G. (1986) Science 232, 985-988. 48. Banks, M.I., Li, T.-B. and Pearce, R.A. (1998) J. Neurosci. 18, 1305-1317. 49. Nufiez, A., Garcia-Austt, E. and Bufio, B. Jr. (1990) Brain Res. 416, 289-300. 50. Alonso, A. and Llinfis, R.R. (1989) Nature 342(6246), 175-177. 51. Alonso, A. and Klink, R.M. (1993) J. Neurophysiol. 70, 128-143. 52. White, J.A., Alonso, A. and Kay, A.R. (1993) Neuron 11, 1037-1047. 53. Eder, C., Ficker, E., Giindei, J. and Heinemann, U. (1991) Eur. J. Neurosci. 3, 1271-1280. 54. Dickson, C.T. and Alonso, A. (1997) J. Neurosci. 17, 6729-6744. 55. Galue, A. and Alonso, A. (1996) Soc. Neuroscie Abs. 22, 61. 56. Klink, R.M. and Alonso, A. (1993) J. Neurophysiol. 70, 144-157. 57. White, J.A., Budde, T. and Kay, A.R. (1995) Biophys. J. 69, 1203-1217. 58. Braun, H.A., Sch/ifer, K. and Voigt, K. (1997) J. Comput. Neurosci. 4, 335-347. 59. Huber, M.T., Krieg, J.C., Dewald, M., Voigt, K. and Braun, H.A. (1998) Biosystems 48, 95-104. 60. Hodgkin, A.L. and Huxley, A.F. (1952) J. Physiol. (Lond.) 117, 500-544. 61. White, J.A., Rubinstein, J.T. and Kay, A.R. (2000) Trends Neurosci. 23, 131-137. 62. Steinmetz, P.N., Manwani, A., Koch, C., London, M. and Segev, I. (2000) J. Comput. Neurosci. 9, 133-148.

CHAPTER 9

Phase Synchronization" From Theory to Data Analysis M. ROSENBLUM, A. PIKOVSKY, and J. KURTHS Department of Physics, University of Potsdam, Am Neuen Palais 10, D-14415 Potsdam, Germany

C. SCHAFER Centre for Nonlinear Dynamics, Department of Physiology, McGill University, 3655 Drummond Street, Montreal, Que. Canada H3G l Y6

P.A. TASS Institute of Medicine (MEG), Research Centre Jfilich, D-52425 Jfilich, Germany



279

Contents

1.

Introduction

.................................................

1.1. Synchronization in biology

1.2. Synchronization and analysis of bivariate data 2.

281

.....................................

281

.........................

282

Phase and frequency locking: a brief review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

284

2.1. Periodic oscillators

284

2.2. Noisy oscillators 2.3. Chaotic oscillators

.......................................... ...........................................

286

..........................................

287

3.

2.4. An example: two coupled noisy R6ssler oscillators . . . . . . . . . . . . . . . . . . . . . . . Estimating phases from data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

287 289

3.1

292

4.

Straightforward analysis of phase difference: application to posture control . . . . . . . . . . .

293

5.

Statistical analysis of phase difference: application to brain activity . . . . . . . . . . . . . . . .

295

5.1. H u m a n brain activity during pathological tremor

298

6.

An example: synchronization via parametric action (modulation) . . . . . . . . . . . . . .

.......................

Stroboscopic technique: application to cardiorespiratory interaction

...............

6.1. Cardiorespiratory interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2. The experimental data and preprocessing 7.

............................

302 302 303

6.3. Cardiorespiratory synchrogram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

304 307

7.1. Is it really synchronization? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

307

7.2. Synchronization vs. coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

312 313

Appendix A. Instantaneous phase and frequency of a signal . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

280

314 319

I. Introduction

Synchronization is a basic phenomenon in science, discovered at the beginning of the modern scientific age by Huygens [1]. In the classical sense, synchronization means adjustment of frequencies of periodic self-sustained oscillators due to weak interaction [2-5]. This effect (also referred to as phase locking or frequency entrainment) is well studied and finds a lot of practical applications [3,4]. During the last 15 years the notion of synchronization has been generalized to the case of interacting chaotic oscillators. In this context, different phenomena exist which are usually referred to as "synchronization", so one needs a more precise description. Due to a strong interaction of two (or a large number) of identical chaotic systems, their states can coincide, while the dynamics in time remains chaotic [6,7]. This effect can be denoted as "complete (identical) synchronization" of chaotic oscillators. It can be easily generalized to the case of slightly nonidentical systems [7], or the interacting subsystems [8]. Recently, the effect of phase synchronization of chaotic systems has been described [9]. It is mostly close to synchronization of periodic oscillations, where only the phase locking is important, while no restriction on the amplitudes is imposed. Correspondingly, the phase synchronization of chaotic system is defined as the appearance of a certain relation between the phases of interacting systems (or between the phase of a system and that of an external force), while the amplitudes can remain chaotic and are, in general, noncorrelated. Of course, the very notion of phase and amplitude of chaotic systems is rather nontrivial. Remarkably, the properties of phase synchronization in chaotic systems are similar to those of synchronization in periodic noisy oscillators [10]. This allows one to describe both effects within a common framework. Moreover, from the experimentalist's point of view, one can use the same methods in order to detect synchronization in both chaotic and noisy systems; we will use this analogy below. Describing particular experiments and searching for phase synchronization, we will not be interested in the question, whether the observed oscillations are chaotic or noisy: the approach we present below is equally applicable in both these cases.

1.1. Synchronization in biology Synchronization phenomena are often encountered in living nature. Indeed, the concept of synchronization is widely used in experimental studies and in the modeling of interaction between different physiological (sub)systems demonstrating oscillating behavior. The examples range from the modeling of the heart in the pioneering paper of van der Pol and van der Mark [11] to investigation of the circadian rhythm [12,13], phase locking of respiration with a mechanical ventilator 281

282

M. Rosenblum et al.

[14] or with locomotory rhythms [15], coordinated movement [13] and animal gaits [16], phase locking of chicken embrion heart cells with external stimuli and interaction of sinus node with ectopic pacemakers [13], synchronization of oscillations of human insulin secretion and glucose infusion [17], locking of spiking from electroreceptors of a paddlefish to weak external electromagnetic field [18], and synchronization of heart rate by external audio or visual stimuli [19]. A very interesting and important example is interaction of human cardiovascular and respiratory systems. Although it is well known that these systems do not act independently [20] and in spite of early communications in the medical literature (that often used different terminology) [21-26], in the biological physics community these two systems were often considered to be not synchronized. So, an extensive review of previous studies of biological rhythms led to the conclusion that "there is comparatively weak coupling between respiration and the cardiac rhythm, and the resulting rhythms are generally not phase locked" (see [13, p. 136]). Recently, the interaction of these vital systems attracted attention of several physics groups, and synchronization during paced respiration [27,28] was investigated. Here, as well as in Refs. [21,23,24,27,28] only synchronous states of orders n: 1 (n heartbeats within 1 respiratory cycle) were found due to limitation of the ad hoc methods used for the analysis of data. In our recent work [29,30] we have reported on cardiorespiratory synchronization under free-running conditions; the proposed analysis technique allows to find out synchronous epochs of different orders n:m. This finding gives some indication for the existence of an unknown form of cardiorespiratory interaction. The notion of synchronization is also related to several central issues of neuroscience (see, e.g., [31]). For instance, synchronization seems to be a central mechanism for neuronal information processing within a brain area as well as for communication between different brain areas. Results of animal experiments indicate that synchronization of neuronal activity in the visual cortex appears to be responsible for the binding of different but related visual features so that a visual pattern can be recognized as a whole [32-34,31]. Another evidence is that synchronization of the oscillatory activity in the sensorimotor cortex may serve for the integration and coordination of information underlying motor control [35]. Moreover, synchronization between areas of the visual and parietal cortex, and between areas of the parietal and motor cortex was observed during a visuomotor integration task in an awake cat [36]. However, as yet, little is known about synchronization between different brain areas and its functional role and behavioral correlates in humans. On the other hand, synchronization plays an important role in several neurological diseases like epilepsies [37] and pathological tremors [38,39]. Correspondingly, it is important to analyze such synchronization processes to achieve a better understanding of physiological brain functioning as well as disease mechanisms. 1.2. Synchronization and analysis of bivariate data As we have argued above, synchronization phenomena are abundant in the real world and biological systems, in particular. Thus, detection of synchronization from

283

Phase synchronization: from theory to data analysis

experimental data appears to be an important problem, that can be formulated as follows: Suppose we can obtain several signals coming from different simultaneous measurements (e.g., an electrocardiogram and respiratory movements, multichannel electro- or magnetoencephalography data, records of muscle activity, etc.). Usually it is known how to attribute these signals to different oscillating objects. The question is whether there are states (or epochs) where these objects oscillate in synchrony. Unfortunately, typically observed oscillations are highly irregular, especially in live systems, and therefore possible synchronization phenomena are masked by strong noise and/or chaos, as well as by nonstationarity. This task is similar to a well-known problem in time series analysis: how to reveal the presence of an interdependence between two (or more) signals. The analysis of such bivariate data is traditionally done by means of linear cross-correlation (cross-spectrum) techniques [40] or nonlinear statistical measures like mutual information or maximal correlation [41-43]. Recently, different synchronization concepts of nonlinear dynamics have been used in studies of bivariate data. Schiff et al. [44] used the notion of dynamical interdependence [45] and applied the mutual prediction technique to verify the assumption that measured bivariate data originate from two synchronized systems, where synchronization was understood as the existence of a functional relationship between the states of two systems, called generalized synchronization. In our previous works [46-49,29], we proposed an ansatz based on the notion of phase synchronization; this implies existence of a relationship between phases of two weakly interacting systems, whereas the amplitudes may remain uncorrelated [9,10]. In our approach we assume that the measured bivariate data originate from two interacting self-oscillatory systems which may either be phase locked or oscillate independently. Generally, we try to access the following problem: suppose we observe a system with a complex structure that is not known exactly, and measure two time series at its outputs (Fig. 1). Our goal is not only to find out whether these signals are dependent or not - this can be done by means of traditional statistical techniques -

(a)

(b)

Fig. 1. Illustration of the synchronization approach to analysis of bivariate data. The goal of the analysis is to reveal the presence of a weak interaction between two subsystems from the signals at their outputs only. The assumption made is that the data are generated by two oscillators having their own rhythms (a). An alternative hypothesis is a mixture of signals generated by two uncoupled systems (b).

284

M. Rosenblum et al.

but to extract additional information on the interaction of some subsystems within the systems. Obviously, we cannot consider the system under study as a "black box", but need some additional knowledge to support the assumption that the content of this "box" is complex, and we indeed encounter several subsystems, that generate their own rhythms, but are, probably, weakly coupled. An advantage of our approach is that it allows to address rather weak interaction between the two oscillatory subsystems. Indeed, the notion of phase synchronization implies only some interdependence between phases, whereas the irregular amplitudes may remain uncorrelated. The irregularity of amplitudes can mask the phase locking so that traditional techniques treating not the phases but the signals themselves may be less sensitive in the detection of the systems' interrelation [46,48]. In the following we briefly review the ideas and results of theoretical studies of the synchronization phenomena that are used in our approach to time series analysis. Next, we present techniques of the bivariate data analysis and illustrate them by examples of physiological data. These examples are given in the ascending order of the signal analysis complexity, and in our presentation we dwell on the analysis itself, but not on the physiological importance of the results. 2. Phase and frequency locking: a brief review

We know that synchronization of weakly coupled oscillators appears as some relation between their phases and frequencies. In the context of data analysis we are going to exploit this fact to tackle the inverse problem: our goal is to reveal the presence of synchronization from the data. To this end we have to estimate from the signals the phases and frequencies and analyse the relations between them. First, we summarize what we know about the interdependence of phases and frequencies of synchronized systems. Definitely, as the experimental data are inevitably noisy, we always have to take fluctuations into account. Therefore, any relation between phases should be treated in a statistical sense. 2.1. Periodic oscillators

Stable periodic self-sustained oscillations are represented by a stable limit cycle in the phase space, and the dynamics of a phase point on this cycle can be described as

d~ dt = o0,

(1)

where COo= 2rt/To, and To is the period of the oscillation. It is important that starting from any monotonically growing variable 0 on the limit cycle, one can introduce the phase satisfying Eq. (1). Indeed, an arbitrary 0 obeys 0 = v(0) with a periodic v(0 + 2rt) = v(0). A change of variables = 030

[v(0)]-' dO

~0~

Phase synchronization: from theory to data analys&

285

gives the correct phase, where the frequency m0 is defined from the condition 2n - m0 fZ~[v(0)j-ld0. A similar approach leads to correct angle-action variables in Hamiltonian mechanics. From (1) it is evident that the phase corresponds to the zero Lyapunov exponent, while negative exponents correspond to the amplitude variables (not written in (1)). If two oscillators are weakly coupled, then in the first approximation one can neglect variations of the amplitudes to obtain equations describing the phase dynamics. In general, these equations have the form d~)l

dt = m, + eg, (~,, (~2),

d(~ 2

dt = 032 -3r-gg2((~2' (~1),

(2)

where the coupling terms gl,2 are 2rt-periodic in both arguments, and e is the coupling coefficient. The phase space of Eq. (2) is a 2-torus, and with the usual construction of the Poincar~ map this system can be made equivalent to a circle map, with a well-known structure of phase-locking intervals (Arnold's tongues) [50]; each of the intervals corresponds to a n:m synchronization region. This picture is universal and its qualitative features do not depend on the characteristics of the oscillations and of the external force (e.g. nearly sinusoidal or relaxational), and on the relation between amplitudes. Analytically, one can proceed as follows. The interaction between the oscillators essentially effects the evolution of their phases if the frequencies COl,2 are in resonance, i.e. if for some integers n, m we have nml ~ ram2.

Then, in the first approximation, the Fourier expansion of the functions 91,2 contains slowly varying terms ~ nq~1 - m q ~ 2. This suggests to introduce the generalized phase difference, (Pn,m(t) = /'/q~l (t) -- m~)2(t ).

(3)

Subtracting Eq. (2) and keeping only the resonance terms, we get d%,m d-----i-= ncol - m~o2 + gG(q~n,m),

(4)

where G(.) is 2n-periodic. This is a one-dimensional ODE that admits solutions of two kinds: fixed points or periodic rotations of %,m" The stable fixed point corresponds to perfect phase locking q~,,~ = const.; periodic rotations describe quasiperiodic motion with two incommensurate frequencies in the system (2). In the analytical treatment of the Eq. (2) we have neglected nonresonant terms, what is justified for small coupling. With nonresonant terms, the condition of synchronization for periodic oscillators should be generally written as a phase locking condition [nq~1(t) - mq~2(t ) - 81 < const.,

(5)

M. Rosenblum et al.

286

where 8 is some (average) phase shift, or as a frequency entrainment condition n~l -- mr22,

(6)

where /d~l.2 / f21.2- (~ dt " We emphasize, that in the synchronized state the phase difference is generally not constant but oscillates around 8. These oscillations vanish in the limit of very small coupling (correspondingly, the frequency mismatch neol - me02 must be also small), or if the coupling depends only on the relative phase: 91.2 = g l , z ( n ~ l - m~2)-

2.2. Noisy oscillators In general, both properties of phase and frequency locking (Eqs. (5) and (6)) are destroyed in the presence of noise ~(t) when instead of (4) one has dq)n,m

d~

= ntol - mr

+ cG(q),.m) + ~(t).

(7)

For small noise the stable phase dynamics is only slightly perturbed. Thus the relative phase q),.m mainly fluctuates around some constant level (former fixed point). These nearly stationary fluctuations may be interrupted by phase slips, where the relative phase changes relatively rapidly by -t-2rt. Thus, strictly speaking, the phase difference is unbounded and condition (5) is not valid anymore. Nevertheless, the distribution of the cyclic relative phase k}ln.m - - (Pn,m

mod 2 rt

(8)

has a dominating peak around the value corresponding to the stable fixed point [51]. Presence of this peak can be understood as the phase locking in a statistical sense.

If the noise is weak and bounded, then the phase slips are impossible and there exists a range of frequency mismatch ncol - m r where the averaged condition of frequency locking (6) is fulfilled. Near the boundaries of the Arnold tongue the noise causes phase slips and the transition out of the synchronous regime is now smeared. If the noise is unbounded, e.g. Gaussian, the probability of a slip to occur is nonzero even for n t o l - meo: --0, so that strictly speaking the region of frequency locking shrinks to a point. As this probability is (exponentially) small for weak noise, practically the synchronization region appears as an interval of nr -me02, where nf~l ~ m~2. Within this region, the distribution of the cyclic relative phase is not uniform, so that one can speak of phase locking. In the case of strong noise, the phase slips in both directions occur very frequently, so that the segments of nearly constant relative phase are very short and time course of r looks like a random walk, that is unbiased in the very center of the synchronization region and biased otherwise. The synchronization transition is

Phase synchronization: from theory to data analysis

287

now completely smeared and, hence, synchronization appears only as a weakly seen tendency. 2.3. Chaotic oscillators

For the periodic oscillator the phase was introduced in Eq. (1) as a variable corresponding to the shift along the limit cycle, and, hence, to the zero Lyapunov exponent. Any autonomous continuous-time dynamical system with chaotic behavior possesses one zero Lyapunov exponent that corresponds to shifts along the flow, therefore we expect that phase can be defined for this case as well. Suppose we can define a Poincar6 secant surface for our autonomous continuous-time system. Then, for each piece of a trajectory between two cross-sections with this surface we define the phase as a piece-wise linear function of time, so that the phase increment is 2rt at each rotation: d~p(t) - 2 rt

t-t~ tn+l -- t~

+ 2 rtn,

tn

o 06 0 04

i01

0 o2

i0 ~

/ I

i0

0

101

I

i0

.... I

2

S

i0

"

3

'[

1

04

AA

I

I0

0

5

O0 I0

0

i0

i

i0

2

S

i0

3

I0

4

Fig. 16. The cluster size distribution (a) is shown for a sequence of 100 sliding-reference subtracted frames. The original frames (resolution 640 x 480) have been coarse grained to frames of resolution 166 x 160 by averaging the gray scales over areas of size 4 x 3 pixels. Subsequently the frames are subject to binary filtering. Calcium activity of less than 50% of the average is considered not calcium active (0). The other sites are considered calcium active (1). Then the frames are stacked in time and clusters decomposition is applied. The histogram of cluster sizes exhibits for small clusters a power law with an exponential cut-off and a slower decay for larger clusters that form the backbone of the calcium pattern. The coverage distribution v(s) has a peak at small clusters and another one at large clusters. This broad distribution of relevant cluster sizes causes the cluster entropy to be relatively large (S - 4.1). rosurgery operating room. Hyperexcitable vs. normal EEG activities are recorded for the tissues that had been characterized in the imaging experiments. Viable cultures which exhibit behaviors characteristic of the diseased tissues can be produced even after two days in transit. Cells cultured from regions of hyperexcitable EEG activity show abnormal responses to neurotransmitter. Cultures have been established for several forms of epilepsy including medial temporal lobe disease, cortical tumors, hyperexcitable regions of the cortex and for the purpose of these studies we have also cultured a Tuberous Sclerosis (TS) case. The Children's Medical Center in Cincinnati supports a clinic dedicated to the treatment of this inherited disorder [50], which is characterized by mental retardation, epilepsy and skin manifestations. So called giant cell astrocytomas may develop within the brain. A tumorous lesion within the cortex of a 17 month old patient was cultured following surgical resection to alleviate epileptiform seizures. Subependymal giant cells are apparent in these cultures (see Fig. 18). The giant cells generally reside on top of a normal appearing astrocyte layer. One such giant cell is localized within an asterix in the left panel of Fig. 20A. Extensive process formation and a large nucleolus, which has stained heavily with the Ca 2+ dye, characterize this cell. These cells are common in these cultures. A second giant cell (asterix in Fig. 20B) has well-developed projections to neighboring cells that are reminiscent of the end feet of normal astrocytes. This cell however is not integrated

346

P. Jung et al.

a

.~

b

c

.~

d

e I~

~l,

~

,~

"

!

--"

*

#I~L,. ~:-:~:~

g

:-

~

h ..% ,, ', ,,~ ~

,~ ,,

Fig. 17. The reconstructed waves (red) after applying a space-time filtering that ignored space-time clusters of size less than 100 is overlaid on the original frames (both coarse-grained to a resolution of 160 • 160). Time proceeds from (a) to (i). The reconstruction based on the cluster decomposition allows to identify coherently connected calcium activity and to eliminate background activity without affecting the larger structures.

into the astrocyte syncytia that sits on top of. Neurons show extensive changes in morphology (dysmorphism) including extensive small processes as seen in the neuron below the giant cell in Fig. 18B. The neuron to the right of the giant cell in this panel exhibits an unusual cell body as well. In contrast, the neuron at the top in this panel has a normal appearance with a rounded cell body and long projection off to the right. Numerous immunohistochemial studies of the giant cells have shown they contain astrocyte as well as neuronal markers [51].

Statistical analysis and modeling of calcium waves

347

Fig. 18. A two-year-old patient underwent neurosurgery for treatment of intractable seizures associated with tuberous sclerosis. Surgical specimens from the posterior temporal lobe were cultured [22] and imaged using confocal microscopy. Fig. A shows relatively normal astrocytes (a) with a reactive astrocyte (*) on the upper surface of the astrocyte syncytium. In Fig. B several neurons and a second reactive astrocyte (*) are also superficial in the culture. The reactive astrocytes share the morphology of both neurons and astroytes (Taraszewska et al. 1997). Extensive process formation (See Fig. A), or elaborate end feet (see Fig. B) are characteristic of reactive astrocytes. The frequent polymorphic astrocytes in the culture are laden with eosinophillic granules. In addition, neurons can exhibit normal morphology (upper left, Fig. B) or varying degrees of abnormal development (B).

,.

9

,

.::

2-

9

o,

.~, ,

Fig. 19. A sequence of subtracted snapshots of calcium activity of a culture of astrocytes from TS-tissue is shown from left to right. The time interval between two snapshots is 1.7 s. Higher calcium activity is coded in larger gray scale. One can observe local islands of calcium activity but no large-scale organization.

Often giant cells respond to neurotransmitter exposure with an increase in intracellular Ca 2+, however a significant number of these cells do not respond at all to neurotransmitter.

348

P. Jung et al.

.i,L

"

:

-

..,

.

,

9 .

,.

,

2

9.

.

:

.~ .

..

,.

-,

.

9

.

,

9 [..

9

,.

, 9

.,.:

.

,-.

.

,o

.

..

9

Q

-

,,

.

.~.~

,.

9

-..

-.

,p. 9 9

.

.

..

-,

.

.

,

..., ..

.

~

.~

...

:.

Fig. 20. A sequence of subtracted snapshots of a calcium activity of a culture of astrocytes from TS-issue is shown from left top to right bottom. The time interval between two snapshots is 1.7 s. Higher calcium activity is coded in larger gray scale. One can observe two dramatic events of a large-scale organization of calcium activity in the form of cylindrical waves.


349

I0 s

0.30

104

9

0 25

,,., v C 102

>

0 15 0 10

10 ~ 0 o5 100

J 100

9149

I

I

I

I

101

102

103

104

S

0 O0 lO ~

lO 1

10 2

s

10 3

10 4

Fig. 21. The cluster size distribution (a) obtained from the TS data at 40 ~m Kainate is shown. All the clusters larger than s = 1000 belong to the rare of dramatic wave events. In comparision with Fig. 16, much less larger clusters are observed indicating mostly local astrocyte signaling. The coverage distribution v(s) (b) differs dramatically from that obtained from the rat-brain tissue. The relative coverage by smaller clusters is almost twice as high. The coverage by large clusters is smaller. If one ignores these rare dramatic wave events, these results imply that the incoherent activity in the epileptic case is higher as in the normal case.

Multiple identical coverslips (n = 32 coverslips) were cultured with the TS case cells described above. All coverslips were treated identically, i.e. they were rinsed with the same saline, stained with the same Ca 2+ dye (2 ~tm Fluo-3AM), imaged within a 48 h period of each other using an identical imaging paradigm (25 frames in saline, 400 frames in neurotransmitter, followed by 50 frames washout) and exposed to equivalent neurotransmitter solutions. Images were collected to the hard drive of our confocal scanning laser microscope (Nikon PCM2000) every 1.7 s. These images were then exported to a writable CD drive. In Fig. 19 a sequence of snapshots is shown. We have applied subtraction of a sliding reference frame (3 frames gap). For most of the time, calcium activity includes only a few cells. This pattern, however, is interrupted by drastic events where one or two waves involving virtually all cells on the coverslip are elicited (see Fig. 20). We have not observed this behavior in cultures of the healthy rat hippocarnpal astrocytes. This observation indicates at least temporary strong coupling between the astrocytes in the culture that leads to a fast and strong wave. During these episodes, the cells in the entire culture appear synchronized in their calcium activity. After these events the synchrony is lost and the calcium activity is local. Applying the cluster decomposition, we correspondingly see a difference in the cluster size distribution compared to that of healthy ran brain cells (see Fig. 21). There are only very few large clusters, corresponding to the dramatic events when a calcium wave occupies the entire culture. The speed of the wave well exceeds the speed of the calcium wave fronts observed in the rat brain tissue in the previous

350

P. Jung et al.

section. This indicates a more rapid response of the diseased astrocytes in comparison with astrocytes from normal tissue. The cluster entropy can be calculated based on the normalized cluster size distribution using Eq. (12). During typical time intervals with no dramatic wave events, the cluster entropy of the epileptic culture was about S = 2.1 and well below the entropy of the cluster entropy of the healthy rat brain cultures (S = 4.5). Including the rare wave events, the entropy of the epileptic culture increased to about S = 3.3, which is still well below the cluster entropy of the rat brain culture. 6. Conclusions

Calcium waves in glial-cell cultures exhibit many features of excitable waves in a noisy environment. We have reviewed previous work on waves in excitable media coupled to a noisy environment. A cluster decomposition method to quantitatively characterize noisy spatiotemporal patterns has been reviewed and applied to computer models as well as to calcium waves in cultured astrocytes stemming from normal and epileptic tissue. The cluster decomposition allows to assign thermodynamic properties such as a cluster entropy to the patterns. Preliminary results indicate that cluster entropy is lower in epileptic than in healthy cell cultures.

Acknowledgements This work has been supported by the National Science Foundation (grant # 0078055) and by the National Institute of Environmental Health (grant # IS08470).

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17.

Jung, P. and G. Mayer-Kress (1995) Phys. Rev. Lett. 74, 2130-2133. Cross, M.C. and Hohenberg, P.C. (1993) Rev. Mod. Phys. 65 851-1112. Winfree, A.T. (1974) Sci. Amer. 230, 82-95. B~ir, M. and Eiswirth, M. (1993) Phys. Rev. E 48, R1635. Lechleiter, J., Girard, S., Clapham, D. and Peralta, E. (1991) Nature 350, 505-508. Charles, A.C., Merrill, J.E., Dirkson, E.R. and Sanderson, M.J. (1991) Neuron 6, 983-992. Cornell-Bell, A.H., Finkbeiner, S.M., Cooper, M.S. and Smith, S.J. (1990) Science 47, 470--473. Sanderson, M.J., Charles, A.C. and Dirkson, E.R. (1990) Cell Regulation 1, 585-596. Guthrie, P.B., Knappenberger, J., Segal, M., Bennett, M.V.L., Charles A.C. and Kater, S.B. (1999) J. Neurosci. 19, 520-528. Kadar, S., Wang, J. and Showalter, K. (1998) Nature 391, 770-772. Sneyd, J., Keizer, J. and Anderson, M.J. (1995) FASEB J. 9, 1463-1472. Jung, P., Cornell-Bell, A.H., Madden, K. and Moss, F.J. Neurophysiol. (1998) 79, 1098-1101. Jung, P. (1997) Phys. Rev. Lett. 78, 1723-1726. Gammaitoni, L., Hanggi, P., Jung, P. and Marchesoni, F. (1998) Rev. Mod. Phys. 70, 223-287. Doughlas, J.K., Wilkens, L., Pantazelou, E., and Moss, F. (1993) Nature 365, 337-340. Levin, J.E. and Miller, J.P. (1996) Nature 380, 165-168. Gluckman, Bruce J., So, Paul, Netoff, Theoden I., Spano, Mark L. and Schiff, Steven J. (1998) Chaos 8 588-598.


351

18. Jung, P., Cornell-Bell, A.H., Kadar, S., Wang, J., Showalter, K. and Moss, F. (1998) Chaos 8, 567-575. 19. Cornell-Bell, A.H. and Finkbeiner, S.M. (1991) Ca 2+ waves in astrocytes, Cell Calcium 12, 185-195. 20. Finkbeiner, S.M. (1993) Glia 9, 83-104. 21. Cornell-Bell, A.H. and Willimson, A. (1993) in: Biology and Pathology of Astrocyte-Neuron Interactions, eds Federoff et al. pp. 51-65, Plenum Press, New York. 22. Lee, S.H., Magge, S., Spencer, D.D., Sontheimer, H. and Cornell-Bell, A.H. (1995) Glia 15, 195-202. 23. Bordey, A. and Sontheimer, H. (1998) Epilepsy Res. 32 286-303. 24. During, M.J. and Spence, D.D. (1993) Lancet 341, 1607-1610. 25. Scheyer, R.D. (1998) in: Progress in Brain Research, eds. O.P. Ottersen, I.A. Langmeon, and L. Gjerstad. Vol. 116, pp. 359-369. Elsevier, New York. 26. O'Connor, E.R., Pizzonia, J.H., Spencer, D.D., de Lanerolle, N.C. (1996) Epilepsia 37, (Suppl. 5), 51. 27. O'Connor, E.R., Sontheimer, H., Spencer, D.D., de Lanerolle, N.C. (1998) Epilepsia 39, 347-354. 28. Scheffer, I.E. and Berkovich, S.F. (1997) Brain 120, 479-490. 29. Wallace, R.H., Wang, D.W., Singh, R., Scheffer, I.E., et al. (1998) Nat. Genet. 19, 366-370. 30. Ronen, G.M., Rosales, T.O., Connolly, M. et al. (1993) Neurology 43, 1355-1360. 31. Singh, N.A., Charlier, C., Staufer, D., et al. (1998) Nat. Genet. 18, 25-29. 32. Biervert, C., Schroeder, B., Kubisch, C., et al. (1998) Science 279, 403-406. 33. Goldbeter, A., Dupont, G. and Berridge, M.J. (1990) Proc. Nat. Acad. Sci. USA. 87, 1461-1465. 34. Goldbeter, A. (1996) Biochemical Oscillations and Cellular Rhytms: The Moleculare Bases of Periodic and Chaotic Behaviour. Cambridge University Press, Cambridge. 35. Lechleiter, J. and Clapham, D. (1992) Cell 69, 283-294. 36. Sanderson, M.J., Charles, A.C., Boitano, S. and Dirksen, E.R. (1994) Molec. Cell Endocrinol. 98, 173-187. 37. Kim, W.T., Rioult, M.G. and Cornell-Bell A.H. (1994), Glia 11, 173-184. 38. Boitana, S., Dirkse, E.R. and Sanderson, M.J. (1992) Science 258, 292-295. 39. Muller, T., Moller, T., Berger, T., Schnitzer, H. and Kettermann, H. (1992) Science 256, 1563-1566. 40. Burnashev, N., Khodorova, A., Jonas, J., Helm, J., Wisden, W., Monyer, H., Seeburg, P.H. and Skemann, B. (1992) Science 256, 1556-1570. 41. Egebjerg, J. and Heinemann, S.F. (1993) Proc. Natl. Acad. Sci. 90 755-759. 42. Goldman, W.F., Yarowsky, P.J., Juhaszova, M., Krueger, B.K. and Blaustein, M.P. (1994) J. Neurosci. 14, 5834-5843. 43. Golovina, V.A., Bambrick, L.L., Tarowsky, P.J., Krueger, B.K. and Blaustein, M.P. (1996) Glia 16, 296-305. 44. Fohlmeister, C., Gerstner, W., Ritz, R. and Van Hemmen, J.L. (1995) Neural Computation 7, 905-914. Myocardium. 45. Kistler, W.M., Seitz, R. and Van Hemmen, J.L. (1997) Physica D 114, 273-295. 46. Jung, P. and Mayer-Kress, G. (1995) CHAOS 5, 458 (1995). 47. Zoldi, S.M. and Greenside H.S. (1997) Phys. Rev. Lett. 78, 1687. 48. Zoldi, S.M., Liu, J., Bajaj, K.M.S., Greenside, H.S. and Ahlers, G. (1998) Phys. Rev. E58, R6903. 49. Jung, P., Wang, J. and Wackerbauer, R., Preprint. 50. Franz, D.N. (1998) Seminars in Pediatric Neurology 5, 253-268. 51. Hirose, T., Scheithauer, B.W., Lopes, M.B.S., Gerber, H.A., Altermatt, H.J., Hukee, M.J., Vandenberg, S.R. and Charlesworth, J.C. (1995) Acta Neuropathologica 90, 387-399.


C H A P T E R 11

Neurones as Physical Objects" Structure, Dynamics and Function C. M E U N I E R Laboratoire de Neurophysique et Physiologie du Syst~me moteur (EP 1848 CNRS), UniversitO RenO Descartes, 75270 Paris cedex 06, France


I. S E G E V Department of Neurobiology, Institute of Life Sciences, and Interdisciplinary Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel


353

Contents

1.

Introduction ............................................................

355

1.1. W h a t e x p e r i m e n t s tell us a b o u t n e u r o n e s . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

355

1.2. A n overview of theoretical a p p r o a c h e s 1.3. O r g a n i z a t i o n o f the p r e s e n t c h a p t e r 2.

....................................

......................................

N e u r o n a l excitability: dealing w i t h time scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1.

T h e e l u c i d a t i o n o f n e u r o n a l excitability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2.2. N e u r o n e s as n o n - l i n e a r systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3. 3.

Discussion

.........................................................

Conduction and distribution of action potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.

Action potential conduction

3.2.

Spike c o n d u c t i o n failures

............................................

..............................................

3.3. P r e s y n a p t i c i n h i b i t i o n . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4. 4.

5.

Discussion

.........................................................

Dendrites and synaptic integration

...........................................

359 363 364 364 370 384 396 396 400 405 410 412

4.1.

H o w do n e u r o n e s utilize dendrites? A l o n g s t a n d i n g q u e s t i o n . . . . . . . . . . . . . . . . . . . .

412

4.2.

Passive cable t h e o r y

416

4.3.

T h e synaptic s h u n t

4.4.

Non-linear membrane properties .........................................

428

4.5.

D e n d r i t i c spines: a m i c r o c o s m . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

434

4.6.

Discussion

.................................................. ...................................................

.........................................................

422

438

Conclusion .............................................................

446

Acknowledgements .......................................................

448

Appendix: The H o d g k i n - H u x l e y model

448

.......................................

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

354

452

I. Introduction 1.1. What experiments tell us about neurones 1.1.1. The classical view o f the neurone

If one looks at pictures of neurones in classical textbooks, one always encounters the same stereotyped view of an exquisite branching structure composed of three morphologically distinct regions: (i) The dendritic tree, (ii) the soma thinning down into the axone hillock, and (iii) the extended axonal tree (see Fig. 1). Most textbooks also assign to each of these three regions a specific role. The dendritic tree is described as a collector of the massive synaptic input that the neurone receives from many other neurones. Synaptic currents are "integrated" in this dendritic tree and in the soma, which leads to the depolarization of the initial segment of the axone. There, if this depolarization is large enough, isolated action potentials or spike trains are initiated. The pattern of action potentials in a train is controlled by a set of voltage-dependent conductances located at the somatic membrane and activated already at potentials below the firing threshold. These spike trains then propagate faithfully along the axone and provoke transmitter release at terminal and en passant synaptic boutons, which leads to post-synaptic events in target cells. The neurone is thus described as a deterministic and "dynamically polarized" device [1,2] that reliably converts the synaptic inputs it receives into a spike train delivered to all the neurones it contacts. This vision, originated in Ramon y Cajal's histological observations and impressive intuition of the workings of neurones (see Fig. 2), was progressively substantiated by electrophysiology experiments. These experiments unravelled the voltage-dependent mechanisms that underlie non-decremental propagation in myelinated and unmyelinated axones [3,4], and elucidated, first on lumbar motoneurones [5,6] and later on different classes of central neurones, in particular hippocampal and neocortical [7] pyramids, thalamo-cortical and Purkinje cells, what were the respective roles of the axone and of the soma in the firing of action potentials and the patterning of the discharge [8]. 1.1.2. Alternative ideas

Over the years many interesting ideas were nonetheless proposed, that departed from the simple and attractive view of the neurone presented above or enriched it. For example, Llinfis and his collaborators suggested that calcium spikes could be fired locally in the dendrites of Purkinje cells, and emphasized the excitable properties of these dendrites [9]. Eccles and coworkers [10] discussed the possibility that action potentials might invade the dendritic tree of motoneurones. Henneman speculated that the action potentials might suffer conduction failures, due to the electrotonic architecture of axones [11]. However all these ideas remained for a long 355

356

C. Meunier and L Segev

Fig. 1. Neurones are the basic building blocks of the nervous system. A pyramidal neurone from the cat visual cortex stained intra-cellularly with a dye and its three-dimensional shape was reconstructed. The axone (red) has a dense local arborization as well as lateral projections. The axone displays a total of 4105 synaptic boutons (yellow), most of which are known to form excitatory synapses on the dendritic trees of other pyramidal cells. The dendritic tree of this cell is shown in green. Courtesy J. Andersen and K. Martin. Calibration: 100 lam.

time unsubstantiated by clear direct experimental evidence. In particular it was impossible to record routinely from dendrites and information on their properties and electrical behaviour was inferred from recordings in the soma. As a consequence the nature of "synaptic integration", a concept introduced by Charles Sherrington [12,13], and its biophysical substrate remained obscure and the most widely accepted view was still that dendrites were passive structures onto which summation of synaptic potentials took place. 1.1.3. New experimental techniques This situation has gradually changed over the last 20 years. The development of new techniques - in particular infra-red D I C video microscopy [14] and i m m u n o l a b e l l i n g -

Neurones as physical objects." structure, dynamics and function

357

A /

11 ~, i'

u

i Fig. 2. Ramon y Cajal's law of dynamic polarization. This "law" states that the nerve impulse flows from the dendritic branches and the soma to the axone (arrows). Shown are the pattern of axo-dendritic connections between cells in the cerebral cortex and the detailed morphology of the dendritic branches (thicker lines) and dendritic spines (the many thin thorns emerging from the apical tufts - on top), the soma and the axonal tree (thinner smooth lines). Dendrites and soma receive input from many cells via terminal arborization of axone collaterals. made it progressively possible to measure in vitro the passive membrane properties and the density and kinetics of voltage-dependent channels at various locations on the neurone membrane, and not only at the soma. It is now becoming feasible to perform such measurements also in vivo [15,16]. One can also record in vitro and in vivo the local variations of the intra-cellular calcium concentration and the membrane potential of neurones at various sites, thanks to calcium and voltage sensitive dyes [17], and two photons microscopy [18,19,20]. Dynamic clamp technique allows experimentalists to investigate the effects of a given voltage-dependent current at the soma on the firing pattern of a neurone [21]. Recording simultaneously from a presynaptic cell and one or two of its target cells in hippocampal and neocortical slices is now possible yielding information about changes in synaptic dynamics during activity [22,23].

358


The results obtained with these new techniques show: 9 that the dendrites of many central neurones are electrically heterogeneous and endowed with a set of voltage-dependent ion channels (see [24,25] for recent reviews), and that these channels can shape the response of neurones to synaptic stimuli. For instance, it was directly demonstrated, both in vitro and in vivo, that full blown calcium spikes can be generated in the dendritic tree of central neur o n e s - such as neocortical pyramidal cells [16,135] - and that they may strongly affect the discharge pattern of these neurones. It was also shown in vitro that action potentials may invade the apical dendritic tree of pyramidal neurones not only because of passive propagation but also thanks to voltage-dependent membrane conductances [26]; 9 that the typical membrane properties, both passive and active, of the soma and dendrites are quite similar. The hypothesis of a large somatic shunt [27] was rejected on the basis of new conductance estimates relying on patch electrodes rather than sharp electrodes [28], and no evidence was found for a particularly high density of sodium channels on the soma and axone hillock [29]; 9 that the stochastic openings and closings of ion channels may impart some variability to the discharge of pyramidal neurones in vitro and affect their response to stimuli [30,101,102]; 9 that the axone does not necessarily convey the same output signal to all its neuronal targets; 9 that neurones are prone to slow modulations of their excitability (neuromodulation, activity-dependent regulations [31,32], circadian rhythms [33], etc.).

1.1.4. Experimental limitations

Still the way neurones behave in physiological (or pathophysiological) conditions in situ in a network remains elusive. How do they handle synaptic inputs in such conditions and respond to them? What is their pattern of activity and how does it depend on the task performed? Can we characterize in some physiologically relevant way the input-output properties of neurones in such conditions? Performing in vivo experiments to answer such questions is difficult but feasible to an extent. Spinal physiology mostly progressed through intra-cellular recordings of neurones in anaesthetized cats that enabled both to largely understand the membrane properties of motoneurones [6], and to unravel the functional connectivity of important spinal circuits. However, anaesthesia strongly reduced background synaptic activity and suppressed neuromodulation (so that motoneurones do not display plateau properties in such an in vivo preparation). In the cortex, intra-cellular recordings of neurones in vivo submitted to an intense synaptic bombardment can now be performed [34,35]. Some in vitro preparations, such as slices of cerebellar cortex [36], display spontaneous activity and can be used to study network activity but most in vitro experiments are mainly useful for investigating cellular properties: nature, kinetics and distribution of ionic channels on the neuronal membrane, voltage transients elicited by stimulation, etc. This can shed some light on possible mechanisms at


359

work at the cellular level in physiological conditions, and helps to identify some basic operating principles. But the functional identification of cells is difficult and extrapolating from in vitro preparations to physiological conditions is unwarranted. For instance, inferring from the response to a single test input what would happen in a physiological situation where numerous synapses are activated is made extremely difficult by the non-linearities of the neuronal membrane (mutual shunt of synaptic inputs, activation of voltage-dependent currents). This set of fundamental problems deserve our attention. Moreover one must address the issue of universality. Many different classes of neurones have been defined on the basis of their characteristic morphological properties or their discharge pattern. Neurones of a given class are often specific to a given nervous structure and they are endowed with the same set of voltage-dependent currents. To what extent can we identify operating principles shared by all these classes of neurones? May we rightfully speak of canonical neurones [37]? We know that single spike generation mechanisms are essentially the same in all neurones that emit spikes. But can we understand synaptic integration on the basis of a few general principles? Moreover the detailed morphology and the intrinsic properties of neurones widely vary even inside a given class [38,39]. How should we then model a neurone? Is it not the case that the behaviour of a model reflects the choice of a specific set of parameters rather than the general operation of neurones in a given class? In view of all these conceptual and technical difficulties, we doubt that unravelling the operating principles of neurones can be achieved by purely experimental approaches. As this may seem an unwarranted statement, let us then examine what lessons we may learn from the past.

1.2. An overview of theoretical approaches 1.2.1. Analytic theories Our present understanding of neurones is not grounded only on morphological and physiological experiments, but also on theoretical approaches. A genuine understanding of spike generation was reached only when Hodgkin and Huxley could write down differential equations that made the non-linear nature of actions potential explicit [3]. This "Hodgkin-Huxley model" does not constitute the first attempt for a mathematical description of neuronal excitability. Indeed a phenomenological model of neural firing was designed by Lapicque as early as 1907 [40]. More recently important progresses were made in understanding the role of subthreshold currents in such phenomena as bursting and post-inhibitory rebounds by relying on theoretical concepts (bifurcations, singular perturbation theory, etc.) and techniques initially developed for understanding dynamical systems with small number of degrees of freedom [41]. Similarly methods introduced to deal with extended dynamical systems may be used to better understand spike conduction in axones [42], while the impact of the random openings and closings of channels on cell excitability could be studied on stochastic variants of Hodgkin-Huxley like models [43].

360


The other important feature of neurones, their capability to "integrate" synaptic inputs, also benefited a lot from theoretical approaches. The first genuine progresses on that issue were due to W. Rall who developed the well-known "cable theory of the neurone" [44]. One may debate on how much of synaptic integration is explained by this linear theory of dendrites, especially now that it becomes clear that many voltage-dependent channels are embedded in the dendritic membrane. One might even consider that developing cable theory was more work than it was worth. We are not of that opinion. Cable theory provided the first solid ground for understanding how post-synaptic potentials would interact and spread in dendritic structures. It introduced extremely useful notions such as space and time constants, emphasized the notion of electrotonic architecture, gave important insights on what could happen in dendrites and made it possible to derive information on dendrites from intra-cellular recordings at the soma.

1.2.2. Neurophysics Much theoretical work on neurones was performed by researchers with a strong background in Physics, using concepts and methods from this scientific discipline. For so much, may we define a physics of neurones? Neurochemistry is recognized as a relatively autonomous branch of Neurosciences. Can we also legitimately speak of Neurophysics? It is widely accepted that the electrical behaviour of neurones is grounded in a specific form of microphysics, the biophysics of excitable membranes. However Hodgkin-Huxley theory [3] or cable theory [44] does not directly deal with such microscopic physical properties of the neuronal membrane as the properties of individual channels. They consider a more macroscopic level of description and address cooperative phenomena that involve the opening of many such passive or voltage-dependent channels: generation and conduction of the action potential, spread of synaptic potentials. Also they use a rigorous and operative mathematical description of phenomena taking place in neurones. They might therefore be considered as applications of mathematics. Still cable theory relies on hypotheses and equations that were introduced almost one century before to analyse the physical problem of electric conduction on telegraphic cables and propagation of a solitary wave in a non-linear medium is a general problem encountered under many forms in Physics. Therefore we think that we can genuinely address dendritic integration or spike conduction as physical problems, in as much as concepts and methods derived from Physics as well as Mathematics are used to investigate these issues. The multi-faceted relationship between Physics and Neurophysiology is better appreciated in a historical perspective. Neurophysiology arose as a science from the first attempts to describe in physical terms the nervous system activity and to unravel the underlying mechanisms. The first important step in this direction was the progressive elucidation of the electric nature of the nervous influx, starting with Galvani's works at the end of the 18th century [45] and parallel to the development of electromagnetism: existence of the "animal electricity", conduction velocity along nerves [46], biophysics of membranes [47]. Physics also provided the appropriate experimental tools for studying neural activity and supported in particular the development of electrophysiology. For instance, nervous influx could be definitively

Neurones as physical objects: structure, dynamics and function

361

assigned to the propagation of action potentials, the nature of which became amenable to investigation, only when high temporal resolution recordings enabled researchers to visualize the shape of these fast events. In France, this occurred only in the 1940s when Alfred Fessard introduced the oscilloscope in neurophysiology laboratories. Metaphors borrowed from Physics were also highly influential in Neurophysiology. In his "Treatise of Man" [48] Ren6 Descartes introduced a number of notions that still shape our way of thinking. Descartes idea that both perception and action relied on the exchange of physical signals between the central nervous system and the periphery (see Fig. 3) had a decisive influence on the development of Neurophysiology, though the electric phenomena taking place in nerves were far remote from his conceptions. Most importantly, his dualistic approach led him to consider the nervous system as a physical machine. This led to the idea that the mental operations of the brain were nothing else than the direct product of its mechanistic activity (see Fig. 4), emphasized one century later by the materialist Julien Offrieu de La Mettrie [49]. Descartes speculations marked neural mechanisms as legitimate objects of scientific investigations, thus signing the birth certificate of Neurophysiology. Much more recently Hopfield [50], capitalizing on an analogy with disordered magnetic materials known as spin glasses, introduced an enlightening and analytically tractable model that demonstrated that many activity patterns could be stored in a large network in a robust and distributed manner by Hebbian modifications of connection weights. This seminal work opened the door to the rigorous study of attractor neural networks and, more generally, of collective phenomena in large systems of neurones.

",

N~]

I

Fig. 3. Descartes, the founder of reflex theory? Descartes proposed in his first work on mind and body, De homine, a mechanism of automatic reaction to external events. This treatise was completed in Amsterdam in 1633 but Descartes renounced to publish it after Marin Mersenne wrote to him of Galileo's problems with the Inquisition. Consequently this treatise appeared only years after the death of Descartes.

362


Fig. 4. Vaucanson's automata. Automata exerted a great fascination throughout the 18th century. The most famous automata maker of this time is certainly Jacques de Vaucanson (1709-1782), who built the tabor player and the transverse flute player shown on this postcard (from Conservatoire National des Arts et M~tiers (Paris), http://www.cnam.fr/museum/ musica_mecanica/a/9lossaire/vaucanson.html).La Mettrie wrote his "Treatise on the Manmachine" in this historical context.

One may still be rightfully reluctant to consider neurones and systems of neurones as genuine physical objects. It is clear that biological systems present striking differences with most inanimate physical systems. For instance they present a historical dimension since they slowly evolve over time, develop from the embryo and undergo plastic changes, all this under the pressure of various environmental constraints. Still such a historical perspective can be encountered also in Physics, in connection with Astrophysics, Geophysics, or when investigating the evolution in time of the mechanic properties of materials, for instance. The major difference with Physics is probably that biological systems are not the mere material substrate of phenomena but perform specific and highly organized functions. Physics may contribute to the design of technological devices but it is badly equipped to deal with the functional properties of neuronal systems. Nonetheless the approaches, concepts and methods developed in Physics can be capitalized on to study the dynamics of neurones and neuronal systems, that is, the nature and evolution in time of the spatial patterns of activity of these structures. The purpose of Neurophysiology is to elucidate how neuronal systems perform their functions. A better understanding of how the dynamical states of activity, that are widely thought to constitute the immediate substrate of functions, are determined by the underlying structural properties may help to solve this structure/function problem.


363

1.3. Organization of the present chapter The present chapter is not a comprehensive review of theoretical works on the neurone. We shall focus on a limited range of Neurophysical problems, and refer little to Computational Neuroscience. As a consequence, such topics as, for instance, information theory will not be dealt with. Also we shall adopt a particular viewpoint on the nervous system where the emphasis is put on action potentials. We are perfectly aware that neurones do not communicate by the sole mediation of spikes and that addressing physiological functions in terms of spiking activity of neuronal populations is not the only way to go. Our current understanding of sleep, for instance, probably owes as much to Neurochemistry as to the study of activity patterns in the brain. Accordingly we shall analyse in what follows some exemplary situations where analytical theory proved eminently useful in the past (Hodgkin-Huxley theory of the action potential, classification of bursters, cable theory, etc.) and discuss open issues that should in our opinion benefit from a rigorous well-managed physically oriented treatment. We shall respect the categories introduced by Ram6n y Cajal [1]: collection of synaptic inputs, firing of action potentials, distribution of action potentials to the target cells, but shall present them in an order reflecting the nature of the non-linear dynamics involved and not signal flow as proposed by the "law of dynamic polarization". We shall discuss first in Section 2 the generation of spike trains, as this constitutes the best established and most dramatic consequence of membrane nonlinearities. In this section we shall put into a historical perspective the contributions of Hodgkin and Huxley [3] - to which the Appendix is also fully d e v o t e d - and shows how the excitability of axones and neurones can be studied in the framework of dynamical systems with few degrees of freedom (that is, without taking into account the fact that neurones are spatially extended structures) using such approaches as bifurcation theory and (multiparameter) singular perturbation theory. So doing we shall emphasize how the existence of several time scales underlie and constrain the behaviour of the neurone. In contrast we shall stress the importance of space scales in Section 3 that discusses the propagation of action potentials on the axone, an extended medium with highly non-linear membrane properties. We shall first discuss this problem in terms of diffusion-driven propagation of an excitability front in a homogeneous or heterogeneous metastable medium. We shall largely dedicate the rest of this section to the physiologically important issue of whether axones distribute the same spike trains to all their neuronal targets or whether selective filtering of these spike trains occurs due to passive membrane properties (electrotonic conduction failures [11]), activation of voltage-dependent currents [51], or activation of axo-axonic synapses [52] (presynaptic inhibition). Section 4 will address the difficult issue of the role of dendrites: What does synaptic integration mean? Are dendrites mere collectors of inputs or do neurones utilize these branched structures for local processing of inputs? How can distal inputs have a significant effect at the soma? What may be the role of active dendritic

364


properties? We have yet no satisfying answer to these questions, in contrast to action potential generation and propagation on axones which were elucidated in the 1950s, and the role of somatic conductances unravelled in the 1980s. This situation largely stems from the fact that dendrites display intermediate features. They constitute neither an electrotonically compact compartment like the soma nor a quasi-infinite medium like axones. Both their passive and active properties very likely play an important role in their function. Moreover dendritic membrane properties are themselves modified by the ongoing synaptic activity. Many misconceptions about dendrites were clarified by passive cable theory, which set the basis for our current understanding of dendrites. Hopefully, the progressive development of an "active cable theory" will pursue this clarification process. All topics regarding dendrites could not be reviewed in depth in the present chapter. In particular, we did not dwell on frequency domain analysis of the dendritic filtering of periodic signals (impedance, resonance in non-linear dendrites, etc.) or stochastic signals. All along this chapter we shall try as much as possible to keep physiological conditions and functional implications as close as possible to the foreground. Accordingly each of the three main sections ends with a discussion of selected questions largely referring to the physiological relevance of the material discussed in the section. Also we shall privilege dynamical phenomena occurring at a macroscopic scale, and analytical studies of idealized models. Much less emphasis will be put on the channel level, and numerical simulations of detailed models of neurones [53]. For instance, we shall describe analytical methods but shall not dwell on numerical methods [54] and software packages (GENESIS [55], N E U R O N [56], SWIM [57], etc.) for simulating the dynamics of neurones. To justify this bias the conclusion of the chapter will be devoted to an unfair comparison of simple mathematically tractable models and detailed supposedly "realistic" models, and to a discussion of their relevance for understanding the experimental "reality" of the neurone. We hope that this chapter will convince the reader that the physiology of the neurone already benefited greatly from metaphors, theories and models which originated from other scientific fields, and particularly from Physics. We also hope that our conviction that genuine progress may still be expected from theoretical studies, and mostly from the further development of analytic theories provided they keep in close touch with physiological issues will be shared by the reader. We shall be greatly satisfied if these aims are fulfilled. If not we still hope that the reader will be interested in this chapter, whether he enjoyed it or was irritated by it. At the least we wish that this chapter will help to correct some misconceptions or prejudices with respect to theoretical work on neurones.

2. Neuronal excitability: dealing with time scales

2.1. The elucidation of neuronal excitability 2.1.1. Spikes Large amplitude events, during which the membrane depolarises by about 100 mV, and which last about 1 ms, constitute the hallmark of neuronal excitability

365


(see Fig. 5). These action potentials are used pervasively throughout the nervous systems of both invertebrates and vertebrates for signalling between neurones, and in particular for long distance communication. All neurones do not display the same firing pattern under current clamp experiments (see Fig. 6), and only a minority of them respond to a current step with a perfectly periodic discharge characterized by a single frequency. Some of them fire only a few spikes at the beginning of current injection (phasic response). Others fire continuously throughout current injection (tonic firing) but display a steep frequency decrease over the first few interspike intervals (adaptation). Even neurones that fire tonically and without significant adaptation cannot be grouped in the same category. While some neurones start firing at high rate as soon as the current threshold is reached, many others display a gradual increase in the frequency with the injected current from some low initial value near the current threshold. In addition many neurones from invertebrates and vertebrates recorded in vitro display periodic bursts of activity, at least in some regimes of activity or when submitted to neuromodulation (conditional bursters). Bursting results from a slow depolarization of the membrane due to an inward current, followed by a repolarization when some threshold is reached where the inward current inactivates or some outward current becomes activated. However, a variety of currents can be involved. The inward current may, for instance, be activated by the hyperpolarization and have a reversal potential close to the resting membrane potential (Ih) or it may be a calcium current de-inactivated by the hyperpolarization (IT). The detailed pattern of bursting also

A

B -" +40

-0

f

-70.;~ ie

A

A

Fig. 5. Action potential recordings by Hodgkin and Huxley. (A) A micro-pipette was inserted axially into a giant axone of the squid (clear space, 0.5 mm in diameter) with small nerve fibres on either side. (B) An action potential, 90 mV in amplitude and lasting 1 ms was initiated following membrane depolarisation. Time scale is marked by a sine wave in bottom, with 2 ms between peaks. Reprinted from [58] by permission of Macmillan Magazines.


366

t!,, I I,

I! l

_-_2__0n A ion to_phoresis

t:ii",; 1 ~'t I~II[,,lI,,~;,~ ...... tl!~

!

!

i~ !! i i:!Ii:!:!i',~!~!~i!!!i

'

"[ ...........

I

z

"

I,~

-20 nA iontophoresis

]

-.

!!,,,, ,,,,,,,,,,,,i!!1!i!1i ' ,~ ,,,: :ii :ili;~i~ii~ ~~'~i t

1~

-45..n A iontophoresis

.....I........

Fig. 6. Firing repertoire of a cortical pyramidal neurone in vitro. Firing was evoked by ejecting the excitatory transmitter glutamate close to the apical dendrite at 407 lam from the soma. Top panel: Repetitive burst of action potentials evoked by minimal glutamate ejection. Middle and lower panels: Increasing glutamate quantity elicited shorter periods of burst firing, followed by longer periods of regular firing [59].

varies from one class of neurones to another. A slow wave, sine-like or sawtooth-like (triangular bursters), may or may not underlie the periodic bursting. Spikes inside each burst may progressively decrease in amplitude or remain of the same height. Their instantaneous frequency may decrease throughout the burst, increase or display a symmetrical pattern (parabolic bursters). How to make sense of this diversity? Understanding the nature of action potentials and neural firing was a long process. At the turn of the century it had been recognized that the electric excitability properties of the membrane were due to ionic gradients between the intracellular and extracellular m e d i a - in particular potassium ion g r a d i e n t s - arising from the limited permeability of the membrane. This is known as Bernstein's "membrane hypothesis" [47]. It was also established that spike initiation was associated with a huge increase in the permeability of the membrane ("membrane


367

breakdown"). It had been noticed that nerves submitted to steady current injection behaved as relaxation oscillators [40]. However, the mechanisms underlying neuronal firing remained elusive, because it was not recognized that the membrane displayed non-linear properties. Understanding this fundamental point constitutes the most impressive contribution of Hodgkin and Huxley to Neurosciences. In a series of works, started in the 1930s and interrupted by World War II, they showed that both the firing of action potentials [58], and their non-decremental conduction in the axone, relied on the opening of voltage-dependent sodium channels [3], followed by the slower repolarization of the membrane. A few years later Coombs et al. [10] suggested that the same mechanisms underlay the firing of central neurones, which was subsequently confirmed. 2.1.2. Neurones as relaxation oscillators

The first attempt to provide an abstract mathematical description of tonic neural firing is much older than Hodgkin and Huxley's works. It dates back to Lapicque, who recognized in 1907 that the membrane response of nerves to small perturbations was linear [40] and proposed on that basis the "integrate-and-fire" model (see [60] for more on this model). During tonic firing the membrane potential of neurones slowly rises until a threshold is reached; the voltage ramp is then interrupted by the abrupt firing of an action potential followed by a resetting of the voltage to its initial value. This is typical of the relaxation oscillations arising from a threshold crossing. Lapicque's integrate-and-fire model is grounded on this observation and displays a sharp voltage threshold Vth. In the subthreshold voltage range (V < Vth) the evolution of the voltage is governed by a linear differential equation dV

Cm - ~ -- Gm(Vrest - V) --[-I,

(1)

where I is the injected current, Cm the membrane capacitance and Gm is the membrane conductance. No dynamical equation is specified in the suprathreshold voltage range: the neurone fires a fast action potential whenever Vth is reached (see Fig. 7), after which the voltage is reset to some fixed value Vreset (often taken to be the resting membrane potential Vrest). This is an extremely non-linear behaviour. In this way the integrate-and-fire model elegantly by-passes the issue of membrane non-linearity and remains analytically tractable. The current threshold Ith where the neurone starts to fire is easily calculated as the current value for which the voltage tends to Vth at large times Ith z Gm (Vth -- Vrest).

The firing frequency, f , is also easily computed for I > Ith by integrating Eq. (1) in the range from Vreset to Vth spanned by the membrane voltage during the interspike interval f -'-

(

z log 1 +

1 Gm( Vth_Vreset)~ .

I-Ith

)

(2)

368


V

action potential (instantaneous)

Vth subthreshold range

Vreset

mterspike interval

#" v

Fig. 7. Lapicque's integrate and fire model. Voltage trajectory of the Lapicque's model for constant current injection above the current threshold. This simple model accounts for many aspects of tonic firing: rheobase, voltage threshold, membrane repolarisation, smooth increase of the firing rate with the injected current, etc. Similarly the phase resetting curve can be readily computed. The amenability of the integrate-and-fire model to both analytical and numerical studies (though the presence of discontinuities sets some numerical problems [61]) explains the popularity of this model in studies of neural network dynamics. However, Lapicque's approach provides no explanation for the firing of action potentials by neurones. Action potential time course (instantaneous in Lapicque's original model), voltage threshold and reset threshold are incorporated "by hand" in the model. Also none of the non-linear effects (membrane rectification, refractoriness) observed experimentally can be correctly reproduced. Moreover, only the oscillatory aspect of discharge is accounted for. The behaviour of Lapicque's model, which is not a smooth dynamical system, near the current threshold departs significantly from what is expected for neurones firing at low frequency (see below): the firing frequency does not grow from 0 as the square root of the bifurcation parameter I - Ith, the voltage lingers near Vth which makes the model very sensitive to perturbations on a large part of the interspike interval. The high frequency behaviour is no more correct. Due to the resetting of the voltage to a fixed value and the inability of the model to account for spike amplitude reduction, the firing rate increases without bound with the injected current. 2.1.3. The H o d g k i n - H u x l e y model revisited

In contrast to the integrate and fire model, Hodgkin-Huxley unravelled the nonlinear properties of the neuronal membrane. Reading their 1952 paper [3] remains extremely enlightening. Hodgkin and Huxley used a very appropriate preparation, the giant axone of the squid, which allowed them to perform in vitro experiments, during which the ionic composition of the extracellular medium could be easily manipulated, and to insert a longitudinal electrode in the axoplasm, thus space clamping a whole section of the axone. This enabled them to study action potential


369

generation per se without having to deal (in a first stage) with the issue of propagation along the axone. In this way Hodgkin and Huxley reduced the study of axonal excitability to a purely temporal problem. We also note that Hodgkin and Huxley were fortunate in the choice of the squid giant axone as neural excitability involves only two voltage-dependent currents in this preparation: a transient sodium current and a persistent potassium current [62]. It was later found, starting with the work of Connor and Stevens [63,64], that other axones display additional ionic currents, in particular transient A-type potassium currents. Very importantly Hodgkin and Huxley did not restrict themselves to a heuristic explanation of action potential regeneration on the basis of the experimental data they obtained. They proposed a quantitative model that correctly predicted the time evolution of the membrane potential in current clamp experiments, both in the subthreshold regime where constant current injections evoke graded depolarizations and in the supra-threshold regime where periodic spiking occurs. This model consists in a system of four ordinary differential equations (see Appendix for how they were derived by Hodgkin and Huxley): dV

Cm--~- --

Gleak(~eak "- V) q- GNam3h(VNa - V) + GKn4(VK - V),

dm "r,m--~ -~ mc~ ( V) - m,

dh

9 u

(4)

( v ) - h,

(5)

T , n - ~ - - ncc(V) -- n.

(6)

dn

-

(3)

Here Cm is the capacitance of the patch of membrane clamped, Gleak, GNa, and GK the maximal conductances of the passive leak current, transient sodium current and delayed rectifier potassium current in this membrane patch, and ~eak, VNa and VK are the reversal potentials of these currents. The voltage dependence of the potassium current is described by the activation variable n: the actual conductance of the potassium current is equal to GKn 4. The variable n is governed by the first-order kinetic equation (6) where both the time constant, ~n, and the equilibrium value of n, n ~ ( V ) , are functions of V, which accounts for the voltage-dependence of the potassium current, n~ (V) is an increasing sigmoid function of V. The behaviour of the transient sodium current is more complex. It is governed not only by the activation variable, m, which plays a role similar to n, but also by an inactivation variable, h with slower kinetics than m. h also follows a first-order kinetics but its equilibrium value, h~ (V) is a decreasing function of V. Accordingly, h is responsible for the slow decay of the sodium current, that follows its fast activation. Finally we emphasize that Hodgkin and Huxley relied on a macroscopic description of the axonal membrane, that was not derived from some microscopic description of the membrane in terms of ion channels. On the contrary they proposed an appealing microscopic interpretation of their results in terms of putative voltage-

370


gated ion channels, that is presented in the Appendix. If many researchers readily accepted the notion of specific ionic channels, experimental confirmation required more than a decade until patch clamp techniques were developed. The existence of these discrete ionic channels in the membrane brings up an interesting issue, at least from the conceptual viewpoint. Patch clamp experiments, and in particular single channel recordings, show that individual channels indeed open and close randomly. How then can deterministic equations, such as the Hodgkin-Huxley system, correctly describe the actual behaviour of the neurone, inasmuch as spike firing is grounded on the non-linear amplification of voltage fluctuations? This issue is discussed in Section 2.3. 2.2. Neurones as non-linear systems 2.2.1. Dynamical systems theory The Hodgkin-Huxley Eqs. (3)-(6) lie at the heart of our understanding of neural excitability. They did not only provide an elegant explanation of the action potential but set the ground for all subsequent work on the excitability properties of nonmyelinated axones, myelinated axones, and neurones. Three major aspects of this model must be emphasized. Firstly, it is highly non-linear because of the voltage-dependence of ionic currents. Hodgkin and Huxley did not only demonstrate that spike generation was due to a massive inward sodium current. Still more importantly, they showed that spike initiation resulted from a non-linear property of the membrane, namely the voltagedependent activation of this sodium current. The steep voltage dependence and fast reaction time of this current, that is, its non-linear features are actually as important as the fact that it is carried by sodium ions. This point is strikingly illustrated by the existence of calcium dendritic spikes, similar to axonal spikes but slower. Despite the different ions involved, they are initiated through the same non-linear mechanism as the sodium spike. More generally it is now clear that many different panoplies of voltage-dependent currents can lead to the same basic pattern of activity in a neurone (see Fig. 6): tonic firing, bursting, subthreshold oscillations, etc. Identifying the abstract non-linear mechanisms that give rise to these dynamical behaviour can help us to understand how similar behaviours may emerge from different combination of ionic conductances and to organize all this diversity. Hodgkin and Huxley were the first to identify neural excitability as a problem in non-linear dynamics. Secondly, Hodgkin and Huxley showed that the existence of well separate time scales was crucial for spike generation. The action potential exists only because the fast activation of the sodium current occurs before slower recovery processes (sodium current inactivation, potassium current activation) may take place. This point was made perfectly clear by subsequent theoretical studies of spiking (see Section 2.2.3). Theoretical studies of neuronal excitability put a great importance on the identification of the time scales involved and capitalize on their separation in several groups. For instance, separation of time scales allows theoreticians to use singular perturbation methods to investigate bursting mechanisms (see Section 2.2.4). Hodgkin and Huxley were the first to show that neuronal excitability was a multiple time scales problem.


371

Thirdly, Hodgkin and Huxley analysed a space clamped situation, and used a simple macroscopic description of the membrane that involves few variables (membrane potential, gating variables, etc., see Section 2.3.1). Most later theoretical works have also relied on space clamped models, that lumped together the axonal spike initiation zone and the soma in a single compartment, and accounted for current leak to dendrites by an increase in the passive membrane conductance. Spatio-temporal aspects were thus dismissed and spiking was more easily investigated as a purely time-dependent phenomenon. Whether firing may or may not be fully understood within the simplified framework of space clamped neurones will be one of the issues discussed in the next subsection (see Section 2.3.5). However it is clear that much of our present understanding of neuronal excitability was gained thanks to this simplification. It enabled theoreticians to analyse excitability phenomena in terms of abstract concepts (control parameters, bifurcations, codimension, genericity, limit cycles, etc.) prior defined in dynamical systems theory (see next section), and to apply in their study of excitability the analytical and numerical techniques developed for systems of non-linear differential equations [65]. In this latter respect it must also be emphasized that Hodgkin and Huxley laid with their model the appropriate ground for the quantitative modelling of neurones (not only of axones), on which compartmental modelling later developed. To summarize Hodgkin and Huxley were the first to show that the theory of dynamical systems with small number of degrees of freedom constituted the natural framework for studying neuronal excitability.

2.2.2. Bifurcation theory applied to neurones The first concept defined for dissipative dynamical systems and obviously relevant for neurones is that of an attractor. For constant current injection below the current threshold where tonic firing starts to take place, the solutions of the HodgkinHuxley all go to a fixed point, which is called accordingly a global attractor of the dynamics. The corresponding value of the voltage V~s(I) is then uniquely given by the I - V curve which is monotonous. The gating variables take their steady-state values: m = m~(V~s), etc. On the opposite, at higher injected current, when the fixed point has become unstable, solutions converge to a stable periodic trajectory called accordingly a limit cycle. Note that the last category of bounded attractors, the socalled strange attractors which present non-trivial topological properties is barely encountered in neuronal models: as a rule Hodgkin-Huxley-like deterministic models display only strictly periodic spiking or bursting under current clamp (see Section 2.3.6 for a discussion of this issue). The experimental notions of rheobase or current threshold correspond to the topological concept of bifurcation [65], which characterizes qualitative changes in the nature of trajectories. Outside bifurcation points in parameter space, differential systems are structurally stable: their solutions smoothly deform as parameters are slightly changed. This is the situation which is encountered both in the subthreshold current range (the value of the potential smoothly depends on the injected current as well as on conductances and kinetic parameters) and in the regime where tonic firing is the sole dynamical state. On the opposite trajectories of the differential system are

372


drastically altered when one crosses a bifurcation point: this happens, for instance, at the onset of tonic spiking in the Hodgkin-Huxley model. However bifurcation is not a mere pedantic rephrasing of the more concrete notion of current threshold. Bifurcations are rigorously defined and can be put to good use to classify the firing mechanisms of neurones. This idea that neurones should be regrouped in a few classes according to their firing pattern actually dates back to the 1930s when Arvanitaki classified the firing patterns of crustacean axones in phasic, tonic and bursting [66]. Different bifurcations exist, that involve fixed point. One of the most important is the Hopf bifurcation [67,68] where a stable fixed point changes stability when merging with a limit cycle. This happens when a pair of complex conjugate eigenvalues cross the imaginary axis while the other eigenvalues describing the linear stability of the fixed point remain all real negative or complex with a negative real part. This bifurcation exists in two different flavours, known as subcritical and super-critical Hopf bifurcations. When the bifurcation is subcritical the stable fixed point becomes unstable when merging with an unstable limit cycle that exists up to the bifurcation point. Above the bifurcation point, the solutions of the differential system converge to some attractor which is far from the fixed point and unrelated to it. This is what occurs in the Hodgkin-Huxley model [69,70] (see Fig. 8). The Hodgkin-Huxley model [3], grounded on the experimental study of the squid's giant axone, cannot account for the ability of most invertebrate and central neurones to fire at very low rate (down to a few Herz). Indeed, even at low temperature (6.3~ firing starts in the squid axone at about 60 Hz. This raises several issues: what is the biophysical substrate of the low frequency firing of neurones? Can this different behaviour also be accounted for by a simple Hodgkin-Huxley-like model? Is this difference a mere quantitative problem or does it stem from some underlying qualitative difference, either biophysical or mathematical? Long interspike intervals were observed in invertebrate preparation by Connor et al. [63,64] who could ascribe them to a fast transient potassium current (as compared to the slow delayed rectifier current present in these preparations). This A c u r r e n t - many different forms of which were later identified in invertebrate and vertebrate n e u r o n e s - de-inactivates under hyperpolarization, activates in a lower voltage range than the delayed rectifier current, and inactivates much more slowly than it activates. These kinetic properties make A currents particularly capable to increase the latency period before the firing of the first spike and to increase the duration of the slow voltage ramp between successive spikes [64]. Although they are not the only currents involved in low frequency firing, it is enlightening to follow Rush and Rinzel's analysis of A-currents [71] and to go deeper in the operating mechanisms of this class of potassium current. As a rule activation and deactivation kinetics of A-currents, and inactivation (which is typically one order of magnitude slower) constitutes the rate limiting process. Still the inactivation properties greatly vary from one preparation to the other and the voltage-independent inactivation time constant lies in the range 1-30 ms in central


373

!

i t

,

Ii

! !

V

I= I1

I'-~ ~

I = 12

I= 13

! w Fig. 8. Schematic bifurcation scheme of the Hodgkin-Huxley model. For the sake of simplicity a simpler two-dimensional situation is described (variables are the voltage, V, and some recovery variable, W) and secondary bifurcations [69] are not shown. Stable (full lines) and unstable (dashed lines) states are shown. The bifurcation parameter is the injected current, I. The onset of spiking at I1 corresponds to a saddle-node bifurcation of periodic orbits [65], which gives rise to a stable limit cycle (in red), and to its unstable counterpart (in blue). At 12 the membrane resting state FP becomes unstable through a subcritical Hopf bifurcation, when it merges with the unstable limit cycle. Between these two values bistability is observed: the membrane can stay in its resting state or display oscillations depending on the initial condition. Spiking ceases at 13, and the membrane again adopts a stable steady state. In contrast with the onset of spiking, vanishingly small oscillations are observed just before this transition due to a high level of sodium current inactivation. Consequently the stable limit cycle is very close to the fixed point near the bifurcation and the Hopf bifurcation is this time supercritical: the unstable fixed point becomes stable when merging with the stable limit cycle.

neurones at physiological temperature. The hyperpolarizing effects of a potassium current that relaxes with a time constant in the tens of milliseconds range m a y be expected to last up to 100 ms, so that these currents might be able to regulate neuronal firing down to frequencies of 10 Hz. But what about firing at still lower frequencies, when the time constants of the potassium currents becomes m u c h smaller than the interspike interval? To u n d e r s t a n d how fast currents (at this time scale) m a y still play a role in regulating the discharge one must distinguish between their steady-state properties

374


and their kinetic properties. Consider a current with fast kinetics (at the time scale of the interspike interval). This current will soon relax to its steady-state value and then follow adiabatically the evolution of the membrane voltage. It will be able to regulate the duration of the interspike interval only if several conditions are satisfied. It must still be operative at steady state (it is then called a window current, as it can be observed only in the limited voltage window where it is at the same time significantly activated and de-inactivated). Its steady-state voltage window must overlap with the voltage range swept during the interspike interval. Finally it must be strong enough to hinder membrane depolarization and maintain voltage long enough in its voltage window. On the opposite a current that slowly inactivates does not require a steady-state window current to be operative. It can take values high enough to oppose membrane depolarization just because it stays in a dynamical state where it is both activated (due to the actual voltage value) and deinactivated (due to the slow de-inactivation kinetics). The potassium A-current may act both ways. When the firing frequency is not too low the current does not fully relax during the interspike interval and the A-current may operate via a kinetic effect. This is no longer possible closer to the current threshold as the A-current has fully relaxed early in the interspike interval. Then it can operate only as a window current. We note also that if the maximal effect on the duration of the interspike interval is generally expected from a current acting all along the interspike interval, a current that acts only transiently may still significantly shorten or lengthen the interspike interval by a phase resetting effect (see for instance the chapter by D. Golomb, D. Hansel and G. Mato in the present book). The existence of a window current can be seen on the steady-state I - V curve of the neurone. The I - V curve of the usual Hodgkin-Huxley model is monotonous. On the opposite it is N-shaped in the model incorporating an A current proposed by Connor et al. [64]. This opens the possibility that the stable resting state disappears via a saddle-node bifurcation, when it coalesces with an unstable fixed point at the first knee of the I - V curve. Such a scenario could not occur in the usual Hodgkin-Huxley model. Still the shape of the I - V curve does not guarantee that spiking will start via a saddle-node bifurcation. The current threshold can be lower than the saddle node bifurcation point. This may happen when the homoclinization affects the unstable fixed point and is unrelated to the saddle-node bifurcation that occurs at a higher current level. In this case the bifurcation is called a saddle-loop. The f - I curve still starts at zero frequency at the bifurcation point but f then grows linearly with I near the bifurcation. In addition, a bistability region then exists between the saddle-loop and saddle-node bifurcation points. The stable fixed point may also bifurcate via a subcritical Hopf bifurcation before the saddle-node bifurcation is reached, in which case the saddle-node will involve two unstable fixed points and will not be associated to a homoclinization. Which scenario will occur depends on all the problem parameters, and in particular on the activation and inactivation time constants of the A current, although changing these kinetic parameters does not affect the steadystate I - V curve [71].


375

The inactivating A current affects the resting membrane potential but its effect progressively subsides as the injected current is increased. It is the subsequent disappearance with increasing injected current of the rectification caused by the A current which makes the I - V curve N-shaped. Non-monotonous I - V curves displaying anomalous rectification may also originate from the steady-state activation of an inward current (persistent, or transient with some window current effect). Correspondingly low frequency firing associated to a saddle-node bifurcation, can be obtained when persistent sodium current rather than potassium A current is added to the Hodgkin-Huxley model. It may even happen within the framework of the Hodgkin-Huxley model provided that the parameters are modified so that the I - V curve displays a strong anomalous rectification [71]. Conversely changing the kinetic parameters of the A-current can make the I - V curve monotonous, in which case the membrane steady state will become unstable through a Hopf bifurcation and firing will start at finite frequency [71]. All these examples illustrate in the simple case of tonic firing an important general fact: the firing pattern of a neurone cannot be readily associated to the presence of some given voltage-dependent current. Similar stationary firing features can be obtained for completely different repertoires of ionic currents (though their transient effects may still be different as exemplified by the brief discussion of phase resetting curves below) while merely changing the relative maximal conductances of two currents or shifting some kinetic parameters can strongly affect the firing pattern of a neurone. On the contrary the abstract concept of bifurcation enables one to regroup many different tonic neurones in a few well-defined classes, each corresponding to a given firing pattern near the current threshold. Moreover analysing the non-linear dynamics of neuronal models through phase plane techniques and bifurcation analysis allows us to progress in understanding the effects of the various ionic currents on the steady-state discharge of neurones [72].

2.2.3. Phase plane analysis Bifurcation analysis enables us to understand through which mechanisms the quiescent state of the membrane becomes unstable and neurones start to fire. However it is, by definition, restricted to a neighbourhood of the current threshold in parameter space. Moreover the analysis is mostly local, and essentially limited to a neighbourhood of the resting state in phase space. The only global feature of the dynamics that is considered is the possible existence of heteroclinic or homoclinic orbits. Therefore many questions regarding neuronal firing cannot be answered in that framework. A good example of that is the issue of the (approximate) linearity o f f - I curves (see Fig. 9). An another approach of neuronal excitability pioneered by FitzHugh [73] and Nagumo [74] gives us a global description of the phase space of neurones, even far from bifurcation points. It is grounded on a description of neurones as twodimensional relaxation oscillators models [73-75]. Models of this type are smooth dynamical systems, at variance with the Lapicque's model, and thus better behaved. They are amenable to phase plane analysis, at difference with the more

376


A

B

130

110

s ,,.-71-

A N

"r

N

:z::

v

,...

-

00

9

9

--

9

A

i thAI

,11

60

I ~-

o

P

J

A

0

A

i

J

(nA)

&

,

D

25

Fig. 9. Gain function ( f - I curve) of lumbar at motoneurones. Intra-cellular recordings from anaesthetised cats. (A) Motoneurone without secondary firing range. (B) Neurone with primary and secondary firing ranges. Note the higher gain in the secondary range. Reprinted from [5] (Figs. 2(C) and I(C)) by permission of the American Physiological Society. The approximate linearity of the f - I curve in the primary firing range is due to the slow AHP conductance. Near the current threshold, this conductance plays no role and the f - I curve is not linear (A). Therefore bifurcation theory can shed no light on the issue of f - I curve (approximate) linearity. complicated Hodgkin-Huxley model, so that the geometric aspects of neuronal excitability can be readily investigated. In view of their simplicity they are also widely used (as well as the Lapicque's model) to investigate such problems as the synchronization of strongly coupled neurones [76-78]. The first model of that type was proposed by FitzHugh [73,79] and independently by Nagumo et al. [74]). It is a variant of the simple undamped oscillator with a non-linear friction term introduced by Van der Pol in the 1920s [65,80] to provide a phenomenological description of heart beat d2x dx dt 2 t-c(x 2 . 1 ) ~ - ~ + x = O . Van der Pol's equations can be recast into the form

dt =C

x + y-

dy= dt

x c

,

(8)

where

1 dx y=c~+5

(7)

X3

x.

FitzHugh-Nagumo equations are barely more complicated


dt

-c

V--

3-

dW _ _V-a+bW

dt

+W+I

'

377

(9)

(10)

c

and the original Van der Pol's equations are even recovered when all three constants a, b and I are set to 0. In these equations V can be interpreted as the membrane voltage and W as some slower "recovery variable" responsible for membrane repolarization (the ratio of the evolutionary time scales of the two equations being given by c2), while I represents the injected current. The phase plane of this system can be readily analysed (see Fig. 10). The V nullcline (the locus of points where dV/dt vanishes) is an N-shaped cubic while the W nullcline is just a straight line. Under appropriate conditions on the constants a, b and c, these nullclines intersect at a single fixed point, which can be thought of as the resting membrane state, so that the I-V curve is monotonous as in the Hodgkin-Huxley model despite the cubic shape of the first nullcline. When I is increased this fixed point becomes linearly unstable at some bifurcation value. The system then performs periodic oscillations, interpreted as tonic firing. These oscillations take a particularly simple form in the limit of infinite c: the system then performs a periodic oscillation on a singular closed trajectory consisting of two slow evolutions on the V nullcline interspersed by two instantaneous transitions at constant W. This asymptotic analysis must then be extended to finite values of c. This is a singular perturbation problem. However, because the differential system is two-dimensional one can use the Poincar6-Bendixson theorem [81] to show that for large but finite c the system displays a smooth stable limit cycle that tends to the singular closed trajectory defined above when c goes to infinity. Phase plane analysis explains why the repolarization of the membrane after the peak of the action potential proceeds in two steps. Just after the peak the membrane voltage decreases as the system operation point follows the depolarized branch of the V nullcline. The subsequent rapid evolution to the hyperpolarized branch of this nullcline is associated with a much steeper voltage drop. Therefore the two slopes observed during the decay phase of the action potential are not the signature of two different active repolarization processes being successively at work, such as sodium current inactivation and potassium current activation, but the natural consequence of the N-shape of the V nullcline. We also note that a voltage threshold naturally appears in this analysis, as a consequence of the N-shape of the V nullcline. In the limit where c vanishes it is merely given by the first knee of this nullcline. This voltage threshold increases with the injected current. However, at variance with the injected current itself, it is a geometric feature of the trajectory rather than a natural control parameter of the differential system. But is there not some hidden but important difference between this simple twodimensional model and the four-dimensional Hodgkin-Huxley model? The relaxation oscillation of the FitzHugh-Nagumo model presents clear differences with the tonic spiking of neurones. The spike duration, for instance, is comparable to the

378


A

1,5

9 '

'

'

I

'

'

'

'

I

'

' ~ '

I

'

'

'

'

I

'

'

'

'

'

'

'

'

dW/dt = 0

W

0,5

-0,5

I

- 3

-2

-1

0

1

2

V

.50"25

.."

U

-75 r -I00

-5O

0

50

V Fig. 10. Phase planes of two-dimensional models. (A) FitzHugh-Nagumo models. Parameters are a = 0.7, b - 0.8, c - 0, and I = -0.4. Nullclines are displayed as bold lines. In the singular limit, shown here, where c = 0, the operation point of the system moves along the V nullcline, the variable V just following adiabatically the slow "recovery variable" W. This is possible only as long as a knee of the curve is not reached. Then the operation point instantaneously jumps to the other branch of the V nullcline (dashed line), which, in turn, is followed till its knee is reached. (B) Reduction of the Hodgkin-Huxley model, using Abbott and Kepler's method [86]. Sodium activation is assumed to be instantaneous. Gating variables h and n are replaced by voltage-like variables Uh and U,, respectively, defined by h = h~(Uh) and n i n~(U~). These two variables are then linearly combined into a single slow variable U. The resulting dynamics is shown in the phase plane (V, U) for I -- 0. Note that the nullcline d U / d t -- 0 is just the straight line V = U. A more systematic treatment can be found in [84]. Reprinted from [86] (Fig. 3).


379

interspike interval in the FitzHugh-Nagumo model whereas it is negligible with respect to the interspike interval in neurones firing at low rates. However this can be fixed by making the recovery time constant voltage-dependent. Oscillations appear at the current threshold through a supercritical Hopf bifurcation [82] in the original FitzHugh-Nagumo model [73]. This means that the amplitude of the oscillations steadily grows from 0 as the current is increased past the threshold, which does not match the subcritical behaviour of the Hodgkin-Huxley model. Still one can adopt other parameters for which spiking onset occurs through a supercritical Hopf bifurcation, with little impact on the global geometry of phase plane. One may then wonder whether it is not possible to reduce the Hodgkin-Huxley model to a simpler two-dimensional FitzHugh-like model. One way to do this is to try to eliminate variables from the Hodgkin-Huxley model with as little impact as possible on the qualitative and hopefully quantitative behaviour of the model. All such reductions schemes rely on the same approach: eliminating the fast activation variable m by assuming that its variations are instantaneous or occur at the time scale of the membrane time constant, merging into a single slow variable the two recovery variables h and n. The first such reduction was done by Kokoz and Krinskii [83] who noticed that the sum h + n remained nearly constant throughout the oscillation. Much later, Kepler et al. showed how to perform more accurate reductions of the Hodgkin-Huxley model and similar differential systems [84]. These two reduction schemes do lead to two-dimensional systems with the desired bifurcation scheme [85]. In addition they approximate much better the shape of the action potentials than the FitzHugh equations [85,86] (see Fig. 10). This is due to the fact that the depolarised branch of the V nullcline barely changes with the injected current is these models [86]. Can we do better and propose a two-dimensional system really equivalent to the Hodgkin-Huxley model? In this model, the resting state of the membrane becomes linearly unstable when a pair of complex eigenvalues of the linearized vector field cross the imaginary axis. Accordingly the unstable manifold of the fixed point, in which the stable limit cycle is embedded, is twodimensional. This manifold is globally invariant under the dynamics and is transversally stable with respect to the other degrees of freedom, which relax exponentially toward it. A perfect approximation of the Hodgkin-Huxley model would require to write down explicitly the equations governing the dynamics on the two-dimensional unstable manifold, near the fixed point (local invariant manifold) and farther in phase space (global invariant manifold, first determined in the vicinity of the Hopf bifurcation, and then extended for larger values of the injected current). Unfortunately this cannot be done analytically, although the Hodgkin-Huxley system is fundamentally bidimensional. Although it is not directly related to bifurcation theory and can be used far from bifurcation points, phase plane analysis can still shed light on the bifurcations of neurones. This is best illustrated by the clever way Hindmarsh and Rose [75] addressed the problem of low firing rates. These authors combine the analysis of experimental data on invertebrates neurones (pond snail Lymnaea stagnalis and crab Cancer magister) with a mathematical approach of the problem. The model they analysed is a variant of the FitzHugh-Nagumo model

380

c. Meunier and I. Segev

dV dt dW dt

= a(-f(v) - b(o(V)-

+ w + 1), w).

The functions f ( V ) = c V 3 + d V 2 + e V + h and g(V) = f ( V ) - qe rv + s were determined by fitting voltage-clamp data (a simpler quadratic function for g(V) give similar results [87]). Just above the current threshold, the two nullclines determine a n a r r o w channel where the trajectory spends a very long time (see Fig. 11). This time increases without b o u n d when I goes to Ith from above. Accordingly the firing rate vanishes like v / I - Ith at the current threshold Ith. At this specific value the stable limit cycle becomes a homoclinic orbit, that is, a trajectory which tends to the fixed point when time goes to +oo. This behaviour corresponds to a saddle-node bifurcation on an invariant cycle [65] (see Fig. 12). Here again this phase plane analysis of the problem is m a d e possible by the two-dimensional nature of the dynamics of tonically spiking neurones. 2.2.4. Multiparameter singular perturbation theory

Singular perturbation theory, such as it is used for studying the F i t z H u g h - N a g u m o or H i n d m a r s h - R o s e models, can to some extent be generalized to a theory of

Recovery side

y(nA)

Action side

6o/

x(mV)

. . . .

/

.'~=0

C

B

Fig. 11. Phase plane of the Hindmarsh-Rose model. The two nullclines and the limit cycle are displayed for I = 0.033 nA. Other parameters are a = 5,400 mV s -1, b = 30 s -1, c = 0.00017, d = 0.001, e -- 0.01, h = 0.1, q = 0.024, r = 0.088 and s = 0.046. The two variables are denoted by x and y in this original figure by Hindmarsh and Rose [75], which corresponds, respectively, to V and W in the text of the chapter. At variance with the FitzHugh-Nagumo model, where the second nullcline is a straight line, the two nullclines of the Hindmarsh-Rose model do not intersect transversely at the current threshold but display a tangential contact. Just above the threshold they remain close together, just separated by a narrow channel (C ~ A). As a consequence, the vector field takes small values all over this region and the evolution of both variables proceeds extremely slowly. This entails a very long voltage ramp before a spike may be fired. Reprinted from [75] (Fig. 3).


381

/

s

,gj m u

m

FP3 m

m

mn

mmm

n

mm

mm

k~

~

n

mm

mmm mmm m m

F::P2

aml jmw 4m~,

9

O N

._

FPI I

I -12

1=13

m w Fig. 12. Schematic bifurcation scheme of a neurone undergoing a saddle-node bifurcation at the current threshold. The phase space displayed here is two-dimensional (variables are the voltage, V, and some recovery variable, W), which is enough to capture the essence of the bifurcation scheme. Bifurcation parameter is the injected current I. Stable (full lines) and unstable (dashed lines) states are shown. The differential system exhibits a stable limit cycle (in red) and three fixed points:/7/1 (light blue), UP2 (green) and FP3 (dark blue). At 11 the pair of unstable fixed points UP2 and FP3 appears through a first saddle-node bifurcation. F& is fully unstable whereas/;P2 is unstable in only one direction. The one-dimensional unstable manifold of FP2 goes to the globally stable fixed point FP1 (resting state of the membrane), thus defining an invariant circle (heteroclinic connection). At I2 F& and FP2 coalesce through a second saddle-node bifurcation. The heteroclinic connection between the fixed points, that existed for I1 ~~ 1), transmission is always possible; R and T both increase with q and tend, respectively, to 1 and 2 when q goes to infinity. The situation is different when the second cylinder is thicker than the first. Then R decreases with q from R - 0 when the two cylinders are identical (q - 1) and impedance match is perfect (no reflection, full transmission: T - 1) to - 1 when the second cylinder becomes extremely thick (q - 0) and no transmission is possible (complete reflection, T - 0). This suggests that a large increase in diameter will lead to a conduction failure as the current spreading axially will then not be able to depolarize enough the axone beyond the discontinuity to activate the transient sodium current. Still this prediction must be checked on the full non-linear Eqs. (11)-(14). Non-linear cable equations for a medium displaying geometric heterogeneities are not readily amenable to analytical studies. Accordingly conclusions on conduction failures were derived from the numerical integration of the equations underlying spike propagation on piecewise homogeneous media. For the extended Hodgkin-Huxley model, single spikes fail to propagate when the diameter ratio between two abutting sections of the axone exceeds 5 [60,158]. Consequently, only very thick varicosities may hinder reliable spike conduction along an unbranched portion of the axone. We note that


404

transmitted spikes are nonetheless delayed by their passage through varicosities [158,159]. We can now understand what happens at branch points, by using the concept of equivalent cylinder introduced by Rall. Suppose one semi-infinite cylinder of uniform diameter dl splits into two semi-infinite branches of diameters dz,a and d2,b, with corresponding space constants ~.2,a and ~2,b- One can then replace these two branches by a single semi-infinite cylinder of diameter d2 and space constant ~2 (note that in the finite geometry such a reduction would also require that the two daughter branches have the same electrotonic length). The (local) condition for the equivalence of the two structures is that their impedance be the s a m e : Z 2 = Z2,a -~ Z2,b. This requires that d2 /2

,43/2 ,43/2 = '~2,a -+ "2,b "

(20)

Then the voltage on a daughter branch at a given electrotonic distance from the branch point will be the same as the voltage on the equivalent cylinder at the same electrotonic distance

and the above analysis of impedance mismatch at geometric discontinuities will still be applicable. This simplified analysis of conduction blocks must now be qualified. Firstly we note that conduction of spike trains is less reliable than single spike conduction. Impedance mismatches that cannot produce a full conduction block may lead to a rhythm transformation where, for instance, only every other spike in a train is transmitted down the axone (see, for instance, [160,176]). Secondly we remark that conduction blocks are favoured by abrupt changes in the geometry of the axone. On the contrary impedance adaptation may be achieved, in spite of large diameter increases, when the axone progressively flares or tapers. A good example is provided by the invasion of the soma (which can be as big as 50-70 I~m in the case of 0t motoneurones) by the IS spike, generated on the much thinner initial segment of the axone. This back-propagation is made possible by the flare of the axone hillock and the electrotonic compactness of the soma. Such smooth changes in the geometric properties can still be addressed in the framework of cable theory (see Section 2). One last aspect that must be considered is finite geometry: For example, short daughter branches lead to enhanced depolarizations (due to the sealed end boundary condition), which makes conduction failures at branch points more difficult. We have focused on the geometric properties of the axone but its electrotonic architecture is also determined by the specific conductance of the membrane. Moreover varicosities associated with en passant boutons are likely to display membrane properties different from the rest of the axone. Therefore one may wonder whether conduction failures might arise from electric rather than geometric heterogeneities, and be controlled by the electric state of the membrane. It is obvious from Eq. (19) that heterogeneities in the specific membrane conductance leads to


405

impedance mismatches: a conductance increase, as well as an increase in diameter, may hinder voltage spread along the axone. Impedance is less sensitive to conductance changes (by a cubic factor) but increasing the conductance also acts by modifying the balance of linear dampening terms and non-linear regenerative terms in the full non-linear problem. It has been recently suggested [51] that the de-inactivation of an A-type potassium current present on the axonal membrane might provoke conduction failures through such a shunt effect, which would explain the differential conduction observed in pyramidal cells in rat hippocampal slice cultures following somatic hyperpolarization [51]. This hypothesis was investigated by the numerical simulation of a detailed model of the axonal arbour where clusters of A-type channels near branching points indeed led to differential conduction [177]. However a high density of A-type channels at hot spots was required, to compensate for voltage decrement between the soma and the hot spot, and the results obtained were very sensitive to the exact value of the channel density. A fine tuning of model parameters seems to be required to account for the experimental observations. All this casts some doubts on the generality of this filtering mechanism, which is unlikely to operate if axones have a small diameter, are poorly myelinated, or branch far from from the soma. Nonetheless this study raises the interesting issue of whether spike trains propagating on axones are affected by voltage-dependent currents, that not directly involved in non-decremental conduction, very much like somatic subthreshold currents pattern the discharge of neurones. Few authors have yet tackled this problem [178], probably due to the dearth of experimental data on such axonal currents and to a relative disinterest for axonal information processing in the past 20 years.

3.3. Presynaptic inhibition 3.3.1. Inhibition mediated by axo-axonic synapses In 1957 Frank and Fuortes [179] demonstrated on anaesthetized cats that the EPSPs induced in spinal motoneurones by the stimulation of proprioceptive Ia fibres could be reduced, without any evidence of direct inhibition of the motoneurones. They suggested that this reduction was due to presynaptic effects taking place on the Ia fibres themselves and accordingly coined the term presynaptic inhibition to describe this phenomenon. Presynaptic inhibition has now been extensively studied in the spinal cord of mammals and was shown to affect not only Ia fibres but more generally all myelinated sensory fibres, that provide peripheral inputs to the spinal circuits involved in motor control (see [180,181] for a review): group Ib fibres innervating Golgi tendon organs [182,183], group II [184] muscle fibres and group A cutaneous fibres (innervating, respectively, the Golgi tendon organs and the secondary endings of spindles). Presynaptic inhibition has been also widely studied in invertebrate species [185-187] where it also affects sensory fibres. Eccles et al. showed [188], still on group I afferents, that presynaptic inhibition was accompanied by a depolarization of the fibres, that they called primary afferent depolarization (PAD). On the basis of electrophysiological and pharmacological experiments, they claimed that presynaptic inhibition of Ia fibres was due to the

406


activation of axo-axonic synapses [189] - which provoked both a local increase in the conductance of the axone and the observed PAD - and that it was mediated by GABA [189,190]. There is now accumulated ultrastructural evidence of such axoaxonic synapses on proprioceptive fibres [191-194] (see [195] for a review) and on group II fibres. The hypothesis that axo-axonic synapses are GABAergic is supported by recent immunolabelling studies [196,197] (see Fig. 17). The sensitivity of presynaptic inhibition to GABAA antagonists indicates that the activation of axoaxonic synapses open post-synaptic chloride channels [198]. Strangely the activation of axo-axonic synapses depolarises the axone, which points to an inverted Nernst potential of chloride ions (see [186] for a study of this issue on the crayfish), perhaps due to the operation of a Na +/K +/C1- pump [199].

3.3.2. Selectivity of presynaptic inhibition For a long time presynaptic inhibition was thought to regulate the gain of sensory pathways [180,181], which cannot be done easily through direct post-synaptic inhibition of target neurones [114] (see Section 3.3), and to gate sensory information so that only the most relevant information in a given physiological context is transmitted to the spinal circuits. For instance, presynaptic inhibition was shown to gate the transmission of proprioceptive information generated by Golgi tendon organs during muscle contraction [183,200]. Consequently, only the onset of contraction and large increases of the force are signalled to motoneurones by inhibitory postsynaptic potentials. The adaptation to the specific requirements of the motor task and to the conditions in which it is executing is made possible by the wide convergence of information - from peripheral, spinal, and supraspinal origin - on the interneurones that mediate presynaptic inhibition [180,181,201-204]. However it became considered only in the last decade that presynaptic inhibition might achieve a selective distribution of sensory information to the different targets of sensory fibres [205-210], and that this control might occur down to the level of the individual fibre [208,211,212]. The first example of selective effects at the segmentary level came from experiments on man: presynaptic inhibition was shown to decrease in the branches of Ia afferents that project to a contracting muscle, and to increase in branches projecting to non-contracting muscles [206,213]. 3.3.3. Mechanisms." shunt and PAD How can the activation of axo-axonic synapses result in a graded and flexible funnelling of information in the axonal arborization of sensory neurones? Eccles and coworkers suggested that the depolarization of terminals reduced the amplitude of action potentials and therefore transmitter release [189] (see review in [180,181]). They also pointed out that the opening of synaptic conductances might contribute to presynaptic inhibition by shunting other ionic currents. Experimental evidence that these mechanisms can indeed explain presynaptic inhibition of afferent fibres cannot be readily obtained in vertebrate preparations and the study of this issue greatly benefited from computer studies. Segev [176] showed that the synaptic shunt could reduce the amplitude of action potentials. More recently it was shown that PAD could further reduce the peak voltage of action potentials by enhancing the in-


407

Fig. 17. Tridimensional reconstruction of a Ib collateral. The collateral (red) bears 12 presumed GABA immunoreactive contacts (yellow spheres), which constitutes only one fifth of the total complement. They are located in terminal fields. D, R, and L, respectively, indicate dorsal, rostral and lateral directions. Length of each axis, 100 lam. Reprinted from [197] (Fig. 1).

408

C. M e u n i e r a n d L S e g e v

activation of the sodium current [214,215]. The fact that membrane depolarizations through current injection does reduce the size of action potentials travelling on sciatic group I fibres brings some indirect experimental support to this numerical result [197]. We note a fundamental difference between these two possible modes of operation. The synaptic shunt increases the linear passive membrane conductance and thus opposes the regenerative processes at work during the upstroke of the action potential. On the contrary the PAD directly interferes with the non-linear kinetics of voltage-dependent currents and decreases (by enhancing inactivation) the effective sodium conductance without affecting the passive membrane conductance. This mechanism which directly exploits the non-linearity of the membrane can be very efficient, provided that steady-state sodium channels inactivation already occurs at the resting membrane potential. These two effects of presynaptic inhibition are easily seen on the simple model presented in Section 1 where a depolarization front rather than a spike travels at constant velocity along a uniform and infinite non-myelinated axone. Let us add a uniform synaptic term gsyn(Esyn- V) to the right-hand side of Eq. (15). It depolarises the membrane by an amount AV, which depends on both gsyn and Esyn, so that the resting membrane potential now becomes Vrest -~- AV, at which value steadystate sodium current inactivation, h~, and steady-state potassium current activation, no~ must now be evaluated. Renormalizing the leak parameters to incorporate now both the potassium current term and the synaptic current term, we still obtain Eq. (17) but now the non-linearity parameter 13 reads _

.

_

_

gNahcx~(grest -~- AN) gleak -+- gsyn -+- gKn4(grest -t- A V)

instead of

gNahee (grest) gleak -+- gK/14 (grest) previously. If synapses are purely shunting [176] (Esyn--0) no depolarization is generated (AV = 0), and the non-linearity parameter ]3 decreases with increasing synaptic conductance. The dependence of ]3 on gsyn is sigmoid. Small synaptic shunt (with respect to the intrinsic passive properties) has no significant effect on spike conduction whereas large shunt strongly reduces ~ which becomes smaller than ]3c. Spike regeneration is then impossible. When the synapses are depolarizing- as in the case in actual presynaptic inhibition where Esyn is thought to be of the order of 30 mV above r e s t - the depolarization A V they create enhances sodium inactivation and, to a lesser extent, potassium current activation, which further reduces 13. For small synaptic conductances (gsyn muchsmaller than gleak), synapses almost function as current injectors and it is easily shown that the effect of the depolarization on the non-linearity parameter is larger than the effect of the shunt by the factor (Es~ - Vrest)h~ (/'/rest)/ho~(Vrest). Still small PADs have obviously a very limited impact on action potentials [214]: a drastic effect is expected ofily for depolarizations of the order of 10 mV [52,197].


409

Two experimental facts suggest that such large depolarizations do occur on proprioceptive fibres: Firstly PAD can reach values high enough to trigger antidromic spikes [202,216]. Secondly depolarizations of the order of 1 mV are frequently observed during intra-axonal recordings of group I fibres on anaesthetized cats. The amplitude of PAD will be one order of magnitude larger near the terminals if they are located at no more than 3 space constants from the recording site. This condition is not very stringent as the intra-spinal portions are almost fully myelinated [193].

3.3.4. Functional compartmentalization of the axone A single axo-axonic synapse should have a conductance of the order of the input conductance of the axone to produce a large depolarization, equal to half its reversal potential. Morphological studies have revealed a substantial synaptic equipment on group Ia fibres [192] and Ib fibres [215]. It is therefore likely that PADs of the order of 10 mV result from the coactivation of axo-axonic synapses. However the selectivity of presynaptic inhibition on axones cannot be consistent with a wide spread of PAD on the axonal arborization. A powerful and selective presynaptic inhibition requires that the axonal arborization be divided in almost independent subdomains and that axo-axonic synapses located within any given subdomain cooperate efficiently to produce a large depolarization locally. This issue, similar to the functional compartmentalization of dendrites (see Section 4.2) was recently investigated analytically [52], using cable theory. Applying cable theory to sensory fibres required to overcome two difficulties. Firstly, these fibres exhibit highly non-linear properties at Ranvier nodes. It could be shown that only the average non-linearity, which is small because of the small size of Ranvier nodes, had to be taken into account in the time-independent regime where no antidromic spikes are generated by the depolarization. Secondly the passive properties of the axones are highly heterogeneous. The main branches are myelinated and display an alternation of leaky Ranvier nodes and insulated internodal zones. The terminal fields are not myelinated and the activation of the discrete set of synapses they bear provokes a local increase of their membrane conductance. Using homogenization methods, originally developed in Physics [217], both myelinated and non-myelinated regions could be replaced by a homogeneous effective medium, the passive properties of which were obtained by spatially averaging the heterogeneous passive properties (see also [218]). This study concluded that the requirements for independent compartments were satisfied on sensory fibres. As synapses are grouped in terminal fields [197], the depolarization they create locally in a given terminal field has little effect on other terminal, despite the myelination of the main branches which favours voltage spread. Terminal fields are electrotonically compact, so that the coactivation of axoaxonic synapses may easily depolarize them. Within a terminal field the amplitude of the PAD depends on the number of active synapses but little on their location. This study suggests that due to a match between the electrotonic architecture, which displays two separate space scales, and the distribution of synapses, the intraspinal arborization of sensory fibres might be divided into independent functional domains, each contacting a specific target population, and each independently con-

410


trolled by presynaptic inhibition according to the requirements of the task performed [52]. Accordingly the views of Chung et al. [219] who suggested that the axone transformed the temporal pattern of activity of the presynaptic neurone into a spatial pattern of excitation or inhibition of its post-synaptic targets seems more and more relevant.

3.4. Discussion 3.4.1. How wide is an action potential?

The saltatory conduction of action potentials on myelinated axones does not entail that, at any given time, the membrane is at rest except in the immediate neighbourhood of some Ranvier node. Conduction velocities as high as 120 m/s are reached on certain fibres of the cat's sciatic nerve (as compared to 1 m/s on nonmyelinated axones). Since the spike duration at a given place is of the order of the millisecond, this entails that in the wake of the spike the membrane potential remains perturbed away from its resting value in a region of about 10 cm. In comparison, the distance between successive Ranvier nodes, which cannot be much larger than the axone's space constant, is no more than 2 mm. If the depolarization was limited to a zone of such a length, action potentials would last only 10 ~ts, 100 times less than what is observed on myelinated axones. This demonstrates that the action potential cannot be considered as a local excitation of the myelinated axone. Action potentials are no more local excitations on homogeneous non-myelinated axones. An action potential travelling at 1 m/s, for instance, and lasting a few milliseconds, depolarises the axone on several millimetres, typically 100 times the space constant. This corresponds to the distance travelled by the action potential during the time it takes for recovery processes (sodium current inactivation, delayed rectifier current activation) to play their role (a few milliseconds). If these voltagedependent recovery processes were not taken into account, action potentials would become excitation fronts (see previous section) that would leave the whole axone depolarized in their wake.

3.4.2. Spike reflection on axones

Spikes travelling on an axone may be reflected by geometric heterogeneities, where the diameter abruptly increases (from the value dl to the larger value d2). Regeneration of a full blown antidromic spike is hindered by the relative refractoriness of the axone in the wake of the orthodromic spike, so that conduction block is not necessarily accompanied with a reflection. However in a range of diameters [220] (when 3dl ~OO~

0.26 ~ o - o A , . C ~

o.os~ I 0

D 4o,-~

m [Rroo

-

At,

o OO(:>OOOOO~3~,, J

&t3

8 ~

I~

~

i

-

fo

......

DENDRITES

. . . . . . .

0'.6 Ptlok

I/1"

.

!

7

\ /

l SINGLE | BRANCH

3.0

C.1

.

A , .

1 ---

......... "..... '0.

(7)

We call t (f) the firing time of the neuron. If there are several neurons we add a lower index to identify the neuron so that t~ ) is one of the firing times of neuron i. Let us write ti "- max{tff)]t} f) < t} for the last firing time of neuron i. In the following we only have a single neuron and we suppress the subscript i. The notation t stands for the last firing time of this neuron. Action potentials in the Hodgkin-Huxley model have the stereotyped time course shown in Fig. 3a. Whatever the stimulating current that has triggered the spike, the form of the action potential is always roughly the same (as long as the current stays in a biologically realistic regime). This is the major observation that we will exploit in the following. Let us consider the spike triggered at time t. If no further input is applied for t > i, the voltage trajectory will have a pulse-like excursion before it eventually returns to the resting potential. For t > i, we may therefore set u(t) = r l ( t - t) + U r e s t , where 11 is the standard shape of the pulse and Urest is the resting potential that the neuron assumes in the absence of any input. Since, without further input, the voltage will eventually approach the resting value, we have v l ( t - t) ~ 0 for t - ~ ~ e~. Let us now consider an additional small input current pulse I which is applied at t' > ~. Due to the input, the membrane potential will be slightly perturbed from its trajectory. If the input current is sufficiently small, we may describe the perturbation by a linear impulse response function ~z. Since the voltage u depends on the last firing time ~, the response kernel ~: does so as well. For an input with arbitrary time course I(t ~) for t~ > ~ we therefore set

t-i u(t) = q(t - ~) + fo

~(t - ~,s)I(t - s ) d s + Urest.

(8)

Eq. (8) will be called the SRM. Note that after an appropriate shift of the voltage scale, the resting potential can always be set to zero, U r e s t --- 0.

480

W. Gerstner

To construct an approximative mapping between the S R M (8) and the H o d g k i n Huxley equations, we have to determine the following three terms: (i) the kernel r I which describes the response to spike emission; (ii) the kernel ~: which describes the response to incoming current; and (iii) the value of the threshold ,9 in Eq. (7).

2.2.1. The q-kernel In the absence of input the membrane potential u is at some resting value Urest. If we apply a strong current pulse, an action potential may be excited. The time course of the action potential determines the kernel 11. To get the kernel rl we use the following procedure. We take a square current pulse of the form

I(t)--c--~q0

for0 ~. The response to the second pulse is u ( t ) - uo(t), hence

481

A framework for spiking neuron models." the spike response model

~2(t- t, t) -- l [U(/) -- r l ( t C

t) -- Urest].

(12)

for t > 0. We repeat the above procedure for various spike times ~. The result is shown in Fig. 7. Since the input current pulse delivers a unit charge during a very short a m o u n t of time A < 0.1 ms, the ~:-kernel jumps almost instantaneously at time t = 0 to a value of 1 mV. Afterwards it decays, with a slight oscillation, back to zero. The decay is faster if there has been a spike in the recent past. This is easy to understand intuitively. During and immediately after an action potential many ion channels are open. The resistance of the cell membrane is therefore reduced and the effective membrane time constant is shorter.

2.2.3. The threshold The third term to be determined is the threshold ,9. Even though Fig. 3b suggests, that the H o d g k i n - H u x l e y equations exhibit some type of threshold behavior, the threshold is not well defined [15,25] and it is fairly difficult to estimate a voltage threshold directly from a single series of simulations. We therefore take the threshold as a free parameter that will be adjusted by a procedure discussed below.

2.2.4. Example: stimulation by time-dependent input To test the quality of the S R M approximation we compare the performance of the SRM (8) with that of the full H o d g k i n - H u x l e y model (2)-(4). We study the case of a time-dependent input current I(t). The input is generated by the procedure discussed in Section 2.1.4. The same current is applied to both the H o d g k i n - H u x l e y and the SRM. In Fig. 8 the voltage trace of the H o d g k i n - H u x l e y model is compared to that of the S R M with the kernels rl and ~: derived above. We see that the

1 ~_.

0.8 !

"L~

0

+

N

7s

~

50

,--,

25

i,, I9

9" 9 9

9

~

I;o

~

~l/

2;0

i

9

,,' i !

s 99

i !

te

9s

240

]l

e

e

! 9.

e

s

! I

e e

9 9 9

i

e

9s

! !

i

e

5 I t

0

',, / \,...,/

-5 180

190

200

210

220

2~-0

~.~o

2~o

,,,.--

time /

ms

Fig. 8. A segment of the spike train of Fig. 6. The inset in the lower left corner shows the voltage of the Hodgkin-Huxley model (solid) together with the approximation of the SRM defined by (8) (dashed line) during a period where no spike occurs. The approximation is excellent. The inset on the lower right shows the situation during and after a spike. Again the approximation by the dashed line is excellent. For comparison, we also show the approximation by the SRM0 model which is significantly worse (dotted line). Taken from [19]. approximation is excellent both in the absence of spikes and during spiking. As an aside we note that it is indeed important to include the dependence of the kernel ~: upon the last output spike time ~. If we neglected that dependence and used ~ ( ~ , s ) instead of ~ ( t - ~,s), then the approximation during and immediately after a spike would be significantly worse; see the dotted line, referred to as SRM0, in the lower right graph of Fig. 8. We have used the above scenario with time-dependent input current to optimize the threshold ~ by the following procedure. The same input was applied to the Hodgkin-Huxley model and the SRM (8) with kernels derived by the procedure described above. The threshold has been adjusted so that the total number of spikes was about the same in the two models; see [19] for details. To check whether both models generated spikes at the same time, we compared the firing times of the two models. About 90% of the spikes of the SRM occurred within -t-2 ms of the action potentials of the Hodgkin-Huxley model [19]. Thus the SRM (8) reproduces the firing times and the voltage of the Hodgkin-Huxley model to a high degree of accuracy.


483

2.2.5. Example: constant input and mean firing rates We study the response of the S R M to constant stimulation using the kernels derived by the procedure described above. The result is shown in Fig. 9. As mentioned above, we take the threshold 8 as a free parameter. If 8 is optimized for stationary input, the frequency plots of the Hodgkin-Huxley model and the S R M are rather similar. On the other hand, if we took the value of the threshold that was found for time-dependent input, the current threshold for the S R M would be quite different as shown by the dashed line in Fig. 9.

2.2.6. Example: step current input Finally, we test the S R M for the case of step current input. For 8 we take the value found for the scenario with time-dependent input. The result is shown in Fig. 5c. The S R M shows the same three regimes as the H o d g k i n - H u x l e y model. In particular, the effect of inhibitory rebound is present in the SRM. The location of the phase boundaries depends on the choice of ~ and would move if we changed I).

2.2.7. Example: spike input In the H o d g k i n - H u x l e y model (2), input is formulated as an explicit driving current I(t). In networks of neurons, input typically consists of the spikes of other, presynaptic, neurons. Let us, for the sake of simplicity, assume that a spike of a presynaptic n e u r o n / , which was emitted at time t/), generates for t > t/) a current input I(t) - ~(t -tJ: ~) to a postsynaptic neuron i. Here ~ is some arbitrary function which describes the time course of the postsynaptic current. The voltage of the postsynaptic neuron

i

changes,

according

to

(8)

by

an

amount

Aui(t) -- fo --~ ~ ( t - ti,s)e~(t- ( ) - s)ds, where ti is the last output spike of neuron i. For reasons of causality, the voltage response Au vanishes for t < t / ) . For t > t/) we define (note that there is no tilde on the left-hand side) 125: ioo

,.~

50

(

,

25

J

i

0 0

5

I0

15

20

I / #Acm -2 Fig. 9. The firing rate of the Hodgkin-Huxley model (solid line) is compared to that of the SRM. Two cases are shown. If the threshold 8 is optimized for the constant-input scenario, we get the dotted line. If we take the same value of 8 as in the dynamic-input scenario of the previous figure, we find the dashed line. Input current has a constant value I. Taken from [19].

484

W. Gerstner

i

-

t-i; (13)

,/o

What is the meaning of the definition (13)? Let us assume that the last output spike of the postsynaptic neuron was a long time back in the past. The voltage response Aui(t) - ~(oo, t - tj(f) ) is the postsynaptic potential of neuron i caused by the firing of the presynaptic neuron j. The time course of the postsynaptic potential can be measured in experiments and has a clear biological interpretation. For excitatory synapses the response of the postsynaptic neuron is positive and called an excitatory postsynaptic potential (EPSP). For inhibitory synapses it is negative (inhibitory postsynaptic potential (IPSP)). The function (13) will play a major role in the formal definition of the SRM in Section 3.

3. Spike response model In this section we collect the results of the previous discussion of the HodgkinHuxley model. We start with a formal presentation of the SRM in Eqs. (14) and (15). We then try to give each of the terms in (15) a biological meaning. To do this we make heavy use of the intuitions and results developed during the discussion of the Hodgkin-Huxley model. Finally, we present some examples and simplifications of the SRM which prepare the transition to a discussion of the integrate-and-fire model in Section 4. 3.1. Definition o f the S R M In the framework of the SRM [19,35,20-24,36-38], the state of a neuron i is described by a single variable u;. Neuron i fires, if ui approaches a threshold ~) from below. The moment of threshold crossing defines the firing time t}f), ui(t) = 9

and

d -;:ui(t) > 0 = ~ t - t } f).

(14)

13/--

In the absence of any spikes, the variable U i would have a value of 0. Each incoming spike will perturb u; and it takes some time before ui returns to zero. The function e describes the time course of the response to an incoming spike. If, after the summation of the effects of several incoming spikes, ui reaches the threshold ,9, an output spike is triggered. The form of the action potential followed by a return to a low value after the pulse is described by a function 11. Let us suppose neuron i has fired its last spike at time ii. After firing the temporal evolution of ui is given by

jE Fi

-t-

# t)(.[) E,~-j

~(t -- ti, S) Iext (t -- S) ds,

(15)

485


where ti is the last spike of neuron i, t~ ) the spikes of presynaptic neurons j and wij is the synaptic efficacy. The last term accounts for an external driving current/ext. The sum runs over all presynaptic neurons j E Fi where (16)

Fi = { j l j presynaptic to i}

and ~-j is the set of all firing times t~ ) < t of neuron j. So far Eqs. (14) and (15) define a formal model. Can we give a biological interpretation of the terms? Let us identify the variable u; with the m e m b r a n e potential of neuron i. The functions qi and eij are response kernels which describe the effect of spike emission and spike reception on the variable ui. This interpretation has motivated the term SRM. Let us discuss the meaning of the response kernels; see Fig. 10. As we have seen in Section 2.2.1., the kernel qi describes the standard form of an action potential of neuron i including the negative overshoot which typically follows a spike. Graphically speaking, a contribution qi is 'pasted in' each time the membrane potential reaches the threshold ~; Fig. 10. Since the form of the spike is always the same, the exact time course of the action potential carries no information. W h a t matters is whether there is the event 'spike' or not. The event is fully characterized by the firing time ti(f). In a simplified model, the f o r m of the action potential may

ui(t)

1'1

A

(t- ti)

, .........

,~'.. I~

4 .................................................................................................

t_•

!

,,.e

i

^

~

~

ti

t'

t"

,~

" "-

-

-- . . . . .

t

Fig. 10. Schematic interpretation of the SRM. The figure shows the time course of the membrane potential of neuron i as a function of time t. A spike of neuron i has been initiated at ~/. The kernel q ( t - ~/) for t > i/describes the form of the action potential (positive pulse) and the (negative) spike after potential that follows the pulse (solid line). If an input current pulse is applied at a time t" a long time after the firing at ~g, it evokes a standard response described by the function gz(e~,t - t") and indicated by the dashed line starting at t" (arrow). An input current pulse at t' which arrives shortly after the postsynaptic spike at ti evokes, due to refractoriness of the neuron, a response of significantly shorter duration. Its time course is described by the response kernel ~:(t- ~g,t - tt); see the dashed line after t'.

486

W. Gerstner

therefore be neglected as long as we keep track of the firing times t/(f). The kernel ]]i describes then simply the 'reset' of the potential to a lower value after the spike at ii. This idea will be exploited later on in Section 4 in the context of the intergrate-andfire model. The kernel ~,(t - ii, s) is the linear response of the membrane potential to an input current. We have already seen in Section 2 that the response depends, in general, on the time that has passed since the last output spike at ii. Immediately after ii many ion channels are open. The resistance of the membrane is reduced and the voltage response to an input current pulse of unit amplitude is therefore reduced compared to the response of an inactive neuron. A reduced response is one of the signatures of neuronal refractoriness. Formally, this form of refractory effect is included by making the kernel ~: depend, in its first argument, on the time difference t - ii. In Fig. 10 we compare the effect of an input current pulse at t' shortly after ii to that of a pulse at t" some time later. The response to the first input pulse is shorter and less pronounced than that to the second one. The kernel sij(t- ti,s) as a function of s = t - t~r) can be interpreted as the time course of a postsynaptic potential evoked by the firing of a presynaptic neuron j at time t~:/). If the synapse from j to i is excitatory, sij is called the EPSP. If it is inhibitory, it is called the IPSP. Similarly as for the kernel ~:, the exact shape of the postsynaptic potential depends on the time t - ii that has passed since the last spike of the postsynaptic neuron i. In particular, if neuron i has been active immediately before presynaptic spike arrival, the postsynaptic neuron is in a state of refractoriness. In this case, the response to an input spike is smaller than that of an 'unprimed' neuron. The first argument of sij(t- ii, s) accounts for the dependence upon the last firing time of the postsynaptic neuron. In order to simplify the notation later on, it is convenient to introduce the total

postsynaptic potential

.jE Fi

E.~-j

t(:f)

f0 C

(17)

Eq. (15) can then be written in the form ui(t) = n i ( t -

i;) +

h sp(tli;).

(18)

3.1.1. Example: refractoriness Refractoriness may be described qualitatively by the observation that immediately after a first action potential it is much more difficult to excite a second spike. In our description two factors contribute to refractoriness; see Fig. 10. Firstly q contributes because, during the spike, the voltage is above threshold. Thus it is excluded that the membrane potential is crossed from below so that emission of another spike is by definition impossible. Moreover, after the spike the membrane potential passes through a regime of hyperpolarization (negative overshoot) where it is below the

A framework jbr spiking neuron models: the spike response model

487

resting potential. During this phase, more stimulation than usual is needed to drive the membrane potential above threshold. Secondly, ~ and ~ contribute because, immediately after an action potential, the response to incoming spikes is shorter and, possibly, of reduced amplitude. Thus more input spikes are needed to evoke the same depolarization of the membrane potential as in an 'unprimed' neuron. The first argument of the ~ function (or function) allows us to incorporate this effect. If tJ/) - ti -----+ o o , then the response of neuron i to a presynaptic spike of neuron j is the standard EPSP. If t~ ) is close to ti, then the postsynaptic potential ~(t- ~, t - tJf )) has a different time course.

3.1.2. Example." experimental results In recent experiments, Stevens and Zador [39] have stimulated a cortical neuron with time-dependent current and measured the response of the membrane potential during repetitive firing. In order to fit their measurements to integrate-and-fire type dynamics, they found that it was important to work with a time-varying time 'constant' r ( t ' - ~). Given that the last output spike was at ~ < 0, the response to input at t - 0 is (for t > 0) approximated by { ~0 t dr' } ~(t'- ~) ' g:(t- ~, t) -- ao exp -

(19)

where a0 is a parameter and ~ ( t ' - ~) is the instantaneous membrane time constant. Immediately after the output spike at ~ the membrane time constant is only about 2 ms; for t ' - ~ - + oc the membrane time constant increases and approaches the standard value "Cm ~ 10--15 ms.

3.1.3. Example: SRMo A simpler version of the SRM can be constructed, if we neglect the dependence of upon the first argument. We simply set =

=

and use (151) in the form "i(t) -- rli(t - ti) + Z wij Z F~O(t-- ( ) ) jEFi tif) Egj

+

f0 ~

~.O(S)IeXt(t- s)ds.

(20)

J

Thus each presynaptic spike evokes a postsynaptic potential with the same time course, independent of the index j of the presynaptic neuron and independent of the last firing time ~; of the postsynaptic neuron. Only the amplitude of the response is scaled with the synaptic efficact wij. This simple version of the SRM has been termed SRM0 [23] and has been used for the analysis of computation with spiking neurons [36,37] and for that of network synchronization [22].

488

W. Gerstner

3.1.4. Example." from action potentials to formal events The shape of an action potential is described by the function r l ( t - t). Since it has a stereotyped time course, the form of the action potential does not transmit any information. What counts is the event 'spike' as such. In formal model, the form of the pulse is therefore often replaced by a 8 function. The negative overshoot after the spike is modeled as a reset to a lower value. One of several possible descriptions is r l ( t - t) - 8 ( t - i) - rl0 e x p ( with a parameter rl0 > 0. The negative overshoot (second term on the right-handside of (21)) decays back to zero with a time constant t. The simpification from a nicely shaped action potential to a formal q-function is illustrated in Fig. 11.

3.1.5. Example: graphical construction of firing times Let us now summarize the considerations of the two preceding examples and proceed to a graphical illustration of the model; cf. Fig. 12. The neuron under consideration receives input from several presynaptic cells. Each input spike evokes a postsynaptic potential of some standard form s0(s). We assume excitatory synapses, so that s is positive. The excitatory postsynaptic potentials are summed until the firing threshold 9 is reached. Each output spike is approximated by a 8 pulse, followed by a reset as in (21). Then the summation of inputs restarts.

3.1.6. Example." coherent versus incoherent input What can we do with the simplified neuron model defined by (20) and (21)? The SRM can provide an intuitive understanding of questions of neuronal coding and signal transmission. For example, we may easily understand why coherent input is more efficient than incoherent input in driving a postsynaptic neuron.

6(t-f)

\ ,, i i

i i

j:'i........................................... -11o. ?~ Fig. 11. In formal models of spiking neurons, the shape of an action potential (dashed line) is replaced by a 8 pulse (thick vertical line). The negative overshoot (spike after potential) after the pulse is included in the kernel r l ( t - t) (thick line). The pulse is triggered by the threshold crossing at ~.

489

A frameworkfor spiking neuron models: the spike response model ' I ~voltage response

,,./

input spikes I

1

ii/

,.~

!/

"spike

u

input spikes Fig. 12. Spike Response Model SRM0. Each input pulse causes an excitatory postsynaptic potential (EPSP) e(s). All EPSPs are added. If the threshold is reached the voltage is reset. The reset corresponds to adding a negative kernel q(s).

To illustrate this point, let us consider an e kernel of the f o r m ~~

jSexp(-~)z

for s > 0

(22)

and zero otherwise. We set J = 1 m V and z = 1 0 ms. The function (22) has a m a x i m u m value of J/e at s = z. The integral over s is normalized to Jz. Let us consider a neuron i which receives input f r o m 100 presynaptic neurons j. Each presynaptic n e u r o n fires at a rate of 10 Hz. All synapses have the same efficacy w0 = 1. Let us first study the case of a s y n c h r o n o u s input. Different neurons fire at different times so that, on average, spikes arrive at intervals of At = 1 ms. Each spike evokes a postsynaptic potential defined by (22). The total m e m b r a n e potential of neuron i is

bli(t) -- l ] ( t - ti) + ~

Z

Wo~.o(t- ( ) )

OG

rl(t- ti) + wo Z eo(t- nat)

(23)

n=0

If neuron i has been quiescent in the recent past ( t - ~; ~ ~ ) , then the first term on the right-hand side of (23) can be neglected. The second term can be a p p r o x i m a t e d by an integral over s, hence

ui(t) ~ -wo ~ ]o"~ e 0 ( s ) d s - woJz At

_- 10 mV.

(24)

If the firing threshold of the neuron is at 8 = 20 m V the neuron stays quiescient.

W. Gerstner

490

N o w let us consider the same number of inputs, but fired coherently at t~) - 0, 100,200,... ms. Thus each presynaptic neuron fires as before at 10 Hz but all presynaptic neurons emit their spikes synchronously. Let us study what happens after the first volley of spikes has arrived at t = 0. The m e m b r a n e potential of the postsynaptic neuron is

ui(t) = r l ( t - ti) + Nwoao(t),

(25)

where N -- 100 is the number of presynaptic neurons. The m a x i m u m of (25) occurs at t = z = 10 ms and has a value of woNJ/e ,~ 37 mV which is above threshold. Thus the postsynaptic neuron fires before t = 10 ms. We conclude that the same numer of input spikes can have different effects depending on their level of coherence. In Fig. 13 we illustrate this effect for a simplified scenario of two groups of presynaptic neurons. Neurons within each group fire synchronously. In a there is a phase shift between the spikes of the two groups, whereas in b the two groups are synchronous.

3.1.7. Example: sliding threshold interpretation The simplified model SRM0 defined in (20) with the rl kernel defined in (21) allows us to give a reinterpretation of refractoriness as an increase of the firing threshold. To see how this works, let us introduce the input potential

hi(t ) ~ Zwij ~ ,)

jGFi

.~f)

F~o(t - tj~.)) q- ~o~

(26)

-c.~.j

We emphasize that hi depends on the input only. In particular, there is no dependence upon ti. With the above definition of hi, Eq. (20) is simply u i ( t ) r l 0 ( t - ti) + hi(t). The next spike occurs if ui(t) = ~) or

h,(t) = a-

n0(t-

(27)

i,).

We may consider ,9 - rl0(t - ti) as a dynamic threshold which is increased after each firing. Eq. (27) has a simple interpretation: The next firing occurs if the input po-

(b)

I

I ..

I

., I

I I

I

I

Fig. 13. Potential u of a postsynaptic neuron which receives input from two groups of presynaptic neurons: (a) Spike trains of the two groups are phase shifted with respect to each other. The total potential u does not reach the threshold. There are no output spikes: (b) Spikes from two presynaptic groups arrive synchronously. The summed EPSPs reach the threshold ~) and cause the generation of an output spike.


tential hi(t) reaches the dynamic threshold ~ - q 0 ( t - t i ) illustration.

491

[40]. See Fig. 14 for an

3.2. Background In contrast to the standard integrate-and-fire model which is usually stated in terms of differential equations, Eq. (15) is based on an 'integral' representation with response kernels. Eq. (! 5) is linear in the spikes and can be considered a starting point of a systematic expansion [21,19]. As we will see later in Section 6, nonlinear effects between pairs of spikes can be included by second-order kernels of the form ~,ijk(t- ti, t - ( I , t - t~)). Higher-order nonlinearities are treated similarly. Effects of earlier spikes of postsynaptic neurons can be treated by kernels q i ( t - ti, t - t~2)), rli(t- "ii,... , t - t}k)) where ti = t}1) is the last spike of n e u r o n / a n d t}k) is the kth spike counting backward in time. The approach by spike response kernels provides a link between simplified neuron models of the form (15) and multi-compartmental models [1-3] and presents an alternative to earlier approaches towards a reduction of Hodgkin-Huxley equations [4,34,32,41]. The remainder of the chapter is organized as follows. In the following section, the integrate-and-fire neuron without spatial structure (point neuron) is reviewed. It is shown that integration of the model leads to (15) or (20). Thus, the integrate-andfire model is a special case of the SRM. In Section 5, a spatially extended version of the integrate-and-fire model with linear dendritic tree is considered. It is shown that integration of the model leads again back to (15). In Section 6 the problem of nonlinearities during synaptic and dendritic transmission is discussed.

4. Integrate-and-fire model We start in the first section with a review of the integrate-and-fire neuron. In the following sections we will show that the integrate-and-fire neuron is a special case of the SRM defined in Section 3.

1.0 el-

0.0

0'

,

,

zoo

t [ms]

Fig. 14. Sliding threshold interpretation. The input potential h(t) (solid line) is generated by the superposition of the EPSPs (solid line) caused by presynaptic spikes. Each spike arrival is denoted by an arrow. An output spike occurs, if h(t) hits the dynamic threshold l~(t) (dashed line). At the moment of spiking the value of the threshold is increased by one. After the spike, the threshold decays exponentially back to its resting value ~1= 1.

492

W. Gerstner

4.1. Definition o f the basic model The basic circuit of an integrate-and-fire model consists of a capacitor C in parallel with a resistor R driven by a current I(t); see Fig. 15, dashed circle. The driving current may be split into two components, I(t) = IR + Ic. The first component is the resistive current IR, which passes through the linear resistor R. F r o m Ohm's law we obtain I R - - u / R , where u is the voltage applied at the resistor. The second component Ic charges the capacitor C. F r o m the definition of the capacity as C = q/u (where q is the charge and u is the voltage), we find a capacitive current Ic = Cdu/dt. Thus

u(t) du I(t) - ~ + C d---t"

(28)

We multiply (28) by R and introduce the time constant ~m = RC of the 'leaky integrator'. This yields the standard form du

T'm dt -- - u ( t ) + RI(t).

(29)

We refer to u as the membrane potential and to ~m as the membrane time constant of the neuron. In integrate-and-fire models the form of an action potential is not described explicitly. Spikes are reduced to formal events and fully characterized by a 'firing time' tg/. The firing time is defined by a threshold process

u(t) -- ~) ~

t = t (f).

(30)

from neuron j _~_

axon ~

8 (t-t (f)) J

I

~ ,'% ,,

synapse

: ~.

' ~(t-t~f)) .- . . . . . . . ..

soma

i ,,"~~_~(t) "",, "'-..

..'""

g (t-t !~ 1

Fig. 15. Schematic diagram of the integrate-and-fire model. The basic circuit is the module inside the dashed circle on the right-hand side. A current I(t) charges the RC circuit. The voltage u(t) across the capacitance (points) is compared to a threshold 0. If u(t) = 0 at time t~ ) an output pulse 8 ( t - t i (f)) is generated. Left part: a presynaptic spike 8 ( t - ( ) ) i s low-pass filtered at the synapse and generates an input current pulse a ( t - t]f)).

A framework for spiking neuron models: the spike response model

493

Immediately after t (f), the potential is reset to a new value Ur < 9, lim u(t (f) + 8) -- Ur.

840+

(31)

For t > t (f) the dynamics is again given by (29) until the next threshold crossing occurs. The combination of leaky integration (29) and reset (31) defines the basic integrate-and-fire model.

4.1.1. Example: constant stimulation and firing rates Before we continue with the definition of the integrate-and-fire model and its variants, let us study a simple example. Let us suppose that the integrate-and-fire neuron defined by (29)-(31) is stimulated by a constant input current I(t) = Io. To keep the mathematical steps as simple as possible we take the reset potential to be Urn0.

As a first step, let us calculate the time course of the membrane potential. We assume that a first spike has occurred at t = t (~ The trajectory of the membrane potential can be found by integrating (29) with the initial condition u(t (~ = Ur = O. The solution is

_ 0[1 exp(

,32)

The membrane potential approaches for t ---, oc the asymptotic value u(ec) = R/0. For R/0 < l) no further spike can occur. For R/0 > 8, the membrane potential reaches the threshold ,9 at time t (~), which can be found from the threshold condition u(t (1)) = 8 or

,33) Solving (33) for the time interval T = t (1) - t (0) yields T - ~mln R/0 R/0 - 8"

(34)

After the spike at t (1) the membrane potential is again reset to Ur = 0 and the integration process starts again. If the stimulus I0 remains constant, the following spike will occur after another interval of duration T. We conclude that for a constant input current I0, the integrate-and-fire neuron fires regularly with period T given by (34). We may define the mean firing rate of a neuron as v = 1/T. The firing rate of the integrate-and-fire model with stimulation I0 is therefore

v-

R/0 I)].1-1 [z,,lnRio_

In Fig. 16b the firing rate is plotted as a function of the constant input I0.

(35)

494

W. Gerstner

(a)

(b) 0.4 1.0

9

...

,

.

,

.

,

N m,,,--=

0

20 t[ms]

0.2

0.0

40

o

0.0

2.0

4.0

I,,

6.0

8.0

Fig. 16. (a) Time course of the membrane potential of an integrate-and-fire neuron driven by constant input current I0 = 1.5. The voltage u(t) is normalized by the value of the threshold ~) = 1. (Resistance R = 1 and membrane time constant rm = 10 ms.) (b) The firing rate v of an integrate-and-fire neuron without (solid) and with absolute refractoriness of ~abs -- 4 ms (dashed) as a function of a constant driving current I0. Current units normalized so that the current threshold is I0 -- 1. (Reset to Ur = 0.)

4.1.2. Example: time-dependent stimulus I(t) The results of the preceding example conditions. Let us suppose that a first lating current is I(t). We allow for an treated as an initial condition for the integration is

u(t)

=

u,.exp( ---~-m t-i ) + Clf'-i

can be generalized to arbitrary stimulation spike has occurred at i. F o r t > ~ the stimuarbitrary reset value Ur. The value Ur will be integration of (29). The formal result of the

exp

(r-~) I(t -

s)ds.

(36)

Eq. (36) describes the m e m b r a n e potential for t > i and is valid up to the m o m e n t of the next threshold crossing. If u(t) = ,9, the m e m b r a n e potential is reset to Ur and integration may restart; see Fig. 17.

4.1.3. Example." absolute refractoriness It is straightforward to include an absolute refractory period. After a spike at t (f), we force the m e m b r a n e potential to a value u = Ur and keep it there during a time ~abs. C u r r e n t I(t) which arrives during the interval [t~f),t (y) + 6 abs] has no effect and is disregarded. At t (/) + ~abs the integration of (29) is restarted with the initial value u = ur. The time interval 6 abs during which the neuron is insensitive to input is called the 'absolute refractory period'. The inclusion of an absolute refractory period does not cause any problems for the integration of the model equations. F o r example, we can solve the dynamics for a constant input current I0. If a first spike has occurred at t = t (~ then u(t) =_ Ur for t (~ < t < t (~ + ~abs and

,,,,

ol, -

for t > t (~ + ~abs.

-

+Urexp(t--t'O'-- abs)--

(37)

495


1.0

r 3

A . 4 . . ~

o 9

.

.

0

9

20

.

.

.

.

.

40

.

.

.

.

.

.

60

t [ms]

J

80

.

.

.

1 O0

Fig. 17. Voltage u(t) of an integrate-and-fire model (top) driven by the input current l(t) shown at the bottom. The input I(t) consists of a superposition of four sinusoidal components at randomly chosen frequencies plus a positive bias current I0 = 1.2 which drives the membrane potential towards the threshold. If R/0 > 9, the neuron will fire regularly. Due to the absolute refractory period the interval between firings is now longer by an a m o u n t ~abs compared to the value in (34). The mean firing rate v = 1/T is

R/0 -

"

(38)

The firing rate of the integrate-and-fire neuron as a function of the constant input current is plotted in Fig. 16b.

4.2. Stimulation by synaptic currents So far we have considered an isolated neuron which is stimulated by an applied current I(t). In a more realistic situation, the integrate-and-fire model would be part of a larger network. The input current I(t) is then generated by the activity of presynaptic neurons. In the framework of the integrate-and-fire model, we may assume that each presynaptic spike generates a synaptic current pulse of finite width. If the presynaptic neuron j has fired at t~f5, spike arrival at the synapse will evoke a current a ( t - tj(f)) f o r t > ( ) . Since several presynaptic neurons contribute to driving the neuron, the total input current to neuron i is

Ii(t)- Z cij ,jEFi

~ a(t- ()). o(f)E~j

(39)

W. Gerstner

496

The factor cij is a measure of the efficacy of the synapse with units of a charge. 1 (39) is a reasonable model of synaptic interaction. Indeed, each input spike arriving at a synapse opens some ion channels and evokes a current through the membrane of the postsynaptic neuron i. Reality is somewhat more complicated, however, since the amplitude of the synaptic input current may itself depend on the membrane voltage ui. In detailed models, each presynaptic action potential evokes a change in the synaptic conductance with standard time course g ( t - t(~)), where tCr) is the arrival time of the presynaptic pulse. The synaptic input current is modeled as

l i ( t - t (f)) = 9 ( t - t(f))[ui(t) -- Urev]-

(40)

The parameter Urev is called the reversal potential of the synapse. The level of the reversal potential depends on the type of synapse. For excitatory synapses, Urev is much larger than the resting potential. The synaptic current then shows saturation. The higher the voltage u;, the smaller the amplitude of the input current. The total input current is therefore not simply the sum of independent contributions. Nevertheless, since the reversal potential of excitatory synapses is usually significantly above the firing threshold, the factor [ui- Urev] is always some large number which varies only by a few percent. Systematic corrections to the current equation (39) will be derived in Section 6. For inhibitory synapses, the reversal potential is close to the resting potential. An action potential arriving at an inhibitory synapse pulls the membrane potential towards the reversal potential Urev which is close to Urest. Thus, if the neuron is at rest, inhibitory input hardly has any effect. If the membrane potential is instead considerably above the resting potential, then the same input has a strong inhibitory effect. This is sometimes described as the 'shunting' phenomenon of inhibition. The limitations of the current Eq. (39) will be discussed in Section 6. In the following we will always work with (39).

4.2.1. Example." pulse-coupling and a-function In this section we will give some examples of the synaptic current a(s) in (39). We start with the simplest choice. Spikes of a presynaptic neuron j are described as Dirac k-pulses which are fed directly into the postsynaptic neuron i. Thus a(s) = 8(s). The total input current to unit i is then

I i ( t ) - Z cij Z jEFi

8(t-(

))"

(41)

t(.l) j c.yT i

As before, the factor cij is a measure of the strength of the connection from j to i. In case of (41), c;j can be identified with the charge deposited on the capacitor C by a single presynaptic pulse of neuron j.

1 c/7 is, of course, proportional to the synaptic efficacy wij as we will see later on.

497

A frameworkfor spiking neuron models: the spike response model

More realistically, the synaptic current 0~ should have some finite width. In Fig. 15 we have sketched the situation where ~(s) consists of a simple exponentially decaying pulse

'exp( )

ors 0

and zero otherwise. Eq. (42) is a first approximation to the low-pass characteristics of a synapse. The exponential pulse (42) can be considered to be the result of some synaptic dynamics described by a first-order linear differential equation. Let us set d

(43)

jeri

t}f) c~j

Integration of the differential equation (43) yields (39) with a(s) given by (42). In (42) the synaptic current has a vanishing rise time which is not very realistic. More generally, we may assume a double exponential which sets in after a transmission delay Aax. For s < Aax we therefore have a(s) = 0. For s > A ax we set ~(s) = 1

"1;s -- -Or

[

exp

(

-

s - A ax) ~ "Cs

- exp

( --~AaX~]s "Cr ,]

.

(44)

Here Zs is a synaptic time constant in the millisecond range and 1;r with "Cr _~ Ts is a further time constant which describes the rise time of the synaptic current pulse. In the limit of Tr --+ "Cs, (44) yields (for s > A ax)

~(s)-s-Aax

----v--- exp

(

-

s-Aax~

.

(45)

"Cs ,/ In the literature, a function of the form x e x p ( - x ) such as (45) is often called an afunction. While this has motivated our choice of the symbol at for the synaptic input current, ~(.) in (39) may stand for any form of an input current pulse. As mentioned before, an yet more realistic description of the synaptic input current would include a reversal potential for the synapse as defined in (40). 4.3. Spike response method (2)." reset as current pulse The basic equation of the integrate-and-fire model, Eq. (29), is a linear differential equation. It can therefore be integrated in a straightforward manner. Due to the threshold and reset conditions, Eqs. (30) and (31), the integration process is not completely trivial. In fact, there are two different ways of integrating (29). The first one treats the reset as a current pulse, the second one as an initial condition. We discuss both methods in turn. In this subsection we focus on the first method and describe the reset as an additional current. Let us consider for the m o m e n t a short current pulse/out = -qS(t) applied to the RC circuit of Fig. 15. It removes a charge q from the capacitor C and lowers the

498

W. Gerstner

-q/C.

potential by an amount Au Thus a reset of the membrane potential from a value of u - 8 to a new value u = Ur corresponds to a negative current pulse which removes a charge q = C ( 8 - Ur). Such a reset takes place at the firing time t/(f). The total reset current is therefore

I?Ut(t)---C(~-Ur) Z t(.f) i EJr

~(t-ti(f)) '

(46)

i

where the sum runs over all firing times. Formally, we may add the output current (46) on the right-hand side of (29) T'm

dui

dt =

(47)

- u i ( t ) .qt_RIi(l) + R/OUt(t).

Here (48)

Ii(t) -- ~ Ciff y ~ O~(t- ~fl))-]-/text(t) jEFi t(f j )E,~ i

is the total input current to neuron i, generated either by presynaptic spike arrival or by external stimulation Iext(t). We note that (47) is a linear differential equation. Integration with initial condition u(-c~) = 0 yields

( s ) [I~

ui(t) -~l

exp -~mm

= Z tlf) E.~

+Z

--(~- Ur)exp - s 8 ( t -

/o Z

exp -~mm a ( t -

jGF~ t~f)E #, 9

+~

lfo~

s) + Ii(t- s)]ds ti~)- s)ds -

/7-.

( S)Iext(t-s)ds.

exp --~mm

(49)

Let us define for s > 0

(so)

q0(S) --" --(0- Ur)exp ( - r-~)

so(s)-

Y)dY

, (s) -~mm

~z0(s)-~exp

(sl) (52)

and r l 0 ( s ) - s 0 ( s ) - ~;0(s)- 0 for s < 0. With the above definitions, (49) may be rewritten in the form


u;(,)= Z

~lo('-t~f))+Zwij

t? )C~i

jcri

-Jr-

~_~ ~~

499

f))

o(f)cd,~j

~,O(s)Iext(t-s)ds.

(53)

with weights wii- cij/C. Eq. (53) is the main result of this subsection. The kernel q0(s) defined by (50) is shown in Fig. 18. We emphasize that the firing times t~ ) which appear implicitly in (49) have to be calculated as before from the threshold condition u~(t) = ,~. In [22], the 'SRM' has been defined by (53). Note that, in contrast to (15), we still have a sum over past spikes of neuron i on the right-hand side of (53).

4.3.1. Examples of eo-kernels If cx(s) is given by (42), then the integral on the right-hand side of (51) can be done and yields (s > 0) 1 -

(54)

[exp ( - ~-~) - exp ( - ~ / 1

1

This is the ~ kernel shown in Fig. 19a. If a(s) is the Dirac 8-function, then we find simply ~0(s) = exp(-s/~m) as shown in Fig. 19b. Note that 0~(s) is the synaptic current. Integration of the synaptic current yields the postsynaptic potential ~o(s). If the synapse is excitatory, e0 is called the EPSP. For an inhibitory synapse, ~0 describes the IPSP.

4.3.2. Short-term memory approximation To keep the discussion transparent, let us set

ui t -

t~i~i'~i

iext

._

0. Eq. (53) is then

(ss)

+

j~Fi (IE~;

On the right-hand side of (55), there is a sum over all past firings of neuron i which does not appear in (15), the equation we are aiming for. According to (50) the effect of the rl0-kernel decays with a time constant ~m. In realistic spike trains, the interval between two spikes is typically much longer than 0.0 /

/

-1.0 0

20 t [ms]

40

60

Fig. 18. The kernel 1"10 of the integrate-and-fire model with membrane time constant "1~m ~

10 ms.

500

W. Gerstner

(a)

(b)

0.1

@

0.1

@ 0.0 0.4

0 ,

.

20 .

.

.

40 .

.

60

.

0.0

0

20

40

60

0

20

40

60

0.4

0.2

0.0

0

~ 20 t [ms]

40

60

0.0

t [ms]

Fig. 19. (a) ~-kernel of the integrate-and-fire model (top) with exponential synaptic input current 0~ (bottom). (b) If the synaptic input is pulse-like (bottom), then the ~-kernel is a simple exponential. the membrane time constant ~m. Hence the sum over the q0 terms is usually dominated by the most recent firing time ti(f) < t of neuron i. We therefore make a truncation and neglect the effect of earlier spikes Z t i(11 E 9

r l o ( t - ti(f)) ~

r l o ( t - ti),

(56)

i

where ti is the last firing time of neuron i. The approximation (56) is good if the mean firing rate of the neuron is low, i.e., if the intervals between two spikes are much longer than Zm. Loosely speaking, we may say that the neuron remembers only its most recent firing. Equation (56) may therefore be called a 'short-term memory approximation' [22]. The final equation is (57) j E Fi

t (/) E.~j )

This is exactly the equation for the model SRM0, defined in (20). Note that we have kept, on the right-hand side of (57), the sum over all presynaptic firing times tj(f). Only the sum over the rl0's has been truncated. Equation (57) can be seen as an approximation to the integrate-and-fire model or else as a model in its own rights. The advantage of SRM0 is that many network results can be derived in a rather transparent manner [21-23]. Moreover questions of computation and coding with spiking neurons can be analyzed in the context of SRM0 [36,37,24]. 4.4. Spike response method (3)" reset as initial condition

In this section, we discuss a method of integration which gives a direct mapping of the integrate-and-fire model

501


du i

"Cm 8--7 z --bli(t) -~-R ~-~cij Z ~(t - t~f )) -Jr-Iext(t) jEFi t~.)E~j

(58)

to the SRM (15). As in (36) we integrate (58) from/i to t with u(/i) - Ur as an initial condition. The result is

u(t)

= Ur exp ( --t-ti).cm +Z ~ jEFi t~f)Eo~j

exp -~mm ~ ( t -

+~1 f0 t-ii exp (-S')lext(t-J)ds'-~m

-

)ds'

(59)

We may now define kernels

rl(t - ti) -- urexp(- t - ti) "Cm ,I

(60)

/0

(61)

a(t-

ti, s) -

exp

--~m a(s - s')ds'

s)~(t-ii-s) ~(t- ti, s) - -~1 exp ( -~mm

(62)

and the synaptic efficacy wij- cij/C. As usual, ~ ( x ) denotes the Heaviside step function which vanishes for x _< 0 and has a value of one for x > 0. The kernels (60)(61) allow us to rewrite (59) in the form

jEFi nt-

jo

~;(t

-

()E~j

t/, s-"ext (t )6

-

s)ds,

(63)

is identical to (15) except for some minor changes of notation. We emphasize that the rl-kernel defined in (60) is not the same as the one defined in (50). In particular, the rl-kernel (60) vanishes if Ur - 0 . which

4.4.1. Examples of e kernels In order to calculate the ~ kernels (61) and (62) explicitly, it is convenient to distinguish two cases. First we consider the case, that the last output spike occurred before presynaptic spike arrival (ti < ( ) ) . Therefore t - t i > t - ( ) - - s . Since a(s - s ~) vanishes for s - s' < 0 we may extend the upper boundary in (61) to infinity without introducing an error. Hence, for tj(f)> ti, we have ~(t-ti, t-t~f )) = ~o(t- t}f)) where t0 has been defined in (51).

w. Gerstner

502

The situation is different, if ~; > tJit, i.e., if the last output spike has occurred after presynaptic spike arrival. In this case only that part of the synaptic current which arrives after ti contributes to the present postsynaptic potential and exp

8(t -- ti, t -- tj(f)) --

- t- r

a({ - tj(f))d{.

(64)

"17m

To be specific, we take a(s) as defined in (42), viz.,

a(s) = "c-~l exp(-s/r,s)~(s). Let us set x -

(65)

t - ~;. The integration of (61) yields [21]

~(x,s) = 1 -' '~

([

exp

-

- exp

~(s)~,Ug(x - s)

-

Tm

+ exp ( _ s - ~ x ) [exp ( - - ~ ) -

exp(-~)]~'~(x)Jg(s-x)).

(66)

The Heaviside functions ~'~(x - s) in the first line of (66) picks out the case ( ) > ti or x > s. The second line contains the factor ~ , ~ ( s - x) and applies to the case t~r / < ti or x < s. See Fig. 20 for an illustration of the result.

4.4.2. Transformation of the ~ kernel W h a t is the relation between the ~ kernel derived in (61) and the e0 introduced in (51)? We will show in this p a r a g r a p h that =

-

exp( ) -

e0 (s - x)

(67)

holds. To see how this comes about we start from (61). We set x - t y = s - s' and find

i; and

0.2 O.1

O.O

~(o [ ~(040 .

t [ms]

80

Fig. 20. The kernel e ( t - ~, t - ( t ) as a function oft for two different situations. If tj(.f) > i, then e ( t - ~, t - t)r/) = e0(t- t~ )) is the standard EPSP (thick solid line). If t~ / < ~, the amplitude of the EPSP for t > ~is much smaller (thin solid line) since the time course of the EPSP is 'reset' to zero at t = i (marked by the long arrow). The time course for t < i is indicated by the dashed line.


G(X, s) --

503

fs ( )

exp -- s -- y a(y)dy

S--X

-

~m

exp -

a(y)dy ~m

exp ~

a(y)dy.

(68)

Tm

In the first term on the right-hand side of (68) we may transform back to the variable s' = s - y, in the second term we set s' = s - x - y. This yields G(x, s) -

/0

exp

~(s - s')ds'

-exp (-rf~)f0~ exp (- ~m)=(~- ~- s')d~'

(x)

= G0(s) - exp - ;c--s G0(s - x).

(69)

The last equality follows from the definition of G0 in (51). By a completely analogous sequence of transformations it is possible to show that

(x)

~z(x, s) - ~:o(s) - exp - ~m ~Zo(s - x).

(70)

The total postsynaptic potential hpsp defined in (17) can therefore be expressed via the input potential hi [23] hpsp(t[ti) - hi(t) - exp ( - t - ti)

(71)

As it should be expected, the reset at ti has an influence on the total postsynaptic potential. We emphasize that the expressions (69)-(71) hold for the integrate-andfire model only. For a general Hodgkin-Huxley type dynamics the transformations discussed in this paragraph would not be possible. 4.4.3. Relation between the two integration methods In order to better understand the relation between the two integration methods outlined in Sections 4.3 and 4.4, we compare the q-kernel in (60) with the q0-kernel defined in (50): q ( S ) - Ur exp (-- T-~) = r l 0 ( s ) + 9exp ( - - ~ ) . Hence with (71), the potential is

(72)

W. Gerstner

504

ui(t) : q ( t - ti) + hpsp(t ti) = qo(t_ ~i) + h(t) _ [h(~i) _ ~)]exp (

--

t--,lTm ti)

.

(73)

The truncation in (56) is therefore equivalent to neglecting the last term in (73).

4.5. Discussion The second of the two integration methods shows that it is possible to map the integrate-and-fire model exactly to the spike response equation (15). The disadvantage of that method is that the ~ kernels look somewhat more complicated. This is, however, no real drawback since the dynamics of a population of spiking neurons can be discussed for arbitrary response kernels 11 and e [21,23]. The integrate-and-fire model is therefore a special case in the general framework of the spike response model. With the first method of integration, the mapping of the integrate-and-fire model to (15) is only approximate. The approximation is good if the typical interspike interval is long compared to the membrane time constant ~m. The main advantage of the approximation is that the ~ kernels do not depend on the state of the postsynaptic neuron. Therefore, the input potential hi(t) can be nicely separated from the effects of reset and refractoriness; cf. (26) and (27). The resulting model SRM0 allows us to discuss dynamic effects in a transparent graphical manner; see, e.g., [22,24,36,37]. The basic integrate-and-fire model is, of course, a rather simple description of neuronal firing. In particular, the neuron has no spatial structure and firing is given by an explicit threshold condition. In the following section we will extend the framework (15) to neuron models with spatial structure.

5. Multi-compartment model 5.1. Definition of the model In this section, the integrate-and-fire model introduced in Section 4 is generalized in two respects. First, we allow for some spatial structure and consider a neuron consisting of several compartments. Second, we refine the reset procedure and include, at the somatic compartment, additional spike currents which generate an action potential. See the chapter of Segev and Meunier in this book for more detailed information about information processing on neuronal dendrites.

5.1.1. Linear dendritic tree We consider a model with n - 1 dendritic compartments 2 _< ~t _< n and a threshold unit at the soma (~t - 1); cf. Fig. 21. Membrane resistance and capacity are denoted by R~ and C ~, respectively. The longitudinal core resistance between compartment ~t and a neighboring compartment v is r uv. We assume a common time constant R~C~--% for all compartments 1 _< ~t _< n. The above specifications define the standard model of a linear dendrite [42].


505

rv~

R"

~r I~

__TC" .k

,,.2

The last two terms on the right-hand side of (86) are the initial conditions for the compartment voltages. For the somatic compartment, we use V l (ti) - ,~. For la _> 2 we may formally use (81) evaluated at t - ti. The sum in the first line on the righthand side of (86) vanishes if the spike currents have stopped before the next spike o c c u r s - as it is trivially the case for a reset current y(s) - -qS(s). In the following we will therefore neglect these terms. We now define r l ( t - ti) - r l 0 ( t - ti) +

~GI1(t- li)

(87)

and, for j 6 Fv _

_

(88/

We use GlV(x + y) = ~-~ Gl~(x)G~V(y) in the Greens function in the second line of (86). With (87) and (88) we find after some calculation:

J

~>2

t (f) G.~j J

t tr) ti some feedback onto the soma.

509

A f r a m e w o r k f o r spiking neuron models: the spike response model

Note the close analogy between (88) and (69). The Greens function G 11 is the generalization of the exponential term in (69). Similarly, the q-kernel (87) is the generalization of (72). We emphasize that for a single-compartment model, the sum in the last line of (89) vanishes. The mapping between the integrate-and-fire model and the SRM (15) is then exact, as we have seen in Section 4.4. For a multicompartment model the mapping to (15) is not exact. The approximation derived in this paragraph is, however, better than the truncation that is necessary to get (85).

5.2.2. Example: two-compartment &tegrate-and-fire model We illustrate the SRM by a simple model with two compartments and a reset mechanism at the soma. The two compartments are characterized by a somatic capacitance C 1 and a dendritic capacitance C 2 = aC 1. The membrane time constant is z0 -- R 1 C 1 - R Z c 2 and the longitudinal time constant T12-- r 12cC1C2 1+c2. The neuron fires, if V 1 = ~). After each firing the somatic potential is reset to V 1 = Ur. This is equivalent to a current pulse 7(s) = -qS(s),

(90)

where q = C 1[ 8 - Ur] is the charge lost during the spike. The dendrite receives spike trains from other neurons j and we assume that each spike evokes a current pulse ~(t - t}f)) with time course ~ ( s ) - 1 exp(--Zs - ~ s ) "

(91)

For the two-compartment model it is straightforward to integrate the equations and derive the response kernels q(s) and ~(s); cf. [8,18,44]. We find q0(s)-

(~--Ur l + a ) eXp (~00)

1

[ l + a e x p ( - T-~2)I

( 0)E1~ Z s ~ ,

t0(s) - (1 + a) exp -

1 - e -82s] -- exp --

Zs~2 J

(92)

with 81 - T s 1 - TO1 and 82 = Zs 1 - Zo 1 - zi-21. In Fig. 23 we show the two response kernels for the parameters z0 = 10 ms, z12 = 2 ms, and a = 10. The synaptic time constant is Zs = 1 ms. The kernel t0(s) describes the voltage response of the soma to an input at the dendrite. It shows the typical time course of an excitatory or inhibitory postsynaptic potential. The time course of the kernel q(s) is a double exponential and reflects the dynamics of the reset in a two-compartment model. In Fig. 23a, the moment of spike firing at t = 0 has been marked by a vertical bar for the sake of better visibility. 6. E x t e n s i o n s a n d d i s c u s s i o n

The strict application of the spike response method requires a system of linear differential equations combined with a threshold p r o c e s s - such as in the integrate-

510

W. Gerstner

(a)

(b) 0.5

0.5 I,d

0.0

o.o

oo

,

.

.

i

.

.

.

.

i

.

200

tiros]

.

.

.

i

.

30.0

.

.

.

,,o.o

.

.

i

o.o

,

,

2o:o

t[msl

30.0

40.0

Fig. 23. Two-compartment integrate-and-fire model. (a) Response kernel Tl(s) of a neuron with two compartments and a fire-and-reset threshold dynamics. The response kernel is a double exponential with time constants ~ 2 - 2 ms and c 0 - 10 ms. The spike at s - 0 is indicated by a vertical dash. (b) Response kernel e(s) for excitatory synaptic input at the dendritic compartment with a synaptic time constant Xs - 1 ms. The response kernel exhibits the typical time course of an excitatory postsynaptic potential. (y-axis: voltage in arbitrary units). and-fire model in Section 4. Naturally the question arises how well real neurons fit into this framework. As an example of a more complicated neuron model we have discussed the effects of a linear dendritic tree. We have also seen in Section 2 that, for the Hodgkin-Huxley model, spike generation can be replaced approximatively by a threshold process. In this section we want to continue our discussion and hint to possible extensions and modifications. To check the validity of the approach, we discuss the two basic assumptions, viz., threshold process and linearity.

6.1. Threshold process The dynamics of spike generation can be described by nonlinear differential equations of the type proposed by Hodgkin and Huxley [14]. Spikes are generated by a voltage-instability of the conductivity. Since the opening and closing of Na and K channels are described by three variables with three different time constants, the threshold depends not only on the present voltage, but also on the voltage in the recent past. In other words, there is no sharp voltage threshold [15,25]. This is most easily seen in a scenario with arbitrary time-dependent input. Let us suppose that, for some ion-based neuron model, there exists a voltage threshold ,~. Even if the potential were already slightly above the formal threshold, there could arrive, in the next moment, a very strong inhibitory current which pulls the potential back below threshold. Thus spiking could still be stopped even though the action potential was already initiated. This consideration points to the general limitations of the threshold concept in the context of time-dependent stimulation. Strictly speaking there can be neither a voltage nor a current threshold if we allow for arbitrary input. Nevertheless, an improvement over the simple voltage threshold is possible. The spike response method does not rely on a specific interpretation of the variable u(t). In principle, it can be any relevant variable, e.g., a current [45,25], a voltage, or some combination of current and voltage variables. To be specific, we may take


u(t) --

f (s) V 1 (t - s ) d s

~0~176

--

f

511

(93)

9 V 1,

where V 1 is the voltage at the soma and f some linear filter with normalization f o f ( s ) d s - 1. Since everything is linear, the response kernels derived in the preceding sections can be transformed ~ ~ f 9a and 11 ~ f * 11 and we are back to the standard form (15). Application of the linear operator f on the voltage u in (8) before it is passed through a threshold would, for example, allow us to match the boundaries in the phase diagram of Fig. 5c more closely to that of the HodgkinHuxley model in Fig. 5b. We emphasize that the formal threshold is constant in our approach. A dynamic threshold ~)(t) which is increased after each spike [40] may always be treated as an additional contribution to the response kernel rl(s) as discussed in (27).

6.2. Adaptation In all the discussion above we have assumed that only the last output spike of the neuron is relevant. This is, of course, an over-simplification of reality. For most neurons, adaptation plays an important role. If a constant input is switched on at to, the interspike interval between the first and second spikes is usually shorter than the one between the 10th and 1 lth. How can adaptation be included in the above framework? One possibility is to make a systematic expansion so as to include the effect of earlier spikes 1

(1)

jGFi

_t~l)

,, _ ( ) )

()E•j

JEFi

~f) Gff j

(94) Here t}l) is the most recent firing of neuron i, t}2) the second last firing, and so forth. If too many terms are necessary, then the approach outlined in (94) is not very handy. On the other hand, we may assume that the major contribution comes from the term rl(Z), 11(3),... and neglect the terms g (2) . . . . Moreover, we may assume, for the sake of simplicity, that rl~/l) = r~i(2) = r l i (3) , . . TI. In fact, adaptation and even bursting can quickly be incorporated if we use a description of the form (1) ( t - t ~ l ) t - ( t~) E~'i

SEI'i

)

()Eo~j

For the kernels 11 we may choose a time-course with a long-lasting contribution which could arise due to, e.g., slow calcium-dynamics [22]. An example is shown in Fig. 24. The neuron is driven with a constant input current. The rl(s)-kernel has a phase of after-depolarization. As a result, a first spike at s = 0 makes a second spike around s ~ 5 ms more likely. A late phase of hyperpolarization in the rl-kernel turns

512

W. Gerstner (a)

(b)

0,5

0.5

g-

> 0.0

0,0 0.0

Fig. 24.

10.0

20.0

rims]

30.0

40.0

0.0

100.0

t[ms]

200.0

300.0

Bursting neuron. Constant stimulation of a neuron model with the q-kernel shown in (a) generates the spike train in (b). Taken from [22].

firing off after a couple of spikes. The results is, for constant input, a bursting behavior as in Fig. 24b.

6.3. Nonlinearities We can distinguish at least three types on nonlinearities of neuronal dynamics. First, there is the nonlinear dynamics of spike generation. These nonlinearities are replaced by the output current ~/(s) which is trigerred by a threshold process as explained above. Second there are shunting effects on the dendrite due to the ion reversal potential, and finally there are potential sources of active currents on the dendrite. The last issue has been a subject of intensive discussion recently. There are indications for dendritic spikes [46], but it is unclear whether this is a generic feature of all neurons. In our approach, all active dendritic currents are neglected. In the following we concentrate on the influence of the reversal potential. In Sections 4 and 5 we have assumed that each input induces a standard current pulse 0~(t- t~)). In more detailed models, however, the input current is due to a conductivity change g(t - t / ) ) at the synapse and the amplitude of the current depends on the present value of the m e m b r a n e potential; see Section 4.2. Specifically, in the context of a compartmental neuron model, the input to c o m p a r t m e n t ~t is

Ila(t)- ~

Z [UrevJ EF~ t(f)E._~j i

Vla(t)]g(t- ( ) ) ,

(96)

where Urev is the reversal potential and wij - 1 for the sake of simplicity. For a further analysis of (96) we write U~ev- V~ - (U~ev -- 12) -- (V ~ - l?) and set ~(s) - (U~ev - V)g(s). For the potential 12 we take some appropriate value between the equilibrium potential V0 and the threshold 8, e.g., V - ( 8 - V0)/2. This yields

jEP

f

Urev- V

]

As long as IV~ - V[ jn ~'U"9 dynamics when the number of patterns is sufficiently small and 13is sufficiently large. Thus the network operates as a distributed memory. When 13 is too small, all attractors become unstable and the firing of the neurons becomes more or less uncorrelated. This behavior is similar to the paramagnetic phase discussed in the SK model. When the number of patterns is too large, the network behaves as a spin glass whose minima are uncorrelated with the stored patterns. This behavior is to a large extent independent of whether the neuron dynamics is sequential or parallel (see Section 3 for the definition of these terms).

7.2. Boltzmann machines Another well-known application of the Boltzmann-Gibbs distribution are Boltzmann Machines [8]. The basic idea is to treat the distribution Eq. (17) as a statistical model, and to use standard statistical tools to estimate its parameters w u and Oi. Let us partition the neurons in a set of nv visible units and nh hidden units (nv + nh = n). Let c( and 13 label the 2 "v visible and 2 "h hidden states of the network,

An introduction to stochastic neural networks

545

respectively. Thus, every state s is uniquely described by a tuple ctl3. Learning consists of adjusting the weights and thresholds in such a way that the BoltzmannGibbs distribution on the visible units p ~ - ~ F P ~ approximates a target distribution q~ as closely as possible. A suitable measure for the difference between the distributions p~ and q~ is the relative entropy [38] K - Z

q~ log q~.

(36)

P~

It is easy to show that K _> 0 for all distributions p~ and K - 0 iff p~ - q~ for all ~. Therefore, learning consists of minimizing K with respect to wi2 and 0i using gradient descent and the learning rules are given by [8,39] 8K ~0 i -- --1] ~//

- - ]] ( ( S i ) c - - (Si)),

8K 6wij -- --]]~wij - T~((siSj)c - (sisj)) i ~ j.

(37)

The parameter 11 is the learning rate. The brackets (.) and (')c denote the 'free' and 'clamped' expectation values, respectively. The 'free' expectation values are defined as usual:

aF s i sj p ~ .

(38)

N

The 'clamped' expectation values are obtained by clamping the visible units in a state a and taking the expectation value with respect to q~: (Si) c -- ~

Si~ q~PFl~,

s i sj q~P~l~, aF

s~~ is the value of neuron i when the network is in state a]3. P~I~ is the conditional probability to observe hidden state 13 given that the visible state is a. Note that in Eqs. (37)-(39), i and j run over both visible and hidden units. Thus, the BM learning rules contain clamped and free expectation values of the Boltzmann-Gibbs distribution. The computation of the free expectation values is intractable, because the sums in Eq. (38) consist of 2" terms. If q~ is given in the form of a training set of p patterns, the computation of the clamped expectation values, Eq. (39), contains p2 ~ terms. This is intractable as well, but usually less expensive than the flee expectation values. As a result, the exact version of the BM learning algorithm cannot be applied to practical problems.

546

H.J. Kappen

We therefore apply the MF approximation as discussed in the previous section. Due to the symmetric weights, the Boltzmann Machine is an equilibrium system and we can improve on our estimates of the correlations between neurons, Eq. (33), using the linear response theorem [40]. The starting point is to observe the exact relations (si) - ~ log_____fZ,

(40)

O0i ~2 log Z Zij =

~Oi~Oj

,

(41)

which follow immediately from the definition of Z. We can combine these equations and obtain O(Si)

~i~ = ~0;

Omi

~ ~0~

(42)

Thus, the correlations are given by the derivative of the equilibrium firing rates with respect to the thresholds. In the last step we have replaced these firing rates by their MF estimates, Eq. (31). We can compute the right-hand side of Eq. (42) from Eq. (31). Having obtained estimates for the statistics, this basically solves the learning problem. For arbitrary wij and 0 i w e can compute the mean firing rates and correlations (both clamped and free) and insert these values into the learning rule Eq. (37). The situation is particularly simple in the absence of hidden units, s In this case, (')c does not depend on wij and 0i and are simply given by the statistics of the data: If the data consist of p patterns with equal probability, s~, ~t = 1 , . . . , p , then ' Thus our task is to find Wij and 0i such that (Si) c = P1~ , s~ and (SiSj) c = p1~ o s~~s). the (MF approximations of the) free mean firing rates and correlations are equal to (si)~ and (sisj)~, respectively: mi--

(43)

(Si)c ,

~,ij -- (SiSj)c --

mimj, i r j.

(44)

Eqs. (43) and (44) are n + 89n(n - 1) equations with an equal number of unknowns wij and 0i and can be solved using standard numerical routines. We can however, make a significant improvement in the learning procedure when we observe that the TAP term in Eq. (31) represents a self-coupling to neuron i. Instead of using the TAP approximation to relate this self-coupling to the off-diagonal weights wij, we propose to introduce additional parameters, diagnonal weights wii, which we estimate in the learning process. We therefore need n additional equations for learning, for which we propose Zii = 1 - m2i . This equation is true by definition for the exact Z, but becomes an additional constraint on wij and 0i when Z is the linear response approximation Eq. (42). Thus our basic equations become 8

The following discussion can be extended to hidden units using an EM-type of iteration procedure.


mi ~,~1

547

0,) ~Oj__

~ij

(46)

= ~mi -- 1 -- m 2 -- wij.

Note, that the sum over j in the equation for mi now also includes a contribution wiimi. From Eqs. (43)-(46) we can compute the solution for wij and 0i in closed form: mi = (Si)c,

(47)

c~j = (sisj) c - ( S ~ ) c ( S j ) c ,

(48)

~)ij

wij = 1 - m

(C_ 1

2

(49)

)ij,

Oi -- tanh -1 (mi) -- f i wijmj. j--1 7.3. C l a s s i f i c a t i o n

(50)

of digits

We demonstrate the quality of the above MF approximation for Boltzmann Machine learning on a digit recognition problem. The data consists of 11,000 examples of handwritten digits (0-9) compiled by the US Postal Service Office of Advanced Technology. The examples are preprocessed to produce 8 x 8 binary images. Some examples are shown in Fig. 8. Our approach is to model each of the digits with a separate Boltzmann Machine. For each digit, we use 700 patterns for training using the approach outlined above. We thus obtain 10 Boltzmann distributions log p(s1141 ~) - - E ( s l W ~) - log z(w~),

a = 0 , . . . , 9,

where W~ - ( w ~ , 0~) are the weights and thresholds for digit a. We then test the performance of these models on a classification task using the same 700 training patterns per digit as well as the 400 test patterns per digit. We classify each pattern to the model ~ with the highest probability. The normalization log Z(W ~) is intractable and depends on a and therefore affects classification. We use its MF approximation given by [41,42]

1

log Z - --~ E wijmimj - Z Oimi u i

1Z((1

2 i

+

mi)log(1

+ mi) + (1

-

mi)log(1 - mi))

The correlation matrix cij in Eq. (48) is (close to) singular. This results in very large weights in Eq. (49) and we should question the validity of the MF approximation. We propose to solve this problem by adding a flat distribution to the training data:

H.J. Kappen

548

ra

, , '.,g

II-.

/ .o. 72 w m.ii

Fig. 8. Sample of patterns of the 8 • 8 handwritten digits of the US Postal Service Office of Advanced Technology. In each row from left to right: the mean digit per class, a nice example and two rather bad examples.

1

q~ -+ (1 - k)q~ + ~ 2-;'

(51)

(Si)c -+ (1 - k)(Si)c,

(52)

(sisj) c ~

(53)

(1

-

~)(SiSj) c + ~'~ij"

In Fig. 9 we show the result o f the B o l t z m a n n M a c h i n e classifier as a function of ~. W e see that the classification error d e p e n d s strongly on the value o f ~. H o w e v e r , there is no overfitting effect in the sense that a value that is optimal on the training set is also optimal on the test set. The optimal ~ on the training set is ~ = 0.24. The classification error on the test set for this value of ~ is 4.62%. In [43,44] this classification p r o b l e m is used on the same data to c o m p a r e a n u m b e r o f algorithms. The

549


0.12 - - trainl

- test I

0.1 ,_

0

W

/

0.08 \

0.06

/ "\

./"

0.04 0.02

0

0.5

1

Z,

Fig. 9.

Classification error of the Boltzmann Machine on the handwritten digits as a function of )v.

Table 1 Classification error rates for the test data set of handwritten digits. The first tree were reported by [43], the fourth was reported in [44] Nearest neighbor Back-propagation Wake-sleep Sigmoid belief Boltzmann machine

6.7% 5.6 % 4.8% 4.6% 4.6%

r e p o r t e d e r r o r rates on the test set are s u m m a r i z e d in Table 1. The result o b t a i n e d with the b a c k p r o p a g a t i o n m e t h o d is rather competitive: I tried to r e p r o d u c e it a n d it requires extensive training times a n d the result is n o t so g o o d in all runs. The three best m e t h o d s in Table 1 are all unsupervised methods. T h e y do density estimation on each o f the classes separately and are not optimized for classification. Therefore, it is e n c o u r a g i n g that these m e t h o d s are capable o f o u t p e r f o r m i n g the multi-layered perceptron. The B o l t z m a n n M a c h i n e yields as g o o d p e r f o r m a n c e as the best unsupervised m e t h o d k n o w n on this data. The m a i n a d v a n t a g e o f the Boltzrnann M a c h i n e is that no hidden structure is needed in contrast to all the other m e t h o d s in Table 1 except for the nearest n e i g h b o r m e t h o d . As a result, the B o l t z m a n n M a c h i n e solution is trained and tested in several minutes, whereas the other m e t h o d s require several hours. 9

A comparison on a larger OCR problem was done in [45] which yields the same conclusion regarding the unsupervised methods. In this case, however, significant improvements have been reported using supervised methods (see h t t p " //www. research, art. com/yann/ocr/mnist / index, html).

550

H.J. Kappen

Abbreviations BM, Boltzmann Machine Eq., equation Fig., figure MC, Monte Carlo MF, mean field PSP, Post Synaptic Potential RMS, root mean square SK-model, Sherrington-Kirkpatrick model TAP-equation, Thouless-Anderson-Palmer equation

Acknowledgements I would like to thank Wim Wiegerinck and Tom Heskes for useful discussions. This research was funded in part by the Dutch Technology Foundation (STW).

Appendix A. T A P equations

In this appendix we present the main steps to derive the TAP equations Eq. (31). We start with the computation of the derivatives in Eq. (29):

e(si) [ eOj q -where mi.q Similarly,

-

-

8(si) [' _

Z

~p(s) ~Oj [q tanh(O q) + q(s)(1 - m2i.q)~ij-- ( 1 - m2i,q)~ij

s

tanh(O q) is the mean firing rate of neuron i in the factorized model q.

(1 -

OWjk q

Using

m2q)~)ijmk.q.

mi = mi.p mi.q because of Eq. (28) we obtain to lowest order =

(A.1) This is equivalent to mi tanh ( ~-~'~jwijmj + ~). In a similar way one computes the second-order derivatives and the result is -

-

~2 E (Si) ] dOj dO, - - 2 mi(1 - m2)(dO)~, jk ~Oj~Ok q

~2Swk,(si>,1,dOj dwk, -- (1 - m2i)Z ((1 - m2) dOg.- 2mimj dO/) dwij, Z 8Oj jkl

q

j

551

An introduction to stochastic neural networks ~2

2 (Si) ] dwjkdw,m - ( 1 m 2) Z ( ( 1 m2)mjdwkjdwik jklm ~Wjk~Wlm q jk + (1 -- mj2)mk dwjk dwij - 2 mi (sjsk) dwij dwik).

Substituting this into Eq. (29) we obtain 0 -- d m i -

(1 - m 2)

i - miA 2 + Z ( 1 j

- m})wijAj - mi Z

w2"(1 - mj)

,

J

w h e r e we h a v e d e f i n e d Ai - d O / § ~-~j dwijmj. Since Ai - 0 § O(w2), according to

Eq. (A. 1), we obtain Ai - mi Z

w2(1 -

2 § mj)

O(W 3)

J

which is equivalent to Eq. (31). References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27.

McCulloch, W.S. and Pitts, W. (1943) Bull. Math. Biophys. 5, 114-133. Rosenblatt, F. (1958) Psychol. Rev. 65, 386-408. Amari, S. (1967) IEEE Trans. on Electron. Comput. 16, 299-307. Rumelhart, D., Hinton, G. and Williams, R. (1986) Nature 323, 533-536. Hopfield, J. (1982) Proc. Nat. Acad. Sci. USA. 79, 2554-2558. Kohonen, T. (1982) Biol. Cybern. 43, 59-69. Sompolinsky, H. and Kanter, I. (1986) Phys. Rev. Lett. 57, 2861-2864. Ackley, D. Hinton, G. and Sejnowski (1985) T. Cog. Sci. 9, 147-169. Amit, D.J., Gutfreund, H. and Sompolinsky, H. (1985) Phys. Rev. A 32, 1007. Amit, D.J., Gutfreund, H. and Sompolinsky, H. (1985) Phys. Rev. Lett. 55, 1530-1533. Mason, A., Nicoll, A, and Stratford, K. (1990) J. Neurosci. 11, 72-84. Kats, B. (1966) Nerve, muscle and Synapse. McGraw-Hill, New York. Koch, Chr. (1999) Biophysics of Computation. Oxford University Press, Oxford. Abbott, L.F., Varela, J.A., Sen, K. and Nelson, S.B. (1997) Science 220-224. Markram, H. and Tsodyks, M. (1996) Nature 807-810. Rail, W. and Rinzel, J. (1973) Biophys. J. 648-688. Little, W.A. (1974) Math. Biosci. 19, 101-120. Peretto, P. (1992) An Introduction to the Modeling of Neural Networks. Cambridge University Press, Cambridge. Kappen, H.J. (1997) Phys. Rev. E, 55, 5849-5858; SNN-96-044, F-96-094. Grimmett, G.R. and Stirzaker, D.R. (1992) Probability and Random Processes. Clarendon Press, Oxford. Saul, L. and Jordan, M.I. (1994) Neural Comput. 6, 1174-1184. Onsager, L. (1936) J. Amer. Chem. Soc. 58, 1486-1493. Sherrington, D. and Kirkpatrick, S. (1975) Phys. Rev. Lett. 35, 1792-1796. Toulouse, G. (1977) Comm. Phys. 2, 115-119. Dfiring, A., Coolen, A.C.C. and Sherrington, D. (1998) J. Phys. A: Math. General 31, 8607-8621. Amit, D. (1989) Modeling Brain Function. Cambridge University Press, Cambridge. Gutfreund, J.D., Reger, H. and Yound, A.P. (1988) J. Phys. A: Math. General 21, 2775-2797.

552

H.J. Kappen

28. 29. 30. 31.

Crisanti, A. and Sompolinsky, H. (1988) Phys. Rev. A 37, 4865-4874. Nfitzel, K. and Krey, U. (1993) J. Phys. A: Math. General 26, L591-L597. Eissfeller, H. and Opper, M. (1994) Phys. Rev. E 50, 709-720. Goles, E. and Vichniac, G.Y. (1986) in: Proceddings AlP conference, ed J.S. Denker. pp. 165-181, American Institute of Physics. Crisanti, A., Falcioni, M. and Vulpiani, A. (1993) J. Phys. A: Math. General 26, 3441-3453. Plefka, T. (1982) J. Phys. A 15, 1971-1978. Amari, S.-I. (1992) IEEE Trans. Neural Networks 3, 260-271. Tanaka, T. (1999) in: Advances in Neural Information Processing Systems 11, eds M.S. Kearns, S.A. Solla and D.A. pp. 351-357, Cohn MIT Press, Cambridge. Thouless, D.J., Anderson, P.W. and Palmer, R.G. (1977) Philos. Mag. 35, 593-601. Kelso, S.R., Ganong, A.H. and Brouwn, T.H. (1986) Proc. Natl. Acad. Sci. 83, 5326-5330. Kullback, S. (1959) Information Theory and Statistics. Wiley, New York. Hertz, J., Krogh, A. and Palmer, R. (1991) Introduction to the Theory of Neural Computation, Santa Fe Institute, Vol. 1. Addison-Wesley, Redwood City. Parisi, G. (1988) Statistical Field Theory. Frontiers in Physics. Addison-Wesley, Reading, MA. Kappen H.J. and Rodrguez F.B. (1999) in: Advances in Neural Information Processing Systems 11, eds M.S. Kearns, S.A. Solla and D.A. Cohn. pp. 280-286, MIT Press. Kappen, H.J. and Rodrguez F.B. (1998) Neural Comput. 10, 1137-1156. Hinton, G.E., Dayan, P., Frey, B.J. and Neal, R.M. (1995) Science 268, 1158-1161. Saul, L.K., Jaakkola, T. and Jordan, M.I.J. (1996) Artificial Intell. Res. 4, 61-76. Leisink, M. and Kappen, H.J. (2000) in: Proceedings IJCNN Submitted.

32. 33. 34. 35. 36. 37. 38. 39. 40. 41. 42. 43. 44. 45.

CHAPTER 14

Statistical Mechanics of Recurrent Neural Networks I - Statics

A.C.C. COOLEN Department of Mathematics, King's College London, Strand, London WC2R 2LS, UK



553

Contents 1.

Introduction

2.

Definitions and properties of microscopic laws

.................................................

555

............................

2.1. Stochastic dynamics o f neuronal firing states

..........................

558

2.2. Synaptic symmetry and L y a p u n o v functions

..........................

562

2.3. Detailed balance and equilibrium statistical mechanics 3.

4.

....................

6.

7.

565

Simple recurrent networks with binary neurons . . . . . . . . . . . . . . . . . . . . . . . . . . . .

569

3.1. Networks with u n i f o r m synapses

569

.................................

3.2. P h e n o m e n o l o g y of Hopfield models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

572

3.3. Analysis of Hopfield models away from saturation

577

Simple recurrent networks o f coupled oscillators

......................

...........................

4.1. Coupled oscillators with u n i f o r m synapses . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.

557

583 583

4.2. Coupled oscillator attractor networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

585

Networks with Gaussian distributed synapses . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

5.1.

Replica analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

592

5.2.

Replica-symmetric solution and AT-instability

.........................

The Hopfield model near saturation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

595 600

6.1.

Replica analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

600

6.2.

Replica symmetric solution and AT-instability

606

Epilogue

.........................

...................................................

Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References

.....................................................

554

615 617 617

I. Introduction

Statistical mechanics deals with large systems of stochastically interacting microscopic elements (particles, atomic magnets, polymers, etc.). The strategy of statistical mechanics is to abandon any ambition to solve models of such systems at the microscopic level of individual elements, but to use the microscopic laws to calculate equations describing the behavior of a suitably chosen set of macroscopic observables. The toolbox of statistical mechanics consists of methods to perform this reduction from the microscopic to a macroscopic level, which are all based on efficient ways to do the bookkeeping of probabilities. The experience and intuition that have been built up over the last century tells us what to expect, and serves as a guide in finding the macroscopic observables and in seeing the difference between relevant mathematical subtleties and irrelevant ones. As in any statistical theory, clean and transparent mathematical laws can be expected to emerge only for large (preferably infinitely large) systems. In this limit one often encounters phase transitions, i.e. drastic changes in the system's macroscopic behavior at specific values of global control parameters. Recurrent neural networks, i.e. neural networks with synaptic feedback loops, appear to meet the criteria for statistical mechanics to apply, provided we indeed restrict ourselves to large systems. Here the microscopic stochastic dynamical variables are the firing states of the neurons or their membrane potentials, and one is mostly interested in quantities such as average state correlations and global information processing quality, which are indeed measured by macroscopic observables. In contrast to layered networks, one cannot simply write down the values of successive neuron states for models of recurrent neural networks; here they must be solved from (mostly stochastic) coupled dynamic equations. Under special conditions ('detailed balance'), which usually translate into the requirement of synaptic symmetry, the stochastic process of evolving neuron states leads towards an equilibrium situation where the microscopic state probabilities are known, and where the techniques of equilibrium statistical mechanics can be applied in one form or another. The equilibrium distribution found, however, will not always be of the conventional Boltzmann form. For nonsymmetric networks, where the asymptotic (stationary) statistics are not known, dynamical techniques from nonequilibrium statistical mechanics are the only tools available for analysis. The 'natural' set of macroscopic quantities (or 'order parameters') to be calculated can be defined in practice as the smallest set which will obey closed deterministic equations in the limit of an infinitely large network. Being high-dimensional nonlinear systems with extensive feedback, the dynamics of recurrent neural networks are generally dominated by a wealth of attractors (fixed-point attractors, limit-cycles, or even more exotic types), and the 555

556

A. C. C. Coolen

/

Fig. 1. Information processing by recurrent neural networks through the creation and manipulation of attractors in state space. Patterns stored: the microscopic states e. If the synapses are symmetric we will generally find that the attractors will have to be fixed-points (left picture). With non-symmetric synapses, the attractors can also be sequences of microscopic states (right picture).

practical use of recurrent neural networks (in both biology and engineering) lies in the potential for creation and manipulation of these attractors through adaptation of the network parameters (synapses and thresholds). Input fed into a recurrent neural network usually serves to induce a specific initial configuration (or firing pattern) of the neurons, which serves as a cue, and the 'output' is given by the (static or dynamic) attractor which has been triggered by this cue. The most familiar types of recurrent neural network models, where the idea of creating and manipulating attractors has been worked out and applied explicitly, are the socalled attractor neural networks for associative memory, designed to store and retrieve information in the form of neuronal firing patterns and/or sequences of neuronal firing patterns. Each pattern to be stored is represented as a microscopic state vector. One then constructs synapses and thresholds such that the dominant attractors of the network are precisely the pattern vectors (in the case of static recall), or where, alternatively, they are trajectories in which the patterns are successively generated microscopic system states. From an initial configuration (the 'cue', or input pattern to be recognized) the system is allowed to evolve in time autonomously, and the final state (or trajectory) reached can be interpreted as the pattern (or pattern sequence) recognized by network from the input (see Fig. 1). For such programs to work one clearly needs recurrent neural networks with extensive 'ergodicity breaking': the state vector will during the course of the dynamics (at least on finite time-scales) have to be confined to a restricted region of state space (an 'ergodic component'), the location of which is to depend strongly on the initial conditions. Hence our interest will mainly be in systems with many attractors. This, in turn, has implications at a theoretical/mathematical level: solving models of recurrent neural networks with extensively many attractors requires advanced tools from disordered systems theory, such as replica theory (statics) and generating functional analysis (dynamics). It will turn out that a crucial issue is whether or not the synapses are symmetric. Firstly, synaptic asymmetry is found to

Statistical mechanics of recurrent neural networks I - statics

557

rule out microscopic equilibrium, which has implications for the mathematical techniques which are available: studying models of recurrent networks with nonsymmetric synapses requires solving the dynamics, even if one is only interested in the stationary state. Secondly, the degree of synaptic asymmetry turns out to be a deciding factor in determining to what extent the dynamics will be glassy, i.e. extremely slow and nontrivial, close to saturation (where one has an extensive number of attractors). In this paper (on statics) and its sequel (on dynamics) I will discuss only the statistical mechanical analysis of neuronal firing processes in recurrent networks with static synapses, i.e. network operation as opposed to network learning. I will also restrict myself to networks with either full or randomly diluted connectivity, the area in which the main progress has been made during the last few decades. Apart from these restrictions, the text aims to be reasonably comprehensive and selfcontained. Even within the confined area of the operation of recurrent neural networks a truly impressive amount has been achieved, and many of the constraints on mathematical models which were once thought to be essential for retaining solvability but which were regrettable from a biological point of view (such as synaptic symmetry, binary neuron states, instantaneous neuronal communication, a small number of attractors, etc.) have by now been removed with success. At the beginning of the new millennium we know much more about the dynamics and statics of recurrent neural networks than ever before. I aim to cover in a more or less unified manner the most important models and techniques which have been launched over the years, ranging from simple symmetric and non-symmetric networks with only a finite number of attractors, to the more complicated ones with an extensive number, and I will explain in detail the techniques which have been designed and used to solve them. In the present paper I will first discuss and solve various members of the simplest class of models: those where all synapses are the same. Then I turn to the Hopfield model, which is the archetypical model to describe the functioning of symmetric neural networks as associative memories (away from saturation, where the number of attractors is finite), and to a coupled oscillator model storing phase patterns (again away from saturation). Next I will discuss a model with Gaussian synapses, where the number of attractors diverges, in order to introduce the so-called replica method, followed by a section on the solution of the Hopfield model near saturation. I close this paper with a guide to further references and an assessment of the past and future deliverables of the equilibrium statistical mechanical analysis of recurrent neural networks.

2. Definitions and properties of microscopic laws In this section I define the most common microscopic models for recurrent neural networks, I show how one can derive the corresponding descriptions of the stochastic evolution in terms of evolving state probabilities, and I discuss some fundamental statistical mechanical properties.

558

A. C. C. Coolen

2.1. Stochastic dynamics of neuronal firing states 2.1.1. Microscopic definitions for binary neurons The simplest nontrivial definition of a recurrent neural network is that where N binary neurons cric { - 1 , 1} (in which the states '1' and ' - 1 ' represent firing and rest, respectively) respond iteratively and synchronously to post-synaptic potentials (or local fields) hi(or), with a = (CYl,...,CYu). The fields are assumed to depend linearly on the instantaneous neuron states:

Parallel: cyi(g + 1) - sgn[hi(~(g) ) + rn~(e)],

hi(a) -- ~

Jij(Yj + Oi.

(1)

J The stochasticity is in the independent random numbers rli(g)E ~ (representing threshold noise), which are all drawn according to some distribution w(q). The parameter T is introduced to control the amount of noise. For T = 0 the process (1) is deterministic: 13"i(g -3I- 1) = sgn[hi(~(g))]. The opposite extreme is choosing T = oc, here the system evolution is fully random. The external fields 0i represent neural thresholds and/or external stimuli, Jij represents the synaptic efficacy at the junction j --~ i (Jij > 0 implies excitation, Jij < 0 inhibition). Alternatively we could decide that at each iteration step g only a single randomly drawn neuron (Yit is t o undergo an update of the type (1):

Sequential: i r ie- (3"i(g+ 1) - (3"i(g)

(2)

i = i~: ~;(g + 1) = sgn[hi(n(g)) + Tq~(g)] with the local fields as in (1). The stochasticity is now both in the independent random numbers rl;(g) (the threshold noise) and in the site it to be updated, drawn randomly from the set {1,... ,N}. For simplicity we assume w ( - q ) = w(rl), and define

g[z] -

2

f0-drlw(rl)

9 g[-z] - -g[z],

:~lim g[z] - +1,

d

-~zzg[Z] ) 0

Popular choices for the threshold noise distributions are l

lrl2

w ( r l ) - (2~)-~e -~

9 g[z]- Erf[z/x/~],

w(rl) - 89[1 - tanh2(q)] 9 g[z] - tanh(z).

2.1.2. From stochastic equations to evolving probabifities From the microscopic Eqs. (1) and (2), which are suitable we can derive an equivalent but mathematically more terms of microscopic state probabilities pe(~). Eqs. (1) system state ~(g) is given, a neuron i to be updated will Prob[cri(g + 1)] = 89 + cy~(g + 1)g[f3hi(c~(g))]]

for numerical simulations, convenient description in and (2) state that, if the obey

(3)

559


with 13 = T -1 . In the case (1) this rule applies to all neurons, and thus we simply get Pe+l (6) - 1--IN1 89 [1 + c~i g[~hi(6(g))]]. If, on the other hand, instead of 6(g) only the probability distribution pe(6) is given, this expression for pe+l (6) is to be averaged over the possible states at time g:

Parallel: N -

-

6/

17[

1[1 + cyzg[13hi(C)]]

(4)

i-- 1

This is the standard representation of a Markov chain. Also the sequential process (2) can be formulated in terms of probabilities, but here expression (3) applies only to the randomly drawn candidate it. After averaging over all possible realizations of the sites ie we obtain:

l

(with the Kronecker symbol: 6ij = 1 if i = j, 8ij = 0 otherwise). If, instead of 6(g), the probabilities pg(6) are given, this expression is to be averaged over the possible states at time g, with the result: Pe+I (6) -- ~1 E . ~1 [1 +

(Yi g[~hi(a)]]pe(6)

l

1 [1 + + ~1 E. -2

(Yi g[~hi(Fi6)]]pe(Fi6)

1

with the state-flip operators F~r = (I)(Cyl,...,cYi-1,--cY;,eYi+I,...,CYN). This equation can again be written in the standard form pe+l (6) = ~'~6, W[6; C]pe(6'), but now with the transition matrix

Sequential: 1 m [ 6 ; 6 t] -- ~ 6 , 6 ' - ~ - ~ Z { w i ( F / 6 ) ~ 6 , F / 6 i

where 66,6, = I-[i 6(Yi,(YI

, --wi(6)~6,6,},

(5)

and

wi(6) - 89I1 - cri tanh[[3hi(6)]].

(6)

Note that, as soon as T > 0, the two transition matrices W[6; 6'] in (4) and (5) both describe ergodic systems: from any initial state C one can reach any final state 6 with nonzero probability in a finite number of steps (being one in the parallel case, and N in the sequential case). It now follows from the standard theory of stochastic processes (see e.g. [1,2]) that in both cases the system evolves towards a unique stationary distribution p ~ ( 6 ) , where all probabilities p ~ ( 6 ) are nonzero.

2.1.3. From discrete to continuous times The above processes have the (mathematically and biologically) less appealing property that time is measured in discrete units. For the sequential case we will now

A. C. C. Coolen

560

assume that the duration of each of the iteration steps is a continuous random number (for parallel dynamics this would make little sense, since all updates would still be made in full synchrony). The statistics of the durations is described by a function 7t~(t), defined as the probability that at time t precisely g updates have been made. Upon denoting the previous discrete-time probabilities as/~e(~), our new process (which now includes the randomness in step duration) will be described by

p t ( u ) - Z rt~(t)/~(u)- Z r o t ( t ) Z We[u; C]po(C) g>~0

g>~0

o'

and time has become a continuous variable. For roe(t) we make the Poisson choice rtt(t) - ~. (~)%-t/A. From (g)~ - t/A and (g2)~ _ t/A + t2/A 2 it follows that A is the average duration of an iteration step, and that the relative deviation in g at a given t vanishes for A---, 0 as V/(g2)~- (g)2/(g)~ = x/~/t. The nice properties of the Poisson distribution under temporal derivation allow us to derive: d A ~p,(6) - Z

W[6; 6']pt(6') - pt(6).

61

For sequential dynamics we choose A - ~ so that, as in the parallel case, in one time unit each neuron will on average be updated once. The master equation corresponding to (5) acquires the form d ~ p t ( 6 ) - Z{wi(F,.6)pt(F,.6) - wi(a)pt(6)}.

(7)

i

The wi(6) (6) now play the role of transition rates. The choice A - ~

implies

I

V/(g2)~ _ (g)2/(g)~ _ x/~/Nt ' so we will still for N ~ cr no longer have uncertainty ,v in where we are on the t axis.

2.1.4. Microscopic definitions for continuous neurons Alternatively, we could start with continuous neuronal variables (3"i (representing e.g. firing frequencies or oscillator phases), where i = 1,... ,N, and with stochastic equations of the form

cyi(t + A) - r

+ A~(6(t)) + x/2 TA~(t).

(8)

Here we have introduced (as yet unspecified) deterministic state-dependent forces f/.(6), and uncorrelated Gaussian distributed random forces ~/(t) (the noise), with (~i(t)) = 0 and (~i(t)~j(t')) = 8ijgt,t,. As before, the parameter T controls the amount of noise in the system, ranging from T = 0 (deterministic dynamics) to T - - e ~ (completely random dynamics). If we take the limit A --+ 0 in (8) we find a Langevin equation (with a continuous time variable): d dt cr/(t) - f-(6(t)) + rl;(t).

(9)

This equation acquires its meaning only as the limit A ~ 0 of (8). The moments of the new noise variables r l i ( t ) = ~i(t)v/2T/A in (9) are given by ( r l i ( t ) ) - 0 and

561

Stat&tical mechanics of recurrent neural networks I - statics

(qi(t)qj(t')) = 2TSijS(t- t'). This can be derived from the moments of the

~i(t). For

instance:

{qi(t)qT(t'))

2•

= limo~(~i(t)~j(t'))

-

1

2 T8i7 lirnox St,t, - 2 TCSijS(t- t').

The constant C is found by summing over t', before taking the limit A ~ 0, in the above equation: oo

0(3

dt'(qi(t)qj(t')) - lim 2 T E A--+0

(~i(t)~J (t')) - 2 T8i7 lim~ E

tt=--cx~

8t,r - 2 TSij.

t' =-oo

Thus C - 1, which indeed implies (qi(t)qj(t'))- 2TSijS(t- t'). More directly, one can also calculate the moment generating function

l e x p ( i / d t E ~ i ( it ) r l ( t ) ) )i- l i m o ~ i t

.

/ vdz / ~ e x p ( - ~ lz2

-+-iz~i(t)~

= lim 171 e-rA*~(t) = e-r f dt ~-]~*~(t).

) (1 O)

A~O t,t .

2.1.5. From stochastic equations to evolving probabilities A mathematically more convenient description of the process (9) is provided by the Fokker-Planck equation for the microscopic state probability density pt(ff) = (5[ff -- ~(t)]), which we will now derive. For the discrete-time process (8) we expand the 8-distribution in the definition ofpt+A(a) (in a distributional sense):

Pt+a(")-Pt(~)- /8 [r ~- -- E

or(t)- Af (r ~

0

~~(t)]

/ 8[" -- l~(t)] [ A f i ( . ( t ) )

)-

(8[r

or(t)])

+ 42 TA~i(t)] )

i ~2

+ TA E.. lj

O(yiO(yj (8[t~ -

t~(t)]~i(t)~j(t)) + (_9(A))

The variables a(t) depend only on noise variables ~y(t') with t' < t, so that for any function A: (A[a(t)]~i(t)) - (A[c~(t)])(~i(t)) - O, and (A[a(t)]~i(t)~y(t)) = 8iy(A[a(t)]). As a consequence: 1

0

[Pt+A(a) -- pt(a)] = -- E ~

(8[a -- a(t)JJ~(a(t)))

i 02 l

0 =

02

+rZ

-

i

i

+ o(a

)

562

A . C . C . Coolen

By taking the limit A ~ 0 we then arrive at the Fokker-Planck equation: a dpt(l~)-

_ Z

~

~)2 [pt(~)fi(~)]

i

_ql_TZ_~2iPt(l~)" i

(11)

2.1.6. Examples: graded response neurons and coupled oscillators In the case of graded response neurons the continuous variable O"i represents the membrane potential of neuron i, and (in their simplest form) the deterministic forces are given by f ( ~ ) - y'~4Jqtanh[7~j]- ~; + 0i, with ? > 0 and with the 0i representing injected currents. Conventional notation is restored by putting (Yi ~ Ui. Thus equation (9) specializes to d

d---~tui(t) -- Z Jij tanh[Tuj(t)] - ui(t) + Oi -q- rli(t).

(12)

J One often chooses T = 0 (i.e. rl;(t) = 0), the rationale being that threshold noise is already assumed to have been incorporated via the nonlinearity in (12). In our second example the variables 13"i represent the phases of coupled neural oscillators, with forces of the form f . ( a ) - ~ j J i j s i n ( ~ j - ~i)+coi. Individual synapses Ji~ now try to enforce either pair-wise synchronization (Jij > O) or pair-wise antisynchronization (Jij < 0), and the f.l)i represent the natural frequencies of the individual oscillators. Conventional notation dictates (3"i --+ ~)i, giving d dt qbi(t) - o3i + Z Jij sin[qbj(t) - ~i(t)]-~- rli(t ). J

(13)

2.2. Synaptic symmetry and Lyapunov functions 2.2.1. No&e-free symmetric networks of b&ary neurons In the deterministic limit T ~ 0 the rules (1) for networks of synchronously evolving binary neurons reduce to the deterministic map (14)

cyi(g + 1) - sgn[hi(t~(g))].

It turns out that for systems with symmetric interactions, Ji2 - Jji for all (ij), one can construct a Lyapunov function, i.e. a function of ty which during the dynamics decreases monotonically and is bounded from below (see e.g. [3]):

Binary & Parallel: Z[6] -- - ~

Ihi(6)[- ~ i

(YiOi 9

i

Clearly L>~ - ~-~i[~-~j [Jij] + 10i[] - ~ i p0/[. During iteration of (14) we find:

(15)


563

Oi[(Yi(e -[- 1) - cr/(g)]

-- Z i = -- Z

(Yi(g)hi(~(g AV l))

[hi(~(e Av l))[ Av Z i

i

-- - Z

Ihi(~(g + 1))1[1 - r

+ 2)cyi(g)] ~ 0); now for large N the number 1) ~ 20~. This is called the saturation of bits stored per synapse will be p N / 8 9 regime. Again numerical simulations, but now with finite 0~, illustrate the main features and complications of recall dynamics in the saturation regime. In our ex-

Fig. 4. Information processing in a sequential dynamics Hopfield model with N = 841, p = 10 and T = 0.1, and with the p = 10 stored patterns shown in Fig. 3. Left pictures: dynamic reconstruction of a stored pattern from an initial state which is a corrupted version thereof. Top left: snapshots of the system state at times t = 0 , 1 , 2 , 3 , 4 iterations/neuron. Bottom left: values of the overlap order parameters as functions of time. Right pictures: evolution towards a spurious state from a randomly drawn initial state. Top right: snapshots of the microscopic system state at times t = 0 , 1 , 2 , 3 , 4 iterations/neuron. Bottom right: values of the overlap order parameters as functions of time.


575

10 0.8

0.6

m 5

0.4

0.2

0.0

0

5

10

15

20

25

0 0.0

0.2

t

0.4

0.6

0.8

L.0

m

Fig. 5. Simulations of a parallel dynamics Hopfield model with N - 3 0 , 0 0 0 and = T = 0.1, and with random patterns. Left: overlaps m = ml(~r) with pattern one as functions of time, following initial states correlated with pattern one only, with m1(~(0))c{0.1,...,0.9}. Right: corresponding flow in the (m,r) plane, with r = ~--1 ~-~g>l mp 2 (~) measuring the overlaps with nonnominated patterns. ample the dynamics is given by (1) (parallel updates), with T - 0.1 and threshold noise distribution w ( q ) - i l l - t a n h Z ( r l ) ] ; the patterns are chosen randomly. Figure 5 shows the result of measuring in such simulations the two quantities m -- ml(t$),

r--

~ - l~mg

2(~)

(42)

g>l

following initial states which are correlated with pattern ~1 only. For large N we can distinguish structural overlaps, where m g ( ~ ) = (9(1), from accidental ones, where m~(n) - (9(N- 89 (as for a randomly drawn ~). Overlaps with nonnominated patterns are seen to remain (9(N- 89 i.e. r(t) - (9(1). We observe competition between pattern recall (m ~ 1) and interference of nonnominated patterns (m ~ 0, with r increasing), and a profound slowing down of the process for nonrecall trajectories. The initial overlap (the 'cue') needed to trigger recall is found to increase with increasing (the loading) and increasing T (the noise). Further numerical experimentation, with random patterns, reveals that at any noise level T there is a critical storage level ac(T) above which recall is impossible, with an absolute upper limit of ~c = maxr ~c(T) = ~c(0) ~ 0.139. The competing forces at work are easily recognized when working out the local fields (1), using (40): l

hi(~) -- ~] ml (($) -J- ~ ~ g>l

~gi Z jr

p

C(N-1

).

(43)

The first term in (43) drives ~ towards pattern {1 as soon as ml (~) > 0. The second terms represent interference, caused by correlations between r and nonnominated patterns. One easily shows (to be demonstrated later) that for N ~ ec the fluctua-

A.C.C. Coolen

576

tions in the values of the recall overlap m will vanish, and that for the present types of initial states and threshold noise the overlap m will obey

m(t + 1) -

/ dzPt(z)tanh[13(m(t) + z)],

I

'im' /I -

9

-

--

--

B>I

j#i

j

j(t)

1)

,44,

If all 6/-(0) are drawn independently, Prob[cyi(0) - +~]] - 1 [1 4-m(0)], the central limit theorem states that Po(z) is Gaussian. One easily derives (z)0 - 0 and (z2)0 - 0t, so at t - 0 Eq. (44) gives re(l) =

~

dz

__k2 7t e - tanh[13(m(0) + zx/~)].

(45)

The above ideas, and Eq. (45) in particular, go back to [11]. For times t > 0, however, the independence of the states (3"i need no longer hold. As a simple approximation one could just assume that the O" i remain uncorrelated at all times, i.e. Prob[cy/(t) - -t-~]]- 89 + m(t)] for all t>~0, such that the argument given for t - 0 would hold generally, and where (for randomly drawn patterns) the mapping (45) would describe the overlap evolution at all times: m(t+l)-

~ dz e - - - ~ 2 tanh[13(m(t) + zv/~)]

(46)

This equation, however, must be generally incorrect. Firstly, Fig. 5 already shows that knowledge of m(t) only does not yet permit prediction of m(t + 1). Secondly, upon working out its bifurcation properties one finds that Eq. (46) predicts a storage capacity of arc - 2/rt ~ 0.637, which is no way near to what is actually being observed. We will see in the paper on dynamics that only for certain types of extremely diluted networks (where most of the synapses are cut) Eq. (46) is indeed correct on finite times; in these networks the time it takes for correlations between neuron states to build up diverges with N, so that correlations are simply not yet noticeable on finite times. For fully connected Hopfield networks storing random patterns near saturation, i.e. with 0t > 0, the complicated correlations building up between the microscopic variables in the course of the dynamics generate an interference noise distribution which is intrinsically non-Gaussian, see e.g. Fig. 6. This leads to a highly nontrivial dynamics which is fundamentally different from that in the l i m u ~ PIN = 0 regime. Solving models of recurrent neural networks in the saturation regime boils down to calculating this non-Gaussian noise distribution, which requires advanced mathematical techniques (in statics and dynamics), and constitutes the main challenge to the theorist. The simplest way to evade this challenge is to study situations where the interference noise is either trivial (as with asymmetric extremely diluted models) or where it vanishes, which happens in fully connected networks when 0 t - limu+~ p/ N = 0 (as with finite p). The latter 0 t - 0 regime is the one we will explore first.


577

1.5

m(O)=0.9

1.0

P(z) 0.5

m(O)=0.1

0.0

-3

-1

1

3

; ~,~cyj, as meaFig. 6. Distributions of interference noise variables Z i T- ~~}-~" g>l ~i ~ .JT: . sured in the simulations of Fig. 5, at t = 10. Uni-modal histogram: noise distribution following m(0)= 0.9 (leading to recall). Bi-model histogram: noise distribution following m(0) = 0.1 (not leading to recall).

3.3. Analysis of Hopfield models away from saturation 3.3.1. Equilibrium order parameter equations A binary Hopfield network with parameters given by (40) obeys detailed balance, and the Hamiltonian H(~) (30) (corresponding to sequential dynamics) and the pseudo-Hamiltonian H(~) (32) (corresponding to parallel dynamics) become

.(o)-

2(~) + ~p,

-

H(~) = -

g=l

~

E

log 2 cosh ]3Z ~m~(~) "

la=l

]

(47)

with the overlaps (41). Solving the statics implies calculating the free energies F and

P:

1 F = - ~ log ~

e-

13B(.)

,

/~

- -

1 -~

log Z e-~q(")"

Upon introducing the shorthand notation m - (ml,... ,mp) and { i - (~],.-., ~/P), both free energies can be expressed in terms of the density of states ~ ( m ) - 2 -N ~ , , 8Ira - m(t~)]"

1 log 2 - ~ 1 log f dm ~(m)e -89 F / N -- - -~ F / N = - ~ l o g 2 - ~ i flog

p -~ 2-N

dm ~(m)e~,=~ Nl o g 2 c o s h [ ~ { i . m ]

(48)

(49)

(note: f d m S [ m - r e ( a ) ] - 1). In order to proceed we need to specify how the number of patterns p scales with the system size N. In this section we will follow [12]

A.C.C. Coolen

578

(equilibrium analysis following sequential dynamics) and [13] (equilibrium analysis following parallel dynamics), a n d assume p to be finite. O n e can n o w easily calculate the leading c o n t r i b u t i o n to the density of states, using the integral r e p r e s e n t a t i o n o f the 8-function a n d keeping in m i n d that according to (48) a n d (49) only terms exponential in N will retain statistical relevance for N ~ c~: lim --l log ~ ( m ) -- lim --'logfdxeiNx'm<e--i~-~iNl'?'i'x> N N---~ N

N-+~

= lim --1 log f

N-~ N

d x e N[ixm+(l~176

J

with the a b b r e v i a t i o n (r -limu~ ~ ~-~U_, r The leading c o n t r i b u t i o n to b o t h free energies can be expressed as a finite-dimensional integral, for large N d o m i n a t e d by that saddle-point (extremum) for which the extensive e x p o n e n t is real and maximal" lim

F/N-

-~

FIN-

- ~ log f [3N

N---,~c

lim i~,- ~

[~N

1

log f dm dx dm dx

e -NI3f(m'x) - -

extrx,m f ( m , x)

e -Nf~f(m'x) --

extrx,m f ( m , x)

with f ( m , x) - - ~1 !112 - i x . m - 13- 1 (log 2 cos[13~ 9x])g f ( m , x) -

_1~-1

(log 2 cosh[13~ 9m])~, - ix- m - ~ - 1 (log 2 cos[13~ 9x])~.

The saddle-point e q u a t i o n s for f a n d f are given by:

f"

x-im,

im - (~ tan[[3~ 9x])g,

f"

x = i(~tanh[13~, m])~,

im-

(~ tan[13~ 9x])~.

In saddle-points x turns out to be purely imaginary. H o w e v e r , after a shift of the integration contours, putting x = ix* (m) + y (where ix* (m) is the i m a g i n a r y saddlepoint, a n d where y c ~P) we can eliminate x in favor o f y c ~P which does have a real saddle-point, by construction. ~ W e then obtain 2

Our functions to be integrated have no poles, but strictly speaking we still have to verify that the integration segments linking the original integration regime to the shifted one will not contribute to the integrals. This is generally a tedious and distracting task, which is often skipped. For simple models, however (e.g. networks with uniform synapses), the verification can be carried out properly, and all is found to be safe. Here we used the equation 8f(m, x)/gm = 0 to express x in terms of m, because this is simpler. Strictly speaking we should have used 8f(m,x)/gx = 0 for this purpose; our short-cut could in principle generate additional solutions. In the present model, however, we can check explicitly that this is not the case. Also, in view of the imaginary saddle-point x, we cannot be certain that, upon elimination of x, the relevant saddle-point of the remaining function f(m) must be a minimum. This will have to be checked, for instance by inspection of the T --+ c~ limit.

579


Sequential dynamics: m

(~ tanh[]3~ 9m])~,

-

Parallel Dynamics: m

--

(~ tanh[[3~- [(~' tanh[[3~' 9m])~,]])~

(compare to e.g. (38) and (39)). The solutions of the above two equations will in general be identical. To see this, let us denote nil = (~, tanh[J3~ 9m])~, with which the saddle point equation for f decouples into" m

(~tanh[13~. th])~,

-

th - (~tanh[J3~.m])~

so [m

-

fia]2 - ([(~. m) - (~. rh)][tanh(13~, tit) - tanh(fl~ 9m)])~.

Since tanh is a monotonicaly increasing function, we must have [ m - r h ] - ~ - 0 for each ~ that contributes to the averages (...)~. For all choices of patterns, where the covariance matrix C,~ - (~,,~,~)~ is positive definite, we thus obtain m - ill. The final result is" for both types of dynamics (sequential and parallel) the overlap order parameters in equilibrium are given by the solution m* of m

-

(~ tanh[[3~ 9m])~,

(50)

which minimizes 3 f ( m ) - ~ lm2 - ~ 1 (log2cosh[13~.m]) ~,.

(51)

The free energies of the ergodic components are l i m N _ ~ F / N - - f ( m * ) and limN_~ F / N -- 2f(m*). Adding generating terms of the form H ~ H + )~g[m(t~)] to the Hamiltonians allows us to identify (g[m(t~)])eq - lirnz_~0 ~F/ 9 = g[m*]. Thus, in equilibrium the fluctuations in the overlap order parameters m(ty) (41) vanish for N --, oc. Their deterministic values are simply given by m*. Note that in the case of sequential dynamics we could also have used linearization with Gaussian integrals (which we will use for coupled oscillators with uniform synapses) to arrive at this solution, with p auxiliary integrations, but that for parallel dynamics this would not have been possible.

3.3.2. Analysis of order parameter equations." pure states and mixture states We will restrict our further discussion to the case of randomly drawn patterns, so _

2-p

-

0,

-

~z{-1,1)~

We here indeed know the relevant saddle-point to be a minimum: the only solution of the saddle-point equations at high temperatures, m - - O , is seen to minimize f(m), since f ( m ) + ~ - 1 log2 = 89 - ]3) + (9(]33).

A. C. C. Coolen

580

(generalization to correlated patterns is in principle straightforward). We first establish an upper bound for the temperature for where nontrivial solutions m* could exist, by writing (50) in integral form: mr

-

m) fo

dk[1 - tanh2[13)~ 9m]] )

from which we deduce 0=m 2-13((~.m)2f0'dk[1-tanh2[13k~.m]])

~> m 2 - 1 3 ( ( ~ - m ) 2 ) r

= !112(1 - ~). For T > 1 the only solution of (50) is the paramagnetic state m - 0, which gives for the free energy per neuron - T l o g 2 and - 2 T l o g 2 (for sequential and parallel dynamics, respectively). At T - 1 a phase transition occurs, which follows from expanding (50) for small Iml in powers of r - 13- 1" 1

m r - (1 + z)m, - - 3 Z mvmpmx(~r~v~P~x)~ +(-9(m5' zm3) vpX

=m r l+'c-m

2+~m

+ C ( m 5,'cm 3)

The new saddle-point scales as m r - ~ r ~ 1/2 + (9(~3/2), with for each g: r~r - 0 or 0 - 1 - m2 + _}mr .-2 The solutions are of the form r~r E { - ~ , 0, r~}. If we denote with n the number of nonzero components in the vector m, we derive from the above identities: r~r - 0 or r~r - + v / - 3 / v / 3 n - 2. These saddle-points are called mixture states, since they correspond to microscopic configurations correlated equally with a finite number n of the stored patterns (or their negatives). Without loss of generality we can always perform gauge transformations on the set of stored patterns (permutations and reflections), such that the mixture states acquire the form n times

p - n times

m -- m , , (~1 , . . . , ~ 1 , 0 , . . . , 0 ) ,

mn -- [ '3/ ]2 3--n2

( 1 3 - 1 1/2) + . . .

(52)

These states are in fact saddle-points of the surface f(m) (51) for any finite temperature, as can be verified by substituting (52) as an ansatz into (50):

v~n ~>n"

0 - (~g tanh [Igm,~Z ~v] ) v~ 0 , n--2"

1-13>0,

n>~3" 1 - 13[1 - (tanh2 [13mn ~ = 3 ~o] )~ 1 > 0 . The n - 1 states, correlated with one pattern only, are the desired solutions. They are stable for all T < 1, since partial differentiation with respect to 13 of the n - 1 amplitude Eq. (53) gives ml -- tanh[13ml] ~ 1 - 1311 - tanhZ[13m,]]- m,[1 -tanhZ[~ml]](Ornl/~) -1 (clearly sgn[m~] = sgn[i~m,/~13]). The n = 2 mixtures are always unstable. For n >_-3 we have to solve the amplitude Eq. (53) numerically to evaluate their stability. The result is shown in Fig. 7, together with the corresponding 'free energies' fn (54). It turns out that only for odd n will there be a critical temperature below which the nmixture states are local minima of f ( m ) . F r o m Fig. 7 we can also conclude that, in terms of the network functioning as an associative memory, noise is actually beneficial in the sense that it can be used to eliminate the unwanted n > 1 ergodic components (while retaining the relevant ones: the pure n = 1 states). In fact the overlap equations (50) do also allow for stable solutions different from the n-mixture states discussed here. They are in turn found to be continuously bifurcating mixtures of the mixture states. However, for random (or uncorrelated) patterns they come

Stat&ticalmechanicsof recurrentneuralnetworksI- statics

583

into existence only near T = 0 and play a marginal role; phase space is dominated by the odd n-mixture states. We have now solved the model in equilibrium for finite p and N --+ c~. Most of the relevant information on when and to what extent stored random patterns will be recalled is summarized in Fig. 7. For nonrandom patterns one simply has to study the bifurcation properties of Eq. (50) for the new pattern statistics at hand; this is only qualitatively different from the random pattern analysis explained above. The occurrence of multiple saddle-points corresponding to local minima of the free energy signals ergodicity breaking. Although among these only the global minimum will correspond to the thermodynamic equilibrium state, the nonglobal minima correspond to true ergodic components, i.e. on finite time-scales they will be just as relevant as the global minimum. 4. Simple recurrent networks of coupled oscillators

4.1. Coupled oscillators with uniform synapses Models with continuous variables involve integration over states, rather than summation. For a coupled oscillator network (13) with uniform synapses J/j - J / N and zero frequencies o~/= 0 (which is a simple version of the model in [14]) we obtain for the free energy per oscillator: N++ ~-~ log N--~oolimF/N--limlpl,

f:f: .-.

d~

We would now have to 'count' microscopic states with prescribed average cosines and sines. A faster route exploits auxiliary Gaussian integrals, via the identity e~y =

Dze yz

(56)

with the shorthand Dx - (2~) -•2e-~1,.2. ~ ax (this alternative would also have been open to us in the binary case; my aim in this section is to explain both methods): N ~ ~-~-/log N---~oolimF/N--limlvi,

z

f~f~ -"

d~

o, exp

N~ ~

log

N~ ~

log

/

cos,,/, +,

Dx Dy

[/-i

dq qe -89

dqb e c~

s'n'*/']

v/~'(~2+y2)/N

..... 1

dqb e ~ [ J [ q c ~

,

A.C.C. Coolen

584

-1

f/T

-2

0

- 1.0

-0.5

0.0

0.5

1.0

'

0.0

0.5

q

1.0

1.5

T/J

Fig. 8. The function f ( q ) / T (left) for networks of coupled oscillators with uniform synapses Jij = J / N , and for different choices of the re-scaled interaction strength J / T (T = f3-1) 9 5 1' '1_ ~-(from top to bottom). The right picture gives, for J > 0, the location of the J/T= 2' nonnegative minimum off(q) (which measures the overall degree of global synchronisation in thermal equilibrium) as a function of T/J. A transition to a synchronised state occurs at v / J = 89

where we have transformed to polar coordinates, (x, y) = q v/f3lJlN(cos 0, sin 0), and where we have already eliminated (constant) terms which will not survive the limit N ~ c~. Thus, saddle-point integration gives us, quite similar to the previous cases (36) and (37): lim F I N - m i n f ( q ) N--,~ q>~O

J > 0: ~3f(q) = 89

2 - l o g [ 2 rtlo(BlJlq)]

J < O" ~ f (q) = 89

2 - l o g [ 2 rclo(iBlJIq)]

(57)

in which the In(z) are the modified Bessel functions (see e.g. [15]). The function f ( q ) is shown in Fig. 8. The equations from which to solve the minima are obtained by differentiation, using ~Io(z) = I1 (z)" I1 (~[J]q)

J > O. q

J < O" q = i II (i~lJ[q)

lo(f31Jlq)'

Io(if31JIq)

(58)

o

Again, in both cases the problem has been reduced to studying a single nonlinear equation. The physical meaning of the solution follows from the identity - 2 8 F / O J = (N -1 ~-~iTkj COS(~ i -- ~)j))"

COS((~i)

lim N---+oc

.

+ lim N - - ~

sin(~/)

= sgn(J)q 2.

.

F r o m this equation it also follows that q~< 1. Note: since Of(q)/Oq = 0 at the minimum, one only needs to consider the explicit derivative of f ( q ) with respect to J. If the synapses induce antisynchronization, J < 0, the only solution of (58) (and

Statbstical mechanics of recurrent neural networks I - statics

585

the minimum in (57)) is the trivial state q = 0. This also follows immediately from the equation which gave the physical meaning of q. For synchronizing forces, J > 0, on the other hand, we again find the trivial solution at high noise levels, but a globally synchronized state with q > 0 at low noise levels. Here a phase transition occurs at T - 89 (a bifurcation of nontrivial solutions of (58)), and for T < 89 the minimum of (57) is found at two nonzero values for q. The critical noise level is again found upon expanding the saddle-point equation, using Io(z) = 1 + (9(z2) and I1 (z) - lz + (9(z3) 9 q - 1 IMq + (9(q3). Precisely at 13J - 2 one finds a de-stabilization of the trivial solution q = 0, together with the creation of (two) stable nontrivial ones (see Fig. 8). Note that, in view of (57), we are only interested in nonnegative values of q. One can prove, using the properties of the Bessel functions, that there are no other (discontinuous) bifurcations of nontrivial solutions of the saddle-point equation. Note, finally, that the absence of a state with global antisynchronization for J < 0 has the same origin as the absence of an antiferromagnetic state for J < 0 in the previous models with binary neurons. Due to the long-range nature of the synapses J;j = J / N such states simply cannot exist: whereas any set of oscillators can be in a fully synchronized state, if two oscillators are in anti-synchrony it is already impossible for a third to be simultaneously in antisynchrony with the first two (since antisynchrony with one implies synchrony with the other).

4.2. Coupled oscillator attractor networks 4.2.1. Intuition and definitions Let us now turn to an alternative realization of information storage in a recurrent network based upon the creation of attractors. We will solve models of coupled neural oscillators of the type (13), with zero natural frequencies (since we wish to use equilibrium techniques), in which real-valued patterns are stored as stable configurations of oscillator phases, following [16]. Let us, however, first find out how to store a single pattern { c [-rt,/1~]x in a noise-less infinite-range oscillator network. For simplicity we will draw each component ~i independently at random from [-Tt, rt], with uniform probability density. This allows us to use asymptotic properties such as IN -1 }-~jeie~sl--(_9(N- 89 for any integer g. A sensible choice for the synapses would be J/j = cos[~/- ~j]. To see this we work out the corresponding Lyapunov function (20): 1

L[~] =

2N 2 ~ cos[~i- ~j] cos[qbi- qbj], /7

L[{] -

2 N 2 Z cOS2 [~i -- ~j] -- -- ~ -Jr- C /7

(the factors of N have been inserted to achieve appropriate scaling in the N ~ ec limit). The function L[~], which is obviously bounded from below, must decrease monotonically during the dynamics. To find out whether the state { is a stable fixedpoint of the dynamics we have to calculate L and derivatives of L at ~ = {:

586

A. C. C. Coolen

~L

1

2N 2 Z sin[2(~i- ~j)], J ~2L

] --

i~2L

1 i

J

~ j'~i~)

j ~

--

1

N 2 COS2[~i- ~].

Clearly l i m u ~ L[~]- -1. Putting t ~ - ~ + A~, with A ~ ; - (9(N~ we find -

--

+ -2 Z i

',~

1

= 4N Z

A~

1

2N 2Z

i 1

1

= ~

A'~iA~j ~ i ~ ) j

-~-

(9(A~3)

ij AdpiAdpj cOs2[~i - ~J] + (9(U- 89A~3)

ij

AO)~ -

~

A4),

--

At~i cos(2

~i)

9 1

-

A~i sin(2 ~i)

+ C(N- 89A~3).

(59)

In leading order in N the following three vectors in ~N are normalized and orthogonal: el

1

- ~ ( 1 , 1,..., 1),

e 2 - - x x/2 / N (cos(2 ~ 1) ~'''~ cos(2 ~N))

x/2 (sin(2 ~ 1) ~'''~ sin(2 ~N)) " e2--v/N We may therefore use A~ 2 >/(A~ .el into (59) leads to

)2 +

(A~. e2)2 + (A~ .e3)2, insertion of which 2

L[~ + A ~ ] - L[~] >/

+

1

A(~i sin(2 ~i)

+(9( N--~, A~3) 9

Thus for large N the second derivative of L is nonnegative at t~ - ~, and the phase pattern ~ has indeed become a fixed-point attractor of the dynamics of the noise-free coupled oscillator network. The same is found to be true for the states = +~ + ct(1,..., 1) (for any at).

587


4.2.2. Storing p phase patterns." equilibrium order parameter equations We next follow the strategy of the Hopfield model and attempt to simply extend the above recipe for the synapses to the case of having a finite number p of phase patterns ~ ' - ( ~ , . . . , ~ ) E [-~, ~]N, giving 1

p

Jij - -~ Z c~ ~t=l

- ~]

(60)

(the factor N, as before, ensures a proper limit N ~ ~ later). In analogy with our solution of the Hopfield model we define the following averages over pattern variables:

(g[~])~-

lim 1 N___+~NZ g[~i]' i

We can write the Hamiltonian H(~) of (34) in the form

-

1

P

2N Z

~t=l

Z

cos[~ - ~]

ij

COS[~)i- ~)j]

2 g=l mc~c(~))2q- m~s(~))2 q- m s c ( * ) 2 -+-m~s(~) 2 } in which

mcc

1 ~ Z

1

mcs(Op)--~Zcos(r

cos(r i

~t(~)_ 1 NZ

m~c

(61)

i

sin(~/~)COS(~)i),

~t(~)_ 1 m~ ~ Z

i

sin(~) sin(qbi).

(62)

i

The free energy per oscillator can now be written as

F / N -- - ~] log / -.-

J d~ e- 'H(r-

|

]3N log

/

...

J d~ e 89~-~"' ~--~"** m'

**(,)2

with ** E {cc, ss, cs, sc}. Upon introducing the notation m * * - (m~.,...,nr~**) we can again express the free energy in terms of the density of states ~({m**}) - (2rt) -N f . . . f d{b I-I** 5[m** - m**({b)]"

F / N - - - - ~ l1o g ( 2 r c ) - - ~1l o g f H din** ~ ({m**})eJl3N~-~**m2*

(63)

Since p is finite, the leading contribution to the density of states (as N ~ e~), which will give us the entropy, can be calculated by writing the 8-functions in integral representation:

A.C.C. Coolen

588

/E

i//d~

lim --1log ~({m**})

f~f

= lim 1 log

N-,ccN

1-I dx**eiNx**m** x **

...

" COS(~)sin(0/) + Xcs

x exp - i Z Z [xc~Cc~ i

(2rt) N

p

+Xs~ sin(~) cos((~/) -+-X~sssin(~) sin(~/)] ) = extr{x..} i ~ x** .m** + " cos(~,) + Xcs

log

~-exp

- i 5-~[x~ cos(r

sin(,) + x~c sin(~.,)cos(,)

+.ssSin,,,sin, ,,)l} r

The relevant extremum is purely imaginary so we put x~ = il3y** (see also our previous discussion for the Hopfield model) and, upon inserting the density of states into our original expression for the free energy per oscillator, arrive at lim F I N -

N--~,oc

extr{m..a,..if({m** y**}) 1

1

2

f({m**, y**})- - ~ l o g ( 2 n ) - ~ ~ m * * + E y * * 9m**

, {lo /d* (

13

~--nexp 13Z[Yc~c cos(~.)cos(,) + Y,~scos(~.) sin(,) ~t

+Y~scSin(~.)cos(d~)+Y~ssSin(~.)sin(d~)])). r Taking derivatives with respect to the order parameters m** gives us y** = m**, with which we can eliminate the y**. Derivation with respect to the m** subsequently gives the saddle-point equations .

mcc

=

fdqb cos[qb]exp(13cos[r ~v[mc~ccos[r + ms~sin[~]] + 13sin[O] Ev[mc"scos[r + msVssin[r

f dd~exp(~c~s[d~]y]~v[m~c~s[~v] + ~sin[~v]] + f5sin[~] y~v[~C~sc~s[~v]+ ~`'ssin[~v]])

\

/r (64)

m~cs= (cos[~.] f d~ sin[~]exp(13cos[~] y~v[mc~cos[~v] + ms~sin[~v]]+ 13sin[~] Y'jv[mcv cos[~v] + ms~ssin[~v]]))

f dd~exp(f5c~s[d~]y-~v[m~c~s[~v] + m~ sin[~v]] + ~5sin[d~]~-'v[mc~c~s[~v] + ms~sin[~v]])

~' (65)

Statbstical mechanics of recurrent neural networks I - statics

589

m~c-- (sin[r ] v v mcs v cos[~v] + mss v sin[~v]]) )~,, fdd~cos[d~]exp(~3cos[d~]Y~v[mccCOS[~v]+mscsin[~v]]+~3sin[d~]~v[ f dqbexp(13cos[qb] ~-~v[mc~cos[~v] + msV~sin[~v]] + 13sin[qb] ~v[m~v cos[~v] + m~ sin[~v]]) (66) m~ = (sin[~t ] f dO sin[qb]exp(13cos[dp] ~-~v[m~V~cos[~v] + m~ sin[~v]] + 13sin[qb] ~-'jv[mcV~cos[~v] + ms~sin[~v]]))~. f dOexp(13 cos[qb] ~--~v[m~)cos[~v] + m~) sin[~v]] + 13sin[qb] ~--]~v[m~ cos[~v] + m~ sin[~v]]) (67) The equilibrium values of the observables m**, as defined in (61) and (62), are now given by the solution of the coupled equations (64)-(67)which minimizes 1

2

'//

13 log

d , exp

(

13cos[,] Z

Imp; cos[~,,] + m,;

sin[~,,] 1

v

+ sinI,l

+ < sinI vll/? 9 v

(6a)

/I

We can confirm that the relevant saddle-point must be a minimum by inspecting the 2 13 = 0 limit (infinite noise levels): limp~0f({m**}) - 89 ~** m** - ~ log(2n).

4.2.3. Analys& of order parameter equations: pure states From now on we will restrict our analysis to phase pattern components ~/~ which have all been drawn independently at random from I-n, n], with uniform probability density, so that (g[~,])~ - (2rt) -p frtn.. "f-~Ttd~, g[~,]. At 13 - 0 (T - c~) one finds only the trivial state m.~. = 0. It can be shown that there will be no discontinuous transitions to a nontrivial state as the noise level (temperature) is reduced. The continuous ones follow upon expansion of the equations (64)-(67) for small {m**},which is found to give (for each p and each combination **): 1

m.~. - ~ 13m.~.+ C({m2**}). Thus a continuous transition to recall states occurs at T -- ~. Full classification of all solutions of (64)-(67) is ruled out. Here we will restrict ourselves to the most relevant ones, such as the pure states, where m.~.- m**8,z (for some pattern label )~). Here the oscillator phases are correlated with only one of the stored phase patterns (if at all). Insertion into the above expression for f({m**}) shows that for such solutions we have to minimize f({m** }) -- ~1 Z

2- ~ m**

~ log

f

dqb exp(13 cos[~] [m~ccos[~] + msc sin[~]]

+ 13sin[qb] [m~ cos[~] + m~ sin[~]]).

(69)

590

A.C.C. Coolen

We anticipate solutions corresponding to the (partial) recall of the stored phase pattern {z or its mirror image (modulo overall phase shifts ~/. ~ ~i + 8, under which the synapses are obviously invariant). Insertion into (64)-(67) of the state ~)i -- ~ Ar-~ gives (m~, msc, mc~, ms~) -- 89(cos 8, - sin 8, sin 8, cos 8). Similarly, insertion into (64)-(67) of i~ i __ __~Xi -'[- ~ gives (mcc, m~c, mcs, mss) -12 (cos 8, sin 8, sin 8 , - cos 8) Thus we can identify retrieval states as those solutions which are of the form (i) retrieval o f { x(ii) retrieval of - { x .

(mc~,m~,mc,,mss) -- m(cos S, - sin S, sinS, cosS) (m~c, re,c, m~,, re,s) -- re(cos 8, sin 8, sin 8, -cos 5)

with full recall corresponding to m - 89 Insertion into the saddle-point equations and into (69), followed by an appropriate shift of the integration variable ~, shows that the free energy is independent of 8 (so the above two ans/itze solve the saddlepoint equations for any 8) and that 1 f as) c o s [ * ] e [3me~ m = ~ f dqb e[3mc~ *]

1 '

f(m) - m 2 -

~

f cos(,] log j dd~ e 13m 9

Expansion in powers of m, using log(1 + z) = z - 892 + (9(z3), reveals that nonzero minima m indeed bifurcate continuously at T - 13-1 - 1 4" . ) m 2 + ~4~3m 4 + (_9(m6). f ( m ) + gllog[2g]-- ( 1 - - 1~f3

(70)

Retrieval states are obviously not the only pure states that solve the saddle-point equations. The function (69) is invariant under the following discrete (noncommuting) transformations: I" (mcc, msc, mcs, m~)

II: (mec, msc, mcs, mss)

--+ --~

(m~c,m~c,-mcs,-mss), (mr

We expect these to induce solutions with specific symmetries. In particular we anticipate the following symmetric and antisymmetric states: (iii) symmetric under I: (iv) antisymmetricunder I:

(mcc,msc,mcs,mss) = v~m(cosS, sinS,0,0), (mcc,m.~ J) or at T = J (if J > J0). The remaining program is: find the saddle-point (q, m) for T < max{J0,J} which for integer n minimizes f , determine the corresponding minimum as a function of n, and finally take the limit n --+ 0. This is in fact the most complicated part of the procedure.

5.2. Replica-symmetric solution and A T-&stability 5.2.1. Physical interpretation of saddle points To obtain a guide in how to select saddle-points we now turn to a different (but equivalent) version of the replica trick (75), which allows us to attach a physical meaning to the saddle-points (m, q). This version transforms averages over a given measure W: n-1

n---~0n 7= 1 ~l...~n

s--1

The trick again consists in evaluating this quantity for integer n, whereas the limit refers to noninteger n. We use (84) to write the distribution P(m) of overlaps in equilibrium as

596

A.C.C.

Coolen

P(m) -- ~-~'~8[m -- 1 ~-~i ~icri]e-f~H('~) ~-~ e-f~H(~) = lim 1 .-

8 m

~i(Y~

-

e-

on

If we average this distribution over the disorder, we find identical expressions to those encountered in evaluating the disorder averaged free energy. By inserting the same delta-functions we arrive at the steepest descend integration (77) and find P(m) -- lim 1 Z n---,0 n

8[m - my]

(85)

7

where {my} refers to the relevant solution of (82) and (83). Similarly we can imagine two systems ~ and ~' with identical synapses {Jij}, both in thermal equilibrium. We now use (84) to rewrite the distribution P(q) for the mutual overlap between the microstates of the two systems E,~,,~' 8[q - ~ E i cricY~]e-13"('~)-13"(e) P(q) =

~-]~.o., e - f 3 H ( n ) - f 3 H ( ~ ' )

=lim

1

[ l~. ] i ~ ~n(~/ 8 q-~ ~)~[ e-

n - ~ o n ( n - 1 ) ~-'~ Z ~r

~l ...or"

Averaging over the disorder again leads to the steepest descend integration (77) and we find 1

(86)

P(q) - - l i m ,,--,o n(n - 1 ) Z 8 [q - qxv], key

where {qxv} refers to the relevant solution of (82) and (83). We can now partly interpret the saddle-points (m, q), since the shape of P(q) and P(m) gives direct information on the structure of phase space with respect to ergodicity. The crucial observation is that for an ergodic system one always has

[

P(m) - 8 m - - ~

.

~i((Yi)eq

]

,

P(q) - 8 q - -~

. (O'i)eq

(87)

9

If, on the other hand, there are L ergodic components in our system, each of which corresponding to a pure Gibbs state with microstate probabilities proportional to exp(-[3H) and thermal averages (...)t, and if we denote the probability of finding the system in component g by ~ , we find P(m) - Z g= 1

W~8 m - ~

~i((Yi)g ' P(q) - Z "

g,g~= l

WeWe 8 q - ~

((Yi) "

597


For ergodic systems both P(m) and P(q) are 8-functions, for systems with a finite number of ergodic components they are finite sums of 8-functions. A diverging number of ergodic components generally leads to distributions with continuous pieces. If we combine this interpretation with our results (85) and (86) we find that ergodicity is equivalent to the relevant saddle-point being of the form: q~13--8oq3+q[1-8~13] ,

ms--m,

(88)

which is called the 'replica symmetry' (RS) ansatz. The meaning of m and q is deduced from (87) (taking into account the transformation c~/---+ ~/cy; we performed along the way): 1 m -- ~ Z

1

~i((Yi)eq '

2 ((3"i)eq"

q- ~ Z

i

i

5.2.2. Replica symmetric solution Having saddle-points of the simple form (88) leads to an enormous simplification in our calculations. Insertion of (88) as an ansatz into Eqs. (81)-(83) gives

1

f(q, m) - - ~ l o g 2

--41 ~j2 (1 -

-~---~log exp

q)2

+-~Joml 2

q~2j2

~

+~JomZ~ ~

+C(n),

(O'1 ~2 exP( 89q~2j2 [~--~'~cya]2+[3Jom ~--2~cya))~ q -m

(exp(lq[ 32J2 [Y'~a cYa]2+[3J0 m E a cya)), (c~l exp(!2 q~2j2 [~-~.~ (y~] 2 +~Jom ~

'

cy~))~

(exp(lq~2j2 l e a cYu]2q-~J0 m E ~ cr~))~ We linearize the terms [ y ~ (3"a] 2 by introducing a Gaussian integral, and perform the average over the remaining neurons. The solutions m and q turn out to be welldefined for n ---+0 so we can take the limit: limf(q, m ) - - ~ 1l o g 2 - ~1 j 2 ( 1 n--+0

'/

13

1 - q)2 _+__~Jom 2

Dz logcosh [IgJom + f3Jzv/-q] ,

q - / Dz tanh2 [13Join + f3JZv~],

m - / Dz tanh [J3Jom + f3Jzx/~ ] .

Writing the equation for m in integral form gives

m - ~ Join

/o' [ / d)~ 1 -

Dz tanh 2 [)~13Join + ~ JZv~ ] 9

(89) (90)

A.C.C. Coolen

598

I

I

I

I

]

I

I

I

I

1.5

T/J 1

i I

0

I

i

i

I

j

i

J

I

J

i\J

1

.5

i

I

J

i

i

J

1.5

I

2

Jo/J

Fig. 10. Phase diagram of the model (72) with Gaussian synapses, obtained from the replica-symmetric solution. P: paramagnetic phase, m = q = 0 (more or less random evolution). SG: spin-glass phase, m - 0, q # 0 ('frozen' equilibrium states without pattern recall). F: recall ('ferro-magnetic') phase, m # 0, q # 0. Solid lines: second-order transitions. Dashed: the AT instability.

From this expression, in combination with (90), we conclude: T>Jo:

m=O

T > J 0 and T > J :

m=q=0.

Linearization of (90) for small q and m shows the following continuous bifurcations: at J0>J: J0 < J :

T<max{Jo,J}:

T=Jo T=J T=Jo[1-q]

from m=q=0 re=q=0 m=0, q>0

to m-f0, q > 0 m=0, q>0 m#0, q>0

Solving numerically equations T = J 0 [ 1 - q] and (90) leads to the phase diagram shown in Fig. 10.

5.2.3. Breaking of RS: the AT instability If for the replica symmetric solution we calculate the entropy S = ~2~F/O~ numerically, we find that for small temperatures it becomes negative. This is not possible. Firstly, straightforward differentiation shows ~S/~[3- [3[(H)~q(HZ)eq]~0 (zero only for ground-state configurations, the number of which we denote by No ~> 1). We now find

599


limS-lim{logEe-PH(")+[~(H)eq}

T--~O

13--~c~

6

= lim[logEe-I~H(~)+13(/t)eq]~>logN 0. 1~----+OO

We conclude that S~>0 for all T. At small temperatures the RS ansatz (88) is apparently incorrect in that it no longer corresponds to the minimum of f(q,m) (81). If saddle-points without RS bifurcate continuously from the RS one, we can locate the occurrence of this 'replica symmetry breaking' (RSB) by studying the effect on f(q, m) of small fluctuations around the RS solution. It was shown [19] that the 'dangerous' fluctuations are of the form qaf~ ~ 8~f~+ q[1 - 8~f~] + rlaf~,

Eq~pP

0 Va.

(91)

in which q is the solution of (90) and rl~p -rlp~. We now calculate the resulting change in f(q,m), away from the RS value f(qrs,mgs), the leading order of which is quadratic in the fluctuations {q~p} since the RS solution of (90) is a saddle-point: ~3j4

[~j2

f(q, m) - f(qrs, mRS)

2

-~-nZ q~

8n Z Z rip,rio~G~O~ ~r pCz

with ( cY~(YvCrpCrZexp ( 89q132j2 [~--]~cr~]2+ 13mJo}-~.acry) }. Gavp~" -

"

. ..(exP( . 89176

Because of the index permutation symmetry in the above average we can write for -r y and p -r ~: G~ypk -- 8o~pS~,k -+- 8~kS~,p -+- 6411 -- 8~p][1 -- 8%][1 - 8~][1 - By0] +

-

+

-

+

-

+

-

with

Ge - f Dz tanhe [13J0m + [3Jzx/~] cosh" [13J0m + [3Jzv~] f Dz cosh" [13Jom + [3Jzx/ff] Only terms which involve precisely two 8-functions can contribute, because of the requirements o~~: % p ~: ~ and ~ p q~p - 0. As a result: ~j2

f(q, m) -- f(qrs, mRS) -- W

[1--

~2j2

(1-2G2+G4)]Eq~

2

v.

The condition for the RS solution to minimize f(q, m), if compared to the so-called 'replicon' fluctuations (91), is therefore

600

A. C. C. Coolen

1 > ~2j2 lirn(1 - 2 G2 + G4). n 0

After taking the limit in the expressions Ge this condition can be written as 1 > ~2j2 f Dz cosh -4 [~Jom + ~Jzx/~] 9

(92)

The so-called AT line in the phase diagram where this condition ceases to be met, indicates a continuous transition to a complex 'spin-glass' state where ergodicity is broken (i.e. the distribution P(q) (86) is no longer a 8-function). It is shown in figure 10 as a dashed line for Jo/J > 1, and coincides with the line T / J - 1 for J0 1. If 7" > 1 we obtain in turn from (110), using tanh 2 (x) < x 2 and 0 _< q _< 1" q - 0 or q _< 1 + ~ 7". We conclude that q - 0 for 7' > 1 + v~. Secondly, for the free energy (108) to be well defined we must require q > 1 - 7". Linearization of (109) and (110) for small q and m shows the continuous bifurcations: at

from

to

cx>0:

T=l+v/~

m=O,q=O

m=O,q>O

~x=O:

T= 1

m=O,q--O

m:/:O,q>O

608

A. C. C. Coolen

The upper bound T = 1 + x/~ turns out to be the critical noise level indicating (for cx > 0) a continuous transition to a spin-glass state, where there is no significant alignment of the neurons in the direction of one particular pattern, but still a certain degree of local freezing. Since m = 0 for T > 1 this spin-glass state persists at least down to T = 1. The quantitative details of the spin-glass state are obtained by inserting m = 0 into (110) (since (109) is fulfilled automatically). The impact on the saddle-point Eqs. (109) and (110) of having a > 0, a smoothening of the hyperbolic tangent by convolution with a Gaussian kernel, can be viewed as noise caused by interference between the attractors. The natural strategy for solving (109) and (110) is therefore to make an ansatz for the nominated overlaps m of the type (52) (the mixture states). Insertion of this ansatz into the saddle-point equations indeed leads to self-consistent solutions. One can solve numerically the remaining equations for the amplitudes of the mixture states and evaluate their stability by calculating the eigenvalues of the second derivative of f ( m , q, ~]), in the same way as for ~ = 0. The calculations are just more involved. It then turns out that even mixtures are again unstable for any T and a, whereas odd mixtures can become locally stable for sufficiently small T and a. Among the mixture states, the pure states, where the vector m has only one nonzero component, are the first to stabilize as the temperature is lowered. These pure states, together with the spin-glass state (m = 0, q > 0), we will study in more detail. Let us first calculate the second derivatives of (108) and evaluate them in the spin-glass saddle-point. One finds, after elimination of r with (110): a2f /am~am,, - 8pv[1 - 13(1 - q)],

a2f /am~aq = o.

The (g + 1) x (g + 1) matrix of second derivatives with respect to variation of (m, q), evaluated in the spin-glass saddle-point, thereby acquires a diagonal form 1 -

[3(1 -

q) ,

82f =

.. 1 - [3(1 - q)

O2f/Oq2

and the eigenvalues can simply be read off. The g-fold degenerate eigenvalue 1 - [ 3 ( 1 - q) is always positive (otherwise (108) would not even exist), implying stability of the spin-glass state in the direction of the nominated patterns. The remaining eigenvalue measures the stability of the spin-glass state with respect to variation in the amplitude q. Below the critical noise level T = 1 + x/~ it turns out to be positive for the spin-glass solution of (110) with nonzero q. One important difference between the previously studied case cx = 0 and the present case cx > 0 is that there is now an m = 0 spin-glass solution which is stable for all T < 1 + x/~. In terms of information processing this implies that for a > 0 an initial state must have a certain nonzero overlap with a pattern to evoke a final state with m r 0, in order to avoid ending up in the m = 0 spin-glass state. This is clearly consistent with the observations in Fig. 5. In contrast, for a = 0, the state with m - 0 is unstable, so any initial state will eventually lead to a final state with m :/: 0.

Stathctical mechanics of recurrent neural networks I - statics

.8

609

f

\k\\\\\ ......

\, \k

.6 m

f

/

.4

~ I

-.55L-

0

0

.2

.4

T

.6

.8

-.6

1

0

i'-i

i\ i \\\

i'\i \\\ ~ :, T, i'xi \\\~

:: \i

.2

~I

.4

T

.6

.8

~[~

1

Fig. 11. Left: RS amplitudes m of the pure states of the Hopfield model versus temperature. From top to bottom: ct = 0.000- 0.125 (Act -0.025). Right, solid lines: 'free energies' f of the pure states. From bottom to top: a t - 0.000- 0.125 (Act = 0.025). Right, dashed lines: 'free energies' of the spin-glass state m = 0 (for comparison). From top to bottom: ct = 0.000 - 0.125 (Act = 0.025).

Inserting the pure state ansatz m = re(l, 0 , . . . , 0) into our RS equations gives m-

Dz tanh 1 3 m + l _ 1 3 ( 1 _ q ) .

'

q-

Dz tanh 2 1 3 m + l _ 1 3 ( 1 _ q ) -

'

(lll)

l m2 f--~ 13

1 [ +~at (l-q)

1 + 13(1- q ) ( 1 3 - 2 ) l l o g [ 1 _ 13(1- q)]] [1-13(1-q)]2 +

Dz log 2 cosh 13m + 1 - 13(1 - q) "

(112)

If we solve Eq. (111) numerically for different values of {z, and calculate the corresponding 'free energies' f (112) for the pure states and the spin-glass state m = 0, we obtain Fig. 11. For ct > 0 the nontrivial solution m for the amplitude of the pure state appears discontinously as the temperature is lowered, defining a critical temperature TM (:t). Once the pure state appears, it turns out to be locally stable (within the RS ansatz). Its 'free energy' f , however, remains larger than the one corresponding to the spin-glass state, until the temperature is further reduced to below a second critical temperature Tc(cz). For T < Tc(Gt) the pure states are therefore the equilibrium states in the thermodynamics sense. By drawing these critical lines in the ({z,T) plane, together with the line Tg(~) = 1 + v ~ which signals the second-order transition from the paramagnetic to

610

A. C. C. Coolen

the spin-glass state, we obtain the RS phase diagram of the Hopfield model, depicted in Fig. 12. Strictly speaking the line TM would appear meaningless in the thermodynamic picture, only the saddle-point that minimizes f being relevant. However, we have to keep in mind the physics behind the formalism. The occurrence of multiple locally stable saddle-points is the manifestation of ergodicity breaking in the limit N ~ oo. The thermodynamic analysis, based on ergodicity, therefore applies only within a single ergodic component. Each locally stable saddle-point is indeed relevant for appropriate initial conditions and time-scales.

6.2.2. Zero temperature, storage capacity The storage capacity mc of the Hopfield model is defined as the largest m for which locally stable pure states exist. If for the m o m e n t we neglect the low temperature reentrance peculiarities in the phase diagram (12) to which we will come back later, the critical temperature TM(m), where the pure states appear decreases monotonically with m, and the storage capacity is reached for T - 0. Before we can put T ~ 0 in (111), however, we will have to rewrite these equations in terms of quantities with well-defined T ~ 0 limits, since q ~ 1. A suitable quantity is C - 13(1 - q ) , which obeys 0 < C a l 3 2 ( / D z cosh-4 i~[m . ~ + z x / ~ )

(121)

The AT line in the phase diagram, where this condition ceases to be met, indicates a second-order transition to a spin-glass state, where ergodicity is broken in the sense that the distribution P(q) (104) is no longer a 8-function. In the paramagnetic regime of the phase diagram, m = 0 and q = 0, the AT condition reduces precisely to T > Tg = 1 + x/~. Therefore the paramagnetic solution is stable. The AT line coincides with the boundary between the paramagnetic and spin-glass phase. Numerical evaluation of (121) shows that the RS spin-glass solution remains unstable for all T < Tg, but that the retrieval solution m r is unstable only for very low temperatures T < TR (see Fig. 12).

7. Epilogue In this paper I have tried to give a self-contained expos6 of the main issues, models and mathematical techniques relating to the equilibrium statistical mechanical

616

A. C. C. Coolen

analysis of recurrent neural networks. I have included networks of binary neurons and networks of coupled (neural) oscillators, with various degrees of synaptic complexity (albeit always fully connected), ranging from uniform synapses, via synapses storing a small number of patterns, to Gaussian synapses and synapses encoding an extensive number of stored patterns. The latter (complex) cases I only worked out for binary neurons; similar calculations can be done for coupled oscillators (see [16]). Networks of graded response neurons could not be included, because these are found never to go to (detailed balance) equilibrium, ruling out equilibrium statistical mechanical analysis. All analytical results and predictions have later also been confirmed comprehensively by numerical simulations. Over the years we have learned an impressive amount about the operation of recurrent networks by thinking in terms of free energies and phase transitions, and by having been able to derive explicit analytical solutions (since a good theory always supersedes an infinite number of simulation experiments ...). I have given a number of key references along the way; many could have been added but were left out for practical reasons. Instead I will just mention a number of textbooks in which more science as well as more references to research papers can be found. Any such selection is obviously highly subjective, and I wish to apologize beforehand to the authors which I regret to have omitted. Several relevant review papers dealing with the statistical mechanics of neural networks can be found scattered over the three volumes [20-22]. Textbooks which attempt to take the interested but nonexpert reader towards the expert level are [8,23]. Finally, a good introduction to the methods and backgrounds of replica theory, together with a good collection of reprints of original papers, can be found in [24]. What should we expect for the next decades, in the equilibrium statistical mechanics of recurrent neural networks? Within the confined area of large symmetric and fully connected recurrent networks with simple neuron types we can now deal with fairly complicated choices for the synapses, inducing complicated energy landscapes with many stable states, but this involves nontrivial and cutting-edge mathematical techniques. If our basic driving force next is the aim to bring our models closer to biological reality, balancing the need to retain mathematical solvability with the desire to bring in more details of the various electro-chemical processes known to occur in neurons and synapses and spatio-temporal characteristics of dendrites, the boundaries of what can be done with equilibrium statistical mechanics are, roughly speaking, set by the three key issues of (presence or absence of) detailed balance, system size, and synaptic interaction range. The first issue is vital: no detailed balance immediately implies no equilibrium statistical mechanics. This generally rules out networks with nonsymmetric synapses and all networks of graded response neurons (even when the latter are equipped with symmetric synapses). The issue of system size is slightly less severe; models of 1 networks with N < ~ neurons can often be solved in leading order in N-~, but a price will have to be paid in the form of a reduction of our ambition elsewhere (e.g. we might have to restrict ourselves to simpler choices of synaptic interactions). Finally, we know how to deal with fully connected models (such as those discussed in this paper), and also with models having dendritic structures which cover a long


617

(but not infinite) range, provided they vary smoothly with distance. We can also deal with short-range dendrites in one-dimensional (and to a lesser extent twodimensional) networks; however, since even the relatively simple Ising model (mathematically equivalent to a network of binary neurons with uniform synapses connecting only nearest-neighbor neurons) has so far not yet been solved in three dimensions, it is not realistic to assume that analytical solution will be possible soon of general recurrent neural network models with short range interactions. On balance, although there are still many interesting puzzles to keep theorists happy for years to come, and although many of the model types discussed in this text will continue to be useful building blocks in explaining at a basic and qualitative level the operation of specific recurrent brain regions (such as the CA3 region of the hippocampus), one is therefore led to the conclusion that equilibrium statistical mechanics has by now brought us as far as can be expected with regard to increasing our understanding of biological neural networks. Dale's law already rules out synaptic symmetry, and thereby equilibrium statistical mechanics altogether, so we are forced to turn to dynamical techniques if we wish to improve biological realism.

Acknowledgements It is my pleasure to thank David Sherrington and Nikos Skantzos for their direct and indirect contributions to this review. References 1. Van Kampen, N.G. (1992) Stochastic Processes in Physics and Chemistry. North-Holland, Amsterdam. 2. Gardiner, C.W. (1994) Handbook of Stochastic Methods. Springer, Berlin. 3. Khalil, H.K. (1992) Nonlinear Systems. MacMillan, New York. 4. Zinn-Justin, J. (1993) Quantum Field Theory and Critical Phenomena. U.P., Oxford. 5. Yeomans, J.M. (1992) Statistical Mechanics of Phase Transitions. U.P., Oxford. 6. Plischke, M. and Bergersen, B. (1994) Equilibrium Statistical Mechanics. World Scientific, Singapore. 7. Peretto, P. (1984) Biol. Cybern. 50, 51. 8. Peretto, P. (1992) An Introduction to the Theory of Neural Computation. U.P., Cambridge. 9. Hopfield, J.J. (1982) Proc. Natl. Acad. Sci. USA. 79, 2554. 10. Hebb, D.O. (1949) The Organization of Behaviour. Wiley, New York. 11. Amari, S.-I. (1977) Biol. Cybern. 26, 175. 12. Amit, D.J., Gutfreund, H. and Sompolinsky, H. (1985) Phys. Rev. A 32, 1007. 13. Fontanari, J.F. and K6berle, R. (1988) J. Physique 49, 13. 14. Kuramoto, Y. (1984) Chemical Oscillations, Waves and Turbulence. Springer, Berlin. 15. Abramowitz, M. and Stegun, I.A. (1972) Handbook of Mathematical Functions. Dover, New York. 16. Cook, J. (1989) J. Phys. A 22, 2057. 17. Sherrington, D. and Kirkpatrick, S. (1975) Phys. Rev. Lett. 35, 1972. 18. Amit, D.J., Gutfreund, H. and Sompolinsky, H. (1985) Phys. Rev. Lett. 55. 19. de Almeida, J.R.L. and Thouless, D.J. (1978) J. Phys. A 11, 983. 20. Domany, E., van Hemmen, J.L. and Schulten, K. eds (1991) Models of Neural Networks I. Springer, Berlin. 21. Domany, E., van Hemmen, J.L. and Schulten, K. eds (1994) Models of Neural Networks II. Springer, Berlin.

618

A. C. C. Coolen

22. Domany, E., van Hemmen, J.L. and Schulten, K. eds (1995) Models of Neural Networks III. Springer, Berlin. 23. Coolen, A.C.C. and Sherrington, D. (2000) Statistical Physics of Neural Networks. U.P., Cambridge. 24. M+zard, M., Parisi, G. and Virasoro, M.A. (1987) Spin-Glass Theory and Beyond. World Scientific, Singapore.

C H A P T E R 15

Statistical Mechanics of Recurrent Neural Networks I I - Dynamics

A.C.C. C O O L E N Department of Mathematics, King's College London, Strand, London WC2R 2LS, UK



619

Contents

1.

Introduction

2.

A t t r a c t o r neural networks with binary neurons

.................................................

621

............................

2.1. Closed macroscopic laws for sequential dynamics

3.

627

2.3. Closed macroscopic laws for parallel dynamics . . . . . . . . . . . . . . . . . . . . . . . . .

632

2.4. Application to separable a t t r a c t o r networks . . . . . . . . . . . . . . . . . . . . . . . . . . .

635

A t t r a c t o r neural networks with c o n t i n u o u s neurons

637

.........................

......................................

637

3.2. Application to graded response a t t r a c t o r networks . . . . . . . . . . . . . . . . . . . . . . .

641

C o r r e l a t i o n and response functions

646

4.1.

...................................

Fluctuation--dissipation theorems

.................................

646

4.2. Example: simple a t t r a c t o r networks with binary neurons . . . . . . . . . . . . . . . . . . .

650

4.3. Example: graded response neurons with uniform synapses

653

..................

Dynamics in the complex regime . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

655

5.1. Overview of methods and theories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

655

5.2. Generating functional analysis for binary neurons

.......................

659

5.3. Parallel dynamics Hopfield model near saturation

.......................

667

5.4. Extremely diluted a t t r a c t o r networks near saturation 6.

623

2.2. Application to separable a t t r a c t o r networks . . . . . . . . . . . . . . . . . . . . . . . . . . .

3.1. Closed macroscopic laws

5.

.......................

623

Epilogue

...................................................

Acknowledgements References

.............................................

.....................................................

620

.....................

675 682 683 683

1. Introduction

This paper, on solving the dynamics of recurrent neural networks using nonequilibrium statistical mechanical techniques, is the sequel of[l], which was devoted to solving the statics using equilibrium techniques. I refer to [1] for a general introduction to recurrent neural networks and their properties. Equilibrium statistical mechanical techniques can provide much detailed quantitative information on the behavior of recurrent neural networks, but they obviously have serious restrictions. The first one is that, by definition, they will only provide information on network properties in the stationary state. For associative memories, for instance, it is not clear how one can calculate quantities like sizes of domains of attraction without solving the dynamics. The second, and more serious, restriction is that for equilibrium statistical mechanics to apply the dynamics of the network under study must obey detailed balance, i.e. absence of microscopic probability currents in the stationary state. As we have seen in [1], for recurrent networks in which the dynamics take the form of a stochastic alignment of neuronal firing rates to postsynaptic potentials which, in turn, depend linearly on the firing rates, this requirement of detailed balance usually implies symmetry of the synaptic matrix. From a physiological point of view this requirement is clearly unacceptable, since it is violated in any network that obeys Dale's law as soon as an excitatory neuron is connected to an inhibitory one. Worse still, we saw in [1] that in any network of graded-response neurons detailed balance will always be violated, even when the synapses are symmetric. The situation will become even worse when we turn to networks of yet more realistic (spike-based) neurons, such as integrate-andfire ones. In contrast to this, nonequilibrium statistical mechanical techniques, it will turn out, do not impose such biologically nonrealistic restrictions on neuron types and synaptic symmetry, and they are consequently the more appropriate avenue for future theoretical research aimed at solving biologically more realistic models. The common strategy of all nonequilibrium statistical mechanical studies is to derive and solve dynamical laws for a suitable small set of relevant macroscopic quantities from the dynamical laws of the underlying microscopic neuronal system. In order to make progress, as in equilibrium studies, one is initially forced to pay the price of having relatively simple model neurons, and of not having a very complicated spatial wiring structure in the network under study; the networks described and analyzed in this paper will consequently be either fully connected, or randomly diluted. When attempting to obtain exact dynamical solutions within this class, one then soon finds a clear separation of network models into two distinct complexity classes, reflecting in the dynamics a separation which we also found in the statics. In statics one could get away with relatively simple mathematical techniques as long as the number of attractors of the dynamics was small compared to the number N of 621

622

A.C.C. Coolen

neurons. As soon as the number of attractors became of the order of N, on the other hand, one entered the complex regime, requiring the more complicated formalism of replica theory. In dynamics we will again find that we can get away with relatively simple mathematical techniques as long as the number of attractors remains small, and find closed deterministic differential equations for macroscopic quantities with just a single time argument. As soon as we enter the complex regime, however, we will no longer find closed equations for one-time macroscopic objects: we will now have to work with correlation and response functions, which have two time arguments, and turn to the less trivial generating functional techniques. 1 In contrast to the situation in statics [1], I cannot in this paper give many references to textbooks on the dynamics, since these are more or less nonexistent. There would appear to be two reasons for this. Firstly, in most physics departments nonequilibrium statistical mechanics (as a subject) is generally taught and applied far less intensively than equilibrium statistical mechanics, and thus the nonequilibrium studies of recurrent neural networks have been considerably less in number and later in appearance in literature than their equilibrium counterparts. Secondly, many of the popular textbooks on the statistical mechanics of neural networks were written around 1989, roughly at the point in time where nonequilibrium statistical mechanical studies just started being taken up. When reading such textbooks one could be forgiven for thinking that solving the dynamics of recurrent neural networks is generally ruled out, whereas, in fact, nothing could be further from the truth. Thus the references in this paper will, out of necessity, be mainly to research papers. I regret that, given constraints on page numbers and given my aim to explain ideas and techniques in a lecture notes style (rather than display encyclopedic skills), I will inevitably have left out relevant references. Another consequence of the scarce and scattered nature of the literature on the nonequilibrium statistical mechanics of recurrent neural networks is that a situation has developed where many mathematical procedures, properties and solutions are more or less known by the research community, but without there being a clear reference in literature where these were first formally derived (if at all). Examples of this are the fluctuation-dissipation theorems (FDTs) for parallel dynamics and the nonequilibrium analysis of networks with graded response neurons; often the separating boundary between accepted general knowledge and published accepted general knowledge is somewhat fuzzy. The structure of this paper mirrors more or less the structure of[l]. Again I will start with relatively simple networks, with a small number of attractors (such as systems with uniform synapses, or with a small number of patterns stored with Hebbian-type rules), which can be solved with relatively simple mathematical techniques. These will now also include networks that do not evolve to a stationary

A brief note about terminology: strictly speaking, in this paper we will apply these techniques only to models in which time is measured in discrete units, so that we should speak about generating functions rather than generating functionals. However, since these techniques can and have also been applied intensively to models with continuous time, they are in literature often referred to as generating functional techniques, for both discrete and continuous time.

Statistical mechanics of recurrent neural networks H - dynamics

623

state, and networks of graded response neurons, which could not be studied within equilibrium statistical mechanics at all. Next follows a detour on correlation and response functions and their relations (i.e. FDTs), which serves as a prerequisite for the last section on generating functional methods, which are indeed formulated in the language of correlation and response functions. In this last, more mathematically involved, section I study symmetric and nonsymmetric attractor neural networks close to saturation, i.e. in the complex regime. I will show how to solve the dynamics of fully connected as well as extremely diluted networks, emphasizing the (again) crucial issue of presence (or absence) of synaptic symmetry, and compare the predictions of the (exact) generating functional formalism to both numerical simulations and simple approximate theories. 2. Attractor neural networks with binary neurons

The simplest nontrivial recurrent neural networks consist of N binary neurons ~; c { - 1 , 1} (see [1]) which respond stochastically to postsynaptic potentials (or local fields) hi(e), with 6 = (Cyl,..., CYN). The fields depend linearly on the instantaneous neuron states, h;(6) = Y~'~yJijcyj+ 0;, with the Jij representing synaptic efficacies, and the 0; representing external stimuli and/or neural thresholds.

2.1. Closed macroscopic laws for sequential dynamics First I show how for sequential dynamics (where neurons are updated one after the other) one can calculate, from the microscopic stochastic laws, differential equations for the probability distribution of suitably defined macroscopic observables. For mathematical convenience our starting point will be the continuous-time master equation for the microscopic probability distribution pt(6) pt(6)

-

Z{wi(Fi.)pt(Fia) i

- wi(")pt(a)},

Wi(6) -- ~[l -- (5"i tanh[[3hi(~)]]

(1) with f/~(6) = ( I ) ( ( y l , . . . , ( Y i - 1 , - o ' i , o ' i + l , . . . , (YN) (see [1]). I will discuss the conditions for the evolution of these macroscopic state variables to become deterministic in the limit of infinitely large networks and, in addition, be governed by a closed set of equations. I then turn to specific models, with and without detailed balance, and show how the macroscopic equations can be used to illuminate and understand the dynamics of attractor neural networks away from saturation.

2.1.1. A toy model Let me illustrate the basic ideas with the help of a simple (infinite range) toy model:

Jij = (J/N)qi~j and 0i = 0 (the variables q/ and ~; are arbitrary, but may not depend on N). For rl; = ~; = 1 we get a network with uniform synapses. For qg = ~,i c { - 1 , 1} and J > 0 we recover the Hopfield [2] model with one stored pattern. Note: the synaptic matrix is nonsymmetric as soon as a pair (ij) exists such

A.C.C. Coolen

624

that flyby ~ qj~i, so in general equilibrium statistical mechanics will not apply. The local fields become h i ( o ) - Jqim(u) with m(u) _ 1 ~-]~k~kCrk 9Since they depend on the microscopic state u only through the value of m, the latter quantity appears to constitute a natural macroscopic level of description. The probability density of finding the macroscopic state m(u) = m is given by ~t[m] = ~ , p t ( u ) 8 [ m - m(u)]. Its time derivative follows upon inserting (1): dt ~@t[m] - Z ~

_

d dm

~--~pt(u)w,(u) 8 m - m(u) +-~,cyk

- 5[m - m(u)]

k=l

Z p , ( u ) 8 [ m _ m ( u ) ] ~ Z ~kcrkwk(u) ~ k=l

+ (9

Inserting our expressions for the transition rates wi(u) and the local fields h/(u) gives:

dt

d{ [ ,N

~

~t[m] m - -~ Z

~* tanh[rlklk/m]

]}

+ (9(N -1).

k=l

In the limit N ~ ~ only the first term survives. The general solution of the resulting Liouville equation is ~@,[m] = fdmo~o[mo]8[m-m(t[mo)], where m(t[mo) is the solution of d 1 N dt m -- lim --~-~ ~k tanh[rlkl3dm] - m, N~ocN k=l

m(0) = m0.

(2)

This describes deterministic evolution; the only uncertainty in the value of m is due to uncertainty in initial conditions. If at t - 0 the quantity m is known exactly, this will remain the case for finite time-scales; m turns out to evolve in time according

to (2).

2.1.2. Arbitrary synapses Let us now allow for less trivial choices of the synaptic matrix {Jij} and try t o calculate the evolution in time of a given set of macroscopic observables f~(u) = (f~l(u),..., f~n(u)) in the limit N ~ c~. There are no restrictions yet on the form or the number n of these state variables; these will, however, arise naturally if we require the observables f~ to obey a closed set of deterministic laws, as we will see. The probability density of finding the system in macroscopic state ~ is given by: ~@t[f~]- ~--~,pt(u)8[~- a(u)].

(3)

Its time derivative is obtained by inserting (1). If in those parts of the resulting expression which contain the operators F~ we perform the transformations u ~ Fiu, we arrive at


d

d~ ~t[n] -

E Pt(6)wi("){8[n - n(F/~r)]-

Z i

6

625

8[n -

n(~r)]}.

Upon writing ~.(Fi6) = ~ . ( 6 ) + Ai~t(6) and making a Taylor expansion in powers of {Ai.(6)}, we finally obtain the so-called Kramers-Moyal expansion:

g-------~.

at ~ t [ n ] - Z g ~> 1

It

involves

"'" ~q=l

conditional

8f~.,

averages

(f(6))n; t

Ajar(") = nB(Fj~r ) -- n~t(@): 2

K(

[n. t] -

(

wj(

8f~.e ~t[n]F([)..,e[n; t ] .

(4)

pe-1

)Aj.l ( - ) ' "

)

and

the

'discrete

derivatives'

f~;t

(s)

(f(6))n: t = E . p t ( 6 ) 8 [ f ~ - n ( 6 ) ] f ( 6 ) E pt(.)6[n- n(.)] "

Retaining only the g = 1 term in (4) would lead us to a Liouville equation, which describes deterministic flow in g~ space. Including also the g = 2 term leads us to a Fokker-Planck equation which, in addition to flow, describes diffusion of the macroscopic probability density. Thus a sufficient condition for the observables g~(6) to evolve in time deterministically in the limit N --+ oo is:

,im >~ 1 ~ g ,.. _

"

Pl=l

~_.~ x

- o.

(6)

pg=l j = l

In the simple case where all observables f~, scale similarly in the sense that all 'derivatives' A j , - f ~ , ( F i 6 ) - f~,(6) are of the same order in N (i.e. there is a monotonic function/~N such that Aj, -- (9(/~S) for all jt-t), for instance, criterion (6) becomes: lim n~NX/~

N---+oo

= 0.

(7)

If for a given set of observables condition (6) is satisfied we can for large N describe the evolution of the macroscopic probability density by a Liouville equation:

~=1

Expansion (14) is to be interpreted in a distributional sense, i.e. only to be used in expressions of the form f df~ ~ t ( ~ ) G ( ~ ) with smooth functions G(f~), so that all derivatives are well-defined and finite. Furthermore, (4) will only be useful if the Aj,, which measure the sensitivity of the macroscopic quantities to single neuron state changes, are sufficiently small. This is to be expected: for finite N any observable can only assume a finite number of possible values; only for N ~ oc may we expect smooth probability distributions for our macroscopic quantities.

626

A.C.C. Coolen

whose solution describes deterministic flow: ~t[~] = f d~0~0[~o]5[l'~- ~(t]~o)] with ~(t[~o) given, in turn, as the solution of d ~(t) - r ( l ) [ ~ ' ~ ( / ) ; t] dt

n(0) = n0.

(8)

In taking the limit N ~ ~ , however, we have to keep in mind that the resulting deterministic theory is obtained by taking this limit for finite t. According to (4) the g > 1 terms do come into play for sufficiently large times t; for N ~ ~ , however, these times diverge by virtue of (6).

2.1.3. The issue of closure Eq. (8) will in general not be autonomous; tracing back the origin of the explicit time dependence in the right-hand side of (8) one finds that to calculate F (1) one needs to know the microscopic probability density pt(a). This, in turn, requires solving Eq. (1) (which is exactly what one tries to avoid). We will now discuss a mechanism via which to eliminate the offending explicit time dependence, and to turn the observables ~ ( a ) into an autonomous level of description, governed by closed dynamic laws. The idea is to choose the observables ~ ( a ) in such a way that there is no explicit time dependence in the flow field F(~)[~; t] (if possible). According to (5) this implies making sure that there exist functions ~ [ ~ ] such that N

lim ~

N--- vc

wj(a)Aj~(a) - ~,[~(a)]

(9)

j=l

in which case the time dependence of F (~) indeed drops out and the macroscopic state vector simply evolves in time according to: d

dt ~ - ~[1"~],

~[~'~] = ((I) 1 [ ~ ' ~ ] , . . . ,

(I)n[~"~]).

Clearly, for this closure method to apply, a suitable separable structure of the synaptic matrix is required. If, for instance, the macroscopic observables ~ depend linearly on the microscopic state variables a (i.e. ~ ( a ) - ~ ~-~'~x_1C%jOj), we obtain with the transition rates defined in (1): d _f l y _

dt

lim 1 N N----~vc N Z j=l

c%j tanh([3hj(a)) - ~

(10)

in which case the only further condition for (9) to hold is that all local fields hk(a) must (in leading order in N) depend on the microscopic state a only through the values of the observables ~; since the local fields depend linearly on a this, in turn, implies that the synaptic matrix must be separable: if Jij = Y-]~Ki, c%j then indeed hi(o) = ~-2~K i ~ ( a ) + Oi. Next I will show how this approach can be applied to

Stat&tical mechanics of recurrent neural networks H - dynamics

627

networks for which the matrix of synapses has a separable form (which includes most symmetric and nonsymmetric Hebbian type attractor models). I will restrict myself to models with 0 / = 0; introducing nonzero thresholds is straightforward and does not pose new problems.

2.2. Application to separable attractor networks 2.2.1. Separable models: description at the level of sublattice activities We consider the following class of models, in which the interaction matrix has the form

1

Jij - ~ Q(~i; ~j),

~i - ( ~ , . . . , ~P).

(11)

The components ~ , representing the information ('patterns') to be stored or processed, are assumed to be drawn from a finite discrete set A, containing nA elements (they are not allowed to depend on N). The Hopfield model [2] corresponds to choosing Q(x; y ) = x . y and A = { - 1 , 1}. One now introduces a partition of the system {1,... ,N} into rflA so-called sublattices In:

{1,...,N}-UIn,

In-{i]{ i-R},

11cA p.

(12)

n

The number of neurons in sublattice I n is denoted by ]In[ (this number will have to be large). If we choose as our macroscopic observables the average activities ('magnetisations') within these sublattices, we are able to express the local fields hk solely in terms of macroscopic quantities:

1

mn(~) - - ~ ] i~ci cyi,

hk(~) - ZPnQ({k;q)mnn

(13)

with the relative sublattice sizes Pn = ]In]IN. If all Pn are of the same order in N (which, for example, is the case if the vectors {i have been drawn at random from the set Ap) we may write Ayn = Cg(nPAN - l ) and use (7). The evolution in time of the sublattice activities is then found to be deterministic in the N ~ e~ limit if limx~p/logN = 0. Furthermore, condition (9) holds, since ~

wj(~)Ajn(~ ) - tanh ~ ~-~Pn'O(ll; II')mn'

j=l

- ran.

n'

We may conclude that the situation is that described by (10), and that the evolution in time of the sublattice activities is governed by the following autonomous set of differential Eqs. [3]:

d

[

]

dt mn - tanh ]3Z p n , Q(ll; II')m n, - m n !!'

(14)

A.C.C. Coolen

628

We see that, in contrast to the equilibrium techniques as described in [1], here there is no need at all to require symmetry of the interaction matrix or absence of selfinteractions. In the symmetric case Q ( x ; y ) = Q(y;x) the system will approach equilibrium; if the kernel Q is positive definite this can be shown, for instance, by inspection of the Lyapunov function 3 5t'{mn}: ~c~a{mn} - 2l ZPnmnQ(R;nn'

R')mn'Pn' - -~1Zpnlogcosh[~y-~Q(q;q,)mn,pn,] n n'

which is bounded from below and obeys: dso__ dt

Z

n ct-tmn Q(R;

)

n' ~ m n '

0 we have in (52) an additional Gaussian noise, absent in the models with binary neurons. Again the pure states are the first nontrivial solutions to enter the stage. Substituting m, = mg,v into (52) gives m

-

f Dz tanh[3'(m + zV/-T)].

(53)

Writing (53) as mZ-3'm~odk[1-fDztanh2[3'(k+zx/-T)]] 0 at T~ = 2/re ~ 0.637. Alternatively, the latter number can also be found by taking limx+oo T(x) in the above parametrization: rc(V -

- xlim x2 1 -

[j" - li+rn

Dz tanh2(zx)

d ]2 [ / ]2 D Zdzz tanh(zx) - 2 Dz 6(z) - Z/re.

644

A. C. C. Coolen

1.0

P T

0.5

0.0

9 0.0

O.5

1.0

0 1.5

0.0

0.3

u~ 0.6

0.9

1/v Fig. 6. Left: phase diagram of the Hopfield model with graded-response neurons and (Z/N) y'~o~,.~~j, o away from saturation. P: paramagnetic phase, no recall. R: pattern recall phase. Solid line: separation of the above phases, marked by a continuous transition. Right: asymptotic recall amplitudes m = (2/N) ~i ~/si of pure states (defined such that full recall corresponds to m = 1), as functions of the noise level T, for ?-l E {0.1,0.2,..., 0.8, 0.9} (from top to bottom).

Jij =

The resulting picture of the network's stationary state properties is illustrated in Fig. 6, which shows the phase diagram and the stationary recall overlaps of the pure states, obtained by numerical calculation and solution of Eqs. (54) and (53). Let us now turn to dynamics. It follows from (52) that the 'natural' initial conditions for ~ and ~ are of the form: ~ ( 0 ) = q. k0 and Z~(0) = Z0 for all q. Equivalently: t = 0"

p,l(u) = [2 rtE~]-' e- 89[u-'lk~

ko E NP, Eo C N.

These would also be the typical and natural statistics if we were to prepare an initial firing state {si} by hand, via manipulation of the potentials {ui}. For such initial conditions we can simplify the dynamical Eq. (51) to

m~t(t)--(~t/Dztanh [?(~-[koe -t +Afotdse~-tm(s)] +zv/T+(EZ-T)e-Zt)])

.

(55)

For the special case of the Hopfield synapses, i.e. A~v - 8~v, it follows from (55) that recall of a given pattern v is triggered upon choosing ko.r, -- koS~,, (with k0 > 0), since then Eq. (55) generates rn~,(t)= m(t)8~v at any time, with the amplitude m(t) following from

m(t) - /Dztanh[y[koe-'+

ftdse'-tm(s)+zv/T+(EZ-T)e-Zt]

1

(56)

which is the dynamical counterpart of Eq. (53) (to which indeed it reduces for t ----+ CO).


645

1.0 I

m 0.5

0.0

0

10

20

30

Fig. 7. Overlap evolution in the Hopfield model with graded-response neurons and Jij = (2/N) ~--~~t~i~j, ~ ~ away from saturation. Gain parameter: y = 4. Initial conditions pn(u) = 8[u- k0rlv] (i.e. triggering recall of pattern v, with uniform membrane potentials within sublattices). Lines: recall amplitudes m = (2/N) ~ i ~si of pure state v as functions of time, for T = 0.25 (upper set), T = 0.5 (middle set) and T = 0.75 (lower set), following different initial overlaps m0 6 {0.1,0.2,..., 0.8, 0.9}.

We finally specialize further to the case where our Gaussian initial conditions are not only chosen to trigger recall of a single pattern ~v, but in addition describe uniform membrane potentials within the sublattices, i.e. k0,~ = k08~v and E0 = 0, so pn(u) = 8 [ u - k 0 q v ] . Here we can derive from (56) at t = 0 the identity m0 = tanh[yk0], which enables us to express k0 as k0 = (27) -1 log[(1 + m0)/(1 - m0)], and find (56) reducing to

m(t)--j'Dztanh[e-tl~ I+I -m:]89 "

z~/T(l_ e_Zt)-] .] (57)

Solving this equation numerically leads to graphs such as those shown in Fig. 7 for the choice y = 4 and T C {0.25, 0.5, 0.75}. C o m p a r e d to the overlap evolution in large networks of binary networks (away from saturation) one immediately observes richer behavior, e.g. nonmonotonicity. The analysis and results described in this section, which can be done and derived in a similar fashion for other networks with continuous units (such as coupled oscillators), are somewhat difficult to find in research papers. There are two reasons for this. Firstly, nonequilibrium statistical mechanical studies only started being carried out around 1988, and obviously concentrated at first on the (simpler) networks with binary variables. Secondly, due to the absence of detailed balance in networks of graded response networks, the latter appear to have been suspected of consequently having highly complicated dynamics, and analysis terminated with pseudo-equilibrium studies [11]. In retrospect that turns out to have been too pessimistic a view on the power of nonequilibrium statistical mechanics: one finds that

A. C.C. Coolen

646

dynamical tools can be applied without serious technical problems (although the calculations are somewhat more involved), and again yield interesting and explicit results in the form of phase diagrams and dynamical curves for macroscopic observables, with sensible physical interpretations. 4. Correlation and response functions

We now turn to correlation functions Cij(t, t') and response functions Gij(t, t'). These will become the language in which the generating functional methods are formulated, which will enable us to solve the dynamics of recurrent networks in the (complex) regime near saturation (we take t > t'):

Cij(t,{) = (~i(t)cyj(t')),

(58)

Gij(t,t') = O(~i(t))/OOj({).

The {cy~} evolve in time according to equations of the form (1) (binary neurons, sequential updates), (22) (binary neurons, parallel updates) or (32) (continuous neurons). The 0~ represent thresholds and/or external stimuli, which are added to the local fields in the cases (1) and (22), or added to the deterministic forces in the case of a Fokker-Planck Eq. (32). We retain 0;(t) = 0/, except for a perturbation 80j(t') applied at time t' in defining the response function. Calculating averages such as (58) requires determining joint probability distributions involving neuron states at different times.

4.1. Fluctuation-dissipation theorems 4.1.1. Networks of binary neurons For networks of binary neurons with discrete time dynamics of the form p~+~(t~) = y]~, W[~;tr162 the probability of observing a given 'path' t~(g') t~(g' + 1) ~ -.. ~ ty(g- 1) ~ t~(g) of successive configurations between step g' and step g is given by the product of the corresponding transition matrix elements (without summation): Prob[t~(g'),..., t~(g)] = W[tr(g); ~(g - 1)] W[t~(g - 1); t~(g - 2)]... x W[t~(( + 1); t~(()]pe(tT(()). This allows us to write

Cij(g, g') - Z

" "Z Prob[t~(g'),..., t~(g)]cyi(g)cyj(g') - Z

~(t')

~(t)

c~c~ Wt-e' [tT;t~']pe (re),

~' (59) (60)

~ t ~tt

From (59) and (60) it follows that both Cij(g, g') and G;j(g, g') will in the stationary state, i.e. upon substituting pe,(tT') =p~(t~'), only depend on g - g ' : Ciy(g,g') Cij(g- g') and Gij(g, g ' ) ~ Giy(g- g'). For this we do not require detailed bal-

Statisticalmechanicsof recurrentneuralnetworksH- dynamics

647

ance. Detailed balance, however, leads to a simple relation between the response function G~i('c) and the temporal derivative of the correlation function Gj(z). We now turn to equilibrium systems, i.e. networks with symmetric synapses (and with all Jii "- 0 in the case of sequential dynamics). We calculate the derivative of the transition matrix that occurs in (60) by differentiating the equilibrium condition Peq(a) = ~--2.~,W[a; aqPeq(a') with respect to external fields:

~Oj Pen (a) -- Z~t

{~w[.;~'] - -~Oj

Peq (a t) -+- W[a; a t] ~-0-TPeq(a t)

9

Detailed balance implies P e q ( a ) - Z - l e -13/4('~) (in the parallel case we simply substitute the appropriate Hamiltonian H -+ H), giving 8Peq(a)/80j -[Z-ISZ/~Oj + f38H(6)/8Oj]Peq(6), so that

Peq(at) -- 13 Z6, m[a; a t] OH(at) ~0j Peq( at ) -- ~~H(a) - - ~ j Peq(6)

~, - -~0j

(the term containing Z drops out). We now obtain for the response function (60) in equilibrium:

Gij(g)-[3ZcyiWg-I[6;gq(ZW[6';g"]~H(6")(6"~H(6~) } p) e q ••,

,,,

O0j

O_s peq(a')

" (61)

The structure of (61) is similar to what follows upon calculating the evolution of the equilibrium correlation function (59) in a single iteration step:

Cij(g) - Cij(g -1) -- Za~r,(yiwg-l [a;at] ( Za. W[6';~"]cy'fPeq(a")-

(3"j.Peq(at) }. (62)

Finally we calculate the relevant derivatives of the two Hamiltonians H(6) -- - ~ i < j J;jeyicyj - ~ i 0icyi and / t ( a ) - - y~'~;0;cy;- 13-1 ~ log 2 cosh[13h;(6)] (with h~(6) - ~jJ;jcyj + 0z), see [1]:

8H(6)/8Oj - -%-,

8/t(~r)/80j - -c~j - tanh[13hj(~r)].

For sequential dynamics we hereby arrive directly at a FDT. For parallel dynamics we need one more identity (which follows from the definition of the transition matrix in (22) and the detailed balance property) to transform the tanh occurring in the derivative of H:

tanh[~hj(g')]Peq(a') - Z girt

cY~~W[a''; 6']Peq(a') - Z

W[6';

att]cyfPeq(att).

~tt

For parallel dynamics g and g' are the real time labels t and t', and we obtain, with

z --t-f:

648

A . C. C. C o o l e n

Binary & Parallel: Gij(r, > O) = -~[Cij(r, + 1) - Cij(r, - 1)],

Gij(r ~< 0) = 0.

(63)

For the continuous-time version (1) of sequential dynamics the time t is defined as and the difference equation (62) becomes a differential equation. For perturbations at time t' in the definition of the response function (60) to retain a nonvanishing effect at (re-scaled) time t in the limit N ~ c~, they will have to be rescaled as well: 60j(t') ~ NSOj(t~). As a result: Binary & Sequential: t = g/N,

a,j(~) -

- f 3 0 ( ~ ) ~d

Cij('c) .

(64)

The need to re-scale perturbations in making the transition from discrete to continuous times has the same origin as the need to re-scale the random forces in the derivation of the continuous-time Langevin equation from a discrete-time process. Going from ordinary derivatives to functional derivatives (which is what happens in the continuous-time limit), implies replacing Kronecker delta's ~t,t' by Dirac deltafunctions according to ~)t,t' "-+ A S ( t - t'), where A is the average duration of an iteration step. Eqs. (63) and (64) are examples of so-called fluctuation-dissipation theorems (FDT). 4.1.2. N e t w o r k s with continuous neurons

For systems described by a Fokker-Planck Eq. (32) the simplest way to calculate correlation and response functions is by first returning to the underlying discrete-time system and leaving the continuous time limit A ~ 0 until the end. In [1] we saw that for small but finite time-steps A the underlying discrete-time process is described by t - - gA,

pgA+A(6) -- [1 + A&~f'n+ 6O(A3)]ptA(6)

with g - 0, 1,2,... and with the differential operator (65)

l

From this it follows that the conditional probability density pea(~l~', CA) for finding state 6 at time s given the system was in state 6' at time g'A, must be peA (616', g'A ) -- [1 + A5% + (9(A3)]t-e'816 - 6'].

(66)

Eq. (66) will be our main building block. Firstly, we will calculate the correlations:

Cij(gA, gtm)

--

(o'i(~A)o'j(gtA))=

S d6d6' erioj.peA(6i6', g'A)pea(6' )

= S d6 cyi[l + A~f'a +

o(A)l' "f d6' or} 8[6-6']pe, A(6')

-- j" d6 trill + A:LP~+ C(A~)]g-t' [6jpg, A(6)].

649


At this stage we can take the limits A ~ 0 and g, g' ~ oe, with t = gA and t' - gA finite, using limA_0[1 + AA]k/a -- ekA"

Cij(t, t') - / d a ~i e (t-t')-~ [c~jpt,(a)].

(67)

Next we turn to the response function. A perturbation applied at time t' = g'A to the Langevin forces f-(a) comes in at the transition a(g'A) ---, a(g'A + A). As with sequential dynamics binary networks, the perturbation is re-scaled with the step size A to retain significance as A ~ 0:

Gij(gA, gtA) - AO0j(g'A) = AS0j(g tA) a d a d a ' (y, pea(a[at, etA)pea(a ') -

-

f

d a d a t da t' (yipga(alat',gtA + A)[

- [da

g'a)]

pea(a t)

da' da" cyi[1 + A5% + (9(A~)]g-e-18[a - a"]

x

[1 +

+

"-

pe,

- - f da d~ tda 't (Yi[1 -71--A~Pa .qt_ (fi(A~)] g-g'-I ~[ff _ fftt]~[fftt _ at] X -

~-Jr-

-/da

(Q(A '-] 2) p/~tA({]r, )

cyi[i + A ~ , + (9(A~)]e-e-' [~-~s

+ e ( a l)

J

We take the limits A ~ 0 and g, g' ~ ~ , with t = gA and t' = gtA finite:

Gij(t, t') - - / d a cyi e (t-t')~ &yjpt,(a). ~

(68)

Eqs. (67) and (68) apply to arbitrary systems described by Fokker-Planck equations. In the case of conservative forces, i . e . f . ( ~ ) = - ~ H ( ~ ) / ~ c y i , and when the system is in an equilibrium state at time t' so that C~j(t,t')= Cij(t-t') and Gij(t, t') = Gij(t- t'), we can take a further step using pt,(~) --Peq(~) -- Z -1 e -~/4(~). In that case, taking the time derivative of expression (67) gives O---~Cij(T,) --

f

dl~ (y i e ~ ~

~~ [o'jPeq (~)] 9

Working out the key term in this expression gives ~G[~jPeq(~)] = - ~ - ~ 5-~ l

(~)- T

[cYjJi(~)] i

[~jPeq(~)] -- Th-~ Peq(~)

650

A.C.C. Coolen

with the components of the probability current density J;(6) - [J~(6) - T ~-~7]Peq(6) In equilibrium, however, the current is zero by definition, so only the first term in the above expression survives. Insertion into our previous equation for OCij(r)/O~, and comparison with (68) leads to the FDT for continuous systems: Continuous: d Gij( ~) - - [30( ~) - ~ Cij( ~) .

(69)

We will now calculate the correlation and response functions explicitly, and verify the validity or otherwise of the FDT relations, for attractor networks away from saturation. 4.2. Example: simple attractor networks with binary neurons 4.2.1. Correlation and response functions f o r sequential dynamics We will consider the continuous time version (1) of the sequential dynamics, with the local fields hi(6) = ~ j J i j ~ j + 0i, and the separable interaction matrix (18). We already solved the dynamics of this model for the case with zero external fields and away from saturation (i.e. p .

(74)

Similarly we obtain from the solution of (71) an equation for the leading order in N of the response functions, by derivation with respect to external fields: O l m~(r with m~(r - N -1 ~ i ~/~(Yi, which are both found to involve Pt(z): dt m =

dz Pt(z) tanh[13(m + z)],

~ r - -~

dzPt(z)ztanh[~(m + z)] + 1 - r.

Next one closes these equations by hand, using a maximum-entropy (or 'Occam's Razor') argument: instead of calculating Pt(z) from (94) with the real (unknown) microscopic distribution pt(r it is calculated upon assigning equal probabilities to all states a with re(a) = m and r(a) = r, followed by averaging over all realizations of the stored patterns with ~t > 1. In order words: one assumes (i) that the microscopic states visited by the system are 'typical' within the appropriate (m,r) subshells of state space, and (ii) that one can average over the disorder. Assumption (ii) is harmless, the most important step is (i). This procedure results in an explicit (nonGaussian) expression for the noise distribution in terms of (re, r) only, a closed two-parameter theory which is exact for short times and in equilibrium, accurate predictions of the macroscopic flow in the (m, r)-plane (such as that shown in Fig. 5 of [1]), but (again) deviations in predicted time dependencies at intermediate times. This theory, and its performance, was later improved by applying the same ideas to a derivation of a dynamic equation for the function Pt (z) itself (rather than for m and r only) [19]; research is still under way with the aim to construct a theory along these lines which is fully exact.

A.C.C. Coolen

658

5.1.3. Exact results: generat&g functional analysis The only fully exact procedure available at present is known under various names, such as 'generating functional analysis', 'path integral formalism' or 'dynamic meanfield theory', and is based on a philosophy different from those described so far. Rather than working with the probability pt(~) of finding a microscopic state tT at time t in order to calculate the statistics of a set of macroscopic observables fl(t~) at time t, one here turns to the probability Probity(0),..., t~(tm)] of finding a microscopic path t ~ ( 0 ) ~ t ~ ( 1 ) 7 . . . ~ t~(tm). One also adds time-dependent external sources to the local fields, hi(ty) ~ h;(t~) + Oi(t), in order to probe the networks via perturbations and define a response function. The idea is to concentrate on the moment generating function Z[~], which, like Prob[t~(0),..., t~(tm)], fully captures the statistics of paths:

Z[,l,] - ( e-' E, :g:"_-'o

(96)

It generates averages of the relevant observables, including those involving neuron states at different times, such as correlation functions Cij(t,t')= (cyi(t)cyj(t')) and response functions Gij(t,t')= i~(~i(t))/~Oj(t'), upon differentiation with respect to the dummy variables {~i(t)}:

(cyi(t))

= i lim ~

'

Cij(t, t')

= -lim (97)

Gij(t, t')

- i lim

Next one assumes (correctly) that for N ~ c~ only the statistical properties of the stored patterns will influence the macroscopic quantities, so that the generating function Z[~] can be averaged over all pattern realizations, i.e. Z[~] ~ Z[~]. As in replica theories (the canonical tool to deal with complexity in equilibrium) one carries out the disorder average before the average over the statistics of the neuron states, resulting for N ~ ~ in what can be interpreted as a theory describing a single 'effective' binary neuron cy(t), with an effective local field h(t) and the dynamics Prob[cy(t + 1) - +1] - 89 + tanh[13h(t)]]. However, this effective local field is found to generally depend on past states of the neuron, and on zero-average but temporally correlated Gaussian noise contributions qb(t): h(t]{~}, {~}) -

re(t) + O(t) + a Z R ( t , { ) c y ( { )

+ x/~dp(t).

(98)

t' < t

The first comprehensive neural network studies along these lines, dealing with fully connected networks, were carried out in [20,21], followed by applications to a-symmetrically and symmetrically extremely diluted networks [22,23] (we will come back to those later). More recent applications include sequence processing networks [24]. 6 For N ~ e~ the differences between different models are found to show up only in the 6

In the case of sequence recall the overlap m is defined with respect to the 'moving' target, i.e. t m(t) = ~1 ~i c~i(t)~i.


659

actual form taken by the effective local field (98), i.e. in the dependence of the 'retarded self-interaction' kernel R(t, t') and the covariance matrix (dp(t)qb(t)) of the interference-induced Gaussian noise on the macroscopic objects C = { C ( s , s ' ) = l i m u ~ 1 ~-~i Cii( S, st) } and G - { G(s, s') - l i m N ~ 1 ~ i Gii( S, st) }" For instance: 7 Model

Synapses Jij

Fully connected, static patterns

1 ~-~1 ~Y

Fully connected, pattern sequence

g=l i

Symm extr diluted, static patterns

-7

Asymm extr diluted, static patterns

cij

~j

~i ~ ~c

c ~-'~t=l ~i~J g la

R(t,t')

@(t)~b(t'))

[(1 -- G)-lGl(t,t ')

[(1-G)-Ic(1-Gt)-II(t,t')

0

Y~'~._>o[(Gt)" CG"] (t. t ' )

G(t.t')

C(t.t')

0

C(t,t')

with the cij drawn at random according to P ( c i j ) - ~vScij,1 + (1-~v)Scij,0 (either symmetrically, i.e. cij = cji, or independently) and where cii = 0, l i m N ~ c / N = O, and c--+ ec. In all cases the observables (overlaps and correlation and response functions) are to be solved from the following closed equations, involving the statistics of the single effective neuron experiencing the field (98):

m(t) = (or(t)),

C(t,t') = (cy(t)cy(t')),

G(t,t')--~(cr(t))/~O(t').

(99)

It is now clear that Gaussian theories can at most produce exact results for asymmetric networks. Any degree of symmetry in the synapses is found to induce a nonzero retarded self-interaction, via the kernel K(t,t'), which constitutes a nonGaussian contribution to the local fields. Exact closed macroscopic theories apparently require a number of macroscopic observables which grows as C(t 2) in order to predict the dynamics up to time t. In the case of sequential dynamics the picture is found to be very similar to the one above; instead of discrete time labels t c {0, 1 , . . . , tin}, path summations and matrices, there one has a real time variable t c [0, tm], path-integrals and integral operators. The remainder of this paper is devoted to the derivation of the above results and their implications.

5.2. Generating functional analysis for binary neurons 5.2.1. General definitions I will now show more explicitly how the generating functional formalism works for networks of binary neurons. We define parallel dynamics, i.e. (22), driven as usual by local fields of the form hi(~; t) - }-~jJijcYj + Oi(t), but with a more general choice of Hebbian-type synapses, in which we allow for a possible random dilution (to reduce repetition in our subsequent derivations): p

Jij

7

cijv~--

- -- ~

c ~t=l

~ ~t

~i ~j ,

p - ~c.

(100)

In the case of extremely diluted models the structure variables are also treated as disorder, and thus averaged out.

A. C. C. Coolen

660

Architectural properties are reflected in the variables c/j E {0, 1}, whereas information storage is to be effected by the remainder in (100), involving p randomly and independently drawn patterns { ~ - ( ~ , . . ' , ~ , ~ u ) C {--1, 1}N. I will deal both with symmetric and with asymmetric architectures (always putting c,/= 0), in which the variables c;j are drawn randomly according to Symmetric:

cij = cji,

Vi < j

c

P(cij) = -~ 8c~j.1 +

1 - -~ 8c~j,o.

(101)

Asymmetric:

V i =fi j

P(cij) = ~ 8~,j.1 +

( 1c-) ~

8~,.~..0

(102)

(one could also study intermediate degrees of symmetry; this would involve only simple adaptations). Thus ckl is statistically independent of cij as soon as (k, l)~ { (i, j), (j, i) }. In leading order in N one has ( ~ j cij) = c for all i, so c gives the average number of neurons contributing to the field of any given neuron. In view of this, the number p of patterns to be stored can be expected to scale as p = ~c. The connectivity parameter c is chosen to diverge with N, i.e. limu__,~ c-1 _____0. If C = N we obtain the fully connected (parallel dynamics) Hopfield model. Extremely diluted networks are obtained when limN_~ c / N = O. For simplicity we make the so-called 'condensed ansatz': we assume that the system state has an C(N ~ overlap only with a single pattern, say g = 1. This situation is induced by initial conditions: we take a randomly drawn ~(0), generated by 1[ 1 -mo]8~,(o),_~] }

I~.{ 1 [1 +

(103)

1 so

m0 i

The patterns ~t > 1, as well as the architecture variables cij, are viewed as disorder. One assumes that for N ~ oc the macroscopic behaviour of the system is 'selfaveraging', i.e. only dependent on the statistical properties of the disorder (rather than on its microscopic realisation). Averages over the disorder are written as =-=.-. We next define the disorder-averaged generating function: Z[l[/]

~

(e-i~-~i~-~t r

(104)

in which the time t runs from t - 0 to some (finite) upper limit tm. Note that Z[0] = 1. With a modest amount of foresight we define the macroscopic site-averaged and disorder-averaged objects re(t) - N -1 }-~i ~ (cri(t)), C(t, t') - N -1 }-~i (~i(t)cyi(t')) and G(t,t') = N -1 ~i~(cyi(t))/~Oi(t'). According to (97) they can be obtained from (104) as follows:

m(t) = lim i ~Z[~J ,~o "N ~. ~j 8~j(t)' J

(105)

661

Statistical mechanics of recurrent neural networks H- dynamics

C(t,

{)

-

-

lim 1 ~

G(t,{)-

~2~[~]

,+oNZT-'. ~ j ( t ) ~ j ( t ' ) '

lim i /~.

~2Z[~]

~,+o-N . ~%(t)~Oj(t')"

(106)

So far we have only reduced our problem to the calculation of the function Z[~] in (104), which will play a part similar to that of the disorder-averaged free energy in equilibrium calculations (see [1]).

5.2.2. Evaluationof the disorder-averagedgeneratingfunction As in equilibrium replica calculations, the hope is that progress can be made by carrying out the disorder averages first. In equilibrium calculations we use the replica trick to convert our disorder averages into feasible ones; here the idea is to isolate the local fields at different times and different sites by inserting appropriate ~5-distributions:

l- Hit/dhi(t)~)[hi(t)- Z Jijcyj(t)- 0i(t)] 9

j

- f{dhdh}exp(i~it [ti(t)[hi(t)-~j Jijcrj(t)-Oi(t)]) with {dh dh} -

I-Iit[dhi(t)dhi(t)/2rc],giving

Z[~/] -- f{dhd]l}eiEithi(t)[hi(t)-~

J~

)pf

in which (...)pf refers to averages over a constrained stochastic process of the type (22), but with prescribed fields at all sites and at all times. Note that with such prescribed fields the probability of generating a path {6(0),..., 6(tm)} is given by

{hi(t)}

6(tm)l{hi(t)}] =P(6(O))exp(Z[f3cyi(t 1)hi(t)-log2cosh[f3hi(t)]])

Prob[6(0),...,

SO

z[,] = f{dh dh}

a(tmP(6(0))eNg[{~}'{h}] )~ Hexp(i/ai(t)[hi(t)it--0i(t)]

--iqti(t)(Yi(t) + ~(Yi(t + l )hi(t) - l o g 2 cosh[~hi(t)])

(107)

with 1 o~[(.}, {h}] = ~log

[e_i~-~it~i(t)~-~/ji/cy/(t) ]

(10s)

662

A.C.C.

Coolen

We concentrate on the term o~[...] (with the disorder), of which we need only know the limit N ~ oc, since only terms inside Z[O] which are exponential in N will retain statistical relevance. In the disorder-average of (108) every site i plays an equivalent role, so the leading order in N of (108) should depend only on site-averaged functions of the {c~i(t),[~i(t)}, with no reference to any special direction except the one defined by pattern {l. The simplest such functions with a single time variable are

a(t; (6}) = - ~1 Z ~ ] c Y i ( t ) ,

k(t; {h)) = ~1Z

i

~l[~i(t) i ,

(109)

i

whereas the simplest ones with two time variables would appear to be 1

q(t, t'; {6}) -- ~ Z ~i(t)CYi(t'),

1

Q(t,t'; {h}) - ~ ~

[~i(t)hi(t'),

x(t,t';

h}) =

(110)

i

i

1

(111) i

It will turn out that all models of the type (100), with either (101) or (102), have the crucial property that (109-111) are in fact the only functions to appear in the leading order of (108): o~[...] - ~ [ { a ( t ; . . . ) , k ( t ; . . . ) , q ( t , t ' ; . . . ) , Q(t,t';...),K(t,t';...)}]

+ . - . (U --~ oc) (112)

for some as yet unknown function q)[...]. This allows us to proceed with the evaluation of (107). We can achieve site factorization in (107) if we isolate the macroscopic objects (109-111) by introducing suitable 8-distributions (taking care that all exponents scale linearly with N, to secure statistical relevance). Thus we insert

lm/

1 - I-I

da(t)

[aItl

-

lt;

t=O

--

a(t) a(t) - -~ Z. ~J cyj(t)

da da exp iN

tmj 1 - II

,

J

dk(t)8[k(t) - k(t; {h})]

t=O

=

N

dkdlcexp

iN~/r

(t)-~

. ~/~j(t)

,

tm / 1 -

H

dq(t,t'l

[q(t,t'l

- q(t,t';

t,d = 0

--

dq dq exp iN Z t,#

O(t' t') q(t, t') - ~ ~

Gj(t)cyj(t') j

,

663


1 -- H

dQ(t, {)8[Q(t, {) - Q(t, {; (h})]

t,tt =O

/

(

[

1

j

1

j

dO d0 exp iN Z Q(t, {) Q(t, {) - ~ Z hj(t)l~j({)

,m/ dK(t, {)8[K(t, {) - K(t,

1= H

t,t ~

1),

{; {a, I]})]

t,t~=O

j

(

dK dl(exp iN Z Is t,t'

[

1).

t') K(t, t') - ~ Z l~j(t)cyj(t')

Insertion of these integrals into (107), followed by insertion of (112) and usage of the shorthand Via, fi, k, k, q, 61,Q, Q, K, K] - i Z[gt(t)a(t) + [c(t)k(t)] t

+ i Z[c)(t, {)q(t, {) + Q(t, {)Q(t, {) + I~(t, {)K(t, {)]

(113)

t ,fl

then leads us to Z[~]- f dadfidkdf~dqdqdQd0 dKdl~exp(NW[a, fi,k,l~,q,q, Q, I~,K,I~] + N*[a, k,q, Q,K] + (9(...)) j {dh dh} ,(~o)"""ff(/m)~P(*(O)) X Hexp(ihi(t)[hi(t) it

- 0 i ( t ) ] - iqli(t)(yi(t ) + ~(yi(t + 1 ) h i ( t ) - l o g 2 c o s h [ ~ 3 h i ( t ) ] )

x HexPi (-i~] Z[~(t)cyi(t)t

+ ~:(t)hi(t)]- i~-~[gl(t,')cyi(t)cyi(t')t,

+ Q(t,{)hi(t)hi({)+K(t,{)hi(t)cyi({)])

(114)

in which the term denoted as (9(...) covers both the nondominant orders in (108) and the (9(logN) relics of the various pre-factors [N/2rc] in the above integral representations of the 8-distributions (note: tm was assumed fixed). We now see explicitly in (114) that the summations and integrations over neuron states and local fields fully factorize over the N sites. A simple transformation {cyi(t),hi(t),fzi(t)) ---+ {~]cYi(t), ~lhi(t ) i , ~]/~/(t)} brings the result into the form

A. C. C. Coolen

664

/{dhdh} ~(~o)"""Z P ( g ( 0 ) ) "(tin) • H exp(ihi(t)[hi(t) -

~)0/(t)] -i~]~li(t)cyi(t )

it

-+-~i(t -+-1)hi(t) • I~exp (-i~]

log 2 cosh[~hi(t)])

~-'~[gt(t)cyi(t)t

+ k(t)]'li(t)]

--i Z[q(t't.t' t')~i(t)~i(t')

+ Q(t, t')hi(t)tti(t') + K(t, t')hi(t)~i(t')])

= eNE[fi.k.iI.Q .R]

with

1

E[fi, l ~ , t ] , 0 , 1 ( ] - ~ Z l o g i

/

{dhdh}

Z

n0(cy(0))

cy(0).-.~(tm)

• exp(Z{ih(t)[h(t)-~]Oi(t) ] i~]~i(t)c~(t)}) • exp ( Z{[3cy(tt + 1)h(t)- log2cosh[[3h(t)]}

-iZ[a(t)cy(t ) +/~(t)h(t)]- i Z[O(t, {)~(t)cy({) t

t,t'

+ Q(t, {)h(t)h({) + Is {)h(t)cy({)])

(115)

in which {dh dh} - I-I,[dh(t)dh(t)/2n] and no(Cy) - 89 [1 + m018~,, + 89 [1 - m018,~,-1. At this stage (114) acquires the form of an integral to be evaluated via the saddlepoint (or 'steepest descent') method: Z[{~(t) }] = f da dfi dk dl~ dq dq dQ dO dK dl~ eN{v[]+*[]+z[l}+e()

(116)

in which the functions V[...], (I)[...] and Z[...] are defined by (112), (113) and (115).

5.2.3. The saddle-point problem The disorder-averaged generating function (116) is for N ~ c~ dominated by the physical saddle-point of the macroscopic surface

T[a, fi, k, l~,q,(], Q, 0, K, I(] + ~[a,k,q, Q,K] + E[fi, i~,l], 0, I(]

(117)

with the three contributions defined in (112), (113) and (115). It will be advantageous at this stage to define the following effective measure (which will be further simplified later):

665


(f[{,,), {h), (~)]), 1

{f{dhd[~}~_,, -~

/'{dh d/~} ~-~'~(O)...,~(tm)M[{cy}, {h}, {h)] f[{cy}, {h), {/~}] f{dh d/~} 2,,(0)...o-(t.,)M[{~}, {h}, {h}] (124)

with

M[{~,}, {h), {~}] - ~0(~,(0)) x exp ( Z { i h ( t ) [ h ( t ) t

0(t)] + ]3cy(t+ 1)h(t)

- l o g 2cosh[~h(t)]} - i Z [ g t ( t ) c r ( t ) +/c(t)/~(t)]) t t,tt

In summary our saddle-point equations are given by (119)-(122), and the physical meaning of the macroscopic quantities is given by (123) (apparently many of them must be zero). Our final task is finding (112), i.e. calculating the leading order of ~-[{~}, {h}]

-

1 log

~

[e_i~-~it~i(t)~-~yj~jcyj(t)]

(125)

which is where the properties of the synapses (100) come in.

5.3. Parallel dynamics Hopfield model near saturation 5.3.1. The disorder average The fully connected Hopfield [2] network (here with parallel dynamics) is obtained upon choosing c = N in the recipe (100), i.e. cij = 1 - 6ij and p = adV. The disorder

668

A. C. C. Coolen

average thus involves only the patterns with ~t > 1. In view of our objective to write (125) in the form (112), we will substitute the observables defined in (109)-(111) whenever possible. Now (125) gives ~[...] : ~log

xp(-iN-' Z

Z t

Z

~t

{~{~h,(t)c~j(t))

iT~j

: iotE K(t,t;{~, h}) - iZ a(t)k(t) t

t

+~176

[~i~,ihi(t)/v/N][~i~,icYi(t)/x/~])]+(9(N-m)" (126)

We concentrate on the last term:

=

/

[ (--~

dx dy d~ d~ ei[x.x+y.y_x.y] exp (271;) 2(t''+l)

_J'dxdyd~d, ( (2 ~-)-2(-3-~-m-~iiexp i[R-x + ~ ' . y -

"

)]

~iE[Yc(t)~i(t) + f~(t)[~i(t)] t

x. y]

+~l~176

_-/dxdYd~d~ ( i[R-x + ~-y _ x (2 ~)2(tm+l) e x p

21N ~,

y]

' [2(t)~i(t)+ f2(t)hi(t)]/ +(9(N -I))

_-fdxdyd'~d:~(

(2X)2(t,,,+l) exp i [ i - x + ~,.y

_ x . y] _'-2Z[x(t)2(t')q(t't')t.t'

+ 22(t)fi(t')K(t',t) + ,f(t)fi(t')O(t,t')] + (9(N-1)). Together with (126) we have now shown that the disorder average (125) is indeed, in leading order in N, of the form (112) (as claimed), with

669


O[a, k, q, Q, K] - is Z

K(t, t) - ia. k + a log / dx dy dz~ d~, t

( 2 n ) 2(tm+l)

( i[R.x+~'.y-x.y]-~

•

1 [~: " q:~ + 2 ' " K:~ + ~' " Q-9])

=i~ZK(t,t)-ia.k t

+ otlog

/

dudv ( l[u.qu+2v.Ku_2iu.v+v. (2 T~)tm-~-I exp - ~

Qv]) (127)

(which, of course, can be simplified further). 5.3.2. Simplification o f the saddle-point equations We are now in a position to work out Eqs. (119) and (120). For the single-time observables this gives ci(t) - k(t) and/~(t) - a(t), and for the two-time ones: 1

f du dv u(t)u(t') e x p ( - 89[u .qu § 2 v. Ku - 2 iu. v + v. Qv]) f du dv e x p ( - 89[u .qu + 2 v. Ku - 2 iu. v § v. Qv])

1

f du dv v(t)v(t') e x p ( - 89[u-qu + 2 v. Ku - 2 iu- v + v. Qv]) fdudvexp(- 89 + 2v. K u - 2iu. v + v. Qv])

(l(t, t') -- -- -~ ~i

Q( t, t') -- - -~ oti

Is

f du dv v(t)u(t') e x p ( - 89[u .qu + 2 v. Ku - 2 iu. v + v. Qv]) {) -- - ~ i ~ f du dv e x p ( - 89[u .qu + 2 v- Ku - 2 iu. v + v. Qv])

-- Ot~t,t,.

At the physical saddle-point we can use (123) to express all nonzero objects in terms of the observables re(t), C(t, t') and G(t, t'), with a clear physical meaning. Thus we find fi(t) - 0,/c(t) - m(t), and 1 f du dv u(t)u(t') e - 89 O(t,t') -- --~o~i fdudve_l[,.cu_2iu.[l_G]v ] t

: 0

l[uCu 21u [1 G]v]

1 .fdudvv(t)v(t)e-~ " - " Q(t,t')---ou ~ - - - - y . ~ 2 f d u d v e-~ [ucu-2Zu'[1-G]v]

=

1

~i

(128) [[(1-G)

1

- C ( 1 - G ~)

-1

](t,t') (129)

/~(t, t') + ~St,t' -- - ~ i f du dv v(t)u(t') e- 89[u'cu-2iu[1-G]v] : f du dve -l[u'Cu-2iu'[1-G]v]

- G) -1 (t, t') ( 30)

(with Gt (t, t ' ) - G(t', t), and using standard manipulations of Gaussian integrals). Note that we can use the identity (1 - G) -1 - 1 - ~e~>0 G e - 1 - Y~'~e>oGe-G ( 1 - G) -1 to compactify (130) to

A.C.C. Coolen

670

(131)

/C(t, t') -- cx[G(1 - G)-l](t, t').

We have now expressed all our objects in terms of the disorder-averaged recall overlap m = {m(t)} and the disorder-averaged single-site correlation and response functions C = {C(t,t')} and G = {G(t,t')}. We can next simplify the effective measure (124), which plays a crucial role in the remaining saddle-point equations. Inserting a(t) = O(t,t') = 0 and k(t) = re(t) into (124), first of all, gives us

+ [3cy(t+ 1)h(t) - l o g 2 cosh[[3h(t)]} -i~Q(t,t')h(t)h(t')).t.t, (132) Secondly, causality ensures that G(t,t')= 0 for t ~< t', from which, in combination with (131), it follows that the same must be true for the kernel/C(t, t'), since

IC(t,t') - cx[G(1 - G)-'](t, t') - cx{G + G 2 + G 3 + - - . } ( t , t'). This, in turn, guarantees that the function M[...] in (132) is already normalized: f{dhd/~}

Z M[{cy}, {h}, {/~}] - 1. ~(0)...o(t,,,)

One can prove this iteratively. After summation o v e r O(tm) (which due to causality cannot occur in the term with the kernel IC(t,t')) one is left with just a single occurrence of the field h(tm) in the exponent, integration over which reduces to 8[/~(tm)], which then eliminates the conjugate field h(tm). This cycle of operations is next applied to the variables at time tm - 1, etc. The effective measure (124) can now be written simply as

/{dhd~} M[{~},{h},{/~}]f[{cy), {h}, {/~)]

(f[{cy}, {h}, {/t}]), = ~(0)---~(t,,,)

with M[...] as given in (132). The remaining saddle-point equations to be solved, which can be slightly simplified by using the identity (cr(t)/~(t')), = iO(cy(t)),/OO(t'), are

m(t) = (o(t)),,

C(t,t') = (cy(t)cy(t')),,

G(t,t') = O(cy(t)),/OO(t').

(133)

5.3.3. Extracting the physics from the saddle-point equations At this stage we observe in (133) that we only need to insert functions of spin states into the effective measure (...), (rather than fields or conjugate fields), so the effective measure can again be simplified. Upon inserting (129) and (131) into the function (132) we obtain (f[{cy}]), - ~,~(0)...~(t~)Prob[{cy}]f[{cr}], with

Statistical mechanics o f recurrent neural networks H -

dynamics

671

Prob[{cy}]- ~0(c~(0))f{d~}P[{~}] tl-I[~[1 + cy(t+ l)tanh[]3h(t {cy}, {qb})]] (134) in which rCo(c~(O))- 1[1 + ~(O)mo], and h(tl{cy}, { , } ) - m(t) + O(t) + ~ Z [ G ( 1 - G)-'J(t, {)cy({) + ~ 89

(135)

tt 0

(14o)

C=I

Similarly we obtain from (1 - G ) -~ - 1 + R that for t' i> t: (1 - G ) - l ( t , t ') - St,t,. To suppress notation we will simply put h(tl..) instead of h(t]{~}, {~}); this need not cause any ambiguity. We notice that summation over neuron variables or(s) and integration over Gaussian variables ~(s) with time arguments s higher than those occurring in the function to be averaged can always be carried out immediately, giving (for t > 0 and t' < t):

re(t) -

~ ~0(cy(0)) f{d~}P[{~}] tanh[13h(t- 1[..)] Or o(0)...o(t- l) xU~t-21 [1 + cy(s + 1) tanh[13h(s[..)]]

(141)

s-----0

G(t,t')

~lC(t,t'+

1)-

t

~

~0(~(0))/{dr

11..)]

cy(0)...t~(t- 1 ) "x

'-2~ 1 [1 + c~(s + 1)tanh[]3h(s I..)]]~ x tanh[13h(t']..)] __ H s--0

(142)

J

(which we obtain directly for t' = t - 1, and which follows for times t' < t - 1 upon using the identity cy[1- tanh2(x)] = [1 + cytanh(x)][cytanh(x)]). For the correlations we distinguish between t ' - t - 1 and t' < t - 1:

C(t,t-1)-

Z

rt0(cr(0))

f {dr162

ll..)]

cy(0)...cy(t-2)

t-3 ~1 [1 + cy(s + 1) tanh[13h(s I"")]], z tanh[13h(t- 21..)] 1-I

(143)

s=0

whereas for t' < t - 1 we have

C(t,t') -

~ cy(0)...cy(t- 1)

rt0(cy(0)) f{dr

tanh[13h(t - 1]..)]cy(t')

,/

xU2t-21 [1 + cy(s + 1)tanh[13h(s I..)]]. s=O

(144)

673


Let us finally work out explicitly the final macroscopic laws (141)-(144), with (135) and (136), for the first few time steps. For arbitrary times our equations will have to be evaluated numerically; we will see below, however, that this can be done in an iterative (i.e. easy) manner. At t = 0 we just have the two observables m(0) = m0 and c ( 0 , 0) = 1.

5.3.4. The first few time-steps The field at t - 0 is h(0 ..) - m0 + 0(0) + a 89 since the retarded self-interaction does not yet come into play. The distribution of ~(0) is fully characterized by its variance, which (138) claims to be (qb2(0)) -- C(0, 0) - 1. Therefore, with Dz m(1)

-

(2re)-1 e -zz2 2 dz, we immediately find (141)-(144) reducing to

/ Dz tanh[J3(m0 + 0(0) + zx/~)],

C(1,0) - m0m(1),

G(1,0)-13{1-/DztanhR[~(mo+O(O)+zv~)]}.

(145)

(146)

For the self-interaction kernel this implies, using (140), that R(1,0) = G(1,0). We now move on to t - 2. Here Eqs. (141)-(144) give us

'

m(2)-5~--~ ~,(o)

/

d*(0)dqb(1)P[qb(0),qb(1)]tanh[13h(ll..)][1 +~(0)mo],

C(2, 1) - ~ ~ ,~(o)

dqb(1) dqb(0)P[qb(0), qb(1)] tanh[j3h(l[..)]

x tanh[13h(OI..)][1+ ~(O)mo],

'

C(2,0) = ~ ~

/

{dqb}P[{qb}]tanh[~h(ll..)]~(0 )

cy(0)cy(1)

x~l [1 + r G(2, 1) - 13 1 - ~ Z ~,(0)

tanh[13h(0[..)]][1 + r dqb(0)dqb(1)P[qb(0), .(1)]

x tanhZ[[3h(ll..)][1 + r

}, )

G(2, 0) - 13 C(2, 1) - ~ Z ~(o)

dqb(0) dqb(1)P[qb(0), qb(1)] tanh[13h(ll..)]

x tanh[13h(O[..)][1 + ~(O)mo] ~ - O.

J

674

A.C.C. Coolen

We already know that (~2(0)) - l; the remaining two moments we need in order to determine P[~(0), ~(1)] follow again from (138): 1

( ~ ( 1 ) ~ ( 0 ) ) -- Z [ ~ l . s --t- ~ o . s R ( 1 , O ) ] C ( s , O )

--

C(1,0) + G(1,0),

S---0 1

1

-

Z[

ls +

+

0R(1,0)]

s=O st:l

= G2(1,0)+ 2C(0, I)G(1,0)+ 1. We now know P[~(0), ~(1)] and can work out all macroscopic objects with t - 2 explicitly, if we wish. I will not do this here in full, but only point at the emerging pattern of all calculations at a given time t depending only on macroscopic quantities that have been calculated at times t' < t, which allows for iterative solution. Let us just work out m(2) explicitly, in order to compare the first two recall overlaps re(l) and m(2) with the values found in simulations and in approximate theories. We note that calculating m(2) only requires the field ~(1), for which we found (~)2(1)) -- O2(1,0) + 2C(0, 1)G(1,0) + 1"

1 /

m(2) - ~ ~ o(0)

d , ( l ) P [ , ( 1 ) ] tanh[13(m(1) + 0(1)

+ ~G(1,0)cy(0) + ocl~(1))] [1 + cy(0)mo] I

1[1 + mo] f Dz tanh[~(m(1) + 0(1) 2 + ocG(1,0) + zv/cx[G2 (1, 0) + 2 mom(1)G(1,0) + 1])] +21 [1

-

too] f Dz tanh[[3(m(1 ) + 0(1) - aG(1 0)

+ zV/~[G2 (1, 0) + 2 mom(1)G(1,0) + 1])].

5.3.5. Exact results versus simulations and gaussian approximations I close this section on the fully connected networks with a comparison of some of the approximate theories, the (exact) generating functional formalism, and numerical simulations, for the case 0 ( t ) = 0 (no external stimuli at any time). The evolution of the recall overlap in the first two time-steps has been described as follows:

Naive Gaussian Approximation: m(1) -- / Dz tanh[13(m(0) + zv/-~)], m(2) - / Dz tanh[13(m(1) + zv~)].


675

Amari-Maginu theory: m(1) - f Dztanh[13(m(0) + zx/~)], m(2) = f Dztanh[J3(m(1) + zErO)I, Z 2 = l + 2 m ( 0 ) m ( 1 ) G + G 2, G - 1311- J" DztanhZ[13(m(0)+zx/~)]

9

Exact solution: m(1) - f Dztanh[13(m(0) + zx/~)], 1

m(2) = ~

'

I1 +

m0] i Dztanh[13(m(1) + aG + z]~ v/-~)]

+~[1 -m0]

/ Dz t a n h [ 1 3 ( m ( 1 ) - ~ O + z Z v ~ ) l ,

E 2 = l + 2 m ( 0 ) m ( 1 ) G + G 2,

[1-/Oztanh I /o(0 +z ll. We can now appreciate why the more advanced Gaussian approximation (AmariMaginu theory, [14]) works well when the system state is close to the target attractor. This theory gets the moments of the Gaussian part of the interference noise distribution at t = 1 exactly right, but not the discrete part, whereas close to the attractor both the response function G(1,0) and one of the two pre-factors 1 [1 + m0] in the exact expression for m(2) will be very small, and the latter will therefore indeed approach a Gaussian shape. One can also see why the non-Gaussian approximation of[17] made sense: in the calculation of m(2) the interference noise distribution can indeed be written as the sum of two Gaussian ones (although for t > 2 this will cease to be true). Numerical evaluation of these expressions result in explicit predictions which can be tested against numerical simulations. This is done in Fig. 8, which confirms the picture sketched above, and hints that the performance of the Gaussian approximations is indeed worse for those initial conditions which fail to trigger pattern recall.

5.4. Extremely diluted attractor networks near saturation Extremely diluted attractor networks are obtained upon choosing limN~oo c / N = 0 (while still c ---+co) in definition (100) of the Hebbian-type synapses. The disorder average now involves both the patterns with l-t > 1 and the realization of the 'wiring' variables c;j c {0, 1}. Again, in working out the key function (125) we will show that for N ---, oc the outcome can be written in terms of the macroscopic quantities (109)(111). We carry out the average over the spatial structure variables {c;j} first:

676

A.C.C.

1.0

1.0

1.0

m 0.5

Coolen

0.5

0.5 q I 4 q I

0.0

0

1

2

0.0

3

0

1

t

2

0.0

3

0

t

1

2

3

t

Fig. 8. The first few time steps in the evolution of the overlap rn(o)= N -1 ~--]icr/~] in a parallel dynamics Hopfield model with 0t = T = 0.1 and random patterns, following initial states correlated with pattern one only. Left: simulations (o) versus naive Gaussian approximation (o). Middle: simulations (o) versus advanced Gaussian approximation (AmariMaginu theory, o). Right: simulations (o) versus (exact) generating functional theory (o). All simulations were done with N -- 30,000.

~[...] - ~log

exp

--Zcij C

~i~

[~i(t)c~j(t)

ir

At this stage we have to distinguish between symmetric and asymmetric dilutions.

5.4.1. The disorder average First we deal with the case of symmetric dilution: cij over the c/j, with the distribution (101), is trivial:

i 1" N- 89 and N- 89~ i ~p/~i(t) - (9(1). Thus the first term in the exponent containing the disorder is (9(c), contributing (9(c/N) to ~-[...]. We therefore retain only the second term in the exponent. However, the same argument applies to the second term. There all contributions can be seen as uncorrelated in leading order, so that ~ir ~gr .... (9(Np), giving a nonleading (9(N -1) cumulative contribution to ~ [ . . . ] . Thus, provided limN__+~c-1 = limN~c/N = 0 (which we assumed), we have shown that the disorder average (125) is again, in leading order in N, of the form (112) (as claimed), with

Symmetric: 9 [a, k, q, Q,K] = - i a . k -

1

-~aZ[q(s,t)Q(s,t) + K(s,t)K(t,s)].

(147)

st

Next we deal with the asymmetric case (102), where Again the average over the cij is trivial; here it gives

i

= 9

,

cij

and

cji

are independent.

1]}

678

A.C.C. Coolen

= ~. .

{ ~tC I~~ 1-~

r162

• 1-~

r162

1 [~ r162~thi(t)(yj(t) ]2-~-(~(C-2)3] }

hi(t)(yj(t)-~--~c2

r162

hj(t)cri(t)

+e(cq/

(in which the horizontal bars of the two constituent lines are to be read as connected) -- H e x p i<j

i -

r C/Z[hi(t)~j(t) + (t)

-

la

] - ~1

r162

]li(t)(Yj(t)

t

1

1

c

Again we separate in the exponent the terms where g - v in the quadratic term (being of the form Y~'~,v-..), and the terms with g - 1, and use the definitions (109)(111): 1

~[. . .] -- -i Z a(t)k(t) - -~ Z q(s, t)Q(s, t) + C(c- 89+ C(c/n) d.,

t

st

+ll~

[~i ~ihi(t)l [~j ~ ~j(t)] , i#j O#v

-~i ~j -~i Cj E hi(s)(YJ(S)hi(t)(YJ(t) st

)}

.

The scaling arguments given in the symmetric case, based on our 'condensed ansatz', apply again, and tell us that the remaining terms with the disorder are of vanishing order in N. We have again shown that the disorder average (125) is, in leading order in N, of the form (112), with

Asymmetric: 1

SIq(s, t)Q(s, t).

(148)

9 [a, k, q, Q, K] - - i a - k - ~ at Z

5.4.2. Extracting the physicsfrom the saddle-point equations First we combine the above two results (147)-(148) in the following way (with A - 1 for symmetric dilution and A - 0 for asymmetric dilution): 1

9 [a, k, q, Q, K] - - i a . k - ~ot E [ q ( s , t)Q(s, t) + AK(s, st

t)K(t,s)].

(149)

Statistical mechanicsof recurrentneuralnetworks H- dynamics

679

We can now work out Eqs. (119) and (120), and use (123) to express the result at the physical saddle-point in terms of the trio {m(t), C(t, t'), G(t, t')}. For the single-time observables this gives (as with the fully connected system) h ( t ) = k ( t ) and /~(t) - a(t); for the two-time ones we find:

O(t,t I) - _li~c(t,t,),

g:(t, t') =

O(t, t') -0,

AG(t, t').

We now observe that the remainder of the derivation followed for the fully connected network can be followed with only two minor adjustments to the terms generated by Is and by Q(t,t I) a G ( l - G ) - I ~ sAG in the retarded self-interaction, and (1 - G)-IC(1 - GTj :1 ---. C in the covariance of the Gaussian noise in the effective single neuron problem. This results in the familiar saddle-point equations (133) for an effective single neuron problem, with state probabilities (134) equivalent to the dynamics Prob[~(t + 1) - + 1 ] - 1 [1 -t- tanh[13h(t)]], and in which rc0(o-(O)) -- 89[1 + cr(0)m0] and h(t]{cy}, {qb}) - m(t) + O(t) + aA E

G(t,t')cr(t') + u 89

d t' > 0, with 'initial conditions' C(t, O) - m(t)mo, where

d~ a d~b f[C] -

2 roy/1 --------~exp - C

(

equation at ac = state. In1, t ' - 1)]

1 , 2 + , 2 _ 2 C*a*b'~ tanh[IgV~*a] tanh[13v~*b].

In the m = 0 regime we have C(t, 0) = 0 for any t > 0, inducing C(t, t') = 0 for any t > t', due to f[0] = 0. Thus we conclude that C(t, t~) = St,t, in the m = 0 phase, i.e. this phase is paramagnetic rather than of a spin-glass type. The resulting phase diagram is given in Fig. 9, together with that of symmetric dilution (for comparison).

5.4.4. Physics of networks with symmetric dilution This is the more complicated situation. In spite of the extreme dilution, the interaction symmetry makes sure that the spins still have a sufficient number of common ancestors for complicated correlations to build up in finite time. We have h(tl{r

{~)) -- m(t) + O(t) + ot Z

G(t,t')cy(t') + ~ ( t ) ,

fl m2 > . . - >

ms},

(29)

and it is known to be absorbing, which means that once the weights are in this configuration they can never leave it. The implication is that if the sets of samples of

Topologically ordered neural networks

707

positive probability exist, then the Markov chain reaches an absorbing configuration in a finite time and hence the neuron weights organize with probability one. Unfortunately for any other values of K,D there is no known absorbing configuration for the neuron weights, and it is widely accepted that none exist, which means the Markov chain is irreducible. This in turn means it is only possible to show that the weights can reach an organized configuration in a finite time with positive probability, but it is not possible to state that they reach an organized configuration with probability one. Cottrell and Fort [58] produced the first theoretical proof of self-organization in the SOM, using the Markov chain method. Their proof was for a one-dimensional SOM with a one-dimensional input x (i.e. K = D = 1) uniformly distributed on [0, 1] and for a neighborhood function whose width W = 1 (i.e. see Eq. (16)) with a constant gain factor a. There was also a restriction on the initial conditions in that it was assumed mi(O) r mj(O), i r j. This initial proof has since been generalized, also using the Markov chain method, for different conditions, but still with K = D -- 1. Erwin et al. [59] assuming that the neighborhood function was monotonically decreasing, i.e. that

h(da(i,j)) < h(da(i,k))

for

da(i,j) > da(i,k)

(30)

and with W = N, were able to outline a similar type proof of organization for a continuous distribution of inputs. Bouton and Pag6s [60] extended Cottrell and Fort's proof to the case of a nonuniform input probability distribution. However, there is still the condition that the diffuse component pc of the probability distribution P of the input x, must be such that its support has a nonempty interior. This in effect means that the proof does not cover the case where the support of P is a set of discrete points. Flanagan [61] has taken a different approach to the same problem, where a set of general conditions are stated which the support of P must satisfy along with the neighborhood function. In [61] it is shown that for W - N and for a monotonically decreasing neighborhood function, the neuron weights self-organize, even if the support of P only consists of two or more discrete points. This proof also includes a proof of the case of diffuse P, and by using a monotonically decreasing neighborhood function the proof even applies to the case where m;(0) = mj(0), i r j. This problem of m~(0) -- mj(0), i ~ j was overcome by Sadeghi [62,63] in a different manner, when he redefined the winner neuron. By redefining the winner he was able to prove, with probability one, self-organization of the neuron weights from all initial conditions and for any decreasing neighborhood function. There is a restriction on the probability distribution P, which must be continuous with respect to the Lebesgue measure, and once again this excludes the case of the support of P being a set of discrete points. Flanagan in [64,65] generalized the proof of [61] for any W and showed that if P is not Lebesgue continuous and its support consists of a set of m discrete points, then sufficient conditions for self-organization to occur require at least that, N

~< log2 m.

(31)

708

J.A. Flanagan

Note that the proofs referred to so far all apply to the K = D = 1 case. What about the cases for K -r 1 and/or D -r 1? There are several problems to analyzing the selforganizing process in this situation. The first is that there is no definition of an organized configuration in higher dimensions which can be easily used in the framework of the Markov chain method. Secondly, even if there does exist a welldefined organized configuration, as mentioned earlier, it is probably not an absorbing configuration and hence the Markov chain is not reducible. Flanagan [61] has shown in a particular case with K - D f> 1 that, by defining what might be considered an intuitively satisfying organized configuration, the neuron weights will reach this organized configuration in a finite time with positive probability which may be less than 1. Fort and Pages [66] have shown in the K - D - 2 case, that for W = 1, the exit time of the weights from this configuration is finite with positive probability. These two analyses of self-organization in the SOM for the multidimensional case using the Markov chain method are the only ones so far, to our knowledge. The ergodicity of the Markov process, as given by the multidimensional SOM, and its convergence to an invariant probability distribution have been analyzed by Sadeghi [63]. Fort and Pag6s [66] have described different "strengths" of self-organization for the general SOM, where they consider a map to be organized if and only if the Voronoi tessellations of the closest neighboring neurons are connected. Their definitions are as follows. Definition 1 (strong organization). There is a strong organization if there exists a set of organized states S such that, 9 S is an absorbing class of the Markov chain M(t). 9 The entering time in S is almost surely finite, starting from any random weight vectors. This definition implies in the K - - D = 1 case, that there is strong organization. The next definition uses ideas from stochastic approximation theory which will be discussed next. Definition 2 (Weak organization). There is a weak organization, if there ex&ts a set of organized states S such that all the possible attracting equilibrium points of the associated stochastic approximation ODE's belong to the set S. A second method used in the analysis of stochastic processes and which has been applied with some success to the analysis of the SOM, is the Stochastic Approximation Method [55,56], which here will also be referred to as the ordinary differential equation (ODE) method. As the name suggests, there is a set of ODEs which are generated from the stochastic process to be analyzed. The general approach of the O D E method is to average, in an appropriate manner, the stochastic recursive equations over all possible input patterns, which results in a set of ODEs. If the process satisfies certain conditions, then the stable stationary points of the ODEs represent the asymptotic limit points of the process. This means that once the ODEs for the stochastic process have been attained (which is not always a trivial matter),

Topologically ordered neural networks

709

then analyzing the convergence properties of the stochastic process becomes an exercise in the analysis of a set of deterministic ODEs. In resume, given the stochastic process M(t + 1) = M(t) + ~(t)SM(M(t), x(t)),

(32)

then a set of ODEs associated with this process is given by dM = f(SM), dt

(33)

where the function f is an ensemble average of 8M and t is a pseudo-time variable. It has been shown by Ljung [55] that all locally asymptotically stable stationary points M ~ of the set of ODEs in Eq. (33) are possible, stable stationary points of the stochastic process M(t). One of the most useful results is embodied in the Kushner-Clark theorem [56], which states that given M(t) is bounded and equicontinuous, and given that the gain function 0t(t) satisfies the Robbins-Monro conditions: 0(3

Z

or(i)

i=0

(34)

~-~ ~2(i) < oc, i=0

then M ( t ) ~ M~,t--+ 2 , with probability one, given that M(t) visits a compact subset of the basin of attraction of M ~ infinitely often. It is now interesting to examine the difference between the ODE method and the Markov Chain method used previously when it comes to analyzing self-organization. The main difference probably is that the Markov Chain method requires a priori knowledge of what you want to show. For example, to prove self-organization, it is first necessary to know what an organized configuration is. Using the ODE method no such a priori knowledge is required as it is known that the process can only converge to stable stationary points of the associated ODEs. This would suggest that the definitions for self-organization and topology preservation would result naturally from knowledge of the configuration of the stationary points of the ODEs associated with the SOM. This sort of idea is used in Definition 2 of weak organization. A second difference is that the ODE method deals with the average paths followed by the stochastic process, while the Markov chain method deals with a subset of possible paths of the process. In some sense this means that the ODE method gives a more general picture of how the stochastic process evolves. Finally, the ODE method deals with convergence and the result of eliminating statistical variations by letting the gain function 0t(t)~ 0, t ~ c~, whereas for the Markov chain method, at least in the analysis of the self-organization phase of the SOM, the gain function must be assumed to be constant or lower bounded to achieve a result.

710

J.A. Flanagan

No analysis of the self-organization phase of the SOM for a(t) ~ 0, t ~ e~ exists in the context of the Markov chain method. The principle of applying the O D E is quite straightforward, although the implementation of the method can be quite difficult because averaging usually involves an integration or summation. For the SOM the ODEs can be written more specifically as dmi_ fifn h(da(i,j))(x-mi)dP(x), d~ j=l ./

(35)

where f~/is the Voronoi tessellation of neuron j. For a one-dimensional input, the Voronoi tessellation corresponds to an interval on the line. For a higher-dimensional input it corresponds to a polytope and the bounding hyperplanes are functions of the weights mi. For each configuration of the neuron weights there is a different formulation for the ODEs associated with the process. These factors complicate the analysis but some general results have been obtained for the convergence phase of learning in the SOM. The following theorem, based on results obtained for a K = D = 1 SOM using the O D E method, and taken from [67] combines the results of [58,66,68]. Theorem 1. Assume that: 9 ~(t) E (0, 1) satisfies the inequalities of Eq. (34). 9 The neighborhood function is such that h(i + 1) < h ( i ) f o r some i < ( N - 1)/2. 9 The input distribution P has a density p such that p > 0 on ]0, 1[ and ln(p) is strictly concave (or only concave, with limo+p and liml-p positive). Then, 9 The mean function f ( S M ) has a unique zero M s in 9 +. 9 The dynamical system dM/d'c = f ( S M ) is cooperative on 9 + (i.e. the nondiagonal elements of ~7f (SM) are positive). 9 M ~ is attracting. S o / f M ( O ) C ~+, M(t) ---, M ~ , almost surely. Where 9 + c ~ is the configuration 9 + = {M: ml < m2 < " - < raN}. The condition of log-concavity of the distribution includes all the usual (truncated) probability distributions (uniform, exponential, gamma distribution with parameter ~> 1). The nature of cooperative dynamical systems is discussed by Hirsch [69]. The O D E method has also been applied to the analysis of the higher-dimensional SOMs, but the results are more restricted. In [70] the ODEs for an SOM with K = D = d >~ 1 have been analyzed, where the probability distribution of the inputs is independent in each coordinate and P = 1~ | P2... | Pd, with [0, 1] as the support of each P~. The neighborhood function is also a product function and corresponds, for example, to the eight nearest neighbors when d = 2. The d-dimensional neuron lattice is defined by I =I1 • I2 • --. x Ia, a d-dimensional lattice with Ij = { 1 , 2 , . . . , N j } , 1 -

cT, u2T,..., uT)T. Note that the indices run from 2 to H. Let

.

,.. , , 2,,. , H,) T C | be a critical point of E,_I, that is, OEH-1 ( 0 ? - l ) ) / a 0 (H-l) -- 0. The function f(H-1)(x; 0~H-l)) can be realized in the parameter space OH in many ways. According to the discussion in Section 5.1, for any v c ~a4 and w r ~L, the points defined by

]tv(0~H-l)) --- (V, ~2. -- V, ~ 3 . ' ' ' ' , ~H*' W2., W2., W3,,---, WH.)

(88)

C~w(0~H-l)) -- (O, ~ 2 . , ' ' ' , ~H., W, W2.,... , WH.)

(89)

and

realize f(x; 0~H-1)). Critical points of E (H) can be found as Theorem 2. (1) I f v = )~g2, for )~ E [~, then 7v(0!H-l)) is a critical point of EH. (2) The point ct0(0!H-~)) is a critical point of EH. For the proof, see Appendix B. In the parameter set giving the same function as f(x; 0!H-l)), the critical points of EH appear in the specific locations. All the points giving f ( x ; 0!H-l)) in OH are not critical points. The critical points given by Theorem 2 can be saddle points or local minima. In the case that the number of the output units is larger than 1, all of them are saddle points; Theorem 3. The critical point a0(0!H-l)) is a saddle point. I f the number of output units M is larger than 1, the critical point 7~r (0~H-l)) is a saddle point. For the proof, see Appendix C. In the case of M = 1, some of the critical points {7~r (0!H-l))] k C ~} can be local minima [22]. Theorem 3 asserts that there always exist saddle points in the symmetry of hidden-unit permutations and of the sign flips.

6.2. Natural gradient escapes plateaus In comparison with other second-order methods, Rattray and Saad [32] showed that the natural gradient works effectively to shorten plateaus, in addition to the rapid convergence around the optimum point. They use the framework of statistical

762

K. Fukumizu

mechanics to analyze the dynamics of learning. This subsection presents another explanation of the dynamics by showing the effect of the natural gradient around the saddle points discussed in Section 6.1. In this section, it is assumed that the loss function g is the least square g(y,z)= 89 2, and the activation function satisfies r and 5q0(x; 0)/Ow ~ 0. For simplicity, the discussion below is limited to networks with two hidden units. However, the following results can be easily extended to a general case, because the critical points in Section 6.1 are defined using only a relation between a network with one hidden unit and a network with two hidden units. Let 0! l) = (~,, u,) be a critical point of El, the empirical loss function of networks with one hidden unit, such that the Hessian matrix of E1 is not singular. By the condition of a critical point, the equalities N

E ( y (~) -f(')(x(~); 0~')))q~(x(~); u,) - 0 (90)

N

E(Y(V) - f(1)(X(V)" 0~I)))T ~* ~q)(X(V); U,) = 0T v=l

~

~lll

hold. A derivative of q~ is considered to be a row vector here. The critical point 0~2)-a0(0! 1)) is given by (Vl,V2,Wl,W2)=(0,~,,0,u,). Assume a point 0 (Vl, Wl, v2, w2) is close to a0(0~l)). Here, a different order of the coordinate components is used for simplicity. Write the components as Vl--ep,

Wl =~q,

vz--~,+~r

and

w2-u,+es,

(91)

where e is a small positive number, and h = (pX, qX,rX,sX)X is an arbitrary vector. The gradient of E2 around 0~2) is given by 5

~ =

p e

~

5

(o!

+

q

r

5

S

= ell(2) (0~2))h,

(92)

where H(2)(0! 2)) is the Hessian matrix of E2 at the critical point 0!2). It is easy to check that the Hessian matrix is 0 A AT 0

where

0 0

0 0 c

0

0

s

0

0

CT D

'

(93)

763

Geometry of neural networks." natural gradient for learning

A -- -- Z ( y (v) -- f(1)(X(v); o!l))) ~(P(X(v); O) ~w

v

'

v

c - ;, E ~( x(~) u~)

&p(x("); u, )

~W

v

D

f(1) (x(V); 0!1))) T

Z(y(V ) -

-

-

v

~2q)(X(v) r

awaw

r162 x-" a~(~(~); ",) a~(x(V); ",)

+

~w

v

aw

; u,)

"

This Hessian is nonsingular in general, and the gradient around the critical point can be written as aE2

~0 (0) - O ( s ) .

(94)

The Fisher information at 0!2) is singular. Intuitively, the multiplication of an almost singular matrix is expected to lengthen the gradient around the critical points. This can be proved as follows. Because the first derivative of the function f(2) is given by a ~1 a b-w71 a f(2) (X; 0) --

~

q~(x; wl) a~(X;Wl)

V1 ~w ~(x; w2) V2 Oq~ww2)

~

aq,(,,;0) sa aw aq,(x;O) gC aw

~(x; u,) ' ~, &p(x;U,o)w

(95)

the Fisher information matrix around the critical point 0!2) has the form

G(O)--(O(82)O(s)

O(1)O(s))"

(96)

The inverse is given by

G(0)-I-

(O(~2) 0(1)

O(1))

o(1)

9

(97)

Therefore, the natural gradient around the critical point has the form

O(0)_l ~g2 --~-(0)--

o(~) 0(1) O(1)

"

(98)

o(1) Eqs. (94) and (98) show a clear difference of the dynamics between the ordinary gradient and the natural gradient learning. As the parameter approaches to a critical

764

K. Fukumizu

point given by 0t0(0,), the change of the parameter becomes very small in the ordinary steepest descent, which causes a very slow decrease of the empirical loss function. On the other hand, in the natural gradient learning the change of the parameter becomes very large in the direction apart from the subset {01wl = 0 or vl = 0}. This strong repulsion avoids the slow dynamics caused by the saddle point. Similar analysis is possible on the critical point 0~2,) -- ~tXr (0~1)) if the networks have a single output unit. Suppose to = (~,1, ~,2,b, 11) be a new parameterization of 0092 -- {f(2)(x; 0(2)) -- 101(p(X; Wl) nt- 102(p(X; W2)}, which is defined by r

-- 101 - - 02,

1

1II - - ~ ( W I

-- W2),

vl + v2 r

--

Vl + 10"~ _~

b

~

(99)

~ WVl I

- 4 - ~ 02 W 2 .

Vl + V2

101 -+- 102

This is well-defined as a parameterization unless vl + v2 = 0. Indeed, the inverse of this transform is given by 101 - -

89( r

-+- r

U2 __ 1(__r

+ r

W1 - - b + -~"2'r W2 -- b

q,

(100)

r162

- -_-T=q"

Let to = (~1,1], ~2, b) be a point close to the critical point to, which corresponds to 0 (2/ Write to as **

9

~1 = (2~,- 1)~, + 8p,

q = 8q,

~2 = ~, + 8r

and

b = u, + 8s,

(101)

where ~ is a small positive number, and h - (pT,qT, FT, sT)T is an arbitrary vector. With respect to this parameterization, the gradient and the Fisher information matrix around the critical point to, have the form
~0 is a fixed response function with m a x i m u m 1, t denotes time, and ,//.i. is called the "strength" or efficacy of a synapse connecting neuron j to neuron i. In cortex we typically have one, at most two connections [4, Section 4.2], [9, Sections 20 and 33]. To simplify the notation and without loss of generality we therefore assume that for each pair {i, j} there is at most one synaptic

776

J.L. van H e m m e n

connection. Synaptic plasticity refers to changes of Ju as time proceeds. All this looks simple and straightforward but there is a stochastic complication, which can be eliminated. We have three ingredients of a synaptic response [4,8]: 9 n presynaptic release sites (or active zones); for cortical synapses, n is a small number near to 1. 9 the probability 0 ~ (bl + c~ + Q)J.

(47)

I n p u t channels i and j whose delays are fixed give rise to a specific matrix element Q~j that takes the delay structure into account. F o r input processes that are Poissonian with periodic intensity, the vln a n d v ~ as defined by (28) are constants w h e n / ] is large e n o u g h , and so are the Q~j. It is i m p o r t a n t to realize that this statement is true on the time scale o f / ] b u t need not hold on that o f the total learning time. Finally, the learning e q u a t i o n (47) allows the analysis o f the influence of noise on long-term synaptic plasticity. L e a r n i n g results f r o m stepwise, infinitesimally small weight changes: "Practice m a k e s perfect". W i t h noise, each weight p e r f o r m s a r a n d o m walk whose expectation value is described by the ensemble-averaged e q u a t i o n (47). F o r an analysis o f noise as a deviation f r o m the m e a n the reader is referred to the literature [47].

5.2. Relation to rate-based Hebbian learning In neural n e t w o r k theory, H e b b ' s ideas [28] have usually been f o r m u l a t e d as learning rules where the change o f a synaptic efficacy Ji d e p e n d s on the correlation between the m e a n firing rate viin o f the ith presynaptic n e u r o n a n d the m e a n firing rate v ~ of a postsynaptic n e u r o n , viz., d ~ . in.v out "-~ d4(viin) 2 + ds (v~ d-'-~tJi "-- do -k dl Vin i + d2 v ~ + u3vi

4

2,

(48)

Dirac's imagination has led to an appealingly simple notation. The idea is this. A Hilbert space a/t~is a vector space with inner product ~ x ;4P 9 {x, y} H (xly) E C, which is taken to be linear in the righthand side, viz., y. The resulting inner product looks like a bracket so that Dirac called (xI a 'bra' and lY) a 'ket'. Vectors in our Hilbert space are kets and written lY)- Then the operator P = Iz><xl is projector-like and bound to operate on vectors lY) in such a way that PlY) = Iz)(xly) c< Iz). See the literature [88, Section 14.4] for additional information.

800

J.L. van Hemmen

do < 0 and d~,..., d5 being proportionality constants. Apart from the decay term do and the "Hebbian" term v iinV~ proportional to the product of input and output rates, there are also synaptic changes which are driven by the pre- and postsynaptic rates separately. The parameters d o , . . . , d5 may depend on Ji. Eq. (48) is a general ansatz with terms up to second-order in the rates; see, e.g., [80-82]. In case do = d4 = d5 = 0 and under the (strong) assumption that S in and S ~ are independent, it is straightforward to derive (48) from (32) directly. Alternatively, one can obtain (48) from (45). Linsker [80] has derived a mathematically equivalent equation to (47) by starting from (48) and using a linear graded-response neuron, a rate-based model. The difference between Linsker's equation and (45) is, apart from a slightly different notation, the term cl] and the fact that (45) has been derived from underlying processes, viz., spikes, whereas Linsker's equation is the result of the rate ansatz (48). The present approach is far more comprehensive. Correlations between spikes on time scales down to milliseconds or below can therefore enter the driving term Q for structure formation. They may be, and I expect are, essential for information processing in neuronal networks such as auditory and electro-sensory systems [83]. The mathematics of (47) has been analyzed extensively by MacKay and Miller [84] in terms of eigenvectors and eigenfunctions of the matrix ]l)(b] + Q with b - bl and c = 0. The matrix (Qij + b + cS;j) in (47) contains e times the unit matrix and thus has the same eigenvectors as (Qij + b) while the eigenvalues are simply shifted by c.

5.3. Synaptic dynamics and self-normalization Normalization of the average synaptic efficacy or of the mean output activity is a very desirable property for any synaptic dynamics. After all, the mean output rate should not blow up during learning but converge to a finite value in an acceptable amount of time. Standard rate-based Hebbian learning, however, can lead to unbounded growth. Several methods have been designed to control this unbounded growth, such as subtractive and multiplicative rescaling of the weights after each learning step so as to impose, e.g., ~ j . J j . - const, o r ~ - ~ j j 2 _ const. [85]. Most of these methods make use of the Jj dependence of the parameters d l , . . . , d5 in the learning equation (48). Mathematically they do what they ought to do but, from the point of view of biological physics, it is unclear where they come from. Hence we will derive self-normalization from scratch. We are going to show under what conditions the arithmetic mean jay__ N-1 }-~ij/. of the synaptic efficacies, and hence the mean neuronal output, converges to a finite limit as t ---, oc. In other words, we focus on the question of how we can get self-normalization. The first result in this direction, an even more pronounced form for integrate-and-fire neurons (see Section 2) in the general context of (32), was found numerically by Gerstner et al. [45]. We will see that, for a realistic scenario, we need ai > 0 and hi < 0, whatever i. The former condition is evident once we realize that naive synaptic efficacies start at J~ m 0; after all, where else? Then (47) is nothing but d J / d t - a. For excitatory synapses starting at Ji = 0 it

Theory of synaptic plasticity

801

would be good to increase. Hence we cannot but require a / > 0. We will assume the excitatory case throughout what follows; inhibition can be treated analogously. The linear equation (47) is of the form d J / d t = M J + a with M being the matrix (]l)(b I + cll + Q). As long as M is fixed, the synaptic dynamics can be solved explicitly through Duhamel's formula, J(t) --- exp[(t - to)M]J(to)+

ft0t ds e x p [ ( t -

s)M]a.

(49)

The right-hand side of (49) satisfies (47) and equals J(t0) at time t = to. Hence it is the solution J(t) we are looking for. In the asymptotic limit t ~ oo, the vector a may depend on s, i.e., time, so that a = a(s). Duhamel's formula is as valid as it was for constant a. To incorporate a time dependence of M one has to replace exp[(t - t0)M] by the corresponding solution operator U(t, to). Since both a(s) and M(t) in general preclude any analytic solution, we will not pursue the issue here but take both constant. Let us, then, suppose first that all eigenvalues of M, the so-called spectrum cr(M), have strictly negative real parts. Accordingly we get e x p ( t M ) J ( t o ) ~ 0 as t ~ oc while in the very same limit the integral gives - M - l a , the fixed point of the differential equation. So it all fits, provided - M - l a has all components nonnegative, which may well happen. We need to keep in mind, though, that the key assumption on or(M) remains a bit hard to verify. We did not require M to be diagonable. In fact, except for or(M), we did not assume anything yet. To get sharp analytic results we now suppose (i) bi = b for all 1 0 the nonzero components of J(t) are bound to blow up or decrease to - ~ as t becomes large. This is of course unrealistic since synaptic resources are finite. For excitatory synapses we therefore assume an upper bound ju, with 0 < j u < oo, and a lower bound 0. If the efficacy of synapse i has reached ju, it will stay there as long as its time derivative J[(t) is positive. On the other hand, once J[ (t) < 0 it may decrease. For the lower bound the argument is just the opposite. We thus see that sooner or later, with the timing depending on ~)~, we get saturation of (53), i.e., of J(t) ~ exp(t)~l)[(gl [J(0)) + ~11 (gl la)]el - m - l a .

(54)

I f M is diagonable, then it has N independent eigenvectors ei that constitute the columns of a matrix T with T - I M T = diag()~l,... ,)~X) (i). Hence T t M t ( T t ) -l = diag(~,~,... ,~'N) (ii) and the columns gi of the matrix (Tt) -1 are eigenvectors of M t with eigenvalues k~. For nondegenerate eigenvalues it is a simple argument to show (gj]ei) = 0 for i r j: (Mtgj]ei) = (gjlMei) so that ~,j(gj]ei) -- (gj]ei)~,i with ki r )~j. Now (gi]ei) r 0 because otherwise gi = 0, so that we can put the inner product equal to 1 and find M = ~,i~,ilei)(gi[. In addition, e x p ( t M ) = y~iexp(t~,i)lei)(gi[. For self-adjoint M = m t we are back at the ordinary spectral representation. The reader may consult Merzbacher [88, Section 14.4] for a detailed account of Dirac's convenient bra-ket notation. In fact, the only condition on M that is needed for a l~iorthogonal expansion is that M be diagonable. Then: (i) says T = ( e l , . . . , ex) and (ii) asserts (T-1)T __ ( g l , . . . , gx). Hence biorthogonality is equivalent with (T -1)T = 4, which is evident. The expansion itself can be verified on a complete set of eigenvectors of M, viz., {ei; 1 0 by assumption, both eigenvalues m+ are positive, if b > 0 and N~ >> 1 so that c is subdominant; we can use c for fine-tuning, however. If on the other hand b < 0, then m_ < 0 but m+ > 0. In both cases the eigenstate e+ belonging to m+ > 0 is dominant. As we are given [e+)(e:~[, the source term a, and the initial condition d(0), we know what the asymptotics looks like; cf. (54). Fig. 9 shows the result of a realistic simulation, a nice academic exercise: 0 < J;(0) - J" for all i. Synapses that are "more equal" than others, win. We can now easily understand why. In the present case a = a l , J(0)-J~l, and j u + a/m+ > 0 so that the vector 1+ := ]e+)(e+ll ) - (1 + (N1/n)(b - Q/2), 1 + ( N 1 / n ) ( b + Q / 2 ) ) T tells us what will happen; (N1/n) does not depend on N1. A nontrivial structure occurs only if b < 0 since the first component of 1+ is then negative whereas the second is positive. This is the case in Fig. 9, where b < 0. For b > 0 both components of the vector 1+ are positive and a trivial saturation occurs.

Theory of synaptic plasticity jU

805

t=103S

104s

2.93xl 04

7xl 04s

Ji

0.08

0 lllll'llllllllllllllllllllll'llllllllllll~l[lllillil:iif~[~iill~lll[l[~]~ 7TTT:I[[[ I 50

0.06

==i'7:7"=i

jav

0.04

0.02

0

2X104 i

.

-

,

4X104 t[s]

6X104

Fig. 9. Temporalevolution of the average synaptic efficacies j~v and j~v as defined in (56), and jay = (j~tv + J~ )/2 as a function of the learning time t in units of 10 4 s. This is a fictitious time to keep the computational time finite. It is in general too fast for biology but can be adapted to it by a simple rescaling without changing the picture. The quantity jav is the average weight of all synapses, jpv and j~v are average weights of the groups Y l and Y 2 , respectively. Synapses i in group ~/~1, where 1 ~~I(0, 0)/> - S dOp(O) log p(O) - S dO9(O)

1 (2xe'~

0-{-67)'

(3),

where the last term at the right-hand side follows straightforward for a Gaussian distribution with variance J(0). When the distribution of the estimator is sharply peaked around its mean value (which implies J(0) >> 1) the entropy of the estimator becomes identical to the entropy of the stimulus. When the estimator has a non-Gaussian distribution, the inequality will be strict.

3. Probabilistic interpretation of population codes The starting point for almost all work on neural population codes is the neurophysiological finding that many neurons respond to a particular variable underlying a stimulus (such as the sensitivity of neurons in visual cortex to the orientation of a luminous line) according to a unimodal tuning function. For neurons involved in sensory perception, the set of variables, which affect the response of a neuron, is usually referred to as the receptive field. However, for neurons involved in movements a better terminology would be "movement field". In order to summarize both types of neurons, and especially neurons in the sensory-motor pathway where neural responses have both sensory and motor components, we will use the term "response field". The value or set of values of the variables underlying the response field, which produce a peak in the tuning function, will be called the "preferred value". The response field plays an important role in interpreting neuronal population codes. For many brain structures, the response fields of neurons are not known. Only for neurons in rather peripheral sensory pathways (such as retina, lateral geniculate nucleus, area V 1 in visual cortex) of motor pathways (for example motor cortex [5]), it is possible to determine the response field. However, for neurons in more central brain structures, the relevant sensory and motor features, which underly the response field, may be very hard to discover. Several authors have used Gaussian white noise as stimulus [6]. The reason for using white noise is that the characteristics of a dynamical system are hard to determine, because what happens now depends on what happened before. Thus all possible stimuli and neural responses have to be considered for a full characterization of the system. The use of Gaussian white noise (GWN) stimuli is attractive, since a GWN-signal has the largest entropy given a particular variance and as such contains all possible combinations of stimulus values in space and time. As a first-order (linear) approximation, the response field Ri(t) of neuron i can be defined by the cross-correlation of the Gaussian white noise stimulus x(t) and the neuronal response ri(t) [6]. This cross-correlation can be shown to be equal to the averaged stimulus preceding an action potential [7] or to the averaged response following a spike. We will refer to this as the averaged peri-spike-event (PSE):

Population coding." efficiency and interpretation of neuronal activity

RPSE /

tri/'ld' =

27

= Z

r

x(t-

"c)~ ~(t- t~)dt

2-@ x(t/7 -1:),

861

/4t (5)

(6)

/7

where the response ri(t) of neuron i is represented by a sequence of ~5-pulses and t, is the time of occurrence of action potential n. As we will see later, this cross-correlation technique can provide a first step to characterize the conditional probability p(rl0 ). However, for neurons with complex properties, the complexity of the G W N stimulus increases exponentially with the number of dimensions of the stimulus. Therefore, this approach to characterize the response field is only useful for neurons with simple, low-dimensional response fields. The characteristic properties of the response field can provide information to answer the question "How is an external event x(t) in the world encoded in the neuronal activity r(t) of the cells". A full characterization of the response field of a neuron (both spatial and temporal properties!) implies that the density function p(rl0 ) is known. The response fields are also indispensible for answering the question about the sensory or motor interpretation of neural activity. The response fields allow the mapping from the set of activities in a neural population r(t), with ri(t) representing the activity of neuron i at time t, to the events in the external world by Bayes' relation: p(Olri) = p(O)p(rilO)/p(ri). Since the generation of action potentials is a stochastic process, the problems described above have to be addressed in a probabilistic way. We will define p(r[x) as the probability for the neuronal activity r given the stimulus x. The simplest models assume that neuronal responses are independent, which gives p(rlx ) = 1-[ip(ri[x). For the time being, we will assume independence of firing. The case of correlated firing between neurons will be discussed later. A Bayesian decoding model specifies the information in r about x by p(xlr) e(p(rlx)p(x),

(7)

where p(x) gives the prior distribution about x. Note that starting with a specific stimulus x, encoding it in the neural activity r, and decoding it results in a probability distribution over x. This uncertainty arises from the stochasticity of the spike generating mechanism of neurons and from the probability distribution p(x). As explained in Section 2.1, the most likely or most plausible stimulus x given a response r is given by the MAP-estimator. However, it can be shown that under some (rather restrictive!) assumptions, a more simple and intuitive interpretation of neuronal activity can be obtained. Suppose that the response of neuron i depends on the projection of the stimulus (or response) to a preferred stimulus (or response) Xi of that neuron (like in visual cortex; see [8] for an overview) or in motor cortex [5], such that

C.C.A.M. Gielen

862

Xi)}, where G(x. Xi) is a continuous, symmetric bell-shaped function and where Z is a normalization factor. If Xi is known (for example, it may be the averaged PSE in Eq. (6)) and if neurons fire independently, then

p(rilx)- 89

p(xlr) - p(rlx)

(8)

1 p(x) = ~ H exp{G(x. Xi)} p(r)" i

(9)

The most plausible stimulus is then found by setting the gradient of p(xlr) with respect to x to zero

1 Vxp(X Ir) - Z II[exp{ " i G(x Xi)}VxG(x" X/)-X;I p(x)p-~+ p(rlx)~ Vxp(X)

= II[p(rilx)VxG(x. Xi)-Xi] p(X) , =0.

p(rlx)Vxp(X)

+ p(,)

(10)

The first term in Eq. (10) gives the most plausible stimulus based on the response fields of the responding neurons. The second term gives a correction for the probability density of stimuli. For a flat distribution, the second term equals zero and the MAP-estimator becomes equal to the ML-estimator. When all Xi are distributed homogeneously in a stimulus subspace, such that correlations between neighboring response fields are the same for all neighboring neurons, then the most plausible stimulus is proportional to y~p(rilx)Xi, which is the well-known population vector (see Section 4). i

4. Models for population codes

4.1. Simple version of population coding The most simple and straightforward interpretation of neuronal population activity is obtained by simple summation of the response fields Ri of all neurons i, weighted by the firing rate ri of each neuron

~-~Nl ri(x)Ri

(11)

This choice corresponds to the so-called center-of-gravity (CG) estimate [9]. CG coding can be statistically optimal. This is the case for perfectly regular arrays of sensors with Gaussian tuning profiles that have an output described by independent Poisson statistics, and for arrays of sensors with a sinusoidal tuning profile for the parameter estimated. However, there are many cases in which CG decoding is highly

Population coding: efficiency and interpretation of neuronal activity

863

inefficient. This includes the important case (which is observed at nearly all parts of the brain), where sensor positions or response fields are not regularly spaced. We will come back on this topic later. Moreover, the CG approach assumes a homogeneous distribution of response fields in the event space and a homogeneous distribution of stimuli x for sensory neurons. Given these assumptions, any deviations between the CG result and the true parameter value are small provided that the noise is small and that the neurons sample the parameter space sufficiently dense. Moreover, the question arises, whether the estimate of this population coding scheme is optimal in the sense that it is unbiased and that the variance in its estimate is small. A good estimator should be unbiased, which is the case when the estimator gives the (expectation value of the) true stimulus x. Baldi and Heiligenberg [10] demonstrated that the CG method is virtually bias free. However, this simplistic version of the population vector is inefficient in the sense that variance of the estimate is much larger than the smallest possible variance. One of the first experimental data demonstrating the importance of the concept of population coding were obtained from motor cortex [5,11]. Neurons in the arm area of primate motor cortex in monkey are broadly tuned in the sense that they increase firing rate for a broad range of arm movement directions. Each neuron appears to have a preferred movement direction (i.e. the movement direction, which corresponds to the largest response modulation of the neuron) and preferred movement directions are approximately homogeneously distributed in 3D space. Georgopoulos et al. [5,11] interpreted the population activity as N

M(r) = ~

rini,

i=1

where Mi is the preferred movement direction of neuron i and M represents the estimated movement direction of the arm. Quite remarkably, the estimated movement direction by the population vector was very close to the actual measured movement direction of the monkey's arm. For a simple array of N independent sensors with unit spacing between consecutive sensors and with a gaussian tuning function f~(0) - e x p

-~

cy

and with Gaussian noise W~ superimposed on the response of neuron n (Rn = fn(0) +/4z~), the Fisher information is given by (see e.g. [9]):

According

to the Cramer-Rao bound, the minimal variance is given by the summation is replaced by integration, which is a good approximation for large N and sufficiently large cy, the minimal variance reduces to

N2/~-.,,,( f ,n' 0( ) ) 2 9When

c.C.A.M. Gielen

864

2crN2/v~.

N o t e , that the m i n i m u m attainable variance increases with the sensor tuning width or, a result which is similar to the M L result (see Section 4.2 and Fig. 1). The results a b o v e show, that the m i n i m a l variance o f the center-of-mass m o d e l is p r o p o r t i o n a l to or, i.e. the m i n i m a l variance increases as a function o f the tuning a

3

~1--

--

i

'

9

'----

-

[

'

'

'

\

e~ 0

i m

_

m

~

E 1,._

m

0 C e,,

..-

i m

1l_ .._1

0o

30 ~

-

"-60

.

.

.

.

~

90 ~

120 ~

Tuning width 3--

b

~

2.5

~'',

,

I

2

~ /

, /, 1.5

\\

", .

,/

/

/

4

f

J

J

J

N

1

0.5

t

I

.

1

1

....

Tuning width (radians) Fig. 1. (A) Fisher information (J/Nfma,,) for the population of neurons with tuning functions according to Eq. (15) for the ML-estimate (solid line and broken line) and for the population vector (dotted line and dashed-dotted line) for ratio's of 0.1 (dashed line and dashed-dotted line) and 0.01 (solid line and dotted line) Offmin and fmin + fmax as a function of tuning width a in degrees (modified after [14]). (B) (modified after Fig. 1 in [12]) shows the variance for the population vector estimate for number of neurons N - 103 (dashed line) and for N = 104 (solid line) as a function of the tuning width of the same set of neurons as in Fig. 1A. Note that variance is related to the inverse of the Fisher information.

Population coding: efficiency and interpretation of neuronal activity

865

width or. Hence, it is advantageous to use narrowly tuned sensors. If we compare the variance of the CG model with that of the Cramer-Rao lower bound, Snippe [9] obtained the result Var(0cR) j- V a r r~~

6v/~Cy 3

~
1 for reasonably effective sets of stimuli. Thus the Poisson model is incapable of representing distributions that are broader than the tuning function, which points to a second problem for the Poisson model. Obviously, the proper way to find the true (set of) stimuli is to estimate the full conditional probability p(xlr).

4.3. Optimum linear estimator (OLE) The simplest possible estimator is an estimator that is linear in the activities r of the neurons, which suggests a solution xest = WTr, where the problem is to find the optimal matrix W, which minimizes the mean square distance between the estimate xest and the true stimulus x: arg min

w -

w

x)-'].

One can think of the linear estimator tron-like neural network with a set of w; to the input r and W is the matrix The OLE is known to be unbiased given x is given by

as being the response of a two-layer percepoutput units, where output unit i has weights with columns w;. for a large number of units [15]. Its variance

N

E[(XOLF -- E { x } ) 2 ] - Z

Wi2(Yi2

i=1

2 and c~] = f-(x) for where cy] - c~], for normally distributed noise with variance cyn, Poisson distributed noise. Note, that the OLE model suffers from the same problem as the CG estimate in the sense that many neurons contribute their noisy output to the population estimate, whereas only few neurons may respond to a stimulus. Therefore, a compro-


869

raise has to be made between small tuning widths for a high resolution versus broad tuning widths to eliminate noise by averaging responses, thereby increasing the signal-to-noise ratio of the estimate.

4.4. Summary In the theoretical approaches discussed so far, the result of the estimation procedure to interpret the neuronal activity, has been a single parameter or feature of a sensory stimulus or movement (such as for the M L estimator) or a distribution over all possible parameters and features, instead of a single parameter or feature. A distribution of probable features is compatible with experimental findings that the neuronal activity is not simply related to a single feature, but rather to a set of stimuli presented simultaneously (see e.g. [18]) or to the expectation about a particular stimulus or movement (see e.g. [19]). The available experimental evidence and the fact that usually several features are represented in the neuronal activity within a particular part of the brain, suggests that we should focus on probability densities, rather than on single features. Also, as long as we do not know precisely what role a particular group of neurons play in the complex sequence of information processing, it might be better to resort to probability densities, rather than to a single feature or parameter.

5. Overlap of receptive fields and correlated noise in neural responses

In the analysis so far, we have made the assumption of independent noise in neighboring neurons. Also, we have demonstrated that the optimal tuning of neurons depends on the type of noise in the neural responses. In this section we will explore this in more detail, in particular in relation to optimal tuning width of neurons and to optimal information content of neuronal activity for various types of (correlated) noise.

5.1. Optimal receptive fields." broad or narrow? One of the central problems with population coding is how the neuronal code can be made as efficient and as accurate as possible. It is a common belief that sharper tuning in sensory or motor pathways improves the quality of the code, although only to a certain point; sharpening beyond that point is believed to be harmful. This was illustrated already in Fig. 1, which shows the Fisher information as a function of the receptive field width of model neurons, which have an orientation specificity, similar to that of neurons in visual cortex. Fig. 1 shows that sharp tuning (small receptive fields) is not efficient for the population coding model, since for very small receptive fields, the number of neurons, that respond to a narrow bar of light, is too small to reduce the noise in the neuronal responses. For broader tuning, more neurons will respond to the narrow bar, which allows noise reduction and improvement of the signal-to-noise ratio. Obviously, the optimal receptive field size depends on several parameters, such as the noise in the neuronal responses, the

C.C.A.M. Gielen

870

number of neurons, the distribution of receptive fields (homogeneous versus nonhomogeneous; see [12]). The best way to proceed is to start with the Fisher information ~2 J - E l - ~ log p(rl0)] , where p(rl0 ) is the distribution of the activity conditioned on the encoded variable 0 and EI.. ] is the expected value over the distribution p(rl0 ). Instead of the Fisher information, one could also have chosen the Shannon information, which is simply and monotonically related to the Fisher information in the case of population coding with a large number of units. Let us consider first the case, in which the noise distribution is fixed. For instance, for the population of neurons from the example in Section 4.2, where we had a population with N neurons with bell-shaped tuning curves and independent G W N with variance cy2, the Fisher information reduces to N t (o) 2 J - --)if. ~2 ,

(19)

i=l

where f ( 0 ) is the mean activity of unit i in response to the stimulus with orientation 0, and f,:(0) is the derivative with respect to 0. Eq. (19) illustrates, that as the width of the tuning curve decreases, the derivative f;'(0) will become steeper and thus the information increases up to infinity for infinitely small receptive fields. Clearly, this corresponds to the ML estimate, discussed in Section 4.2, where narrow tuning is better than broad tuning. Note, that for the same noise the minimal detectable change, which is inversely proportional to the square root of the Fisher information, reveals that narrow tuning may not be optimal for the population coding model (see Fig. 1). It should be noted, that the results on the optimal tuning of receptive fields has been a cause for much confusion, because the results critically depend on the type of noise and on the model used. For example, Fitzpatrick et al. [20] reported the opposite result (i.e. that sharp narrow curves allow better discrimination for the center-of-mass model than broad tuning); however, these authors had a different, biologically implausible noise model. Pouget et al. [21] demonstrated that when the noise distribution is not fixed, the results become different. They considered a two-layer network with an input layer and an output layer, with feedforward connections from input to output neurons and with lateral inhibitory connections in the output layer to sharpen the tuning curves (see Fig. 2). This case is particularly relevant for neurophysiologists. Since the output neurons can never contain more information than the input neurons, this model shows an example where broad tuning contains more information than narrow tuning. However, sharpening is done by lateral interactions, which induces correlated noise between neurons. The loss of information has to be attributed to this correlated noise. The results above demonstrate that the answer to the question whether broad or narrow tuning is best, depends on the noise. In most neurophysiological experiments


[

,

,

-.

.

.

.

.

.

orientation

871

I

output layer

input ~ ~ T layer ("~ ~ ~i~ (~ ~ c:&cj (..~ ~

orientation Fig. 2. Two-layer neural network with feedforward excitatory connections between input layer and output layer and with lateral connections in the output layer. For visibility, only one representative set of connections is shown in each layer. The tuning of the input units was chosen broad, whereas lateral "Mexican-hat"-like connections in the output layer create narrowly tuned neurons in the output layer. Since information cannot increase, this provides an example, where broad tuning in the input layer provides more information (or at least as much) as narrow tuning in the output layer (adapted with permission from [22]). measuring single-unit activity it is impossible to detect correlated noise and in most cases it is not even possible at all to make a good estimate of the type of noise in the neuronal response. Therefore, usually independent noise is assumed. In the example above, this would lead to the erroneous conclusion that the output layer contains more information than the input layer. This simple example demonstrates that a proper characterization of the noise distribution is essential for a proper estimation and interpretation of the neuronal activity in a population. Multi-unit recording techniques may be an excellent tool for this purpose. Many studies have convincingly demonstrated, that noise in a population of neurons is correlated. When the fluctuations of individual neurons about their mean firing rates would be uncorrelated, the variance of their average would decrease like 1/N for large N. In contrast, correlated fluctuations cause the variance of the average to approach a fixed limit as the number of neurons increases (see Section 5.2). The inverse of the Fisher information is the minimum averaged squared error for any unbiased estimator of an encoded variable. It thus sets a limit on the accuracy with which a population code can be read out by an unbiased decoding method. The analysis above has illustrated how the optimal coding (broad versus narrow tuning) depends on the noise in the neuronal responses. However, another parameter, which is relevant in this context is the dimension of the encoded variable. Suppose, that the encoded variable is a D-dimensional vector x. Under the as-

872

C . C . A . M . Gielen

sumption of independence of the components of the vector x, the Fisher information J(x) is defined by D

For a set of N neurons with radial symmetric response functions, which are assumed to have the same shape for all neurons and where neuronal activity is assumed to be independent, with tuning function

'x-~'22)cy

f-(x)-Fdp(

(20)

the Fisher information is given by J(x) --

rl(cya-2Koo(F,z,D)),

where 1"1is the number of neurons whose tuning centers fall into a unit volume in the D-dimensional space of the encoded variable and z is the time window for the neuronal information under consideration [22]. This illustrates that the Fisher information scales with the tuning width in arbitrary dimension D. Sharpening the tuning width helps only for D = 1, has no effect when D = 2, and reduces information encoded by a fixed set of neurons for D >/3. Although sharpening makes individual neurons appear more informative, it reduces the number of simultaneously active neurons, a factor that dominates in higher dimensions where broad tuning functions give rise to more substantial overlap between neighboring units. One could ask how the information content of an action potential is affected by tuning. This can be addressed using the Fisher information per spike. If all neurons have the same tuning parameters, the total number gspikes of spikes within a time window z is 3C t

~

Nspikes = r I /

z f ( x ) d x l .... dxD -- qcYD

FzQ4,(D),

--"2C

where f ( x ) is the mean firing rate in Eq. (20). When the neurons have Gaussian tuning functions, independent Poisson spike distributions, and independent distributions of peak firing rates and tuning widths, the Fisher information per spike is given by I~. Therefore, they are able to fire at an arbitrarily small rate. The f - I curve of the WB model is shown in Fig. lB. In the following we will use this model for representing inhibitory neurons. To describe excitatory neurons, we will modify this model by adding a slow potassium current, which will induce spike adaptation, a phenomenon known to be widespread among excitatory cortical neurons. Other parameters will be kept the same. The f - I curve of this modified WB model is also shown in Fig. lB.

3.1.2. Integrate-and-fire neurons Integrate-and-fire models constitute another class of models frequently used in modeling [77]. These models do not rely on a biophysical description of firing and their simplicity makes them more easily amenable to analytical studies than conductance-based models [81-83]. In the simplest integrate-and-fire model, the Lapicque model, the membrane potential of a neuron satisfies the differential equation dVi C d--7= --gL(~ -- VL) + Ii + Isyn,i(t)

(2)

for 0 < V < 0. As before, Ii is an external current and Isyn,i is the synaptic current which is due to the interactions with all the other neurons in the network. A spike is

D. Golomb et al.

896

fired whenever the membrane potential reaches the threshold 0. If at time to a spike is fired, the membrane potential is immediately reset to Vr namely ~(t0~)

-

if Vi(to)

Vr

-

O.

(3)

For simplicity we chose: V~ = VL. One can also introduce in this model a refractory period, if necessary, by imposing that V(t) remains equal to 0 for a time Tr after the firing of a spike. If the neurons are not interacting they emit spikes periodically with a period Ti-- T r - v01n(1 - g L ( 0 - - VL)/Ii) for Ii > gL(0-- VL). where the passive time constant of the membrane is denoted by t0 = C/gL. In the following we will chose: C = l g F / c m 2, gL = 0 . 1 m S / c m 2 and 0 - VL ---- 20 mV. This corresponds to passive membrane time constant: t0 = 10 ms and a current threshold Ith = 2 ~tA/cm 2. The refractory period will be a parameter, the influence of which on the network properties will be discussed. Without loss of generality, we assume that VL = 0. It is often convenient to write the I&F model using the reduced dimensionless variables: = V~- VL O-V L '

/,.-

(4)

c ( o - vL)'

/syn,i /syn,i-- C ( 0 - VL)"

In terms of these reduced variables the dynamics can be written

dVi

d---~= - ~ +/"

+ Isyn.i,

(5)

where the time is measured in units of t0 and the resetting conditions read /2/i(t~-) -- 0

if ~(to) -- 1.

(6)

We will use this formulation in Section 8.

3.2. Models for synaptic interactions The synaptic current, Isyn.i(t), which the ith cell receives, is the sum of all contributions Isyn,ij(t) from all the presynaptic cells with index j impinging on the the ith neuron

Isyn,i(t) -- Z l s y n , i j ( t ) . J

(7)

897

Mechanisms of synchrony of neural activity in large networks The current Isyn,ij(t) is modeled by

Isyn,ij(t)--Gsyn,ij(t)(Vi- Vsyn),

(8)

where Gsyn,ij(t) is the total conductance at time t of the synapse from cell j to cell i with a reversal potential Vsyn. The v a l u e Gsyn,ij is given by 0syn,/j(t) - Gsyn,ij sj(t),

(9)

where Gsyn,ij is a constant which measures the strength of the synapses that neuron j makes on neuron i, and sj evolves with time. Various models are used for the dynamics of the synaptic conductances. They differ by their faithfulness to the details of the biophysics. One possibility is to assume that the variable sj. obeys the kinetic equation [84,85]

dsj = kfS~(Vj)(1 - sj) - krsj, dt 9

(10)

where Vj is the presynaptic potential,

S ~ ( V ) - {1 + exp[-(V - 0~)/cy~]}-',

(11)

where 0~ = 0 being the presumed presynaptic threshold for transmitter release and cy~ = 2 mV. After termination of the presynaptic spike, the decay rate of s is 1/kr. The parameter kf affects both the synaptic strength and its tendency to saturate. If 1/kf is much smaller than the spike time width, the variable s saturates to the value

k f / ( k f + kr). In a more phenomenological description, the variable s is represented by

Sj(t) -- ~

tspike,j)

f(t-

(12)

spikes the summation being performed over all the spikes emitted by the presynaptic neuron with an index j at times tspike,j- The synaptic interaction is usually classified according to whether Vsyn is larger or smaller than the threshold potential, Vth, at which the postsynaptic neuron generates spikes. For Vsyn> Vth the interaction is called excitatory, while for V~y~< Vtth it is called inhibitory. The function, f , is normalized such that its integral is 1. Then, Gsyn,ij, is the total synaptic conductance induced by one presynaptic spike. Several forms can be used for the function f ( t ) . A standard choice is

f (t )

_

1

T1 -- T2

exp

-

-~l

-exp

-

~2

|

"

(13)

Here, | is the Heaviside function and the normalization o f f ( t ) has been chosen so that the integral of f ( t ) is one. The characteristic times ~1 and ~2 are the rise and decay times of the synapse, respectively. In case ~l = "~2 - - T one obtains the so-called "alpha function" [86]:

f (t) --t-~exp ( - ~)

(14)

898

D. Golomb et al.

Analytical studies of integrate-and-fire networks are often easier if one adopts a more simplified model for the synaptic current in which one neglects the effect of the driving force V/- V~yn.In that case, the synaptic current has the form Isyn,ij(t) -- Gsyn.ij sj(t),

~ f (t -

sj(t) --

(15)

tspike,j),

spikes

where the function f(t) is given by Eq. (13) or (14) and the summation is done over all the spikes emitted prior to time t by all of the presynaptic neurons. Note that in this formulation, Gsyn,ij has the dimension of a density of current. In this model, excitatory (resp. inhibitory) interactions correspond to Gsyn.ij > 0 (resp. Gsyn,ij < 0). In such a model, one neglects the fact that the synapses are changing the effective integration time constant of the neurons. This approximation is more justified for excitatory interactions (Gsyn.;j > 0) since no description of the spike is incorporated into the model and the driving force V~yn- V, remains approximately constant in the subthreshold regime. For inhibitory neurons, where shunting effects are more important, this approximation is more crude.

3.3. Network architecture and inputs In its more general form, the network model we consider in this chapter consists of two populations of neurons, one is excitatory (E) and the other is inhibitory (I). The synaptic current, which a cell i of the 0tth population receives from a cell j from the 13th population, is ZY n'i(t) -- Z

(16)

Isyn.ij (t)'

J

where 0t = E, I, i = 1 , . . . , N~, j -- 1 , . . . , NI3, and N~ is number of neuron in the 0tth population. The corresponding elements of the synaptic conductance Gsyn,ij are proportional to the connectivity matrix w~~ defined as 1

if neuron j from population 13 is presynaptic to neuron i from population 0t,

0

otherwise.

w~~ -

(17) We will assume that the network has no spatial structure (except for the segregation into two populations), and define M ~ to be the average number of synaptic inputs a cell from the 0tth population receives from the cells from the [3th population. A neuron, j, ]3 is making one synapse on neuron i, 0t at random with a probability M~/Nf~. Therefore, Prob(w~f - - 1 ) _ M ~ ,

Prob(w;~- 0)-

1

M~I3

(is)

The number of synaptic inputs from population 13onto population at fluctuates from neuron to neuron in population at with an average M~. In part of the examples

Mechanisms of synchrony of neural activity in large networks

899

treated below we further simplify the architecture by assuming all-to-all connectivity. In that case, M~fi = N~. For the sake of simplicity, we assume that all the existing synapses between the neurons of two populations have the same strength

Gsyn,ij

Gsyn w~~.

(19)

The synaptic rise time and decay time will depend only on the nature, excitatory or inhibitory, of the synapses and will be denoted by Zl~,Z2~, 13- E,I. We neglect axonal propagation delays. In many cases it is convenient to model the external input to the network as an external current. In the analytical studies presented below, we adopt this approach since it significantly simplifies the calculation without modifying substantially the specific results presented in this chapter. However, one should keep in mind that depending on the issues which one wants to address, such a description may or may not be suitable. A more plausible description of the external input is in term of conductances. This is the approach we adopt for the simulations presented in Section 5. In that case, the input network consists of No excitatory neurons. Since we are not modeling the architecture and the dynamics of this network in detail, we assume that the input neurons are firing spikes independently, with a Poisson statistics characterized by a rate v0. We also assume that the number of afferent synapses and their strength are the same for all the neurons in the same population. The number of afferent synapses on the excitatory (resp. inhibitory) neurons will be denoted by n~E (resp. nIa) and their strength by gae (resp. g~). Under these assumptions the external input on the neuron i, ( i - 1,... ,N~) in population ~ - - E , I is written

I~(t) = g~ Z

f (t - tspike)[VE -- Via(t)l ,

(20)

spikes

where the sum is extended over all the spike times of the na~ afferent input neurons to neuron (a, i) which have occurred before time t. Here the function f is given by Eq. (13) with rise time and decay time which for simplicity will be taken to be the same as for the recurrent synapses of the excitatory population. Note that the time average conductance of the external input on the population ~ is simply :

n a g a V 0.

4. Nature of the network state

4.1. Thermodynamic limit The dynamic behavior of a network may be sensitive to the precise details of the parameters that characterize its dynamics and connections. One of the parameters is the size of the network. Studying the size dependence of the results is extremely important for several reasons. First, systematic numerical simulations of networks, such as those described below, are limited to sizes of up to several thousands of

D. Golombet al.

900

neurons. This limitation is more severe for more complex networks, e.g., networks with spatially extended cells. Although the relevant scale of local circuits in cortex is not known, it is quite possible that it involves a larger number of neurons. More importantly, understanding the size dependence is crucial for understanding the qualitative nature of the dynamical states in networks and the phase transitions between them. This is because frequently, these states and the transitions between them can be defined in a rigorous way only in the thermodynamic limit, i.e., when the number of neurons goes to infinity. In order to discuss the size-dependence of the dynamic behavior of the network we have to specify how the network parameters vary when the numbers of neurons of the two populations, N~'s, are increased. Here we assume that the single cell properties, including its time-constants, the resting and threshold potentials, as well as the external current, are independent of N~. This implies that the total synaptic input Isyn must remain of order unity as N~, ~ -- E, I grow. Two extreme cases are frequently considered: (1) Massively connected network. In this case the average number of synaptic inputs per neuron varies proportionally to the size of the system. Excluding precise cancelations, the size of the amplitude of most of the individual synaptic inputs are kept fixed. In our case this means that the synaptic strengths, have to vary in inverse proportion with the system size such that Gsy~n = gs~y~n

N~'

(21)

where the normalized conductances, gsyn ~ are independent of the N~'s. (2) Sparse network. In this case the average number of synaptic inputs per neurons, M~, are O(1) compared to the size of the network, and they should not be scaled in the thermodynamic limit. Therefore in this limit, these parameters remain finite. We want, however, that the dynamics of the network would have a defined value also at the limit when the values M~[~ vary and become large. That is why, in most of this chapter, we scale the conductances between cells such that Gsy~n = gs~.

M~

(22)

We will come back to this issue in Section 7.2. Note also that if a cancelation is expected (balanced state) between the excitatory and the inhibitory inputs, the synaptic strengths should be scaled with the square root of the connectivity (see [87-90] for more details on that issue).

4.2. Classification of degree of synchrony 4.2.1. Definitions of synchrony As was already mentioned in the introduction, roughly speaking, the activities of two neurons are synchronized if their spike trains display some level of correlations. According to this definition, in the simplest case, the experimental signature of


901

synchrony is a nonflat cross-correlogram of the spike trains or the membrane potentials of the two neurons. This definition does not assume anything regarding the sign of these correlations, i.e., neurons with positive or negative correlations in their firing times are considered as synchronized. However, in this sense, in any system of interacting neurons, some synchrony exists. In a more restrictive way, two neurons are said to fire synchronously if they tend to fire spikes with positive correlations at zero or small time delay. An example of synchronous firing following this definition would be a pair of identical neurons, periodically firing in a phase-locked manner with a small phase-shift. Full synchrony is the extreme, ideal case in which the two neurons fire simultaneously. Note that some authors are restricting the definition of synchrony to this last situation and call ~-synchrony the more general case where the firing times of the two neurons differ slightly [91]. With these definitions, synchronous firing requires synaptic interactions between neurons with suitable properties [92,93,31,84,94] (see also Section 6.3). These definitions are well suited to the study of spatio-temporal patterns of firing in a small system of neurons which depend on the detailed pattern of connections between the neurons. However, when dealing with large systems of neurons as found in the central nervous system, these definitions are not sufficient. Therefore, a more appropriate definition of neuronal synchrony is needed. To be rigorous, this definition requires the thermodynamic limit. It allows us to differentiate between synchronous activity which would result from the anatomical connectivity and synchronous states which, independent of the detailed connectivity patterns, are cooperative in their origin.

4.2.2. Cooperative synchrony in large networks Massively connected systems. Given the above scaling of the synaptic conductances, there are several ways in which the system size can affect the dynamics of the network. However, as discussed in Ref. [95] there are two simple generic cases. These cases are termed asynchronous and synchronous states, and they differ in the way the temporal fluctuations of the total synaptic conductance change with the size of the system, N. In asynchronous states the total synaptic excitatory and inhibitory conductances generated by the network on a neuron approach a time-independent limit as N ~ ~ . This reflects the fact that the action potentials of the individual neurons are very weakly synchronized. Summing N temporally uncorrelated (or weakly correlated) contributions results in a total synaptic conductance whose fluctuations have an amplitude of the order of 1/~/N. Such a state can be self-consistent because the weak temporal variation in the "common input" to the different neurons may be insufficient to synchronize them. Characterizing this state can be done in two ways. One way is by evaluating a global variable, e.g., the spatially averaged instantaneous activity. In the asynchronous state the variance of such a quantity vanishes as N increases, typically as 1IN. Alternatively, one may evaluate the cross-correlation (CCs) functions between the activity of pairs of neurons [9]. The magnitude of the typical CCs will be of the order of 1IN.

902

D. G o l o m b et al.

A specific type of an asynchronous state is the "splay state" [96], in which neurons fire consecutively with a phase difference of 27tiN between firing of two successive neurons. In this state, g scales like 1IN. This state, however, is hardly seen even in simulations of homogeneous, all-to-all networks. In contrast to asynchronous state, in synchronous states there are temporal fluctuations on a global scale. The variance of the global activity, as well as the variance of the total synaptic conductance of a neuron, remain of order unity even for large N. This implies, of course, that the degree of synchrony generated by the common input from the rest of the network is itself of order unity even in the limit of large N. Sparsely connected system. In this case one cannot discriminate between asynchronous and synchronous states on the basis of the temporal fluctuations of the total synaptic current received by one neuron. Indeed, even if the action potentials of the individual neurons are weakly synchronized, their total effect on the synaptic current received by a given neuron are inversely proportional to M, the number of inputs this neuron receives. Therefore, the temporal fluctuations of the synaptic input on a specific neuron remain finite in the thermodynamic limit. However, one can still define synchronous states in a sparse network as states in which coherent temporal fluctuations of the synaptic inputs are occurring on a global scale. In such a state, the variance of the global activity, as well as the variance of temporal fluctuations of the synaptic currents averaged on the macroscopic spatial scale, remain of order unity even for large N. This implies that the degree of synchrony generated by the common input from the rest of the network is itself of order unity, and that the CC of a pair of neuron is dominated by this common input. This behavior differs from what happens in the absence of coherence in the temporal fluctuations of the inputs. Indeed, in that case the temporal fluctuations of the macroscopic spatial averages of the synaptic currents are vanishing in the thermodynamic limit as 1IN. The CC of the activity of a pair of neurons is strongly dependent on their direct interaction.

4.2.3. Population averages in asynchronous and synchronous states The above criteria of synchrony are difficult to check directly in experimental systems since it requires reliable estimates of parameters, such as the size of the network, the connectivity and the strength of connections. An alternative criterion, which works for massively connected as well as for sparsely connected systems, is based on the behavior of population averages [13]. Let us denote by xi(t) a local observable, e.g., the instantaneous rate of the ith neuron. Let us suppose that we can measure the means of this quantity over a subpopulation of size K, where K Mc, the nature of the network state for weakly synchronized initial conditions. For example, if n~ - 1 (resp. nc - 2), the system is expected to settle in a "smeared" 1-cluster (resp. 2-cluster) state in which the phase distribution is broad but monomodal (resp. bi-modal). Examples are shown below.

6.6. Sparse networks of integrate-and-fire neurons Using Eq. (67) together with Eqs. (38) and (39) one can study in a systematic way the stability of the asynchronous state as a function of the network parameters [33].

926

D. Golomb et al.

1.0-

0.5

0.0 n

n

2

4

Fig. 14.

0 O~

~

4

n

2

A graph of the function ]3(a). Adapted from [33].

Here we will describe some of the results for inhibitory coupling. We focus on the way the critical connectivity required to have an unstable asynchronous state depends on the the firing rate of the neurons, on the synaptic time constants and on the refractoriness. No refractoriness, Tr = 0: In this case, one can show that Mc varies n o n m o n o t o nously as f0 decreases; see Fig. 15A. At high fo, nc = 1, and Mc decreases with decreasing f0. At a specific value of f0 it reaches its minimum Me,1 (f0) and then increases again. As f0 continues to decrease, a value of f0 for which n~ = 2 is reached, and Mc decreases again until it reaches a second minimum. Then, it increases until nc = 3, decreases again and so on. More generally, Mc oscillates infinitely many times as f0 decreases towards 0, i.e., as I decreases toward the threshold 0. It is also possible to analyze how Mc depends on the synaptic kinetics. If rl -- 0, and in the absence of refractoriness, a, < - n / 2 for all n. Therefore, the asynchronous state is stable for every M. As rl increases from 0 (note that ~1 ~ Mc, the system is in a smeared 1-cluster state, as demonstrated in Fig. 17B. The values of Mc in this case are on the order of a few tens, which are much smaller than that for an integrateand-fire model at the same firing rate. Slowing down the kinetics of the active current (the factor ~ in the W a n g Buzsfiki model) changes the dynamics remarkably. With ~ reduced to 1 (from its reference value ~ = 5), the index of the unstable mode has changed from n =- 1 to n = 2; see Fig. 17C. As expected, the system exhibits a smeared 2-cluster state for M > Mc (Fig. 17D). The values of M~ are now larger, on the order of a few hundreds. When qb decreases in the WB model, the spikes become broader and the refractory period of the neuron increases. Therefore, these results for the WB are


A

T~ - 2 m s

I:1=1 ms

1:,=5.9 ms

4000

i

ii ili / U

3000 O

929

2000

iil

1000

I i /'

II i/]/ ,

od

z~=l

'

'

I

50

i

n=l .......... n=2 - - - - n=3 -

'160'

Frequency, fo (Hz)

ms, fo=102 Hz

i

1

i

,

40001

" '9' ' "

~%lo.oo~oo~176176

50 100 Frequency, fo (Hz) B

.//

\l

II 1

i

2000

0

'

'

'

I

2

'

'

'

I

4

'

'

'

I

6

'

'

i

T r (ms) Fig. 16. Effects of refractory period, Tr. The first three critical numbers of inputs Mc,, are plotted for an I&F model with inhibitory coupling and ~2 = 6 ms. The graphs of Mc,, are denoted by the solid, the dotted and the dashed lines for n = 1,2, 3, respectively. (A) Mc,n versus f0 for Tr = 2 ms and Vl = 5.9 ms (left) or ~l = 1 ms (right). (B) Mc,, versus Tr for ~] = 1 ms, f0 = 102 Hz. Adapted from [33].

g o i n g a l o n g with the c o r r e s p o n d i n g b e h a v i o r f o u n d a b o v e for the i n t e g r a t e - a n d - f i r e m o d e l . H o w e v e r , in c o n t r a s t to the b e h a v i o r o f i n t e g r a t e - a n d - f i r e n e t w o r k s , Mc for c o n d u c t a n c e - b a s e d n e t w o r k s is n o t very sensitive to the value o f ~l (result n o t shown).

930

D. Golomb et al.

4~-~

Ae

r

C.

200 -~

400 1

150

300 -.

~oo

200 ~

~

~

100

50

i 0 0.0

'

sd.0 . . . . .

~06.0

0o.o ~

.

Firing Rate (l/see)

DQ

T~

To

9...... :~

::"'

0.4

"

~ ~-' ~-.l ~~~ !9 ! ,, - ,

~:"~ .[

.

s6.o .

.

.

.

~o6.o .

"~sb.o

.

Firing Rate (l/see)

BO

~oo ~I

.

. ~.'

.

"I!

.'i

~oo .:i. :, ; . , , . ;

~!;,':~

... ,..~ .,..~.~..~..~'.,

~

, , - , , ~'""'~.'; ~.;.,,,~.;..,,. i l ."". , - ~''~

-~ ,oo. ~ ~ ~,.,~.,~~ ~.~, ,,.~ , ~ i ':.

50 ms

50'ms'

0.4t~J~,LLLLLLJL~ZJL

a. 0.2 0.0

0.0

Fig. 17. Synchronization properties of the Wang-Buzsfiki model. Parameters are given in Appendix A and B except that in C and D, the kinetics of the active currents are slowed down by reducing the parameter ~ from 5 to 1 [79]. (A) and (C) The critical number Mc as a function of the neuronal firing rate, calculated using the phase reduction technique. For q 5 - 5, the asynchronous state is destabilized by the first mode, and for ( l ) - 1 it is destabilized by the second mode. (B) and (D) Simulations of network of inhibitory Wang-Buzsfiki neurons. Rastergrams of 200 neurons out of N = 1000 are shown in the upper panel; M = 100 in (B) and M -- 400 in (D). A bar denoting the time period, To, is shown above each rastergram. In each lower panel, P, the number of firing neurons within a time bin of 1 ms is displayed as a function of time. For ~ -- 5 (B), the system is in a smeared one-cluster state whereas for q5 = 1 (D), the it is in a smeared two-cluster state.


931

7. Synchrony in one population of spiking neurons - beyond weak coupling

7.1. Fully connected, heterogeneous networks In order to assess the range of validity of the conclusions reached in the weak coupling limit one has to perform numerical simulations. In Fig. 18A and B, respectively, we show RE and R1 as a function of the coupling strength for integrate-and-fire networks, and a fixed value of the average firing rate in the uncoupled network, i.e., for fixed external inputs. For sufficiently small coupling strength, gsyn, the robustness varies linearly with gsyn with a slope which is correctly predicted by the phase model limit. WB networks behave in a similar way, although the asynchronous state is unstable in heterogeneous excitatory networks only in a much more restricted range of firing rates. Therefore, for both models and both types of interactions when the external input is kept fixed and the strength of the interaction is varied, the conclusions derived using the phase reduction method remain qualitatively valid on a finite range of coupling strength. In this range, excitation is more efficient than inhibition at synchronizing heterogeneous networks at low firing rates. The opposite is true at large firing rates. If the coupling strength is too strong, deviations from the weak coupling behavior for fixed firing rate are observed. The robustness RE, starts to decrease for larger values of gsyn. The robustness RI also decreases with increasing gsyn (although it may eventually increase again at larger coupling strength). This is expected since for fixed external input, increasing the excitatory interaction tends to increase firing rate and inhibitory interaction tends to decrease it. This change has an important effect on synchrony. An increase of the firing rate makes more difficult to synchronize the system because the synaptic interaction is less temporally modulated. For this reason, strong excitation (that leads to high firing rates) is not useful to synchronize the network. One can also ask the question of the relative efficacy of strong excitation and inhibition at a given firing rate, where the firing rate is kept constant independently of the coupling strength by the injection of a suitable bias current. When the synaptic interaction is modeled according to Eq. (15), this can be achieved by making the drive Ii on neuron i depend on gsyn as I i ( f , gsyn) -- I i ( f ) -- gsyn f ,

(68)

where/*;(f) is the constant drive required for neuron i to fireat frequency f and f is the average frequency of neurons across the uncoupled network. In the asynchronous state the total synaptic input on a neuron due to the rest of the network is then exactly balanced by the coupling-dependent term of the applied current, so that the frequency remains that of the single neuron. Let us first consider the case of a network of excitatory neurons for a fixed average firing rate f . Let us suppose that the average firing rate and the synaptic time constants are such that the AS state is unstable in the homogeneous case. When the coupling strength is increased and the external input decreased according to Eq. (68), the robustness increases. However if the coupling becomes too large an

932

D. Golomb et al.

0.05 9

0.04

R

m

m

[]

A

0.03 E

-

II

m

0.02

0.01 0.00 0.0

0.1

0.2

gsy,,

0.3

0.4

0.020

B 0.015

RIO.OlO 0.005

0.000 0.0

0.6

1.2

1.8

2.4

gsyn Fig. 18. Integrate-and-fire neurons. The robustness vs. the coupling strength 9syn (in nC/ cm2). The average free firing rate is f = 50 Hz. Filled squares: numerical simulations. Solid line: results from phase model. (A) Excitatory interactions, ~1 = 3 ms, x2 = 1 ms. (B) Inhibitory interactions, T 1 = 7 ms, x2 = 1 ms. The straight line in each figures denotes the prediction of the weak-coupling theory. Adapted from [32].

instability occurs because the inputs become subthreshold for too many neurons. The activity of the network cannot be controlled in spite of the fact that condition, Eq. (68), is fulfilled. This is due to the fact that the asynchronous state with average firing rate, f , is unstable through a "firing rate" instability. This instability will be analyzed in the following Section. The smaller the average rate f , the smaller the


933

excitatory coupling for which this instability occurs. Therefore, strengthening the interactions can improve to some extent the robustness RE but this is limited by the fact that on the one hand the firing rate must be sufficiently small, and on the other hand small firing rates cannot be controlled when the coupling strength is increased. For an inhibitory network, the situation is different since when the coupling is increased, the external input has to be increased to compensate for the reduction in firing rate. Moreover, when the coupling is sufficiently strong, a precise tuning of the external (depolarizing) input parameters and inhibitory synaptic strength is not essential to control the firing rate. In fact, as shown in [32], if the external and synaptic currents are large enough it is sufficient to assume that both currents are proportional in order to have a control of the firing rate without need of fine tuning of the parameters. In this balanced state the firing rate of the neurons depend only on the current-frequency relation, being independent on the detailed dynamics of the model. The constant of proportionality will be denoted by K. The robustness, RI, of an inhibitory WB network in the balanced state is shown in Fig. 19A and B as a function of K for two values of the average frequency in the large K limit, f - 50 Hz (A) and f - 200 Hz (B). In both cases a much larger robustness than at weak coupling can be achieved by sufficiently increasing K. This is because, in spite of the averaging effect of the high firing rate, the synaptic current remains strongly modulated because the coupling strength is large. The conclusions of these simulations study beyond weak coupling is that as a general trend, inhibitory interactions are more advantageous than excitatory ones for neuronal synchrony in fully connected networks, except, may be in some intermediate range of firing rate which depend on the details of the model.

7.2. Finite-size effects in sparse networks In general, for finite N, even in the asynchronous state, z(N) is not equal to 0. However, for finite but large N, according to Eq. (31), synchronous states differ from asynchronous state in the way z(N) scales with N, i.e., by the way the finite size of the system affects the level of synchrony of the neuronal activity. As explained above (Section 4.1), these finite size effects can be used, in numerical simulations, to characterize the regimes of parameters in which the network states are asynchronous or synchronous. If in the thermodynamic limit, a continuous transition occurs between asynchronous and synchronous regimes when a parameter is changed (e.g., M), this transition is replaced for finite N by a crossover between the two different scalings of Z with N. These finite size effects occur no matter what the connectivity. In networks with finite M and N, sparseness is another source of finite-size effects. The reason is that the desynchronizing effect due to the sparseness depends not only on M, but also on the relative values of M and N. For instance, in Eq. (60), the synchrony level in the system is determined by the competition between the term that includes [~ that may be synchronizing, and the distribution of mi/M that tends to desynchronize the system. If M is sufficiently large, this binomial distribution of mi/M is approximated by a Gaussian with a variance 1/M (Eq. (62)). When N is finite but large enough

934

D. Golomb et al.

0.20

"A

0.15

R I O.lO 0.05

0.001P, 0

= 5

9' . ' , ' . ~ 10 15 20 25

K

9' 30

9 35

0.030 0.025 0.020

Ri o.ols 0.010 0.005

5

10

15

K

20

25

30

35

Fig. 19. Inhibitory network of WB neurons. Robustness, Rl vs. the scaling factor K (see text), vl -- 7 ms, ~2 - 1 ms, f -- 50 Hz (A) and 200 Hz (B). Same conventions as in Fig. 18. Adapted from [32].

(say, N > 30), and N - M is also large enough, the binomial distribution of m i / M is well a p p r o x i m a t e d by a G a u s s i a n with a variance

Var

- M'

' N"

/69)

This suggests t h a t the level of synchrony should depend on M and N t h r o u g h the effective variable, Meff, defined as


1

1

1

Mefr

M

N

935

(70)

R e l y i n g o n this h e u r i s t i c a r g u m e n t , we expect t h a t Mofr is a m o r e a p p r o p r i a t e par a m e t e r t h a n M to r e p r e s e n t s i m u l a t i o n results. N u m e r i c a l s i m u l a t i o n s c o n f i r m this c o n c l u s i o n as s h o w n in Fig. 20.

1.0

A

II

0.8 0.6

I

,"'

i

.'"'"

,,,,

9

" x

0.2

I #x

N=800 N=1600 N=3200

X

0.0 0.0

N=200

x N=400

0.4

1.0-

,,"

I

1

'

1000.0

B

I

'

2000.0 M

I

'

3000.0

"1

4000.0

0.8 0.6 0.4 9

0.2

,~i-~'

~_x x ~" ~ ' ~ "

0.0 0.0

'

~'

o~/

|

I

5000.0

|

u

!

u

I

10000.0

Meff Fig. 20. Simulations of the I&F model with Tr = 0, zl = 1 ms, Z2 = 3 ms, I = 4 ~tA/cm 2 (f0 = 144.3 Hz), 9syn = - 0 . 4 ~tA/cm 2. Simulations are carried out with N = 200, averaged over 100 realizations of the network connectivity and random initial conditions (dots), N = 400, 50 realizations (crosses), N = 800, 25 realizations (dashed line), N = 1600, 20 realizations (dotted line) and N = 3200, 10 realizations (solid line). The integration time step At - 0.25 ms; The runs were performed over 4 x 104 time steps. The synchrony measure was computed after discarding a transient of 2 • 104 time steps. (A) The synchrony measure Z versus M. (B) The synchrony measure Z versus Mort (Eq. (70)). The lines for N - 800, 1600, 3200 almost coincide with each other, and the deviations of the line with N = 400 and N = 200 is small. The analytical estimate for Mc is denoted by the arrow. Note that for Meff ~ 2500, Z decreases with N " Z (x 1/v/N. Adapted from [33].

936

D. Golomb et al.

7.3. Sparse networks beyond weak coupling Here we are interested in how Mc depends on the coupling strength in the integrateand-fire model beyond the weak coupling regime. In Fig. 21, we plot results from simulations showing the synchrony measure g vs. Meff for a fixed value of the external input, I - 4 gA/cm 2, and different coupling strength, 9syn- F o r sufficiently low values of 9syn, z(M) depends weakly on 9syn, and Mc remains close to the weak coupling estimate, although it starts to increase. For larger -gsyn (e.g., 1.0 laA/cm2), the limit of stability of the asynchronous state becomes substantially different from its weak-coupling limit. Moreover, the system displays multistability: 1-cluster state (upper branch) coexists with asynchronous or 2-cluster state (lower branch). Therefore, increasing gsyn opposes synchrony instead of helping synchrony, as one would expect at first sight. This is also clear from Fig. 22A (solid line). Here we have

1.0 / /

oO

-gsyn

(l.tNcm2)

~

0.2

o

0.5

- -9- " --'"

0.4 0.6 0.8

---

J

0.0

0

~'

.

/

/

/

, 1000

1.0 1.2

J

/

" .........

J

,

, 2000

Me. Fig. 21. Beyond weak coupling: simulations of the inhibitory I&F network. The synchrony measure Z is plotted vs. Merf for different synaptic strength. Parameters are: Tr = 2 ms, Zl - 1 ms, 1:2 - 3 ms, I = 4 gA/cm 2, (f -- 112 Hz) N - 800. Simulations with 4 • 104 time steps with At -- 0.25 ms. Time averages were computed after discarding a transient of 2 x 104 time steps. The results were averaged over 100 realizations. The arrow denotes the analytical estimate for Mc in the weak coupling limit. In the asynchronous regime, the average firing rate in the network changes from f ~ 105 Hz for g s y n - - -0.2 gA/cm 2 to f ~ 80 Hz for gsyn -1.2 gA/cm 2. Adapted from [33]. =


937

plotted Z versus gsyn for a given value of M and I. Not only does one find that the synchrony of the system decreases when the coupling increases, but one also sees that there exists a critical coupling, go(M), above which the network settles into an asynchronous state. In these simulations, I was kept fixed. Therefore, the firing rate decreased when the inhibitory coupling was increased. In order to compare networks with comparable average firing rates but different coupling strength, we have performed another set of simulations in which I and gsyn increase in such a way that the average firing rate, f , in the asynchronous state was kept fixed. The results are shown on Fig. 22A (broken dashed line). Here also, Z is a decreasing function of-gsyn, although it varies more slowly than when the firing rate is not fixed. For a sufficiently strong coupling, Igsyn[ > gc(M), the network settles into a state in which neurons fire in an asynchronous and (almost) periodic fashion. By further increasing gsyn, one can keep the population average firing rate constant by increasing I. However, the neuronal activity becomes more and more irregular. This is shown in Fig. 23 where the population average of the coefficient of variation (CV) of the interspike distribution of the neurons is plotted as a function of the synaptic strength. Interestingly, it varies monotonously. Note that it varies very smoothly and that there is no apparent transition between a regime in which the neurons are firing periodically and another one in which they are firing irregularly. When the coupling is sufficiently strong, the neurons are firing with a very high CV. Moreover, the distribution across a network of the time average firing rates of the neurons becomes broad. An example of the traces of the neurons in the strong coupling regime is plotted in Fig. 24. The firing pattern of the neurons is irregular and the four neurons represented have very different firing rates. The firing pattern in the strong coupling regime is investigated further in Fig. 25 for f = 100 Hz, and gsyn - - 5 0 laA/cm 2. In this figure, we present results from the simulations of networks of different sizes (N - 400, N = 800 and N = 1600), but the same Meff (Meff = 800). Fig. 25A shows the distribution of the firing rates of the neurons. The coefficient of variation of the interspike distribution (CV) of the neurons versus their firing rates is displayed in Fig. 25B. Remarkably, the curves for the three sizes superimpose. In Fig. 25B, the three curves are indistinguishable. This means that for M and N sufficiently large, in the strong coupling regime, the dynamic state of the network depends, on M and N, through Meff. Note that the variability in the firing pattern is stronger for the neurons with the lower rate. For fixed 9syn, the lower the population average firing rate or the lower Meff, the larger the variability of the neuronal activity (result not shown). This is in agreement with results obtained by previous authors [87,88,123].

8. Stability of the asynchronous state of integrate-and-fire networks: theory at all coupling strength

8.1. The method For the integrate-and-fire networks with all-to-all connectivity and arbitrary number of populations, it is possible to analyze the stability of the asynchronous state at

938

D. Golomb et al.

A

,.

1.0-

....

fixed

I

fixed

f

0.8 0.6 0.4 0.2 0.0 0.0

'

210

-gsyn

4.'0

'

(i Ncm2)

6'.0

95=0.15 ~Ncm 2

5=0.3 gNcm

2

0.4 0.2 0.0 0.0

,

,

2.' 0

'

4.0 '

610

-gsyn (gNcm2) Fig. 22. The synchrony measure X is plotted vs. the coupling strength gsyn for M - 400, N = 800 (Merr = 800). Parameters are: Tr = 2 ms, zl = 1 ms, z2 -- 3 ms, N = 800. Simulations were carried out with 4 x 104 time steps and At = 0.25 ms. (A) N o heterogeneity. Solid line: The external current is fixed at I = 3.632 laA/cm 2. The firing rate decreases with increasing -gsyn from 100 Hz for gsyn = 0 to 56 Hz for gsyn - - 2 pA/cm 2. Dashed line: The external current increases when -gsyn increases such that the average firing rate in the asynchronous state remains constant (f = 100 Hz). (B) Heterogeneity. the external current varies from neuron to neuron and is homogeneously distributed between [ - 6 and [ + 6. The average, i,


939

0.80.6

>

o 0.4 0.2

0.0

,

0

,

.....

,

[

t

4O

'-

~

I

8O

-gsyn (1aNcm2) Fig. 23. The coefficient of variation of the interspike distribution as a function of the synaptic strength. Parameters are: Tr = 2 ms, ~l - 1 ms, "C2 - - 3 ms, f - 100 Hz. The level of external current is varied together with the synaptic strength such that the population-averaged firing rate remains constant: f = 20 Hz. Averaged are over 160,000 time steps (At = 0.125 ms). any coupling strength. This was shown by A b b o t t and van Vreeswijk [81] for a network consisting of one population of fully connected identical neurons. In the following, we present a generalization of this approach for the case of heterogeneous fully connected networks comprising both excitatory and inhibitory neurons. For simplicity, we first consider a one-population network consisting of N interacting neurons. For convenience we use the formulation in terms of reduced dimensionless parameters, see Eq. (4). To simplify the notation we drop the bar in the notations; time is measured in units of t0 = 10 ms. The firing threshold of the neurons is t? = 1. We assume that the external input is an external current, constant in time, but different from one neuron to the other. We denote by Ii, (i = 1 , . . . ,N) the external current received by neuron i. We consider the case where N is very large and where the external current is drawn at r a n d o m from a distribution P(I). The synaptic interactions will be given by Eq. (15). They will be characterized by a

is determined such that the population average of the firing rate, f , in the asynchronous state is kept constant, f = 100 Hz. Solid line: the width of the distribution is 8 = 0.15 laA/cm2; The network activity is synchronized for 1.0gA/cm2 ~ 1. In the weak coupling regime, with both S = Slin and S = Slog, we find a stable fixed point A0 whose absolute value increases monotonically with increasing input I (Fig. 6a and b). The only difference between the two gain functions is that with Slog, A0 saturates at higher values of I. In the strong coupling regime, with S = Slin, there is either one stable fixed point at A0 - 0, or the activity increases beyond all limits. With S = Slog, depending on I either one stable fixed point near 0, one stable fixed point near maximum activity, or both of these fixed points coexist. This behavior results in hysteresis: with intermediate I, depending on the initial or previous activation level A, one of the fixed points either at the low activity or at the high activity level is reached (Fig. 6c and d).

978

U. Ernst et al.

b

< 0 0

0

,//

/

/0 and Ai >/0, or the activation diverges because the interaction is too strong. With Slog, there is the possibility of multiple hysteresis phenomena. Increasing the constant input, one finds either one, two, or three stable fixed points existing simultaneously (Fig. 7a and b). The existence of hysteresis is very important, because it can implement a form of short-term memory: brief pulses of external input can excite a column, which remains activated after the input has decayed, due to the dynamics of the internal couplings. Additionally, there is a parameter range where the model can exhibit (damped) oscillations in the population activity. These solutions of the differential equations correspond to the existence of limit cycles in phase space. Limit cycles occur if there is only a single unstable fixed point of the dynamics, and if the input is sufficiently

Emergence of feature selectivity from lateral interactions in the visual cortex

b

0.4

0

A.

979

0.5

0.4

-0.5

0

I

I

1

e

Fig. 7. Hysteresis in the Wilson-Cowan model. (a) shows intersecting isoclines for three different excitatory input currents Ie (red, Ie - 0; green, Ie = 0.5; blue, Ie = 1). Fixed points are marked as in Fig. 6. (b) Depending on the excitatory input current Ie, either one (blue), two (green), or three fixed points (red) are stable, and the initial conditions determine which one is selected. Adapted from Wilson and Cowan [19], parameters were Wee = 13, Wie - - 4, Wei - - 2 2 ,

Wii "- 2, Se - - 1.5,

Si = 6, If, e ---- 2 . 5 ,

If, i = 4 . 3 ,

re = ri - - 1, "1;e - - 10, ~i = 5,

and Ii = 0. high. It can be shown that limit cycles occur naturally in coupled neuronal populations [19]. It has been speculated, that these oscillations explain the rhythmicity seen in the frequency bands of E E G recordings, or other oscillatory phenomena in brain activity. In the following, we will concentrate on nonoscillatory solutions.

2.3. Coupled columns To simulate more than a local cortical column, Wilson and Cowan extended their model and examined a chain of coupled neuronal populations [23]. The activation A is now a function of time and space, A(t) ~ A(r, t), and the synaptic input now depends not only on the activities of the populations in the same column, but also on the activities in all other columns. The products in Eqs. (1) and (2) therefore have to be replaced by the convolution of the activities with the corresponding coupling kernels Wee (r - r'), Wei (r - l't), ~/ie (r - r'), and H/ii(r - r'), where [W 9A](r, t) := j c

W(r - r')A(r')dr' /7(

Te

~Ae(r,

t)

~-----------~- -Ae(r, t) + (ke - reAe(r, t))Se([Wee * Ae] (r, t)

-[g(e ~i

* Ai](r, t ) + Ie(r, t)),

(4)

8Ai(r, t) ~----------~ = -Ai(r, t) + (ki - riAi(r,t))Si([~i * Ae](r,t)

-[Wii * Ai](r, t ) +

Ii(r, t)).

The delay of synaptic transmission from r to r ~ has hereby been neglected.

(5)

980

u. Ernst et al.

The choice of the coupling functions Wxx is crucial for the dynamics of the system. A common assumption is that excitatory couplings prevail on short distances [[r - r']], while inhibitory interactions dominate on larger distances. This leads to a coupling function having the shape of a Mexican hat (Fig. 8). It is questionable if this assumption is really fulfilled in the visual cortex. It has been shown that longrange horizontal connections spanning several hypercolumns exist [24--31], while inhibitory interactions have a limiting range of about one hypercolumn [32,33]. These long-ranging axons, however, are not distributed homogeneously but form dense clusters in columns having a similar orientation preference as the neuron from which they originate. Due to the typical structure of an orientation map in the visual cortex, it may still be possible that the interaction profile has indeed the shape of a Mexican h a t - at least in the young animal, where long-ranging excitatory connections have not been developed yet. For convenience, we will also assume that the coupling functions are chosen such that fc

d r ' W ~ ( r - r') = Wx~. TX

In many cases, it is reasonable to reduce Eqs. (4) and (5) introducing the following simplifications: first, let us assume that the neurons have a vanishing absolute refractoriness, r i - - r e - - 0 . Second, axons originating from one population should contact excitatory and inhibitory neurons in a given distance with equal relative probability, Wee cx Wei and Wie o( g~i- Our last assumption is that the remaining parameters and the gain functions in Eqs. (4) and (5) are identical for inhibitory and

r,.

x !

x v

-50

0 r-r x

50 x

Fig. 8. Excitatory couplings We(rx- r'x) (green) having a shorter length scale than inhibitory couplings ~ ( r x - r'x) (red) lead to a coupling function W ( R ~ - r'~)= We- N (black) having the shape of a mexican hat. Parameters of the coupling functions chosen (see Eqs. (7) and (8)) are We = 14, wi = 12.5, ere = 5.6, cyi = 10, and d = 1.


981

excitatory populations, re = r i , ke = ki, and Se = Si. Under these assumptions, Eqs. (4) and (5) are redundant and can be replaced by a single integro-differential equation

~A(r, t)

17

~---7---= -A(r, t) + S([We * A](r, t) - [ < 9A](r, t) + I(r, t)).

In the following section, we will assume a threshold-linear gain function S make the following choice for the coupling functions We and N:

_ We(r - r')

We

(2 g)d/2(Y d e x p

__

Wi

(2 rt)d/zcr~ exp

(r - r')

(,r-r',,2) 2 cy2

--

(r-r',,2) --

2 cy2

(6) =

Slin

and

(7) '

(8)

"

The results are qualitatively identical for other choices, exceptions will be discussed. s will be termed the gain, If the firing threshold, We the excitatory coupling strength, wi the inhibitory coupling strength, and Cre and cri, with ~e < c~i, the excitatory and inhibitory coupling length scales, respectively, d denotes the dimensionality of the neuronal tissue.

2.3.1. Dynamics of coupled columns Constant input." Linear, marginally stable, and diverging regimes. For simplicity, let us first consider a one-dimensional chain of length lx, with periodic boundary conditions. This chain is stimulated with a constant external input I ( r , t ) = I0 = const. > If. What activation dynamics do we expect from the model in Eq. (1.6)? It is simple to calculate that there exists one single spatially homogeneous fixed point A0 > 0, if wi > We - 1/s (for arbitrary coupling functions) A0(r, t) = A0 =

(Io - l d

1 - S(We -

wi)"

Is this fixed point stable? After linearization of the dynamics and solving the characteristic equation by applying the Fourier transformation, one obtains the following spectrum of eigenvalues )v(k):

1 I

a(k) = --+-

,17 ,17

weexp

If there exists one k for which stable. For our choice of eye < )~(kl) is always negative for V ' - c r e2/cr2 < 1. Therefore it is This is the case if

2

-wiexp

( 2 2)1 -

2

.

)v(k) > 0, then the homogeneous fixed point is un~i, 2 has one or two extrema at kl = 0 and k2 > 0. A0 > 0, and k2 does not exist for wi < we/V, crucial to know if)v(k2) is also negative ifwi > we/V.

U. Ernst et al.

982

wi > We ~" V. [(1 - V)-s] -}-1.

(9)

Summarizing these considerations, we have three stability conditions B1-B3: 9 B 1: Existence of a homogeneous fixed point for Wi > We - - 1Is. 9 B2: The homogeneous fixed point is stable for wi < we/V. 9 B3: For wi > we/V, the homogeneous fixed point is stable if condition (9) is fulfilled. Due to these conditions, the phase diagram of the activation dynamics depending on the lateral coupling strengths shows three different regions (Fig. 9a). If the inhibitory coupling is strong enough, the homogeneous fixed point is stable and every perturbation decays exponentially (linear regime) (Fig. 9b). If the excitatory weights become much stronger, the fixed point becomes unstable and the smallest perturbation of the homogeneous external input leads to an exponentially increasing activity. Due to the threshold in the gain function S, however, there is a subregion where the activity converges into an inhomogeneous stable state, the so-called marginally stable regime (Fig. 9c), which will be described later. If the excitatory interaction becomes too strong, the nonlinearity of the coupling function does not suffice to limit the diverging activity (diverging regime). We want to emphasize that this divergence could be avoided by using a saturating gain function as e.g., 5'1o8. In cortical neurons, however, neurons barely operate near their limits - so the distinction between the divergent and marginally stable states could help to find an operating regime where the activity level of the cortical neurons is regulated only by the network, and not by the internal dynamics of a single element.

Nonconstant input in the marginally stable regime. In the linear regime, spatially inhomogeneous input leads to a similar activity distribution, because afferent input dominates over lateral feedback. In the marginally stable state, however, the activation dynamics becomes much more interesting. Let us assume that we have a neuronal chain of length lx with periodic boundary conditions, and that there is a positive perturbation in the input at the position r~x = lx/2. Two possible realizations of this input would be:

' 1 = 10(1 +

cos(2

(lo)

(rx -

Due to the excitatory interactions prevailing on short distances, this perturbation becomes enhanced, while the activity in the surround becomes suppressed by the inhibition. This leads to a localized activation blob centered around 1~/2. If the afferent input is suprathreshold everywhere, other blobs appear in a specific distance which is determined by the length scales of the excitatory and inhibitory interactions. In Fig. 9c, this distance is about half of the size of the chain, such that two activation clusters appear. This picture does not change significantly in higher dimensions: in a two-dimensional cortex, the activation clusters typically


983

marginally stable regime

100

0 -2

-4 0

1

2

0

linear regime

1

2

50 divergent regime 0 0

I

n

,

i

20

30

W

b

20

0

,

i

40

50

e

1

2

60

c

200 I

n

m

m

m

m

m

|

m

u

m

u

m

u

Neuro-informatics and Neural Modelling (Handbook of Biological Physics)

Neuro-informatics and Neural Modelling (Handbook of Biological Physics)

Neuroinformatics