Self-Organization in the Evolution of Speech (Studies in the Evolution of Language)

This page intentionally left blank Self-Organization in the Evolution of Speech STUDIES EVOLUTION IN THE OF LAN...

Author: Pierre-Yves Oudeyer

71 downloads 1219 Views 2MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form

DOWNLOAD PDF

This page intentionally left blank

Self-Organization in the Evolution of Speech

STUDIES

EVOLUTION

IN THE

OF

LANGUAGE

General Editors James R. Hurford, University of Edinburgh Frederick J. Newmeyer, University of Washington PUBLISHED 1 The Origins of Vowel Systems Bart de Boer 2 The Transition to Language Edited by Alison Wray 3 Language Evolution Edited by Morten H. Christiansen and Simon Kirby 4 Language Origins Evolutionary Perspectives Edited by Maggie Tallerman 5 The Talking Ape How Language Evolved Robbins Burling 6 Self-Organization in the Evolution of Speech Pierre-Yves Oudeyer translated by James R. Hurford IN PREPARATION The Origins of Language Jean-Louis Dessalles translated by James Grieve PUBLISHED

IN

ASSOCIATION

WITH THE

SERIES

Language Diversity Daniel Nettle Function, Selection, and Innateness The Emergence of Language Universals Simon Kirby The Origins of Complex Language An Inquiry into the Evolutionary Beginnings of Sentences, Syllables, and Truth Andrew Carstairs McCarthy


Pierre-Yves Oudeyer Translated by James R. Hurford

3

3 Great Clarendon Street, Oxford OX2 6DP Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide in Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With offices in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam Oxford is a registered trade mark of Oxford University Press in the UK and in certain other countries Published in the United States by Oxford University Press Inc., New York c Pierre-Yves Oudeyer 2006 c James R. Hurford 2006 English translation The moral rights of the author have been asserted Database right Oxford University Press (maker) First published 2006 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, or under terms agreed with the appropriate reprographics rights organization. Enquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above You must not circulate this book in any other binding or cover and you must impose this same condition on any acquirer British Library Cataloguing in Publication Data Data available Library of Congress Cataloguing in Publication Data Data available Typeset by SPI Publisher Services, Pondicherry, India Printed in Great Britain on acid-free paper by Biddles Ltd., King’s Lynn, Norfolk ISBN 0–19–928914–X 978–0–19–928914–1 ISBN 0–19–928915–8

978–0–19–928915–8

To Cécile and Arthur


Contents

Preface List of Figures

X xii

1. The Self-Organization Revolution in Science 1.1 Self-organization: a new light on nature 1.2 Language origins 1.2.1 Interdisciplinarity 1.2.2 Computer modelling 2. The Human Speech Code 2.1 The instruments of speech 2.2 Articulatory phonology 2.3 The organization of the speech code: universals 2.3.1 The speech code is discrete and combinatorial 2.3.2 The speech code is a classification system shared by the whole linguistic community 2.3.3 Statistical regularities in the phoneme inventories of human languages 2.4 The diversity of speech codes 2.5 Origins, development, and form 3. Self-Organization and Evolution 3.1 Self-organization 3.1.1 Rayleigh–Bénard convection 3.1.2 Ferro-magnetization 3.2 Self-organization and natural selection 3.2.1 Classic neo-Darwinism 3.2.2 Self-organization: constraining the search space 3.2.3 Evolutionary explanations: function is not enough 3.2.4 Exaptation 3.3 Explaining the origin of living forms

1 1 6 7 9 14 14 16 21 22 24 25 28 29 32 32 32 35 38 39 40 42 48 51

viii

Contents

4. Existing Theories 4.1 The reductionist approach 4.2 The functionalist approach 4.3 Operational scenarios 4.4 Going further 5. Artificial Systems as Research Tools 5.1 What is the scientific logic? 5.2 What is the point of constructing artificial systems? 6. The Artificial System 6.1 Mechanism 6.1.1 Assumption 1: neural units 6.1.2 Assumption 2: perceptuo-motor correspondences 6.1.3 Assumption 3: perception and plasticity 6.1.4 Assumption 4: production 6.1.5 Assumption 5: initial distribution of preferred vectors 6.1.6 Assumption 6: no coordinated interactions 6.1.7 What is not assumed 6.2 Dynamics 6.2.1 The case of uniform initial distribution 6.2.2 The case where the initial distribution is non-uniform 6.3 Categorization and acoustic illusions 7. Learning Perceptuo-motor Correspondences 7.1 The articulatory synthesizer and a model of vowel perception 7.2 Dynamics: predicting human vowel systems 8. Strong Combinatoriality and Phonotactics 8.1 Temporal neurons and their self-organized death 8.2 The dynamic formation of phonotactics and patterns of combinations 8.3 The impact of articulatory and energetic constraints 9. New Scenarios 9.1 Compatibility with neuroscience 9.2 Contribution to scenarios of the origins of speech 9.2.1 An adaptationist scenario: an origin linked to the evolutionary advantage of linguistic communication systems

53 53 56 58 65 68 68 70 75 75 76 77 80 83 84 86 86 86 86 96 97 106 111 113 123 124 126 133 139 140 143

143

Contents

ix

9.2.2

Another adaptationist scenario, with the exaptation of discreteness, shared categorization, and combinatoriality 9.2.3 An exaptationist scenario in which the origin of the whole speech system results from architectural side effects 10. Constructing for Understanding

147 150

Bibliography Index

155 163

146

Preface

The extraordinary capacities of the human brain have fascinated me for a long time. It is without doubt the most complex system we know. Reading the works of the founding fathers of artificial intelligence (von Neumann, Turing, Minsky among others) made me aware that computers could be a crucial instrument in our quest to understand the brain. These machines have the potential to play the same role for the cognitive sciences as particle accelerators play in physics: they make it possible to re-create in a controlled environment simpler versions of the brain, while keeping to an interesting level of complexity. The use of the computer, a calculating machine, to simulate and study natural phenomena is not new: Pascal used his little calculator to simulate the behaviour of mathematical series, Lorenz used the first computers to study the behaviour of climatic models, Fermi to study the interactions between magnetized particles, Turing to imagine how processes of morphogenesis could be self-organized, von Neumann to study self-replication. Later, my meeting with Luc Steels, who invited me to work in his research team, made me realize that there is one subject whose study could show itself particularly useful for understanding cognition, namely language, and especially the origins of language. In addition, after several years, research into the origins of language underwent spectacular development and mobilized the energies of researchers from very different scientific cultures: linguists, biologists, philosophers, anthropologists, ethnologists, primatologists, neuroscientists, and researchers in artificial intelligence. It was therefore quite natural for me to undertake research in this area, obviously with the idea that using a computer would be the touchstone. Just as quickly, I chose to focus on studying the origin of speech, the physical supporting medium of language, which seemed to me an ideal compromise between complexity and generality. By way of example, Jakobson’s work in phonetics in the first half of the twentieth century established the bases of structuralism which were, and still are, highly influential in all domains of Western thought.

Preface

xi

This book is a synthesis of the results of these years of research. It is bound together by the concept of self-organization, a property by which complex systems spontaneously generate organized structures, and by the role which selforganization could have played in the evolution of speech. From a theoretical point of view, it participates in the revolution in the sciences of complexity which took place in the second half of the twentieth century, and which has already made possible the reconceptualizing of whole sections of the physical and biological sciences. An example of this is the new understanding of the architectural and social structures of insect societies. My aim is to contribute to this momentum, showing how the concept of self-organization makes possible a better understanding of a fundamental phenomenon of human culture, the origin of speech. In this book, however, the reader will not find a definitive answer to this question. We are still a long way from resolving it. The book tries, rather, to explore and structure the space of hypotheses, and to develop conceptions and intuitions about the complex dynamics of speech. In short, it should be read as a tool for thinking about the origin of speech, which I hope will find a place in the construction of theories about the origin of language. I thank Luc Steels for welcoming me into his team at the Sony Computer Science Laboratory in Paris, as well as for the confidence and support which he has extended to me in this research work. His visionary papers on the origin of language are among the main sources of inspiration for my work. I also thank Frédéric Kaplan for stimulating discussions which enabled me to explore many aspects of my work, as well as for his critical commentary on the text. I have learned a lot from him on how to present my ideas. Thanks to Nicole Bastien for the time she took to reread this text and the articles which preceded it. I am also grateful to Michael Studdert-Kennedy, Bart de Boer, Louis Goldstein, and Jim Hurford, who helped with the elaboration of my arguments by the constructive rejoinders they made on careful reading of my work. The encouragements, kindness, and open-mindedness of Michael Studdert-Kennedy have been a precious help. I am indebted to Jim Hurford for his editorial work, which improved the quality of the text, as well as for his faithful and precise translation, and to John Davey for his amiable comportment. Finally, I give special thanks to my wife, Cécile, for the energy she gives me each day, and which has motivated me during the writing of this book.

List Of Figures

1.1. 1.2. 1.3. 2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 2.7. 2.8. 2.9. 2.10. 2.11. 2.12. 2.13. 3.1. 3.2. 3.3. 3.4. 3.5. 3.6. 3.7. 3.8. 3.9. 5.1. 6.1. 6.2. 6.3. 6.4.

The forms and patterns in inorganic nature The self-organization of ice crystals The self-organization of termite nests. The vocal tract The cochlea The basilar membrane The constriction variables in articulatory phonology The places of constriction An example of a gestural score: the word ‘pan’ The relation between gestures and phonemes The three representational spaces of speech Distribution of counts of vowel inventories Distribution of counts of consonant inventories The perceptual magnet effect Distribution of consonants in UPSID Distribution of vowel systems in UPSID Self-organization of parallel stripes in Bénard convection Convection currents in Bénard liquids for different temperatures Self-organization of hexagons in Bénard Convection The edge of chaos in ferromagnetic structures Magnetization dynamics in a 2D ferromagnetic model The growth and form of shell shapes The growth and form of fishes’ morphology Hexagonal cells in beehives and in packed water bubbles Non-adaptive stripes in molluscs Cellular automaton with growth of crystal-like structures Neurons have a Gaussian activation function General architecture of a full system A simplified architecture The perception of dynamic vocalizations

2 3 4 15 16 17 18 19 20 20 21 23 23 26 27 27 33 34 35 37 37 45 46 47 49 72 78 79 80 81

List of Figures 6.5. 6.6. 6.7. 6.8. 6.9. 6.10. 6.11. 6.12. 6.13. 6.14. 6.15. 6.16. 6.17. 6.18. 6.19. 6.20. 6.21. 6.22. 7.1. 7.2. 7.3. 7.4. 7.5. 7.6. 7.7. 7.8. 7.9. 7.10. 7.11. 7.12.

Updating of activation function when a stimulus is perceived Producing a vocalization Initial distribution of preferred vectors in two agents The crystallized distribution of preferred vectors Examples of crystallized distributions of preferred vectors Evolution of the entropy of preferred vectors distributions Evolution of the kl-distance of preferred vectors distribution Varying σ 2 The phase space Examples of systems generated for different values of σ 2 Initial biased distribution of preferred vectors in two agents Crystallized distributions with biases Examples of crystallized biased distributions Initial distribution of preferred vectors in two dimensions Perceptual warping at the beginning of the simulation Corresponding neural maps and basins of attraction after crystallization The categorization mechanism with the coding–decoding cycle Examples of landscapes of basins of attraction after crystallization Agents’ architecture when learning the perceptuo-motor correspondences. The perceptual neural maps of two agents shortly after the beginning of the simulation Crystallization of neural maps on a common five-vowels system Another example of a system that can be obtained. This is “an eight vowel system” Another example “of a vowel system containing five vowels.” Another example “of a vowel system containing six vowels.” Evolution of entropy and kl-distance of preferred vectors distributions Further examples of vowel systems The UPSID database Identifying patterns in the artificial vowel systems Distributions of sizes of vowel systems obtained in the simulations and in the UPSID database Distribution of vowel inventories obtained in the simulations and in the UPSID database

xiii 82 84 87 88 89 90 90 93 94 95 97 97 98 101 102 103 104 105 107 114 115 116 116 117 117 118 119 119 121 121

xiv 8.1. 8.2. 8.3. 8.4. 8.5. 8.6.

List of Figures

The spatial and temporal maps of two agents Crystallized temporal map if the pruning mechanism is not used Crystallized temporal map using the pruning mechanism Another example of a crystallized temporal map Evolution of the number of surviving temporal neurons Another example of evolution of the number of surviving temporal neurons 8.7. Initial biased distributions of preferred vectors in the spatial map 8.8. Representation of the initial biases due to non-linearity and energetic cost 8.9. Distribution of surviving temporal neurons in 500 simulations.

127 128 129 130 131 132 134 136 137

1. The Self-Organization Revolution in Science

1.1 Self-organization: a new light on nature Nature, especially inorganic nature, is full of fascinatingly organized forms and patterns. The silhouette of mountains is the same whether one views it at the scale of a rock, a summit, or a whole mountain range. Sand dunes often arrange themselves in long parallel stripes. Water crystallizes into symmetrical serrated flakes when the temperature is right. And when water flows in rivers and hurtles over cataracts, trumpet-shaped vortices appear and the bubbles collect together in structures which are sometimes polyhedral. Lightning flashes draw plant-like branches in the sky. Alternating freezing and thawing of the rocky ground of the tundra leaves polygonal impressions in the earth. The list of these forms rivals many human artefacts in complexity, as can be seen in Figure 1.1. And yet they have no designer, not even natural selection, Dawkins’s (1982) ‘blind watchmaker’. What, then, are the mysterious factors that explain their existence? In fact, all these organized structures have a feature in common: they are the macroscopic outcomes of local interactions between the many components of the system from which they emerge. Their global organizational properties are not to be found at the local level. Indeed, the properties of the shape of a water molecule, as well as of its individual physico-chemical components, are qualitatively different from the properties of ice crystals (see Figure 1.2), whirlpools, or polyhedral bubbles. The polygonal impressions in the tundra do not correspond with the shape of the stones composing them, and have a spatial organization quite different from the temporal organization of freezing and thawing. This is the hallmark of a newly discovered phenomenon—self-organization. This fundamental concept is the touchstone of the paradigm shift driven by the sciences of complexity in the twentieth century, developed by brilliant researchers such as William Ross Ashby, Heinz Von Foerster, Ilya Prigogine, Francesco Varela , René Thom, and Stuart Kauffman. Ever since Newton,

2


FIGURE 1.1. Nature is full of organized forms and patterns without there being anywhere any plans which might have served to build them; they are said to be selforganized. Here, parallel stripes running through sand dunes, water bubbles on the surface of liquid which has been stirred up and the polyhedral structures which are left when they dry out, an ice crystal, mountains whose shapes are the same whether one views them on the scale of a rock or a whole peak. (Photos: Nick Lancaster, Desert Research Institute, Nevada; Burkhard Prause, University of Notre Dame, Indiana; Bill Krantz, University of Colorado.)

The Self-Organization Revolution

3

self-organization

water molecules

snow crystal

FIGURE 1.2. The self-organization of ice crystals.

good science has been supposed to be reductionist, and has consisted in decomposing natural systems into simpler subsystems. For example, to understand the functioning of the human body, it was appropriate to study the respective parts, such as the heart, the nervous system, or the limbic system. It did not stop there: study of the nervous system, for example, was subdivided into study of the cortex, of the thalamus, or of the peripheral motor innervations, and each of these subparts was studied by hyper-specialists in separated, dedicated university departments. This method has obviously enabled us to accumulate an impressive bank of knowledge. But the prophets of complexity have broken up this paradigm. Their credo is: ‘The sum of the parts is greater than the parts taken independently.’ This is because nature is composed of complex systems with many interacting subsystems, and complex systems have a very strong tendency to self-organize. This includes even systems in biology, in which the ascendancy of natural selection is not total but must work alongside self-organization (Kauffman, 1996; Ball, 2001). This is why it now seems that many natural systems cannot simply be explained by a reductionist study of their parts. One of the most emblematic examples is that of the collectively built artefacts of insect societies (Camazine et al., 2001). For example, termites make immense nests, rising several metres above the ground and with an architecture which is reminiscent of human structures, as Figure 1.3 shows. To try to explain how these structures are built, the study of individual termites, for example the precise study of all their neural wiring, is absolutely not sufficient. One could know everything about the anatomy and the brain of a termite without ever understanding how their

4


FIGURE 1.3. The architecture of termite nests is the self-organizd result of local interactions among thousands of individuals.

nests are built; because no termite has the equivalent of a plan, even partial, of the superstructure. The knowhow that they possess is infinitely more basic; it is of the type ‘if I come across a lump of earth, pick it up and place it where the pheromone signal is strongest’. The superstructure is rather the result of the dynamic interactions in the environment of thousands of termites, in the same way as the symmetrical structure of ice crystals is the result of the interactions of water molecules, and not a projection to the macroscopic level of structures already present at the microscopic level. Such concrete examples of the use of the concept of self-organization, and of explanations of natural forms in terms of systemic properties, are now abundant, and are at the heart of the most advanced research of more and more physicists and biologists. Further examples include hunting or foraging patterns of bees and ants, the dynamic shapes of shoals of fish or flocks of birds, symmetrical patterns on butterfly wings, the regular spots on a leopard’s skin, the stripes of fish and shellfish, the magnetization of magnets, the formation of whirlpools in rivers, the birth of galaxies, demographic oscilla-


5

tions in predator–prey ecology, the formation of patterns in bacterial cultures and chemical reaction–diffusion systems, crystallization, lasers, superconductivity, the distribution of the sizes of avalanches, auto-catalytic chemical systems, the formation of lipid membranes, and the dynamics of traffic jams on freeways (see Ball, 2001). The sciences of complexity have thus demonstrated the fundamental usefulness of the concept of self-organization for the explanation of natural phenomena involving physical structures and certain biological structures characterizing the morphology or behaviour of simple animals like insects. We are now at the dawn of a new and decisive phase in this scientific revolution: researchers in complexity theory are beginning to tackle the understanding of mankind itself, using these new tools. The understanding of the vital functions of the human body was the first to be transformed by the emerging wave of what is called “integrative” or “systemic” biology (Chauvet, 1995; Kitano, 2002). Rather than concentrating on each organ in isolation, there is now an attempt to understand their complex interactions in an organism considered as a whole in which each element is integrated with the others. This has opened up new theoretical vistas, for example the understanding of cancers (Kitano, 2004) or of morphogenesis (Kupiec and Sonigo, 2000), which, according to the authors, is seen not as the serial execution of a genetic programme but as the self-organized dynamic of the whole ecosystem formed by the cells competing for nourishment. The advocates of self-organization do not stop there: the human brain, and thus the phenomena of sensation and thought, are also under the strong influence of features of spontaneous organization in their structure. Indeed, the brain, composed of billions of neurons dynamically interacting among themselves and with the outside world, is the prototype of a complex system. For example, as I will show in this book, self-organization could be at the heart of the capacity of our brains to categorize the perceived world, that is, to organize the continuous flux of perceptions into atomic psychological objects. But the main subject of this book goes beyond speculation about the brain as a self-organized system. We are today on the brink of a major advance in science: that of a naturalized understanding of what makes humans so exceptional, their culture and language. Indeed, if culture and language have been the subjects of investigation by social sciences for centuries, scientists have not yet succeeded in anchoring an understanding of them in terms of

6


their material biological substance, i.e. the set of human brains in ongoing complex dynamic interactions. Now, the tools of complexity begin to make it possible. This is what I will illustrate in this book, concentrating on a quite precise example, that of the origin and shaping of one of the pillars of human language—speech, the outward form and vehicle of language—seen as systems of shared and combinatorial sounds particular to each language community. To understand the ongoing revolution on this particular question, I will first outline the main trends in its history.

1.2 Language origins It is an obvious fact, only matched by the mystery that follows from it: humans speak. It is their main activity, an activity which, moreover, sets them off from the rest of the animal kingdom. Human language is a communication medium of unequalled complexity. It is a conventionalized code which lets one individual share his ideas and emotions with others, talk of the colours in the sky and also of distant landscapes, of past events, even of how he imagines the future, of mathematical theorems, of invisible properties of matter, and of language itself. Besides that, each language defines a system which is peculiar to its speakers, an original way of organizing sounds, syllables, words, and sentences, and of spelling out the relationship between these sentences and the concepts which they convey. Today there are thousands of languages spoken in human communities. Over time, some languages die and others are born. The number of languages which have existed is estimated at over half a million. It is hard to imagine humanity without language. And yet, a long time in the past, humans did not speak. This raises one of the most difficult questions in science: how did humans come to talk? A further question follows naturally: how do languages evolve? These two questions, concerning the origin of the language faculty and the evolution of languages, have been focused on by many thinkers in centuries gone by, particularly in the nineteenth century. They are prominent in Darwin’s speculations (Darwin, 1999[1859]). Many such theories were developed without the benefit of any empirical or experimental constraint. They were equally devoid of reasoned arguments and scientific method, to the point where the Linguistic Society of Paris ruled that such questions should be raised no more in the context of scientific discussion. This ruling initiated a century of almost total lack of progress in research in this domain.


7

Advances in neuroscience, cognitive science, and genetics towards the end of the twentieth century have put these questions back into the centre of the scientific arena. On the one hand, modern neuroscience and cognitive science have made enormous progress in understanding the general functioning of the brain, and especially the way in which language is acquired and processed. These developments have allowed the study of language to relate to the natural sciences, that is, to ground the abstract systems which linguists describe in the biological matter of which humans and their environment are composed. In short, natural sciences have taken over questions previously in the domain of the social sciences. This new light on the workings of language and the brain have provided researchers with the constraints whose absence undermined the speculations about the origins of language of the nineteenth century. On the other hand, progress in genetics has turned the spotlight on neoDarwinian theories of evolution, both confirming some of its foundations (with the discovery of genes, for example, along with their mechanisms of variation) and allowing its predictions to be tested, often successfully, thanks to the sequencing of the genomes of different species of animals so that we can reconstruct their phylogenetic trees and trace their evolutionary history. In particular, the sequencing of the human genome, along with that of other animals such as chimpanzees and monkeys, makes it possible to specify the relationships between humans and their ancestors. Thus, driven by vigorous evolutionary biology, which simultaneously provides an impressive body of observations and a solid explanatory framework, the question of human origins has become a central theme in science. And, quite naturally, the origins of language (this being one of the distinctive features of modern humans) has become, as in the nineteenth century, a beacon for research.

1.2.1 Interdisciplinarity There is an emerging consensus among researchers who are today getting down to questions of the origin of the human language faculty and the evolution of languages: this research must be interdisciplinary. In fact it poses a puzzle with immense ramifications which go beyond the competence of each individual discipline. First, the two big questions must be decomposed into subquestions which are themselves already quite complex. What, in fact, is the language faculty? What is a language? How are sounds, words, sentences, and representations of meaning related to each other? How does the brain

8


represent and process these sounds and sentences and the concepts which they convey? How do we learn to speak? What are the respective roles of nature and nurture? What is language for? What is its role in a community? How does a language form and change in the course of successive generations of speakers? What do we know of the history of each particular language? Why are the language faculty and languages the way they are? Why do we see universal tendencies and at the same time great diversity in languages? How does language influence the way we perceive and understand the world? What do we know of the history of the human capacity for speech? Is it mainly the result of genetic evolution, like the evolution of the eyes, or a cultural invention, like writing? Is language an adaptation to a changing environment? An internal change in an individual which increased its chances of reproduction? Is it an exaptation, a side effect of changes which were not at first tied to communicative behaviour? What are the evolutionary prerequisites which paved the way for the capacity of speech? And how did these prerequisites appear? Independently? Genetically? Culturally? Ranged against the diversity of these questions is an even greater diversity of research disciplines and methods. Linguists, even though they may continue to provide crucial data on the history of languages, are no longer the main actors. Developmental and cognitive psychology and neuropsychology carry out behavioural studies of language acquisition and language pathology, and these often reveal cognitive mechanisms involved in language processing. Neuroscience—especially with equipment for brain imaging allowing us to see which brain regions are active for given tasks—attempts to find neural correlates of verbal behaviour, to discover its organization in the brain. Some researchers also study the physiology of the vocal tract, to try to understand how we produce speech sounds. The physiology of the ear, the essential receptor in the speech-decoding chain (or vision, in the case of signed languages), is also a focus of research. Archaeologists examine fossils and artefacts left by the first hominids, and try on the one hand to deduce our anatomical evolution (especially of the larynx) and on the other to get an idea of what activities they were engaged in (what tools did they make? and how did they use them? and what can these tools tell us about the degree of cognitive development?). Anthropologists do fieldwork on isolated peoples and report on cultural differences, especially those related to languages and the meanings they convey. Primatologists try to report on the communicative capacities of some of our ancestors and to compare them with our own. Geneticists on the one hand sequence the human genome and that of potential ancestral species when it is


9

possible to specify their phylogenetic relatedness, and on the other hand use genetic information from different people across the planet to help in reconstructing the history of languages, which is often correlated with the genetic history of their speakers. Thus language involves a multitude of components interacting in complex ways in parallel on several time-scales: the ontogenetic time-scale, characterizing the growth of an individual person; the glossogenetic or cultural timescale, which characterizes the evolution of cultures; and the phylogenetic time-scale, which characterizes the evolution of species. Moreover, not only is it essential to study each of these components independently, to reduce the complexity of the problem; it is also necessary to study their interactions. In fact, as I stressed in the first part of this chapter, the sciences of complexity have taught us that in many of the complex systems in nature, there are global phenomena that are the irreducible result of local interactions between components whose individual study would not allow us to see the global properties of the whole combined system. Thus, a growing number of researchers think that many properties of language are not directly encoded by any of the components involved, but are the self-organized outcomes of the interactions of the components. Yet these self-organizational phenomena are often complicated to understand or to foresee intuitively, and to formulate in words.

1.2.2 Computer modelling It is for this reason that we find, in addition to the scientific activities already mentioned, and in the framework of language origins research, work by researchers in artificial intelligence, mathematicians, and theoretical biologists, who construct operational models of these interactions between the components involved in language. An operational model is one which defines the set of its assumptions explicitly and above all shows how to calculate their consequences, that is, to prove that they lead to a certain set of conclusions. There are two main types of operational model. The first, used by mathematicians and some theoretical biologists, consists in abstracting from the phenomenon of language a certain number of variables, along with the rules of their evolution in the form of mathematical equations. Most often this resembles systems of coupled differential equations, and benefits from the framework of dynamic systems theory. The second type, which allows for modelling of more complex phenomena than the first, is that used by researchers in artificial intelligence:

10


it consists in the construction of artificial systems implemented in computers or in robots. These artificial systems are made of programs which most often take the form of artificial software or robotic agents, endowed with artificial brains and bodies. These are then put into interaction with an artificial environment (or a real environment in the case of robots), and their dynamics can be studied. Building artificial systems in the context of research into language origins and the evolution of languages is enjoying growing popularity in the scientific community, exactly because it is a crucial tool for studying the phenomena of language in relation to the complex interactions of its components. These systems are put to two main types of use: (1) they serve to evaluate the internal coherence of verbally expressed theories already proposed by clarifying all their hypotheses and verifying that they do indeed lead to the proposed conclusions (and quite often one discovers errors in the assumptions as well as in the conclusions, which need to be revised); (2) they serve to explore and generate new theories, which themselves often appear when one simply tries to build an artificial system reproducing the verbal behaviour of humans. A number of definitive results have already been obtained, and have opened the way for the resolution of previously unanswered questions: the decentralized generation of lexical and semantic conventions in populations of agents (e.g. Steels, 1997; Kaplan, 2001); the formation of shared inventories of vowels or syllables in groups of agents, with features of structural regularities greatly resembling those of human languages (e.g. de Boer, 2001; Oudeyer, 2001b; 2001c; 2001d); the formation of conventionalized syntactic structures (e.g. Batali, 1998); the conditions under which combinatoriality, the property of sytematic reuse, can be selected (e.g. Kirby, 1998).1 The work to be presented in this book belongs in this methodological tradition of building artificial models. It will concentrate on the origin of one particular aspect of language: speech sounds. Sounds, as I will explain in detail in the next chapter, constitute a conventional code providing each language with a repertoire of forms for conveying its messages. This code, which has an acoustic and an articulatory side, organizes sounds into categories which are special to each linguistic community, and regulates the ways in which they can be combined (these rules of sound-syntax are also cultural conventions). The system is thus discrete and combinatorial. Without such a code, which could also be implemented in a manual modality for signed languages, there 1 This list is in no way exhaustive, and more examples can be found in Cangelosi and Parisi (2002).


11

could be no form, no content, and hence no linguistic communication. How could such a code have arisen? In particular, how were the first codes able to form themselves before there was any conventional linguistic communication, of which they are prerequisites? Why are the sound patterns of human languages the way they are? Such are the questions which drive the work in this book. They are simultaneously very ambitious, as they include many complex aspects, both individual and social, and very modest in relation to the overall programme of research into the origin of language and the evolution of languages. In fact, they only concern the origin of one prerequisite of language among many others (such as the capacity to form symbolic representations, or the pragmatic capacity to infer the intentions of others by means of behavioural signals). I do not claim to suggest direct or definitive answers. This book is rather of the type of exploratory philosophy which drives the construction of artificial systems. In this spirit, I construct a population of agents, endowed with welldefined mental, vocal, and perceptual capacities, based more or less exactly on their real human counterparts, which will enable us to establish the sufficient conditions for the formation of speech codes like those of humans. It will be shown that these sufficient conditions are interesting when their generality and simplicity is compared with the structures that they generate, namely speech codes. It is the phenomenon of self-organization which enables this linking between qualitatively different properties of the same system both on a local and on a global scale. This enables us to to define and suggest a possible and original kind of mechanism for answering the questions posed above, and to show its internal coherence. It will also be explained why these conditions are interesting in that they not only support a classical adaptationist, neo-Darwinian scenario of the origins of speech, but also open new perspectives, and in particular make possible a scenario in which the speech code, or at least some of its properties, might be an exaptation2 . I will stress that the relationship between the artificial system and the human system is not a relationship of identity or close modelling; it is a relationship of analogy. I will not attempt to show that these assumptions correlate with precise features of the real world,3 but rather that they are useful for defining the dimensions of the space of possible explanations which have already been proposed, and indeed for generating new types of explanation. 2 We will explain in detail the differences between adaptations and exaptations in Ch. 3. 3 This is largely infeasible today, due to the modest level of our knowledge of the phenomena of

speech, particularly its neural correlates.

12


To be precise, the goal of this book is not to propose direct answers, but rather to engage in the structuration of the theoretical thinking involved in the research on the origins of speech. For this reason, the main evaluation criterion for this work is the impact it will have on the thinking of researchers in this domain. In summary, this book should be read while asking, not whether what is written is true or false, but whether it is useful or not. Chapter 2 will introduce the way the speech code works, and will specify the questions to be asked about its origins. Chapter 3 locates the problems of the origin of speech in the general framework of the origins of form in biology: I will explain the phenomenon of self-organization along with that of natural selection, both of which are features of the mechanisms of creation of forms in the living world. In particular, I will present a linkage between the concepts of self-organization and natural selection, which will lead us to discuss how one should structure the arguments explaining the origin of living forms. Chapter 4 makes a detailed survey of the literature, to review what answers have already been proposed, and will support the broad outline of the approach at the centre of this work. Chapter 5 details my methodology, that is, the building of artificial systems, along with the goals and scientific philosophy which drive it. Chapter 6 gives a formal description of a first version of the artificial system, and illustrates its dynamics, which involves the formation of a discrete speech code shared by a population of agents who initially only pronounce unstructured and holistic vocalizations, and do not follow any rules of coordinated interaction. I will discuss, in particular, the role that morphological constraints on the vocal and perceptual apparatus may or may not play in the formation of speech codes. Chapter 7 presents a variant of the artificial system in which, by contrast with Chapter 6, it will not be assumed that the agents are capable from the outset of retrieving articulatory representations of the sounds which they hear; this capacity will be learned, thanks to a quite generic neural architecture. I will also use a model of the human vocal tract for producing vowels, which will enable us to specify the analogy between artificial and human systems: it will be shown that the statistical regularities which characterize the vowel systems of populations of artificial agents are very similar to those of the vowel systems of human languages. Chapter 8 presents an extension of the artificial system of Chapter 6, and shows how non-trivial combinatoriality and rules of sound-syntax, that is, phonotactic rules, can arise. Chapter 9 discusses the results obtained, and in particular the contribution that the system makes to research on the origins


13

of speech. I show that the generality and simplicity of the assumptions built into the artificial system, and the fact that they suffice for a speech code to selforganize, allow us to build up a convincing adaptationist scenario in which the neural structures have been chosen by natural selection driven by pressure for linguistic communication. Moreover, I show that the generality and the nonspecificity of these assumptions suggest several alternative scenarios in which the speech code could have arisen independently from linguistic communication. Chapter 10 winds matters up.

2. The Human Speech Code

Language, and more generally communication, involves the transmission of information between individuals. This needs a physical medium. The medium we currently use most is the sound of the human voice, and it is structured in a very particular manner. The capacity for producing, perceiving, and structuring the stream of sounds is called ‘speech’. 1 Other media are manual signs (for sign languages) and writing. The sounds which humans use for speaking are organized into a code, the speech code, which provides a repertoire of forms that are used as a physical vehicle for information. This code is a prerequisite for communication in language. Without such a repertoire of forms, whether acoustic or gestural, there is no way of transferring information between individuals. The speech code is mainly a matter of convention: there is enormous diversity among languages. The code regulates the manner in which vocalizations are organized (which is discrete and combinatorial), the way in which sounds are categorized, and the ways in which they can be combined (defining the rules of sound-syntax). I will now describe the way in which we produce and perceive speech sounds, and the organization of the speech code.

2.1 The instruments of speech We are dealing with a complex musical instrument: the vocal tract. This is organized into two subsystems (see Figure 2.1): one generates a sound wave, the other shapes it. The first system is subglottal. Together, the lungs and the diaphragm cause air to pass through the trachea, which vibrates the larynx. The larynx is an assemblage of cartilages and muscles which generates a sound wave when it vibrates. The generated sound is made up of a great number of frequencies. The second system, the supralaryngeal, is a tube stretching from the larynx upward and forward, dividing into the nasal and oral passages, and 1 Note that the term ‘speech’ is reserved here for acoustic and articulatory events independent of meaning and is thus not a synonym of ‘language’.

The Human Speech Code

15

NASAL CAVITY SUPRALARYNGEAL VOCAL TRACT

PALATE VELUM

ORAL CAVITY

TONGUE

PHARYNX EPIGLOTTIS LARYNX

TRACHEA SUBGLOTTAL SYSTEM

LUNGS

DIAPHRAGM

FIGURE 2.1. The vocal tract is organized into two subsystems: the subglottal system, which produces a sonorous source, and the supralaryngeal system, whose changeable shape makes it possible to shape this sound wave. (Adapted from Goldstein, 2003.)

ending at the lips and nostrils. The organs of the glottis, the velum, the tongue (tongue body and tongue tip), and the lips make it possible to modify the shape of this tube, in particular its length and volume. This change in shape results in the weakening or strengthening of certain frequencies in the sound signal. Thus the production of sound with the vocal tract resembles the production of sound with a flute: you blow at one end, and air passing through the ‘whistle’ makes a sound composed of many frequencies, which can be modified by blocking the holes in the body of the instrument. Thus, speech reduces to moving the various organs of the vocal tract. The perception of sounds is carried out by the ear, in particular the cochlea (Figure 2.2). The cochlea is the device which lets us register a certain number of parameters of sound. Among these parameters is the decomposition of a sound into its component frequencies, or harmonics. In fact, every complex sound can be seen as a superposition of sine waves, each with a given frequency and amplitude. In mathematical terms, this is called a decomposition into a Fourier series. The cochlea performs an approximation to this decomposition,

16

Self-Organization in the Evolution of Speech Unrolling of cochlea

Basilar membrane

Cochlear base

Basilar membrance 'Unrolled' cochlea

FIGURE 2.2. The cochlea, speech perception organ. Its basilar membrane can decompose sounds into a Fourier series, i.e. it can calculate the amplitude of its various harmonics. (Adapted from Sekuler and Blake, 1994.)

due to the basilar membrane. This membrane increases in thickness, and in its thinnest parts it responds better to high frequencies, while in its thicker and heavier parts it responds rather to low-frequency stimuli, corresponding better to its own inertial properties (Figure 2.3) . Nerve cells called ‘hair cells’ are linked to this membrane to gather information about the stimulus. This information is passed by a network of fibres to the central nervous system.

2.2 Articulatory phonology How do the organs of the vocal tract control the flow of sound? What representations does our brain use to produce sounds? How is the connection between production and perception managed? These are the questions addressed by articulatory phonology (Browman and Goldstein, 1986). This is the approach adopted in this book. The central concept of articulatory phonology is the gesture. A gesture, a unit of action, is the coordination of a certain number of organs (e.g. the

The Human Speech Code Oval window

17

Direction of travelling wave

Base Basila

r mem

brane

Apex t1

t1

t2

t2

t3

t3

t4

t4

Time

t5

t5

Base

Apex

Response to high-frequency tone

Base

Apex

Response to low-frequency tone

FIGURE 2.3. The basilar membrane decomposes the signal into its harmonics: it increases in thickness, and in its thinnest parts responds better to high frequencies, while in its thicker and heavier parts it responds rather to low-frequency stimuli, corresponding better to its own inertial properties. (Adapted from Kolb and Wishaw, 2001.)

tongue, the lips) to bring about a constriction in the vocal tract. A constriction obstructs the passage of the sound wave. It is a narrowing of the vocal tube. For example, the words boat, parameter, and metre all begin with a closure of the lips. A gesture is specified not by the trajectory of one or more organs but by a constriction target defined by a relationship between organs. For example, the opening of the lips is a constriction variable which can be implemented by the movement of three organs, the upper and lower lips and the jaw (each controlled by a set of muscles). An articulatory target (a constriction defined by a relationship among organs) can be realized by several different combinations of movements by the organs.

18

Self-Organization in the Evolution of Speech tract variable

articulators involved

LP LA

lip protrusion lip aperture

upper and lower lips, jaw upper and lower lips, jaw

TTCL TTCD

tongue tip constrict location tongue tip constrict degree

tongue tip, tongue body, jaw tongue tip, tongue body, jaw

TBCL TBCD

tongue body constrict location tongue body constrict degree

tongue body, jaw tongue body, jaw

VEL

velic aperture

velum

GLO

glottal aperture

glottis

TBCL

velum

TTCL VEL

tongue tip

+

+

TBCD TTCD

LA LP

tongue body center

+

+ upper lip + lower lip

jaw

GLO

+ glottis

FIGURE 2.4. Constriction variables are used to specify gestures. Each of these variables can be controlled by the shifting of several organs and many muscles which activate them. (Adapted from Goldstein, 2003.)

The constriction variables, or systems of constriction, which are used to specify gestures are: the position of the larynx, the velum, the tongue body, the tongue tip, and the lips. Each of these constriction variables can be controlled by the movement of several organs and the many muscles which activate them. Figure 2.4 schematizes the constriction variables along with the organs which can implement them. Each constricting organ can produce gestures whose constriction varies along two continuous dimensions: place and manner of articulation. Among the places of articulation, i.e. the places in the vocal tract where the narrowing occurs, are: bilabial, dental, alveolar, palatal, velar, uvular, and pharyngeal. Figure 2.5 gives further examples. Among the manner in which the narrowing is realized, there are stops (e.g. [d]), fricatives (e.g. [z]), and approximants (e.g. [r]). Gestures which involve a severe narrowing or a total blockage are

The Human Speech Code palato-alveolar palatal

19

velar

alveolar dental labiodental bilabial

uvular

interdental retroflex

glottal

FIGURE 2.5. Places of constriction range between the larynx and the lips. (Adapted from Bickford and Tuggy, 2002.)

called consonantal, and those which involve a wider articulation are called vocalic, i.e. they are vowels. When we speak, several gestures can be performed in parallel. Our vocalizations are thus the parallel temporal combination of several gestures. This combination can be represented by a ‘gestural score’, by analogy with orchestral scores. Gestures of five constricting systems (velum, tongue tip, tongue body, lips, glottis) are represented on five different lines. Figure 2.6 gives an example of a gestural score. The shaded boxes represent the time intervals during which the gestures of each constrictor are active in the vocal tract. The labels on the boxes indicate the place and manner of constriction. Some phonologists use the concept of a phoneme to describe the sounds found in words. This presupposes that words can be segmented into sequences of units called phonetic segments (although these do not necessarily correspond to the letters of the word when written). These units are characterized by the fact that they distinguish two words such as bar and par. It is possible to define phonemes in terms of gestures and their organization: a phoneme can be seen as a set of gestures (often just one) which systematically recurs in many words in a regular scheme of coordination. Figure 2.7 gives an example of the correspondence between the gestural score of the word pad and its phonemic transcription.

20

Self-Organization in the Evolution of Speech pan closed

wide

VELUM

GLOTTIS

closed

narrow phar

TONGUE BODY LIPS

closed

clo alv

TONGUE TIP

closed

clo lalo wide 100

closed 200 TIME (ms) (c)

300

400

FIGURE 2.6. An example of a gestural score: the word pan. (Adapted from Goldstein, 2003.)

FIGURE 2.7. The correspondence between the description of the word pad in terms of a gestural score and in terms of phonemes. (Adapted from Goldstein, 2003.)

Articulatory phonology theory hypothesizes that gestures and their coordination are represented in the brain not only for controlling the production but also for the perception of sounds. At first, when we produce an utterance, it is gestures which are specified. Commands are sent to the organs so that they can implement the corresponding constrictions. Thus, speech production is organized on two levels: the level of the commands which define the gestural score,


21

and the level of implementation. The first level is intrinsically discrete (but not necessarily discrete in the same sense as vocalizations as a whole are discrete, as we will see). The second level is intrinsically continuous, since it corresponds to a trajectory of the organs. The acoustic wave which is produced is related in a fixed but complex manner to this trajectory. In fact, the physics of the vocal tract are such that many different vocal configurations produce the same sound; yet again, certain configurations which are very close in articulatory terms are very different acoustically. The electromechanical properties of the cochlea also complicate the function which computes the correspondence between the perception of a sound and the motor program which implements it. In fact, gestures are the representations which make a connection between perception and production possible. According to the Motor Theory of Speech Perception (Liberman and Mattingly, 1985), the brain of a speaker of a language, when perceiving a sound, reconstructs the configurations of constrictions which produced it. Thus the brain should be capable of transforming gestural representations into muscular representations (to control the speech organs) and vice versa. Figure 2.8 summarizes this organization.

2.3 The organization of the speech code: universals Comparison of the gestural scores forming words shows striking regularities both within one language and between languages. Cochlea activation

Gestural representation = Places and manners of constrictions

Muscle activation

FIGURE 2.8. The brain handles three representations in the perception and production of speech: an acoustic representation, a muscular representation, and a gestural representation, which is the representation used to classify sounds. The brain can pass from one representation to another, according to the theory of articulatory phonology. (Adapted from Goldstein, 2003.)

22


2.3.1 The speech code is discrete and combinatorial The complex vocalizations that we produce are coded phonemically. This has two implications: (1) in each language, the articulatory and acoustic continuum which defines gestures is broken into discrete units; (2) these units are systematically reused to construct the representations of the next higher linguistic level, such as the syllable level. It would be possible to imagine that each syllable is specified by a gestural score consisting of unique gestures or unique combinations of gestures. For comparison, there are writing systems which use a unique holistic symbol for each syllable (Nakanishi, 1998). These are syllabaries, as opposed to alphabets. In fact, by contrast with writing systems, all human languages have repertoires of gestures and combinations of gestures which are small in relation to the repertoires of syllables, and whose elements are systematically reused to make syllables. In the languages of the UPSID451 database (UCLA Phonological Segment Inventory Database) initially elaborated by Maddieson (1984), containing 451 languages, the average is about thirty phonological segments per language. More precisely, twenty-two consonants and five vowels are the most frequent counts, as Figures 2.9 and 2.10 show. This small number (twentytwo) has to be contrasted with the considerable number of phonemes that we can possibly produce: for one thing, the gestures can vary the place of their constriction continuously from the larynx to the lips; they can also continuously vary the manner of articulation (i.e. the shape and degree of narrowing). Thus, as phonemes are combinations of gestures, it is clear that the combinatorial possibilities are immense. Moreover, some languages use a great number of phonemics (!Xu, of the Khoisan family, uses 141), although such examples are rare. The total number of phonemes in the UPSID database is 920. This phenomenon of systematic reuse does not stop there. Gestures themselves are recursively constructed on this principle. In fact, we have just explained that the place and manner of constriction which specify a gesture may vary continuously. In a given language, only a small number of places and manners occur and are reused (varying the combinations, to be sure) to make the gestures. Gestures could in principle have places of articulation which are peculiar to each, while still being reused in many syllables, but this is not what happens. For example, for each separate manner of articulation, 95 per cent of languages only use three places.


23

30

25

20 % 15 in UPSID 10

5

0

0

5

10 15 Number of vowels per system

20

25

FIGURE 2.9. Distribution of counts of vowel inventories in the UPSID database. (Adapted from Escudier and Schwartz, 2000.)

0/00

60

40

20

0

0

25

50

75

95

Number of consonants per system

FIGURE 2.10. Distribution of counts of consonant inventories in the UPSID database. (Adapted from Escudier and Schwartz, 2000.)

24


Thus, from the continuous space of possible gestures, speech carves out basic building blocks which it reuses systematically. The phonemic and gestural continuum becomes discretized. Speech, already discrete in its manner of control involving motor commands specifying articulatory targets, is now also discrete in terms of the systems it uses (i.e. the possible articulatory targets in a language are limited to small, finite numbers, although physically they could be distributed across the whole articulatory continuum) and combinatorial. There are thus two remarkable points: first, the discretization of the continuous space of gestures; second, the two levels of systematic reuse of resources: • Places and manners of articulation are reused to make gestures. • Gestures and their combinations are reused to form syllables. 2.3.2 The speech code is a classification system shared by the whole linguistic community A further property of speech is this: it appears that all the speakers of one language perceive and classify sounds in the same way. This is a striking property, as each language differs in the classification it imposes. For a start, speech perception is characterized by a psychological uniformity, despite the great variability of speech. Physically different sounds associated with different trajectories of the organs of the vocal tract can in fact correspond to the same sound, psychologically speaking, as the [d] in idi, ada, or udu. The sounds are different because the gestures which specify the [d] are superposed on the gestures which specfy the [i], [a], and [u]: the articulatory targets are temporarily in competition with each other, and the organs have to make a compromise to satisfy them as well as possible. The result is that the specifications of the gestures are not exactly met, but are approximated. This is the phenomenon of coarticulation, which explains the acoustic and motor variability of sounds. Coarticulation occurs not only when the contexts of a segment change, but also when the rhythm of speaking changes. Thus each phoneme in a language can be realized in several different ways while still remaining identifiable. The variants of a phoneme which speakers are able to recognize are called allophones. It is in the space of gestures that variations are the weakest: coarticulation can bring into play very variable muscular or acoustic trajectories, which nevertheless more or less preserve the articulatory targets in terms of the relationships among the organs. This is why gestural representation is central to speech. However, even if the level of motor


25

commands is invariant, the level of their realization involves variability which is managed in precise and particular ways in each language. In a given language, all speakers decide in the same way which sounds are variants of the same phoneme and which are variants of different phonemes. This organization of the space of sounds is culturally specific to each language. For example, native speakers of Japanese identify the [r] of read and the [l] of lead as allophones, i.e. they classify these two sounds under the same heading, whereas for English speakers these are two distinct categories. Not only do speakers of the same language share a common way of classifying sounds which is specific to them, they also share a common and different way of perceiving sounds, from a sensory point of view. This is shown by the “perceptual magnet” effect (Kuhl et al., 1992). When subjects are asked to judge the similarity between two sounds (on a scale of 1 to 10), and where these sounds are regularly spaced at a given distance D in a physically defined space (e.g. a spectrum of amplitude), it is noticed that when two sounds belong to the same phonemic category, the similarity reported by the subjects is greater than that which they report for two sounds which do not belong to the same phonemic category but which are nevertheless the same distance apart on the physical scale. To summarize, intra-categorial perceptual differences are diminished and inter-categorial differences are augmented. It is a kind of perceptual deformation or acoustic illusion (‘perceptual warping’), in which the centres of categories perceptually attract the elements of the categories like magnets. Once again, this a culturally specific phenomenon: the perceptual deformations are particular to each language. Figure 2.11 shows this effect.

2.3.3 Statistical regularities in the phoneme inventories of human languages Statistical study of languages shows universal tendencies in the inventories of phonemes and the gestures which compose them. Certain phonemes are very frequent, while others are very rare: more than 80 per cent of languages in UPSID have the vowels [a], [i], or [u], while only 5 per cent have [y], [œ], or [W]. More than 90 per cent of languages have [t], [m], and [n] in their inventories, as Figure 2.12 shows. It is the same with gestures, and in particular the places and manners of articulation: 15.3 per cent of consonants in UPSID317 are alveodental, but only 3.9 per cent are retroflex. Similarly, 38.6 per cent of consonants in UPSID317 are plosives, while only 3.9 per cent are trills, taps, or flaps (Vallée et al., 2002).

26

Self-Organization in the Evolution of Speech Physical World

Formant 2

A.

Formant 3 B. Perceptual World: Americans

/ra/

/la/

C. Perceptual World: Japanese

/ra/

FIGURE 2.11. The perceptual magnet effect. Kuhl et al. (1992) required subjects to rate pairs of consonants for similarity on a scale of 1 to 10, where these consonants varied continually between [r] and [l] (they were followed by the [a] vowel); and their values are represented by the circles in the figure above. One group of subjects were American, the other group were Japanese. It is possible to derive from Kuhl’s results a graphic representation showing the subjective way in which they perceived each consonant. This graph of their subjective perception is given in B for the Americans and in C for the Japanese. Note that the Americans subjectively perceive two categories of sound in this continuum, whereas the Japanese only perceive one. Moreover, within the neighbourhoods of [r] and [l] for the Americans, the sounds are subjectively more similar to each other than they are when measured objectively in physical space. (Adapted from Kuhl et al., 1992.)

The regularities apply not only to phonemes individually but also to the structure of phoneme inventories. This means, for example, that if a language has a front unrounded vowel of a certain height, like the [e] in pet, it will also usually have a back rounded vowel of the same height, which would be here the [o] in pot. Thus the presence of certain phonemes is correlated with the presence of other phonemes. Some vowel systems are also very common,

The Human Speech Code C

%

C

%

t

97.5

g

56.2

m

94.4

27

52.7

n

90.4

?

k

89.5

tf

41.8

j

84.0

f

41.6

p w

83.3 76.8

F dz

40.0 34.9

s

73.5

48.0

31.3 s

d

64.7

t

29.3

b

63.8

kh

22.9

h

62.0

ph

22.4

l

56.9

vr

21.1

FIGURE 2.12. Distribution of consonants in the UPSID database. (Adapted from Escudier and Schwartz, 2000.)

3.2%

0.6%

5 vowels

3.7%

1.9%

6 vowels

7 vowels

8 vowels

9 vowels

28%

6.4%

6.3%

1.9%

2.0%

1.9%

3.1%

1.6%

0.9%

0.6%

u

W

4 vowels

o

o

c

3 vowels

i

e

i

e e

I e

a

FIGURE 2.13. Distribution of vowel systems in the UPSID database. The triangle represents two dimensions characterizing vowels, the first and second formants (formants are the frequencies for which there is a peak in the energy spectrum of the signal, where the harmonics have the greatest amplitude). (Adapted from Escudier and Schwartz, 2000.)

while others are rarer. The five-vowel system /[i], [e], [a], [o], [u]/ is found in 28 per cent of languages, as Figure 2.13 shows. There are also regularities governing the ways in which phonemes combine. In a given language, not all possible phoneme sequences are allowed. Speakers

28


know this, and if they are asked to invent a new word, it will be made up of non-arbitrary phoneme sequences (some will never be used). For example, in English spink is a possible word, while npink or ptink are not possible. Here again, the rules governing the possible ordering of phonemes, called phonotactics, are cultural and particular to each language. In Tashliyt Berber, tgzmt and tkSmt are allowed, but they are not allowed in English. Moreover, the set of allowed phoneme combinations within syllables is organized into patterns. This means that, for example, one can summarize the allowed phoneme sequences of Japanese syllables by the patterns CV/CVN/VN, where CV for example defines syllables composed of two slots, and in the first slot only the phonemes belonging to a group that is called ‘consonants’ are allowed, while in the second slot, only the phonemes belonging to the group that is called ‘vowels’ are allowed (and N stands for ‘nasals’). Finally, there are phoneme combinations which are statistically preferred over others in the languages of the world. All languages use syllables of the CV type, while many do not allow consonant clusters at the beginnings of syllables. Statistically, languages prefer CV syllables, then CVC syllables, then CCV and CVV, then V and VC, then CCVC and CCVV, and then others. Syllables tend to begin with a phoneme with a high degree of constriction, and then to let the degree of constriction decrease until the middle of the syllable, and then to increase the level of constriction again up until the last phoneme of the syllable (this is called the principle of the Sonority Hierarchy).

2.4 The diversity of speech codes in human languages Some of the regularities mentioned in the previous section are systematic and common to all languages. This is the case with the reuse of phonemes and gestures, as well as their discretization; it is also the case with the shared classification of sounds by speakers of each language, as well as with acoustic illusion phenomena related to speakers’ knowledge of the phoneme inventories of their languages. On the other hand, regularities in the inventories of phonemes and gestures, as well as the phonotactic preferences, are only statistical. This sheds light on a striking aspect of the speech systems of the world—their diversity. Recall that the languages of the UPSID database include 177 vowels and 645 consonants (and again, this classification groups together phonemes which are not exactly identical). While the average is five vowels per language, some have more than


29

twenty; while the average is twenty-two consonants per language, some have only six (e.g. Rotokas, a language of Papua New Guinea) while others have as many as ninety-five (e.g. !Xu). As seen above, the ways in which sounds are classified also varies greatly; Chinese, for example, uses tones, i.e. musical pitch, to differentiate sounds, something which the ears of English speakers have difficulty in catching.

2.5 Origins, development, and form In the previous sections I have described the speech code and its role in human languages today. Possession of a speech code is a basic attribute of modern humans. It is a structure (one could also call it a trait) whose origin is a fundamental question that research into the evolution of language tries to answer, just as biologists try to explain the origin of a morphological structure such as the hands or the origin of some trait such as bipedalism. The speech code is very complex, and seeking to explain its origin implies answering several questions which are dependent on one’s point of view. For a start, we have seen that it is a conventional system—it is linked to norms formed by the interaction of individuals during the course of their lives. We now understand quite well—especially since works such as Steels (1997), Kaplan (2001), and de Boer (2001)—how a linguistic norm can be formed in a population of agents living in an environment in which a conventionalized system of communication already exists, and thus in which there are already norms like ritualized interactions which frame the interactions in language games. The question of how the very first norms were established, when ritualized interactions did not exist and thus no language games were possible, is still unexplored. And this question applies particularly to the formation of the first speech codes. We have seen how speech codes 2 provide a repertoire of forms for transmitting information within a framework of conventionalized communication. Now, without a speech code (or manual sign code), it is impossible to have ritualized communicative interaction (because this requires a shared repertoire of forms). Thus the question arises how it is possible that a speech code could arise without there already being conventionalized or ritualized interactions, i.e. without presupposing the existence of any convention whatsoever. 2 Or codes of manual signs; for this problem, the issue of modality is not crucial.

30


Next, the question arises why humans speech codes are the way they are. Looking at such systems from the outside (as a linguist does), how does it come about that speech is discrete, and carves basic building blocks out of the articulatory continuum and reuses them systematically? How come sounds are grouped into categories? Why are there preferences for places and manners of articulation and for phonemes in the repertoires of the world’s languages? Why are there syntactic rules governing the formation of syllables? Why are some rules preferred over others? Why do we have both regularity and diversity at the same time?3 Looking at such systems ‘from the inside’, as a psychologist or neuroscientist does, parallel questions arise: how can a speaker acquire a sound system? What sensorimotor or cognitive mechanisms do they use? In fact, the problems facing the speaker are a priori difficult: could a speaker learn to transform gestural commands into muscular commands? Or learn to relate the perception of a sound to the gestural score which generates it? Must this ability be innate? Even if a speaker could manage to learn to transform an acoustic trajectory into an articulatory trajectory, a continuous sequence of constrictions, how would he know what are the articulatory targets which are the key points in these trajectories? The question of the origin of the first conventionalized speech code, the question of the general form of this code in contemporary languages, and the question of the ontogenetic acquisition mechanisms for this code are normally dealt with by quite independent research communities, and are not taken on together. This, however, is what I will attempt here, in the modest framework of the artificial system that I will construct. One of the things I will try to show, moreover, is that taking these problems into account simultaneously can be done in a reasoned way, and can yield theories which are not too complex and illuminate them with a new and original light. It is by trying not to isolate the separate questions too much from the start, keeping in mind the systemic and complex nature of speech, that one may show that the complexity of these problems can after all be reduced4 . The questions which arise concerning the origin and form of the speech code are analogues of the questions which arise in general for biologists on 3 As far as the properties of the speech code are concerned, I will stay with these general questions, and will not try to grapple with their particular instantiations; e.g. the questions which interest us are of the sort ‘Why are there statistical preferences for certain phonemes?’ rather than of the sort ‘Why do languages prefer [t]s to [d]s?’ 4 Keeping a certain level of complexity in problems at a global level evidently leads to reducing complexity at the local level: in Ch. 5 I will discuss the advantages and disadvantages of this approach.


31

the origin of the shapes, structures, and characteristics of living organisms. It is essential to consider these questions within a general framework of the origin of forms in nature. This allows one to specify what kinds of answer are required and what pitfalls one should avoid. For this reason, before continuing to spell out the problems of the origin of the speech code and to develop precise theories about it, I will take a step back and reflect on the mechanisms responsible for the origin of forms in nature in general. The next chapter will therefore set out the phenomenon of self-organization, characterizing a certain number of form-creating mechanisms particularly responsible for shaping living organisms. Above all, I will attempt to present a reasonably argued relationship between the concept of self-organization and that of natural selection. This will allow us to outline the structure of the arguments necessary for explaining the origin of living forms.

3. Self-Organization and Evolution

3.1 Self-organization The organized form and patterns with which we are concerned possess global organizational properties not seen at the local level: the form and physicochemical properties of a water molecule are qualitatively different from the global properties of ice crystals, whirlpools, or polyhedral collections of bubbles. The polygonal impressions in the tundra do not correspond to the shape of the rocks which compose them, and have a spatial organization very different from the temporal organization of the cycle of freezing and thawing. This is the trademark of self-organization, typical of systems in which organized patterns and forms develop at a global level and whose properties are qualitatively different from those of the entities found at the local level. Self-organization is not a mechanism, as is often claimed in the literature; it is a property, in the same way as the growth of a child is a property. It is a fundamental concept of modern science typical of a whole variety of natural systems, whose form-creating mechanisms are not necessarily instances of natural selection (e.g. the mechanisms involved in the formation of inorganic structures). There are certainly no universal principles which enable us rigorously to unite all self-organizing systems, but nevertheless a number of components do recur with some frequency: breaking of symmetry, tension between forces, positive feedback loops, the presence of attractors in dynamic systems, a flow of energy through dissipative structures, non-linearities and bifurcations, and noise. I will give two classical examples of systems which have this property of self-organization. 3.1.1 Rayleigh–B énard convection The first example is the formation of so-called Bénard cells.1 This phenomenon arises when a thin layer of liquid is placed on the level top of a stove. 1 In the English-language literature, this phenomenon is usually called by the name of its English co-discoverer, Lord Rayleigh; hence ‘Rayleigh–convection’ or ‘Rayleigh–Bénard convection’. Here, I will continue to use the name of the French co-discoverer of the phenomenon.

Self-Organization and Evolution

33

If the liquid is not heated, then it is in a state of equilibrium in which none of its particles move. The properties of the system are homogeneous; the temperature is the same throughout. The system is symmetrical at the macroscopic level. If the stove is gently heated from below, the heat will be transferred from the bottom to the top by a process of thermal conduction. In other words, there is no macroscopic displacement of fluid, but rather an increase in the thermal agitation of the particles which, from one neighbourhood to another, move to the colder surface. Layers of liquid at different temperatures acquire different density and hence different mass; under the effect of gravity, there is thus a force which pushes the higher (colder) layers toward the bottom and the lower (warmer) layers toward the top. This force is dependent on the difference in temperature between the bottom and the top of the liquid. The force is in competition with the viscosity of the liquid, which itself inhibits movement. This is why, when the temperature difference is small, the liquid itself does not move and only thermal conduction takes place. But if the temperature passes a certain threshold, then the liquid suddenly begins to move at the macroscopic level, giving the appearance of convection currents. What is interesting is that these currents are not random, but organize themselves into quite particular structures which break the symmetry of the liquid (but not so much as to make the symmetry disappear altogether), and whose typical dimensions are several orders of magnitude greater than those of the forces applying at the molecular level. At first, just above the critical temperature threshold, parallel rectangular stripes are formed (see Figure 3.1). Two adjacent stripes circulate the liquid in opposite directions to each other. The initial symmetry is minimally broken:

Cool

Hot

FIGURE 3.1. If a thin layer of liquid is heated on a stove, then given a certain minimum temperature difference between the top and the bottom of the liquid, there is selforganization of convection currents in parallel stripes. (Adapted from Ball, 2001.)

34


since it has two dimensions, the system stays symmetrical along the dimension which is parallel to the stripes. If the temperature is raised further, then stripes at right angles to the first stripes appear. The liquid is this organized into square convection cells. If the temperature is raised still further, then polygonal forms appear, which can sometimes cover the whole surface with regular hexagons. Figures 3.2 and 3.3 show these different states. If the temperature is raised very high, the regular pattern becomes chaotic and turbulent. If the heating is stopped, so as to equalize the temperature at the top and the bottom, the convection patterns disappear and the liquid returns to its equilibrium state, with a uniform temperature and no macroscopic displacement of molecules. This is an example of a system in which the formation of patterns requires that the system be pushed far from its equilibrium by a continuous flow, in and out, of energy (thermal agitation in this case). Such systems are called dissipative systems. However, self-organization is not

a

b

c

FIGURE 3.2. Representation of convection currents in Bénard liquids when the temperature is raised: at first parallel stripes are formed, then there are square cells, and for higher temperatures polygons appear. If the temperature is raised even further, the regular patterns dissolve into turbulence. (Adapted from Tritton, 1988.)


35

´ FIGURE 3.3. At a certain temperature, the Benard cells tessellate the surface with regular hexagons. (Photo: Manuel Velarde, Universidad Complutense, Madrid.)

concerned only with systems pushed out of equilibrium, but can equally take place when a system evolves toward its equilibrium state. 3.1.2 Ferro-magnetization Another example of a natural system with the self-organizing property is that of iron plates. An iron plate is an assembly of atoms each of which is a sort of microscopic magnet. Each atom can have two possible magnetic orientations, termed −1 or +1. The state of each atom depends on and evolves as a function of two parameters: the states of its neighbours, whose majority orientation it tends to adopt, and temperature, which makes it randomly change state all the more often when it is raised (and which thus has no effect when it is zero). First of all, note that the behaviour of a lump of iron, a macroscopic arrangement of its atoms, is interesting at zero temperature. Whatever the initial state of the atoms, the system self-organizes in such a way that after a certain time, all the atoms are in the same state, which can be +1 or −1. That is, even if initially each atom is in a random state, a kind of global consensus is formed according to which even two quite distant atoms end up with the same orientation. The two equilibrium states, ‘all atoms are +1’ or ‘all atoms are −1’, are called the attractors of the dynamical system formed by the set of atoms. If the orienting of each atom is carried out asynchronously and randomly, which is a good approximation to reality, it is not predictable which

36


equilibrium state will be reached: this depends on the particular history of each system. The outcome is most uncertain when the atoms are in a random state, so that there are equally many in the −1 and +1 states. This is a globally symmetrical state of the system, as neither orientation is favoured. This initial state is an equilibrium, since just as many atoms will switch from +1 to −1 as in the opposite direction, but it is an unstable equilibrium. In fact the random updating of the states of the atoms causes fluctuations which make the ratio of one state to the other vary around 1. Then, for example, the more +1s there are, the greater is the probability that atoms with this state will convert others to the same state as themselves. This can ‘snowball’: it is what is called a positive feedback loop. At a certain moment, one of the random fluctuations in the ratio of states is amplified by a positive feedback loop. This is how one particular magnetic orientation is ‘chosen’ by all the atoms and magnetizes the lump of iron at a macroscopic level. Symmetry is thus broken. Now if the temperature is not zero, and randomly changes the states of the atoms more often as it gets higher, there are three possible situations. First of all, if the temperature is low, then its effect only slows down the convergence of the iron lump toward a state where all the atoms share the same magnetic orientation. On the contrary, if it is very high, it becomes the dominant factor among the forces affecting the local magnetic interactions between atoms. Here, no order appears and the state of each atom evolves randomly over time. The lump of iron is demagnetized. What is more interesting is an intermediate situation, corresponding to a very narrow temperature band: large regions appear in the lump of iron with complex but well-defined forms, composed internally of atoms which are mostly in the same state. It is a state between order and disorder, corresponding to the ‘complexity at the edge of chaos’ often mentioned in the popular science literature. By changing the temperature, one can see phase changes: from a completely ordered state, after a certain critical threshold, a state with complex patterns is reached, after which total disorder soon appears. Figures 3.4 and 3.5 represent these phase transitions in a two-dimensional model of ferromagnetic plates. These two examples of self-organizing systems have some points in common which we will see again in the artificial system set out in this book. They are both characterized initially by symmetry at the macroscopic level, which is then disrupted. Nevertheless the final self-organized state is still characterized by certain symmetries which make it an ‘organized’ system; it is possible to predict the overall form of this global state qualitatively but not quantitatively,


T = 1.20

T = 2.24

37

T = 4.00

FIGURE 3.4. Representation of the states of atoms in a two-dimensional ferromagnetic structure. The points are black or white according to whether the atoms they represent are in state +1 or −1. The left-hand square shows a typical configuration starting from an initial random state with low temperature (the atoms are almost all in the same state); the right-hand square shows a typical configuration when the temperature is raised (the atoms are in random states); the middle square shows a typical configuration in an intermediate temperature band (the atoms form regions with complex shapes within which all are in the same state). (Figure: Christian Sigg) 80⫻80 Ising Model Simulation: Magnetization versus Temperature

1.5

Magnetization

1 0.5 0 -0.5 -1 -1.5

0

0.5

1

1.5 2 2.5 Temperature

3

3.5

4

FIGURE 3.5. Representation of the magnetization of a two-dimensional ferromagnetic model, after settling from a random state, and as a function of temperature. At low temperatures, the metal self-organizes and all atoms adopt the same magnetic orientation. Two orientations are possible, corresponding to two opposing magnetizations, as can be seen in the figure. At high temperatures, the final state also consists of atoms in random states, and yet globally there are not more atoms oriented in one direction than in the other: the iron fragment is not magnetized. Between the magnetized and non-magnetized states, it can be seen that the transition is rapid and non-linear. This diagram is also a way of showing the phenomenon of bifurcation at a branching point.

38


because it depends on the history of the system subject to random fluctuations. There is competition between forces pushing the system in different directions. The system has a control parameter whose values determine several types of behaviour or ‘phase’, and continuous linear variation along this parameter is mapped to rapid non-linear transitions between the different phases.

3.2 Self-organization and natural selection The examples of the previous section were chosen deliberately from inorganic systems to show that the property of self-organization can be found in systems subject to laws which have nothing to do with natural selection. However, selforganization applies similarly to living systems. It is a concept widely used in several branches of biology. It is particularly central to theories which explain the capacity of insect societies to build nests or hives, to hunt in groups, or to explore in a decentralized and effective way the food resources of their environment (Camazine et al., 2001). In developmental biology it is used, for example, to explain the formation of coloured patterns on the skins of animals like butterflies, zebras, jaguars, or ladybirds (Ball, 2001). It seems possible, then, that there are shape- and pattern-forming mechanisms in biological systems which are orthogonal to natural selection, just because they have the property of self-organization. Now natural selection is at the heart of almost all the arguments of biologists when it comes to explaining the presence of a shape, a pattern, or a structure in an organism. What, then, is the relationship between the the theory of natural selection and self-organization? Some researchers have suggested that self-organization casts doubt on the centralilty of natural selection in explaining the evolution of living organisms. Waldrop (1990: 1543) explains: Complex dynamical systems can sometimes go spontaneously from randomness to order; is this a driving force in evolution? ... Have we missed something about evolution—some key principle that has shaped the development of life in ways quite different from natural selection, genetic drift, and all the other mechanisms biologists have evoked over the years? ... Yes! And the missing element ... is spontaneous self-organization: the tendency of complex dynamical systems to fall into an ordered state without any selection pressure whatsoever.


39

However, rather than seeing self-organization as a concept which minimizes the role of natural selection by suggesting competing form-creating mechanisms, it is more accurate to see it on the one hand as belonging to a somewhat different level of explanation and, above all, on the other hand as describing mechanisms which actually increase the power of natural selection by an order of magnitude. Mechanisms with the self-organizing property are completely compatible with the the mechanism of natural selection in explaining evolution.2 3.2.1 Classic neo-Darwinism To see the matter clearly, it is first necessary to recall what the mechanism of natural selection, or neo-Darwinism, comprises. It is a mechanism characterizing a system composed of individuals each having particular traits, shapes, or structures. In addition, the individuals in this system are capable of replication. This replication must occasionally produce individuals which are not exact copies of their ancestors, but are slight variants. These variations are the source of diversity among individuals. Finally, each individual has a greater or lesser capacity for replication, according to its surrounding environment. Moreover, most often the environment is such that not all individuals can survive. Thus, differential replication of individuals in an environment which does not allow everyone to survive gives rise to ‘selection’ of those who are most capable of replicating themselves. The combination of the processes of variation and selection means that, over the generations, the structures or traits of individuals that help them to reproduce themselves are preserved and improved upon. Now there is one crucial point on which the theory of natural selection is neutral: it is the way in which variation is generated, and more generally the ways in which the individuals with their shapes, traits, and structures are produced. Some researchers say that the answer to this is simple for living organisms: individuals are defined by their genes, which build the organism, and mutations and crossover (in the case of sexual reproduction) are the engines 2 The explanation of the relationship between natural selection and the concept of self-organization

presented in this section does not reflect the position of any particular researcher or school of thought, but is the outcome of my own personal synthesis from readings on the subject. In this sense, the vision that I present here is original, and will certainly attract support from one body of researchers as well as rejection by another. Like other attempts to extend the theory of natural selection, and moreover like the classical version of that theory in neo-Darwinian thinking, it remains a theoretical vision which is still a long way from being empirically verified.

40


producing variation of traits. This explanation would be sufficient if the relationships between genes and traits or shapes of the organism were simple, direct, and linear. In this case, in fact, exploration of the space of phenotypes (which determines, along with the environment, the relative effectiveness of the individuals at replicating) can simply be carried out by studying the way things change in the space of genotypes. Now the mechanisms of mutation which actually bring about these changes are rather straightforward and of little amplitude (most mutations only affect a very small part of the genome when replication succeeds). What this means is that, under the hypothesis that phenotypic and genotypic space have the same structure and can be mapped approximately linearly, the space of possible biological forms can be searched quasi-continuously, by successive little modifications of pre-existing forms. Incidentally, this is how many researchers consider the mechanism of variation in natural selection to be. Fortunately for the appearance of complex life forms, this is not the case. In fact, although this mechanism of small successive variations in form is notably effective in the delicate regulation of the structures of organisms, it would make the search for forms as complex as those of human organisms equivalent to the search for a needle in a haystack (Keefe and Szostak, 2001).

3.2.2 Self-organization: constraining the search space It is here that the concept of self-organization comes to the rescue of this naive search mechanism in the space of phenotypic forms in the neo-Darwinian theoretical framework. In fact, the relation between genes and the forms of organisms is complex and strongly non-linear. Organisms are constructed starting from a stem cell containing a whole genome. This stem cell can be seen as a dynamic system parameterized by its genome and under the influence of perturbations imposed by the environment. This dynamic system is, crucially, a self-organizing system, with the same sorts of properties as the self-organizing systems described in the previous section. The genome is a set of parameters analogous to temperature and the viscosity of liquids in Bénard systems, and the environment is analogous to noise (but evidently highly structured noise). Thus the development of an organism from a stem cell is analogous to the the self-organized formation of Bénard cells: shapes, structures, and patterns appear at the global level, and are qualitatively different from those implementing functioning at the local level, i.e. different from the patterns characterizing


41

the structure of the stem cell and its genome. The hexagonal pattern which can appear as a result of a simple difference in temperature in a homogeneous liquid gives an idea of the way in which a simple sequence of nucleotides enclosed in a system of molecules that transforms them automatically into proteins can generate a bipedal organism endowed with two eyes and ears and an immensely complex brain. Crucially, as with Bénard systems or ferromagnetic plates, dynamic systems defined by the cells and their genomes are characterized by a landscape of attractors: there are large regions in the parameter space within which the dynamic system systematically adopts behaviour which is more or less the same. For Bénard systems, there is a range of temperatures giving rise to parallel stripes which is wide enough to locate easily. For ferromagnetic plates the temperature range in which the system settles to global magnetic coherence is also very wide. Thus for living organisms not only it is possible to generate self-organizing structures with complex global properties, but in addition these structures are generated by genomes belonging to broad subspaces of genome space, called basins of attraction. The structuring of genome space into basins of attraction by this kind of dynamic system facilitates the evolutionary search of the space of forms so that it does not resemble a search for a needle in a haystack. As in ferromagnetic systems, structured noise imposed by the environment on the development of the dynamic system can lead it to take different developmental pathways. For pieces of iron at low temperatures, this corresponds to magnetization in one direction or another. For a living organism, this corresponds to its possible shapes; this is how it happens that even monozygotic twins can show quite important morphological differences. This is also the reason why the relationship between genes and the forms of organisms is not only complex and non-linear, but also non-deterministic. As in Bénard systems where search of the parameter space of temperature can sometimes lead to fast and qualitative changes in the behaviour of the system (e.g. the change from parallel stripes to square cells), which have been called phase transitions, the search within genome space can also lead to fast qualitative changes. This possibly corresponds to many observations of rapid form changes in evolution, as shown by the fossils studied by anthropologists, and which are the basis of the theory of punctuated equilibrium proposed by Eldredge and Gould (1972). To summarize, the self-organizing properties of the dynamic system composed by the cells and their DNA brings crucial structuring to the

42


phenotypic space by constraining it, making the discovery of complex robust forms by natural selection much easier. On the one hand, these properties enable a genome to generate complex, highly organized forms without the need for precise specification of each detail in the genome (in the same way as Bénard’s polygonal shapes are not specified precisely, or encoded in a plan, in the properties of the liquid’s molecules). On the other hand, the self-organizing properties structure the landscape of these possible forms into basins of attraction within which they resemble each other greatly (here is where gradual evolution happens, involving fine-tuning of existing structures), and between which there can be substantial differences among forms (transitions from one basin to another are what provide abrupt and powerful innovations in evolution). To give a simple picture, self-organization provides a catalogue of complex forms distributed over a landscape of valleys in and between which natural selection moves and makes its choices: selforganization proposes, natural election disposes.3 3.2.3 Evolutionary explanations: function is not enough This view of the relationship between natural selection and the concept of selforganization not only allows a unification of the concerns of many researchers who often only work on one of the two aspects, but also shows up the need to have explanations of the form of organisms which are more complete than those often proposed in the literature. In fact, one often sees studies explaining the presence of a shape or trait in an organism in terms of the reproductive advantage that it confers. This reproductive advantage is sometimes transformed into a survival advantage, or even more abstractly into an advantage in realizing some function, but such arguments are just alternative ways of talking about reproductive advantage. For example, bipedalism in humans is explained by the fact that in the savannah environment it helps them to survey their environment better, for predators and food alike, and thus to survive 3 Obviously this is only an image to facilitate understanding, because with its movements natural selection actually enables new mechanisms, themselves self-organized, to appear, and these in turn structure the space of forms within which it moves. Thus natural selection participates in the formation of these mechanisms which help it to move effectively in the space of forms; vice versa, the mechanism of natural selection certainly appeared in the history of life due to the self-organized behaviour of systems which were as yet completely unconnected to natural selection. Natural selection and selforganizing mechanisms thus help each other in a sort of spiral which enables complexity to increase during the course of evolution.


43

more easily and logically to reproduce more effectively. Another example, much closer to the questions which concern us in this book: vowel systems, made up of elements sufficiently distinct from each other, are explained by the fact that they allow information to be passed from one individual to another with minimum risk of mixing up sounds and not understanding, thus maximizing communicative capacity (Lindblöm, 1992). These explanations contain information which is correct and essential, but which does not constitute a sufficiently complete explanation to be satisfactory. In fact, they locate themselves, often without admitting it, in a classic neo-Darwinian vision which makes the simplification set out above of the relation between genotypic and phenotypic space: they say nothing about the way in which natural selection was able to find such a solution. Now according to the view presented in the preceding paragraphs, it is actually this aspect which can be crucial to understanding the origin of a shape or trait. It is as if a team of Martian researchers were to land on earth and ask how it comes about that humans take planes to cross the oceans. A first response, which would be that of classical neo-Darwinism, would be: ‘Because it is the fastest and most effective way of crossing the oceans.’ This response is accurate, even necessary, but incomplete and therefore unsatisfactory. The neo-Darwinian Martian might add: ‘and aeroplanes have structures which were discovered by natural selection. For a long time humans tried out many structures, first at random, and then kept the best and made small random variations, replacing a bolt here with a bolt there, which they then selected, and so on, until they hit upon working aircraft.’ But obviously, it can be seen immediately that this is a long way from a satisfactory understanding of the cultural and non-linear history of the invention of aeroplanes. Maybe this explanation is to some extent valid as an explanation for how engineers today adjust the exact shapes of wings to increase speed or reduce fuel consumption (by the method of trial and error with simulation programs), but it is far from revealing the aeronautical revolution between the end of the nineteenth century and the beginning of the twentieth4 So it seems that an explanation for the origin of numerous forms in biology requires much more than simply establishing that they enhance the reproduction of the organisms that have them. One needs to identify how the form 4 Moreover, a parallel is found here on the cultural level between gradual and abrupt changes in shapes and structures during the course of evolution, due precisely to the non-linearity of selforganizing phenomena which work in tandem with natural (or cultural) selection.

44


was generated, and to understand the structure of the relationship between genotypic space and phenotypic space constraining the functioning of natural selection. In the first place this involves understanding the formation of the structure on the scale of ontogeny, i.e. the development of the organism from a stem cell to an adult form, to try to see what are the bases from which it could self-organize. In practice, this consists in trying to find in the organism some structures much simpler than those which one is trying to explain, and in showing how their interactions can lead to the appearance of global structure. The formation of a certain number of macroscopic patterns on the skins of animals, like the stripes of the zebra or the regular round patches of the leopard, can be explained as the almost inevitable attractor outcomes of microscopic molecular interactions between the chemical components present in their epidermis, in the same way as hexagonal Bénard cells (Ball, 2001). In this way the problem of the origins of these patterns is considerably clarified: evolution did not have to search through all the mathematically imaginable patterns, coding them particle by particle, but only had to find how to produce some chemical molecules whose interaction contrives to make stripes (and it seems that the combinations of chemical elements, just as the numbers of patterns that they can produce on the skins of zebras, for example, is effectively very limited: see Ball (2001)). Next, it is necessary to understand how variation in the genomes of organisms, and so in the parameters of the organisms seen as dynamic systems, causes changes in form, and in particular at what speed and of what type. D’Arcy Thompson was one of the pioneers of this work, fundamental to understanding the origin of the forms of living organisms. In particular, he studied the impact of parameters in the growth of structures like the shells of molluscs. These are built by self-organizing processes of cellular division, whose parameters are speed and orientation. Thompson showed in On Growth and Form (1961[1917]) how, with a constant mechanism, the simple numerical variation of these parameters, which from a modern point of view we can easily imagine to be fairly directly controlled by the genes, can lead to surprising non-linear and very diverse variations in the generated forms. Figure 3.6 gives some examples of this. He repeated this work for many animal species, and in particular with fish, where he shows the same phenomenon: Figure 3.7 shows the variety of forms which can be obtained with a constant growth mechanism simply by varying the parameters of speed and orientation. These studies considerably clarify the way in which the space of phenotypic forms can be searched and constrained, thus allowing natural selection to find complex and effective forms without much difficulty.

Self-Organization and Evolution Large a

45

Small a

-b Small b Large b b

Large g Small g

FIGURE 3.6. D’Arcy Thompson showed that with a constant growth mechanism, simple numerical changes in the parameters of cell division, such as speed and orientation, can result in the generation of very diverse forms. (Adapted from Thompson, 1961[1917].)

Thompson has another example which illustrates very well the way in which self-organizational phenomena can facilitate the discovery of complex effective structures by natural selection.5 This is the case of the hexagonal cells 5 Of course, in his time Thompson did not describe this example in the same way as I do, because the concepts of self-organization and dynamic systems had not yet been invented. Nevertheless, a reading of On Growth and Form, which describes many mechanisms that we could now call ‘self-organizing’, without having been conceived as such at the time of writing, suggests that his intuition was moving in the direction in which I interpret his work here.

46


Fig. 146. Argyropelecus alfersi.

Fig. 147. Sternoptyx diaphana.

Fig. 148. Scarus sp.

Fig. 149. Pomacanthus.

Fig. 150. Polyprion.

Fig. 151. Pseudopriacanthus altus.

FIGURE 3.7. Thompson extended his work on molluscs to all kinds of living species, such as fish. (Adapted from Thompson, 1961[1917].)

formed on the walls of beehives.6 This form of cell is remarkable because the hexagonal shape is optimal: hexagons use less wax than any other possible 6 I take the position in this book that structures built in one way or another by organisms, like the wax walls made by bees, are themselves aspects of the characteristic form of the organisms. This point of view corresponds to the idea put forward by Dawkins (1982). In the same way, the speech code will be considered as belonging to the characteristic form of humans.


47

shape to cover the same surface area. There are two ways of accounting for this form. The first takes a classical neo-Darwinian point of view and considers no other mechanism than classic natural selection. The bees would have tried out a whole range of possible shapes, starting with random shapes and selecting those whose construction used up less energy, varying them little by little, and selecting again, and so on, until finally coming across the hexagonal shape. This amounts really to a search for a needle in a haystack, taking into account on the one hand the huge number of possible cell shapes and on the other that this view presupposes identity between the genotypic space and the space of cell shapes; in other words the search is unconstrained. Fortunately for the bees, their search is helped by a providential phenomenon of self-organization. D’Arcy Thompson noted that if the cells are taken to be of roughly the same size and of a shape that is not too twisted, and if the temperature generated by the bees makes the wax walls flexible enough, then cells packed close next to each other will behave more or less like drops of water in the same situation if surrounded by a viscous fluid. Now, the laws of physics work so that each water-drop in such an assembly of drops will take on a hexagonal shape. (See Figure 3.8.) So the bees don’t need to work out how to design a regular hexagonal tessellated pattern, which would require abilities worthy of a young mathematician armed with compasses and rulers, but, much more simply, they have to work out how to make cells which are roughly the same size and not too twisted, packed close to each other. Physics does the rest. And thus, in explaining the origin of hexagonal shapes of wax

FIGURE 3.8. The figure on the left shows the regular hexagonal tessellation in the walls built by bees in their hives. The right-hand figure shows the shape taken by drops of water when they are packed together: it is exactly the same shape as seen in the walls of beehives. (Photos: Scott Camazine, Pennsylvania State University (left); B. R. Miller (right).)

48


cells in beehives, the role of self-organization of physical structure is just as important as the metabolic advantage that this structure confers on the bees.7 3.2.4 Exaptation We have just seen that sometimes the explanation of a shape or a trait in an organism requires much more than establishing its usefulness in promoting successful reproduction. Indeed, it can even happen that such usefulness is not even directly involved in an explanation of the origin of a form. This is again possible because of the existence of self-organization. The interaction of several structures appearing in an organism for Darwinian reasons (each structure helping to replicate its genes more effectively, and each independently of the others) can lead to the self-organizing formation of a new structure which may have no usefulness at all for the organism. One of the most striking examples is that of zebra-like or leopard-like striped or spotted patterns in certain species of molluscs (Ball (2001: 89), and see Figure 3.9). Although one can easily imagine a function for these patterns in zebras and leopards (camouflage), it is hard to imagine one for these molluscs because they live buried in the depths of the oceans, where visual patterns could not be useful for anything at all. These patterns are formed by growth processes in the shells. They correspond to the pigmentation produced during the continuous gradual calcification of the cells at the edges as the shell grows by a process of cell division with a dynamic very similar to that of the well-known Beloutzov–Zabotinsky reaction, a classic example of a self-organization phenomenon (Ball, 2001). In fact, we need not appeal to self-organization to give evidence of forms whose origin is not directly linked to the potential (dis-)advantage they provide for the reproduction of the organism. Some structures can be side effects of the formation of other structures which themselves are useful in the reproduction of the organism. These side effects arise from architectural constraints, a topic developed by Stephen Jay Gould in his many books and papers. We will use two technological examples to illustrate the concepts of side effect 7 This does not mean that nowadays honey bees do not have a precise, innate, hard-wired neural structure which allows them to build precisely hexagonal shapes, as has been suggested in further studies such as those of von Frisch (1974). The argument of Thompson just says that initially the honey bees might have relied simply on the self-organization of heated packed wax cells, which would have led the them to ‘find’ the hexagons; but later on in their evolutionary history, they might have incorporated in their genome schemata for building those hexagons directly, in a process similar to the Baldwin effect (Baldwin, 1896), in which acquired features become innate (the variant is that features are here initially self-organized instead of being learnt).


49

FIGURE 3.9. Certain species of mollusc living in darkness at the bottom of the ocean have striped patterns on their shells. These patterns result from the pigmentation produced during the continuous gradual calcification of cells at the edges of shells as these grow by a process of cell division with a dynamic very similar to that of the well-known Beloutzov–Zabotinsky reaction. These self-organized forms have no adaptive value for the molluscs. (Photo: Hans MeinHardt.)

and architectural constraint. In the first case, the design of electric light bulbs, whose purpose is to provide light, involves passing a current through a metal wire. Now this has a systematic consequence which can sometimes be inconvenient for users of the light bulb: heat is emitted. Electric light bulbs thus have as one of their characteristics the property of generating heat. It is the same with oil lamps. Human engineering cannot make lamps which do not also produce heat. Likewise, it is highly probable that evolutionary engineering is sometimes obliged to tolerate the cost of inconvenient side effects for the benefit of other useful structures from which they result. Gould (1982) gives the example of the tibial sesamoid bone (part of the tibia) of the panda which tends to hinder its walking, but results from a growth pattern which allows the radial sesamoid (its equivalent on the forelimb) to serve as an opposing thumb for handling bamboo stalks. In fact the coupling of the growth of these two bones has brought it about that the adaptation of the radial bone on the forelimb for grasping has also produced a morphological change in the tibia. The first change is positive for the panda, while the second is an inconvenient side effect (but one which is worth the cost because this arrangement has been selected). Obviously the explanation of such ‘side effect’ structures also requires a precise explanation of why they are the consequences of the formation of other structures which must be identified, and whose origin one is also required to explain. These examples of side effects are mostly useful for avoiding getting

50


trapped when trying to explain certain characteristics of organisms. In fact, for the sake of simplification, scientists often try to isolate particular traits which are highly characteristic of an organism, and then to explain them. There is a risk of seeking a utilitarian explanation for such a trait, which perhaps does not exist unless it is linked to the totality of the organism. This structure or trait which one wants to explain is possibly a side effect of another structure which one has not considered. And this danger is equally important when the structure concerned has a useful function for the organism in the eyes of the researcher: this is the case when the function is more recent than the structure. This is the phenomenon of exaptation (Gould and Vrba, 1982): some property N was taken from an earlier state (hence ‘ex’) to be used (hence ‘apt’) in a new role. To illustrate the situation, I will use the example of very long suspension bridges. Imagine that a Martian scientist discovers such a structure. He sees that the pillars of the bridge are extremely high. Why so high? He also sees that each pillar carries a collection of antennae, serving as radio relays for telecommunications. So he says that this must be the reason for the height of the pillars: they enable the relaying of radio waves over long distances. In fact, he is completely wrong: the pillars were built without anyone knowing that they would be used to house radio antennae. They are very high simply because it was desired to minimize the number of pillars along the bridge. Besides, in order for each pillar to be able to support the immense weight of the long roadways, the cables supporting this horizontal part need great strength as well as great resistance. And the closer they are to the vertical, the more effectively they support the roadway. And the higher the pillars, the closer to the vertical are the cables. This is a pure example of an architectural constraint. So the Martian scientist’s error comes from his isolating the pillar from the rest of the structure, and from the pillars having, by chance, found a supplementary use after their construction. In biology, Gould and Vrba (1982) give the example of birds, which initially evolved feathers for regulating their body temperature and recruited them later for flight. They also give the example of certain species of snail which have a space inside their shell in which they ‘hatch’ their eggs. There are also species of snail which have this space but do not use it, and these latter species appeared in evolution before those which use the space. This space is in fact a result of the process of construction of the shell complying with architectural constraints similar to those which require the pillars of suspension bridges to be very high. This space in the snails’ shells is an architectural side effect, with


51

no particular function initially, and was only later recruited for use as a shelter for eggs.

3.3 Explaining the origin of living forms It has been shown in the previous sections that the genesis of a form is a complex process, one which could involve several causal factors. This means that in seeking an explanation for the origin of a living form it is necessary to provide several types of answer, representing complementary visions of the same phenomenon from different points of view. A first type of answer concerns the utility of a form, structure, or trait in terms of the reproductive effectiveness of the organisms which possess it. This is classical neo-Darwinian argumentation, and is often fundamental to an evolutionary explanation. It often takes a short cut by replacing reproductive effectiveness with survival value, or often even with usefulness for some given function. In addition to the caveats for this type of explanation set out above, this kind of short cut, which can be useful, can also sometimes be misleading, because, as Dawkins (1982) has shown, what counts in the mechanism of natural selection is reproductive or replicative effectiveness: it does not always follow that a trait enabling individual organisms to survive better enables them to to reproduce better (and moreover some organisms ‘commit suicide’ in order to perpetuate their genes, like certain species of insect which die to provide food for their offspring: see Dawkins (1982)). It does not necessarily follow that a function that to us seems intuitively useful to an organism helps it to reproduce more effectively: this is the case sometimes with communication and more particularly with the sharing of information, as Gintis et al. (2001) have shown. This pure neo-Darwinian argumentation rounds off its explanation by proposing that the optimal form was produced by the action of the optimization mechanism constituted by natural selection, and often stops there without giving more details on this process of formation. Now we have seen that on the one hand natural selection is not a complete mechanism, in the sense that it does not specify the way in which individuals are formed or, above all, the way in which variation arises, and on the other hand that, if this form of explanation is filled out with a ‘naive’ version of the search of phenotypic space, then it is of limited power and many of the evolutionary problems it is confronted with are seen to be equivalent to the search for a needle in a haystack. This is

52


why a second type of answer, which props up the answer in terms of utility and optimization, explains how natural selection can find solutions, and in particular how it can be helped by the self-organization of the systems on which it operates, and by architectural constraints on the structures which it builds. In practice, this second kind of answer can consist in identifying simpler structures than those which one is trying to explain, corresponding to a phenotypic space which is easier for natural selection to explore, and whose self-organizing dynamic alone generates the global structure in question. For example, if one tries to explain the hexagonal shape of cells in beehives, the first type of answer consists in explaining that hexagons are the shapes that require least energy on the part of the bees (optimality), while the second type of answer consists in explaining that these hexagonal shapes appear spontaneously once the bees pack together cells which are not too twisted in shape and of roughly the same size, and warm them. Thus the space of shapes explored by the bees is significantly simplified, and the chance of happening upon a genome leading to the construction of such cells is much greater. It is exactly this second type of argumentation that provides the drive for this book. I will show that speech codes with the complex properties that we saw in the last chapter can be generated by self-organization starting from much simpler structures. In particular, these simpler structures which will constitute the axioms of our system have a twofold interest. (1) First, it will be easily seen that their complexity, of a completely different order from that of the speech code, makes the discovery of the speech code by natural selection much more understandable than if one starts from the ‘naive’ search mechanism used in classical neo-Darwinian theory. (2) On the other hand, the generality and simplicity of these structures will enable me to explain in Chapter 9 how they could have appeared even without any connection to the function of linguistic communication: in particular, I will use the concept of an architectural constraint, and will propose that they could be side effects of the construction of other structures whose function is not linked to linguistic communication. In short, this will enable me to suggest that the speech codes we use nowadays could be exaptations, whose first versions could have been the outcome of the self-organization of structures whose origin had nothing to do with language. Before presenting these arguments and the system that is at the heart of this study, in the next chapter I will describe the different theories proposed in the literature in response to the questions posed in Chapter 2. I will interpret them in the theoretical context of the origin of living forms which I have just presented in this chapter.

4. Existing Theories

Some of the questions I described in Chapter 2 have already been the focus of much scientific research. Different approaches have been proposed, coming from various scientific cultures, and each with its own methodology. Here I will present the state of the art as reflected in the most representative works. I will also demonstrate that several of the questions posed in this study have not been addressed by existing research.

4.1 The reductionist approach A prime approach taken by the scientific community can be called ‘reductionist’. It tries to reduce the properties of speech to certain of its parts. This approach consists in trying to find physiological or neural structures whose characteristics suffice to deduce from them the characteristics of speech. ‘Cognitive nativism’ (Pinker and Bloom, 1990) defends the idea that the brain has a specific neural disposition for language, (the Language Acquisition Device) and in particular for speech, which knows at birth the properties of speech sounds. This knowledge is supposed to be pre-programmed in the genome. A limitation of this approach is that its proponents have remained rather imprecise on what exactly it means for the brain to know innately the properties of speech. In other words, this is a hypothesis that has not been naturalized, which is to say that the link to biological matter, the issue of its implementation, has not really been addressed. Other researchers defend an approach that could be called morphoperceptual nativism. They focus their attention on the physics of the vocal tract and on the electromechanical properties of the cochlea. They think, for example, that the sound categories appearing in human languages reflect nonlinearities of the system mapping sounds and percepts to articulatory trajectories. Two theories propose different ways of exploiting these non-linearities. First, there is the quantal theory of speech, proposed by Stevens (1972). Stevens observes that there are certain articulatory configurations for which

54


small changes produce small acoustic changes, and other articulatory configurations for which small changes produce large acoustic changes. The phonemes used by languages are, then, located in zones of stability, and unstable zones are avoided. Then there is the Distinctive Region Model of Mrayathi, Carré, and Guérin (1988). This approach, which uses arguments from information theory (Shannon, 1948), proposes on the contrary that speech prefers to use zones in articulatory space for which small changes produce large acoustic modifications. Both Stevens and Carré have carried out simulations with models of the vocal tract and made rather good predictions on possible places of articulation. The fact that their theories are based on contrasting assumptions is very interesting. Now, even if there is no doubt that general properties of the articulatory and auditory apparatus influence the form of speech sounds, an approach based purely on morphology and perception has its limitations. For a start, strong and obvious non-linearities are not found in all regions of articulatory space, especially where the production of vowels is concerned. Moreover, a certain number of perceptual non-linearities are completely cultural and not perceptible to speakers of other languages: Japanese speakers cannot hear the difference between the l of lead and the r of read in English. Therefore such non-linearities do not explain the great diversity of sounds across human languages (Maddieson, 1984), and are of no help in understanding how a given language ‘chooses’ its phonemes. The theories of Stevens and Carré attack the question of why there is suchand-such a phoneme rather than some other, but do not get to grips with the more basic question of why there are phonemes at all. Put briefly, they do not deal with the fundamental problem of phonemic coding: discreteness and combinatoriality. Among reductionist approaches, that of Studdert-Kennedy and Goldstein (2003) addresses one aspect of this question. This is the organization of utterances into independent parallel gestural tracks,1 making possible the delivery of information fast enough for humans to convey complex messages effectively (Studdert-Kennedy, 2005). They note that the vocal tract is composed of independent articulatory organs, such as the tongue, the lips, and the velum. This implies on the one hand a discrete aspect to the physiology of speech, and on the other, since there is only a small number of such 1 A gestural track corresponds to a row in a gestural score, specifying the trajectory of one organ during one vocalization. The gestural score is thus the set of gestural tracks. See Ch. 2 for more details.

Existing Theories

55

organs, a systematic reuse in complex utterances, at least from the point of view of which organs move. Studdert-Kennedy is undoubtedly right on this point. However, other aspects of phonemic coding, and in particular discreteness, remain to be explained. In fact, as they have noted elsewhere (StuddertKennedy and Goldstein, 2003), each organ or set of organs can be used to make a constriction in the continuous space of places and manners of articulation. How is this space discretized? Browman and Goldstein (2000) propose an answer to this question, which I will examine later in the chapter because it is not reductionist, but a mixture of self-organization and functionalism. Also, only certain combinations of gestures from among the available repertoire are used, even though many more are possible. How is the space carved up into possible and impossible combinations? The work that I will present in the next chapters will focus on these questions. Generally speaking, these reductionist approaches study the properties of speech, and in particular phonemic coding and the common features in phoneme inventories in the languages of the world. By contrast, they do not deal with the diversity of speech sounds, nor the cultural formation of specific systems shared by language communities. A fortiori, they propose no solution to the chicken-and-egg problem of the formation of the first conventional codes at a time when they did not already exist. Such was not, however, their goal. Besides, more generally, this kind of research does not really attempt to explain the origin of the speech code, but rather attempts to discover some of its physiological, morphological, and neural correlates. This is thus less of an excursion into causal explanation than into the naturalization of linguistic observations, i.e. grounding the speech code in its biological substance. Nevertheless, through being reviewed in the theoretical framework presented in the previous chapter, these groundings allow us to clarify the problem, exactly by naturalizing it. Studdert-Kennedy, for example, points out an essential biological aspect of the combinatoriality of the contemporary speech code, namely the independent control of the different organs which can modify the shape of the vocal tract. This permits him to formulate a hypothesis according to which this independent control is a result of co-opting the control structures of the facial muscles, in the functional context of imitation (StuddertKennedy, 1998), involving a transfer of information from the visual to the auditory modality. I will not pursue this theory, since it concerns an aspect of phonemic coding which I will not deal with in this study, namely the organization of utterances into independent gestural tracks. As far as phonemic coding in general is concerned, Studdert-Kennedy’s theory is compatible with,

56


and independent of, the theory that I will put forward in this study: the one theory does not rely on the other, and one fills out the other in explaining the origin of phonemic coding.

4.2 The functionalist approach The functionalist approach provides answers of the first type which we identified at the end of Chapter 3: it tries to explain the properties of speech sounds by relating them to their function. It uses the cover-all notion of function, which is why it is labelled ‘functionalist’, but it could equally well have been labelled the classical neo-Darwinian approach. To the question ‘Why do sounds have such-and-such a form?’, functionalists reply ‘They have property N because that helps them to carry out function F.’ To the question ‘How did sounds come to have this form?’, they reply ‘Property N of speech was formed by a broadly Darwinian evolutionary process (biological or cultural) under a selection pressure proportional to its contribution to performing function F.’ This approach could also be labelled ‘adaptationist’:2 property N of speech was put in place for (‘ad’) its present usefulness (‘apt’). Note that functionalist approaches also take into account constraints due to cerebral, perceptual, and vocal structures. The function of the speech code which is typically invoked to explain its form (and implicitly its formation according to a naive version of natural selection) is ‘communication’. The speech code provides a repertoire of forms which should be as efficient as possible, in order for the individuals who use it to understand each other. This efficiency is evaluated by many criteria. The criterion which is always appealed to is perceptual distinctiveness. That is to say, that sounds should be distinct enough from each other for them not to be confused and for communication to take place. The other criteria are often tied to the costs of production, like the energy needed to articulate sounds, or perceptual salience. A repertoire of sounds is thus a set of forms which is quasi-optimal for communication and simultaneously minimizes the costs in terms of energy. This approach differs from the reductionism of the previous section because, instead of looking at phonemes individually, it considers the system they belong to, studied as a whole according to its structural properties. Also, it is truly in the business of explaining the origin of the modern speech 2 Here I use the term ‘adaptationism’ in its general sense: adaptation can be realized at either the genetic or the cultural level.

Existing Theories

57

code, giving answers in terms of ‘optimality’, which I have called answers of ‘type 1’ in Chapter 3. Some researchers have produced precise models of this idea. Lindblöm was the pioneer (Liljencrants and Lindblöm, 1972; Lindblöm, 1992). His model concerns the prediction of vowel systems in human languages. Given a number n of vowels, he defines the energy in a system of n vowels by En =

n n

1/r i,2 j

i =0 j =0

where r i, j is the perceptual distance between two vowels. Each vowel in his simulation is a point in a space defined by the first two formants.3 The possible points are articulatorily confined to a triangle, the vowel triangle (see Chapter 7). This energy is used to measure the perceptual distinctiveness of the whole system. If the vowels are very similar to each other, then the r i, j are small and the energy is high. If they are distant from each other, the energy is low. Lindblöm, using numerical optimization techniques, thus looked to see which were the vowel systems which had least energy, i.e. were minimal. He then found a certain number of resemblances with the most frequent vowel systems of human languages, as far as the systems with less than six vowels were concerned. He then improved the predictions of the model by adding a term modelling the articulatory cost. However, Lindblöm’s results were not very life-like for systems with more than six vowels, and gave a large number of high peripheral vowels between [i] and [u], compared to the languages of the world. A second model, incorporating a new criterion, perceptual salience, was then developed within the framework of Dispersion-Focalization Theory (Schwartz et al., 1997a). This perceptual salience characterizes the relative nearness of the formants of vowels: the nearer they are, the more the energy in one region of the spectrum is reinforced, thus giving the vowel a focal quality. The authors of this theory propose that this property is registered by the brain. This makes it possible to improve on Lindblöm’s predictions. This research method was taken up by Redford et al. (2001), working on syllable structure. Given a repertoire of phonemes, Redford looked to see which were the syllable systems, sequences of these phonemes, that represented the best compromise between such criteria as minimization of word length in the lexicon, minimization of number of syllable types in the repertoire, perceptual 3 Formants are the frequencies where there is a peak in the energy spectrum.

58


distinctiveness between adjacent phonemes, and maximization of the difference in jaw opening between adjacent phonemes. She was then able to predict some phonotactic regularities in the world’s languages, like the preference for syllables of type CV (Consonant–Vowel) over syllables with consonant clusters, or the iterative principle of syllable construction (simple syllables are more frequent than complex syllables in the same repertoire, in an organized distribution). These works, which tried to explain certain properties of the speech code by trying to show that they are there because they make the code quasi-optimal for communication, should not lead one to forget that there are traps such as those I described in the previous chapter. For a start, the definition of such optimality is far from clear, as is shown by the different criteria used by different authors. Evaluation of perceptual distinctiveness depends on one’s model of speech perception; some take energetic costs into account, but we know the speech production system very poorly, so that these costs are inevitably modelled very crudely; some add psychological constraints, such as a preference for salient sounds, i.e. a preference for sounds whose formants are sufficiently separated from each other. In addition, not only is our knowledge of these different criteria very rough, but it is also hard to know how to weight them in relation to each other. For this reason, while it is interesting to show that one can devise definitions of criteria and combinations of criteria that yield qualitative predictions about the form of the speech code, the exact predictions must be approached with more caution. Moreover, I have shown in the previous chapter that forms could sometimes be explained without necessary recourse to their utility in a given function. It will be shown in this study that this can be the case for certain properties of phoneme inventories: the system that I will build leads one to predict the same properties of vowel systems as those predicted by Lindblöm, but it will not appeal to any explicit pressure for efficient communication.

4.3 Operational scenarios As was explained in the previous chapter, all explanations of the type given by Lindblöm and those inspired by his work should be accompanied by an explanation of the second type, which shows how the optimal solution for communication, embodied in the speech code, can be found. In fact, while Lindblöm has shown that one can predict the most frequent phoneme systems

Existing Theories

59

of human languages on the basis of optimizing a certain number of criteria, he says nothing about the way in which this optimization could have taken place in reality. Furthermore, as with reductionist theories, he doesn’t undertake to explain how a community of speakers can come to share a sound system. Accordingly there are now a number of works which partially respond to this lack. These are inspired by the pioneering works of Steels (1997), which consist practically of constructing populations of artificial agents, sometimes in robots, sometimes simulated, and providing them with a certain number of capacities and mechanisms in such a way that they manage, culturally and in a decentralized way, to form a linguistic norm shared by all the members of the community. This type of model has been used for the formation of shared lexicons (e.g. Kaplan, 2001), shared systems of categorization (e.g. Kaplan, 2001), conventionalized syntactic rules (e.g. Kirby, 1998), and inventories of sounds. Here I will survey those which are particularly directed at inventories of sounds and which I judge to be the most representative. I will come back to their presuppositions, and show what are their strong points and their limitations, from the viewpoint of the problems I face in this study. De Boer (2001) carried out a simulation of the formation of vowel systems. In this simulation, which used artificial agents, the same mechanism explains both the acquisition of vowels and their emergence: this mechanism is imitation. Thus, he undertakes in addition to answer the question: ‘How are vowel systems learnt?’ De Boer’s model does not deal with the problem of phonemic coding: discreteness is built into the simulation; the agents only produce static vowel sounds, and thus do not have occasion to compose sounds, and even less to reuse them. Nevertheless, de Boer’s model is very interesting because it shows a process of conventional sound formation in a population of agents. The agents in de Boer’s simulation interact according to rules of a language game (Steels, 1997) which is called the imitation game. Each agent has a vowel synthesizer, so that, given a point in articulatory space (tongue position, height or manner of articulation, lip-rounding), it can produce a vowel. Vowels are points in a space defined by the first two formants. Each agent also has a repertoire of prototypes which are associations between a point in articulatory space and the corresponding (acoustic) vowel. This repertoire is initially empty. It expands either by random invention or in the course of the learning which takes place during an episode of interaction. In one interactive episode between two agents, one of them, called the speaker, chooses a vowel from its repertoire and pronounces it to the other agent, called the hearer. Then, the

60


hearer looks to see which prototype in its repertoire is closest to what it has just heard, and pronounces it in imitation of the other agent. Then the speaker categorizes this sound by looking in its own repertoire to find the closest prototype. If it is the same as it used in its own initial utterance, it judges the imitation a success and makes this known to the other agent by telling it that it was good. Otherwise, it tells it that it was bad. Each prototype in a repertoire has a score which is used to reinforce the vowels leading to successful imitation, and to eliminate the other vowels. In a case of poor imitation, depending on the score of the prototype used by the hearer, either this is modified so as to resemble more closely the sound made by the speaker, or a new sound is created, as close as possible to that of the speaker. It can be seen from the description of this game that it requires a certain number of complex abilities on the part of the agents. For a start, the agents have to be able to follow the conventionalized rules of a game with successive turn-taking and asymmetric roles (speaker and hearer). Then, they need to be able voluntarily to copy the sound productions of other agents, and to be able to evaluate such copies. Finally, as speakers, they have to recognize when someone tries to imitate them intentionally, and give a feedback signal to the hearer to tell it whether or not it has succeeded. The hearer must be able to understand this feedback, to understand whether its imitation was successful as seen by the other agent. It seems that the level of complexity necessary for the formation of shared vowel systems in this model is characteristic of a population of agents that already has complex ways of interacting socially, and in particular already has a primitive system of communication (which enables them to know, for example, which is the speaker and which is the hearer, and what signal means ‘imitation succeeded’ or ‘imitation failed’). The imitation game itself is a conventional system (the rules of the game!), and the agents communicate when they play it. This in effect necessitates the intentional transfer of information from one agent to another, and thus requires a system of shared forms making such information transfer possible. The vowel systems do not emerge from an entirely non-linguistic situation. This does not imply a flaw in de Boer’s model, but rather implies that it deals with modelling the evolution of particular languages,4 and not with the evolution of language in general. De Boer has in fact presented very interesting results concerning the changes which can take place in a vowel system in the course of the cultural history of a 4 Of the sounds of languages, to be precise.

Existing Theories

61

population of agents. This results from the stochastic nature of his model and the properties of the learning mechanisms in successive generations of agents. But the model does not engage with the chicken-and-egg question of how the first repertoire of shared forms could have appeared without prior complex communication or complex patterns of social interaction. In particular, the question of why the agents try to imitate each other in de Boer’s model is left unanswered (it is programmed in). De Boer’s work, like those of Kaplan, Steels, and Kirby, takes a crucial step forward in showing how conventional shared linguistic structures on the scale of a whole population of agents can emerge through self-organization starting from the properties of individual agents. In other words, this work constitutes a naturalization of the cultural phenomenon of language formation (of sound-system formation). Now, the biological capacities with which each agent is equipped in these simulations are already language capacities. The capacity to play a language game, a conventionalized and structured ritual whose objective is communication, is the most striking example of this. The question of knowing how these capacities could have arisen is fundamental to understanding the origin of language, and is so far unanswered. De Boer’s work, which only dealt with the production of static vowels, has been extended to syllables composed of a wider range of phonemes, including consonants, by Oudeyer (2001b; 2001c; 2001d; 2002b). By analogy, this work is to Redford’s simulations on syllables what de Boer’s model is to Lindblöm’s simulations of vowels. Agents played the same imitation game in these simulations, but their repertoire consisted of syllables. At the beginning of the simulation, they were given a shared repertoire of phonemes (imagined as the result of an earlier game, such as one of de Boer’s simulations). The scores of the agents’ prototypes no longer depend only on success or failure in imitation, but also on the energy necessarily taken to produce them. It was possible to predict two regular properties of syllabic structure in human languages with this model. The first is a preference for CV syllables, followed by CVC, then CCV and CVV, then V and VC, then CCVC and CCVV, and finally other syllable types. The Sonority Hierarchy is also predicted by this model: syllables tend to begin with a phoneme with a high degree of obstruction to airflow, then to allow the obstruction to diminish to a certain point, and then to increase the obstruction again up until the final phoneme of the syllable. Oudeyer (2001d; 2005e) presented a study on the learnability of syllable systems in the framework of the imitation game. It was shown that, while agents have no problem learning a syllable system already constructed by a

62


population of agents,5 this does not happen given a randomly invented syllable system: if this random system is inserted by hand in the heads of a population of agents, any other agent who did not already know it could not learn it. This is due to the fact that the agents’ learning procedures, like all learning procedures (Duda et al., 2000), are biased (even despite the fact that it is a general learning procedure that can be used for very different tasks, like learning to classify flowers according to their shape, or chemical substances according to measurements by electronic instruments). The fact that there are biases implies that certain types of data are easier to learn from than others. What is interesting in the simulations of Oudeyer (2001d; 2005e) is that populations of agents culturally select syllable systems that they are capable of learning. In other words, syllable systems adapt to the ecological niche provided by the brains and morpho-perceptual organs of the agents. This proposal reverses the perspective taken by researchers in the nativist cognitive tradition presented earlier, who propose a different scenario to explain the fact that children learn a language and its grammar so easily, and especially the sound patterns, despite the poverty of the stimulus that they receive (Gold, 1967). These researchers in effect argue (and certainly with reason) that it is impossible for a generic learning device, with no a priori knowledge of language, to learn any arbitrary language it is presented with. They conclude from this that the brain must know innately how language is organized, especially in its sound patterns, in order to be able to learn it. Thus, the brain itself would have adapted to language during the course of biological evolution in order to be able to learn it (Pinker and Bloom, 1990). Oudeyer (2001d; 2005e) suggests another explanation: one should not imagine that just any arbitrary language could be presented to a learning system. In fact, the process itself by which languages are formed develops and selects just those which can be learnt. It is languages which adapt to a generic brain, and not the contrary. Just as in de Boer’s model, the works of Oudeyer (2001b; 2001c; 2001d; 2002b; 2005e) do not deal with the chicken-and-egg problem. They take for granted the existence of quasi-linguistic means of interaction, and thus are only concerned with modelling the evolution of languages. They bring a supplementary dimension to de Boer’s works in that they attack the problems of the grammar of sounds, and are no longer restricted only to vowels. Nevertheless, they make certain assumptions about the bases of this grammar: 5 The agent is simply made to interact with agents who already ‘speak’ the syllable system.

Existing Theories

63

agents are endowed from the start with a shared and discrete repertoire of phonemes. That is to say, that discreteness, which is one of the central issues of the present study, is taken for granted in these models, and is thus not explained. By contrast, Oudeyer (2002b) presented an experiment in which discreteness and combinatoriality was not pre-programmed in order to study a prediction made by Lindblöm (1992) which suggested that combinatorial systems are optimal compromises between perceptual distinctiveness and ease of articulation. Thus, just like the regularities in vowel systems or the phonotactic regularities emerging from the models of de Boer (2001) and Oudeyer (2001d), one might have expected that among the phonemes put at the disposition of the agents, certain ones would be chosen and systematically reused in constructing syllables. However, that is not what happened. Oudeyer (2002b) carried out a more precise study of this matter, varying the numbers of phonemes given to the agents as well as the degrees of freedom in their vocal organs (a model of the vocal tract in the form of an acoustic tube) and their perceptual organs (a model of the cochlea). When the number of phonemes given to agents was small in proportion to the number of syllables that their repertoire could contain, then necessarily they were systematically reused. But putting a small number of initally shared phonemes in the heads of agents obviously comes down to pre-programming combinatoriality, and leaves open the question of where these phonemes, a radical discretization of the articulatory continuum, come from. When the number of phonemes was increased, for all configurations of the vocal tract and the perceptual apparatus of any reasonable dimensions, the syllable systems which emerged (consisting of several hundred elements), were never combinatorial, even with a lot of added noise, which raises the effectiveness of optimization achieved in the imitation game. Thus, agents in the simulation arrived at constructing very large shared syllable systems, without there necessarily being any combinatoriality (systematic reuse). Another model in the spirit of those of de Boer (2001) and Oudeyer (2001d) is that proposed by Browman and Goldstein (2000). Their work approaches the question of the origin of discreteness (they speak in terms of the emergence of discrete gestures). They constructed a simulation in which two agents can each produce two gestures, parameterized along a dimension of constriction whose values are taken from a one-dimensional continuum (typically, this space is the space of place of articulation). The agents interact following the rules of a game called an ‘attunement game’ (this could be paraphrased as a ‘game in which each agent adjusts to the adjustments of the others’).

64


In one episode of the game, the two agents produce their two gestures, each using a value of the parameter taken from the continuum with a certain probability. At the beginning of the simulation, this probability is the same for all values: in other words, all values on the continuum are equally usable. Next, each agent reconstructs the parameter used by the other agent for its first gesture, and compares it with the parameter which it has used itself. If the two values correspond, within a certain tolerance interval, then two things happen: the probability of using this value of the parameter for the first gesture is increased, and the probability of using the same value for the second gesture is decreased. This simulates the idea that the two agents are each trying to produce their gestures differently (so that they can be differentiated and contrasted), and at the same time in a way similar to the other agent (so that a shared conventionalized usage is established). At the end of the simulation the agents converge on a state where they use a single value for each of their gestures; thus the space has been discretized, and the pairs of values of the agents are the same in each simulation, but different from one simulation to the next. Browman and Goldstein carried out simulations both using and not using non-linearities in the function from sounds to articulatory parameters (this is implemented by modelling the noise added when an agent reconstructs the parameters used by the other agent). When non-linearities were not used, the set of parameters used across all simulations covers the space uniformly. When non-linearities were used, then certain parameters in regions of stability were statistically preferred. As in the simulations by de Boer (2001) and Oudeyer (2002b), in this model the agents have coordinated interactions: they follow the rules of the game. Every time, indeed, they have to produce their gestures together during one interactional episode. Thus, as in the imitation game, a pressure to differentiate sounds is built in, as is a pressure to copy the parameters used by the other agent. This implies a presupposition that the agents already live in a community in which a complex communication system exists. However, this is certainly not a consideration taken into account by Browman and Goldstein in their work in the framework of phonological research, whereas the study presented in this book is instead formulated within the framework of research into the origins of language. So it still remains to be seen how the discreteness of speech, which seems to be crucial for the birth of language (Studdert-Kennedy and Goldstein, 2003), could have appeared without assuming that a complex communication system was already in place. More exactly, how could a discrete speech system have

Existing Theories

65

appeared without an explicit pressure to contrast sounds? This is one of the problems which I propose to resolve in this book, but I will come back to it. In addition, in Browman and Goldstein’s model, one assumption is that the agents directly exchange the articulatory targets which they use in producing gestures (noise is added, but nevertheless the agents receive articulatory targets). However, human vocalizations are continuous trajectories, first in acoustic space and secondly in the articulatory space of relations between organs (henceforth ‘organ relation space’). Thus what a human retrieves from the vocalization of another is not articulatory targets which were used to specify gestures, but rather the realization of these gestures, which is a continuous trajectory between the starting state and the target. And because several targets are aimed at in sequence, vocalizations do not stop when one target is reached, but continue on their path toward the next target. To retrieve the targets from the continuous trajectory is very difficult, and a task which speech recognition engineers have not conquered. Possibly the human brain is equipped with an innate capacity to do this, by detecting events in the flux of sounds which correspond to the targets, but this is strongly speculative and is thus a strong assumption in any model (although certainly an interesting one). In this book, I will not make this assumption: agents will produce complex continuous vocalizations specified by sequences of articulatory targets, but they will not at first be capable of detecting acoustic events which might help them in identifying articulatory targets. Rather, they will use a temporal resolution filter which has the effect that each point on the trajectory is considered as an articulatory target (even though in fact only some of them really are). This introduces a very high level of noise (not white noise, but with a particular structure). However, I will show that the population of agents converges on a state in which they have broken down the continuum of possible articulatory targets into a discrete repertoire of gestural commands shared by the population. Thanks to the structure of the activations of the neural nets of the agents, at the end of a simulation it will in addition be possible to retrieve the articulatory targets in a continuous trajectory (although this will be a result rather than an assumption).

4.4 Going further We have seen that the operational models developed in the literature show how the cultural phenomenon of language formation can be made to seem

66


natural, by demonstrating that languages can emerge by self-organization out of the decentralized interaction of agents. I have also explained that the capacities attributed to the agents in these simulations are already quasi-linguistic, with, for example, the capacity to play language games, conventional interactions which are themselves norms structurally equivalent to primitive linguistic norms. Moreover, these capacities are so complex that they do not really make it clear how natural or cultural selection could have formed them. In this book I will ally myself with the research tradition that builds artificial systems supporting explanations of type 2. However, the structures which will constitute the initial biological baggage of the agents will be much less complex.6 In particular, the agents will not have the capacity to interact in structured ways, they will not have access at the outset to any conventional norm, and they will not have any inbuilt pressure aimed at motivating them to communicate. (To compare more precisely with the imitation game of de Boer (2001) and Oudeyer (2001c), they will not try to imitate each other, there will be no positive or negative reinforcement signals, no mechanism verifying whether the categorizations are the same or not, and no mechanism driving the agents to differentiate sounds.) In addition, the speech codes which will be generated through selforganization during the interaction of these structures will be more complex than those generated by de Boer (2001) or Browman and Goldstein (2000). The self-organized systems of complex vocalizations will be characterized by the following properties: the vocalizations will be discrete and combinatorial; the repertoires of emerging units will be shared by all the individuals in a population and variable from one population to the next; the ways in which phonemes are put together will be organized according to phonotactic rules. In addition, the use of morphological constraints will lead us to make the same predictions about vowel systems as those already made by de Boer, but on the basis of assumptions of a different order of evolutionary complexity. These less complex structures which, then, will constitute the assumptions of my system are interesting in two ways: 6 It is crucial to note that we are speaking here of ‘evolutionary’ complexity, which is not related to the complexity of the programme implementing the artificial system, which itself depends on the degree of detail one wishes to model. Thus, the programmes used in this work are definitely more complex than those used by de Boer, for example, but this is because I am modelling the capacities of agents at the neuronal level, while his model was at a higher level. By way of example, in neuroscience, there are models of individual neurons whose implementation is much more complex than that of the artificial system described in this book: nevertheless, a neuron is evolutionarily a much less complex system than a network of neurons controlling, e.g., the perception of sounds.

Existing Theories

67

1. One could consider these structures within the framework of a classic neo-Darwinian theory of the origin of language which would defend the idea that the biological structures which are the basis of the capacity for language, and thus include the bases of the capacity for speech, are the outcome of natural selection under pressure of communication, a function that helps individuals to replicate their genes better. Seen in this light, the artificial system in this book will make it more understandable how natural selection could have arrived at the biological bases of speech: in effect, I will show how these bases can be relatively simple by comparison with the speech systems that they generate. I will thus propose an explanation analogous to D’Arcy Thompson’s explanation for the formation of hexagonal cells in beehives: the biological structures that natural selection arrives at for the bees are not those of a mathematician armed with a compass and rules and exact plans for dividing up the surface regularly, but simpler structures allowing for cells of roughly similar size, not too misshapen, to be piled together. 2. On the other hand, the generic nature of these structures will allow me to explain in Chapter 9 how they could have appeared without any connection to the function of linguistic communication: in particular, we will use the concept of architectural constraint, and propose that they could be side effects of the construction of other structures whose function is not connected to language. In brief, this will allow us to suggest that the speech codes that we use nowadays might be exaptations, the first versions of which might have been the outcome of the self-organization of structures whose origin has nothing to do with language. In both cases the artificial system presented here will attempt to illuminate the issue of the origin of the language faculty (rather than the origins of specific languages), by showing how the evolution of one its fundamental prerequisites, the speech code and its biological bases, could have been crucially facilitated by self-organization phenomena. Before presenting the artificial system, I will turn in more detail in the next chapter to the research method based on the construction of artificial systems.

5. Artificial Systems as Research Tools for Natural Sciences

Theories of the origin of language, and of speech in particular, are necessarily complex, and just expressing them in words is not enough to make them convincing. In fact, verbal theories remain approximate about their assumptions because they use natural language, and are not capable of proving that their premises lead to their conclusions because they necessarily involve complex dynamic systems whose behaviour is difficult to predict exactly by intuition alone. Building artificial computational systems is a way of avoiding these pitfalls and evaluating the logical coherence of verbal theories; it is also a way of generating and formulating new theories. This has been called ‘the methodology of the artificial’ (Steels, 2001). This approach, very widespread in biology (Langton, 1995), has been used more and more in linguistics in recent years (Cangelosi and Parisi, 2002). For example, artificial systems have been developed showing the formation of shared vowel systems (de Boer, 2001), of syllables (Oudeyer, 2001b), of lexicons (Kaplan, 2001), of semantic categories (Kaplan, 2001), or of syntax (Kirby, 1998; Batali, 1998). I will now explain this method in detail, while laying out its epistemic foundations.

5.1 What is the scientific logic? There is a great variety of conceptions of science (Bachelard, 1865; Bernard, 1945; Chalmers, 1991; Feyerabend, 1979; Kuhn, 1970; Popper, 1984). I will adopt the constructivist point of view defended by Glasersfeld (2001). There are several types of activity in science: 1. Some scientists do experiments and make observations of reality. 2. Others study existing theories and try to derive predictions about reality from them, and devise experiments which could falsify or confirm a theory.

Artificial Systems as Research Tools

69

3. Others study the correspondences between observations and the predictions of theories. 4. Others study the internal coherence of theories, and the compatibility or incompatibility between theories, while trying to build bridges between them. 5. Yet others construct theories. There are several ways of arriving at a new theory. One of them is called ‘abduction’, a term invented by Peirce (1958 [1931–5]). This consists, given the state of a system, in figuring out an initial state as well as a mechanism which could have led to the observed state. More succinctly, it is a matter of finding a set of premises which leads to a certain conclusion. It is in some way the inverse of ‘deduction’ (Peirce, 1958[1931–5]). For example, if one sees the end of a pair of shoes sticking out from behind some curtains, one can generate the hypothesis that there is someone behind the curtains. This is abduction. Obviously, the theories and hypotheses generated could be false: maybe the shoes are empty. Consequently, once a theory has been formulated, it is necessary that researchers engage in activities 2, 3, and 4, to evaluate the correspondence between the theory and reality (although, as I shall explain, a theory can still be useful even if it corresponds poorly with reality). The study presented in this book is based on this logic of abduction. This is almost inevitable, given the problem I am attacking. In effect, I am participating in a quest for a theory about the origin of certain aspects of speech; but the state that humans were in when speech arose has left very few traces! The development of the artificial system in this work will make us carry out activities 5 (I will construct a new theory and present a formal description of it), 4 (I will run the system and show that it leads to conclusions, and I will use this theory to evaluate the internal coherence of other theories), and 3 (I will present comparisons between the behaviour of the system and reality). Note that I avoid the use of the term ‘model’ to describe the artificial system presented in this book. This reflects the fact that its goal is not to mimic reality realistically, but to help us to understand reality. I prefer to use the terms ‘mechanism’ or ‘artificial system’. I also use the term ‘theory’. I believe that artificial systems are themselves narratives, albeit not verbal, which help scientists to understand the world. We will not take the point of view that considers artificial systems as implementations of verbal theories. They are theories expressed in a formal language. To be sure, many artificial systems are inspired by verbal theories, but they are formal variants of them, and not implementations.

70


5.2 What is the point of constructing artificial systems? Before going into the details of the mechanism, some criticisms can be addressed touching on the usefulness of the approach. First of all, how can we link abstract entities in a computer program with the real world? How do we verify the hypotheses embodied in the program? Is it even justifiable to attack a problem like the origin of speech, since after all there is a high likelihood that we will never be able to verify our theories, since no traces of the origin are left? These questions can be summarized as: ‘What is the point of an artificial system? How can it be verified?’ In reply, I will first set out my view of science. I consider it to be a highly constructivist activity (Glasersfeld, 2001). The goal is to form representations (often in the form of machines) which help us to understand the world we live in. In other words, theories are narratives about the world which help us to put some order into our view of it. To be sure, there are constraints on how we construct these narratives, with the result that certain theories like the Bible are not scientific theories. Another important constraint is that there must be no ‘miracles’. Furthermore, these narratives must form a coherent whole, and it should be possible to pass from one to the other by logical ties. There may be gaps in the picture they paint, and in fact a large part of scientific activity is guided by the goal of filling these gaps. These narratives must also be compatible with one’s observations, but the observations depend on the theoretical context in which they are made. From time to time, new theories are seriously at odds with the set of narratives accepted by scientists of one era, but scientists replace the old narratives with new ones, because they judge them to be more useful in understanding the world (this is what Kuhn (1970) calls a paradigm change). An example is the shift from Newtonian physics to the theory of relativity. This vision is summarized by Einstein: Physical concepts are free creations of the human mind, and are not, however it may seem, uniquely determined by the external world. In our endeavor to understand reality we are somewhat like a man trying to understand the mechanism of a closed watch. He sees the face and the moving hands, even hears its ticking, but he has no way of opening the case. If he is ingenious he may form some picture of a mechanism which could be responsible for all the things he observes, but he may never be quite sure his picture is the only one which could explain his


71

observations. He will never be able to compare his picture with the real mechanism and he cannot even imagine the possibility or the meaning of such a comparison. (Einstein and Infeld, 1967[1938]). Consequently, the central principle is: The object of all science, whether natural science or psychology, is to co-ordinate our experiences and to bring them into a logical order. (Einstein, 1955[1922]). On this basis, I defend the idea that a theory can be useful even if its connection with reality is tenuous or if observations show it to be false. For a start, there are phenomena that we understand so poorly that it is already useful to be able to imagine what sort of mechanism might explain them. Let’s return to the example of ice crystals. In physics, there are well-established theories on the behaviour of water molecules. In addition, we know quite well the shape of ice crystals, and even under what conditions of temperature, pressure, and humidity these shapes appear. However, there is a gap in the narratives of physics between the level of water molecules and the level of crystals. The presence of ice crystals was a ‘miracle’ for several decades. But researchers (see Wolfram, 2002) did computer simulations which transformed this miracle into something understandable. They used cellular automata—grids in which a cell can be in an ‘on’ or an ‘off ’ state. Their state evolves as a function of their neighbouring cells and according to rules of the type: ‘if one neighbour is on, then switch on too’. Wolfram (2002) presents rules of this type such that starting from a group of four cells switched on at the centre of the grid (the ‘seed’ molecule which starts the crystalline growth), a shape similar to that of ice crystals appears spontaneously by growth and self-organization. See Figure 5.1 for an example. To be sure, this cellular automaton has nothing to do with real water molecules. However, this simulation suggests a particularly stimulating vision of the formation of ice crystals: after having seen it, it is easy to imagine that only the interaction of water molecules, with their known properties—even if they have nothing to do with the symmetry of crystals—can lead spontaneously to their formation without the need to appeal to other forces. Thus one aspect of the properties of ice crystals is no longer a miracle. Next, a theory, especially a formal or computational theory, can be useful in evaluating the coherence of other theories or logical compatibility among several theories. A theory which is at some distance from reality or unverifiable can still bring order to the set of existing theories. In fact, many of the

72


FIGURE 5.1. This figure was obtained after 13 iterations of the rule ‘if a neighbour is on, switch on too’, starting from an initial state with 4 central cells switched on and all the others off. The different colours correspond to different steps in the iteration (the black cells correspond to stages 0, 5, and 10). This shows that rules of local interaction can lead to the growth of shapes resembling ice crystals, thanks to self-organization. (For more information, see http://www.lps.ens.fr/˜weisbuch.)

narratives which scientists construct, especially in the social sciences, are verbal. Because they make only approximate assumptions or depend on intuition to derive their conclusions, they may contain errors, or on the other hand not be convincing enough for part of the scientific community. I will give examples of simulations in computer science which allowed the subject to advance in these two kinds of case. For example, it is sometimes suggested that when babies imitate the arm movements of another person with a movement of their own arm, this shows that they must have a concept of the ‘other’ as opposed to the ‘self ’—i.e. a simple form of theory of mind—and a fortiori that the baby possesses primitive representations (Guillaume, 1925). However, Andry et al. (2001) have presented a robotic model in which a robot copies an arm movement of someone else, even though it has no concept of ‘self ’ as opposed to ‘other’ and uses no representation. The copying is an effect of what they call perceptual aliasing: the robot thinks that the arm that it sees is its own, and corrects the error between this perception and the state of its motors. This shows that the observation of an apparently complex imitation behaviour does not require theory of mind. To be sure, this does not say whether the baby has a theory of mind or not when it does this action; but this experiment constrains the intuitive


73

conclusions that one may reach on observing its behaviour. The way in which the robot copies could be very different from the way in which the baby copies; but this robotic simulation acts as a very powerful cautionary tale. This kind of cautionary tale is interesting because we know exactly what is in the robot or the program, and this opens the way to suggestive comparisons based on behavioural analogies between the robot and living beings (many researchers also use comparisons between animals, which has its advantages, but also the disadvantage that we don’t know much more about the ways in which animals function). A second example of a computer simulation concerns the case of the Baldwin Effect. In the nineteenth century, Baldwin (1896) and Morgan (1896) developed a theory that genetic evolution could be modified by learning. More exactly, they proposed that certain acquired behaviours could become innate (genetically hard-wired) with the passage of generations and at the level of the whole population. This theory was mainly verbal and was ignored by the scientific community until recently (French and Messinger, 1994). This was not surprising: this theory is quite close to Lamarckism, which says that evolution consists in the inheritance of acquired characteristics (an idea well discredited since Darwin). However, Hinton and Nowlan (1987) presented a computational model proving that the concept of the Baldwin Effect is compatible with Darwinian theory.1 Their model consists of a population of sequences of 0 and 1 and ‘wild cards’, postulated to be the genes of a population of agents. Each gene codes for a trait which can have the value 0 or 1. This value can be innately specified in the genome or acquired (by using the wild cards). The agents are evaluated according to a selection function: if all the agent’s traits, after using its wild cards, are 1, then the agent has the maximal adaptive value; otherwise it has zero adaptive value. This adaptive value is used to determine which agents reproduce the most. The simulation showed that when wild cards, modelling learning, are allowed, the population converges much faster on the situation in which everyone has all traits set to 1 than it does when learning is not allowed. Moreover, at the end, all the wild card alleles disappear and all the traits are set to 1 innately. This model is very distant from real biology. Its assumptions are even false (for example, it is assumed that there is no structural difference between the genotype and the phenotype). However, it has changed evolutionary biology: the idea of the Baldwin 1 As stated above, Hinton and Nowlan’s artificial system is itself a theory, compatible with Baldwin’s theory, but more precise, so that its coherence can be evaluated.

74


Effect is now considered as plausible by biologists. No one knows yet whether the phenomenon actually occurs in nature, but science has taken a step forward. To summarize these different examples, abstract models distant from reality can be useful for drawing and constraining the contours of the research space of theories. They can also be useful for understanding by analogy and can make possible the transformation of apparent miracles into intuitively understandable phenomena. The mechanism presented in this book will be evaluated within the framework of this methodology. I do not intend to propose direct or definitive answers. The aim is exploratory: I will try to formulate and suggest a possible and original type of mechanism responding to the problem issues presented in Chapter 2. The relationship between the artificial system and human systems is not a relationship of identity or of modelling, but a relationship of analogy. I will not try to show that its assumptions can be identified with precise aspects of the real world,2 but rather that they are useful for spelling out the contours of the space of possible explanations that have so far been proposed, as well as generating new types of explanation. To be precise, the aim of this work is to participate in the organization of theoretical thinking on questions of the origin of speech. For this reason, the work should mainly be evaluated by judging the impact it will have on the thinking of researchers in this domain. To summarize, this work should not be read while wondering whether it is true or false, but whether it is useful or not.

2 This is in any case largely infeasible today, due to the low level of our knowledge of speech phenomena, especially their neural aspects.

6. The Artificial System

In this chapter I will describe a preliminary version of the artificial system. This is a computer program simulating a world composed of entities called agents. These agents are a kind of robot in a virtual world. They are endowed with mechanisms for the perception and production of sounds. They are located randomly in the virtual world.

6.1 The mechanism To construct the agents’ speech production and perception mechanisms, we will adopt the point of view of articulatory phonology (Browman and Goldstein, 1986), and more generally of a whole section of the research community in mammalian motor control (Kandel et al., 2000), which defends the idea that gestures (or more generally relations between organs) are the central representations. Here, to be precise, I will call a ‘gesture’ a command specifying an articulatory target, itself defined by a relation between several organs (like place of articulation or the distance between the lips).1 I make the following simplification in relation to the description of vocalizations made by articulatory phonology: instead of having vocalizations defined by several parallel tracks of gestures, there will only be a single track. This means that a vocalization will be a sequence of gestures strung together, but there will be no other gestures to carry out in parallel. In addition, for the purpose of visualizing the results, I will only use one, two, or three dimensions to define the articulatory targets (which could be seen as place and manner of articulation as well as liprounding, for example). One vocalization will consist of a sequence of two, three, or four gestures. These gestures, which are commands, are executed by a lower-level control system which makes the organs move continuously in such a way that the articulatory targets are reached at the relevant times. 1 However, the results presented here are also compatible with the idea that articulatory targets are configurations defined by the individual positions of the organs, or else defined by a set of formants, for example.

76


In the first version of the system, I will assume that agents are capable of retrieving the trajectory of relations between organs corresponding to a sound that they hear.2 In this chapter I also assume that agents are capable of finding the muscular activations which will move the organs in such a way that the relations between organs corresponding to a given sound are realized. Papers in the literature have, moreover, already presented mechanisms explaining how these translations from one space to another can be learned and realized (Bailly et al., 1997; Oudeyer, 2002a). I will give an example in the following chapter. For the moment, I simply assume that agents can do this. This will allow us to use just a single representation in this chapter, namely that of relations between organs. Thus, I will not use acoustic or muscular representations, which will give us a more intuitive visualization and understanding. I will use the term ‘sound’ to refer to a vocalization, but I will only work with the trajectory of relations between organs (thus I allow myself in this section a slight extension of the term ‘sound’). What an agent perceives directly in the vocalization of another agent is the continuous trajectory of relations betwen organs (but not the gestures which produced it). I will now give the details of the assumptions of the mechanism. We present them arranged in six groups. 6.1.1 Assumption 1: neural units The brains of agents are composed of neural units, in the notation ni , and the set of all neural units is called a neural net or a neural map. A neural unit (which I will also sometimes call a ‘neuron’, since it is inspired by the neurons in the human brain) is a box which receives input signals, corresponding to measurements, and integrates them to calculate a level of activation. Typically, the integration is carried out by first of all comparing the vector of all the inputs with an internal vector, notated as v i and labelled the ‘preferred vector’, peculiar to each neural unit.3 Then, the result of this comparison is filtered

2 I use the expression ‘trajectory of relations between organs’ instead of ‘sequence of gestures’ because I will not assume that the agents are initially capable of retrieving the articulatory targets from which the continuous trajectory was generated (articulatory targets will be points like others on this trajectory). In fact, any point on this trajectory could be a target used in specifying the production of the vocalization. This means that initially the agents are not capable of detecting the ‘sound events’ which might correspond to articulatory targets. 3 This internal vector corresponds to the weighting of neurons often used in the neural network literature.

The Artificial System

77

by what is called an ‘activation function’, which calculates the activation, or response, of the neuron. The activation function here is a Gaussian, whose width is a parameter of the simulation. Gaussian activation functions ensure that there is one input to which the neuron responds maximally: this is the input with the same value as v i . This is why it is called the preferred vector. When inputs are distant from the preferred vector in the space of inputs, the level of activation decreases according to a Gaussian function. When the width of the Gaussian is broad, this implies that it is not very specific, i.e. that there are many different inputs to which the unit responds significantly. If we denote by G i,t the activation function of neuron i at time t, and if s is a stimulus vector, then the form of the function is 1 1 2 2 e − 2 |vi,t −s | /σ G i,t (s ) = √ 2πσ

where vi,t defines the centre of the Gaussian at time t and is the preferred vector of the neuron. This means that when a perceptual stimulus is sent to a neuron i , then this neuron will be activated maximally if the stimulus has the same value as vi,t . The parameter σ determines the width of the Gaussian, and so if it is large the neurons are broadly tuned. A value of σ 2 = 0.001, as used below, means that the neural units respond significantly to about 10 per cent of the space of inputs; see Figure 6.1. This value corresponds to the default level of noise that was used in de Boer’s thesis (2001). The preferred vectors of neural units change as and when new inputs are perceived. This is how learning happens, and is described below under the heading ‘Plasticity’. The width of the Gaussians does not change when stimuli are perceived; that is fixed. Thus the centres of the activation functions evolve over time, but not their width. 6.1.2 Assumption 2: perceptuo-motor correspondences As I have already said above, I assume that an agent, given an acoustic signal, which is a continuous trajectory, is capable of retrieving the corresponding trajectory of relations between organs (which produced the sound).4 4 The function mapping organ relation space to the space of muscular activations is not an isomorphism: a target in organ relation space can often be realized by several organs or combinations of organs. I assume that at least one possibility can be found. How the choice between several possibilities can be made is described in the literature (Bailly et al., 1997).

78

Self-Organization in the Evolution of Speech Activation of 40 neuron 35 30 25 20 15 10 5 0 -0.2 -0.15 -0.1 -0.05

0

0.05

0.1

0.15

0.2 vi-s

FIGURE 6.1. The default Gaussian activation function used in the simulations has σ 2 = 0.001. It is not very specific: it allows neurons to respond significantly to about 10 per cent of the space of possible inputs.

By contrast, the agents are not initially able to retrieve the articulatory targets which were used to construct the trajectory of relations between organs. I also assume that given a sequence of articulatory targets in organ relation space, agents are capable of finding a trajectory in the space of muscular activations which will move the organs in such a way that the targets are reached.5 This will allow us to use only the level of representation dealing with relations between organs, and will thus give a more intuitive visualization and understanding. If I had used the three representations, as schematized in Figure 6.2, I would have had three neural networks, each composed of neural units coding a space. There would be a perceptual neural network, composed of neurons receiving their inputs from the activation of the cochlea. These inputs could be formants, for example. Next, there would be neural units in a network of relations between organs, receiving their input from activations 5 Note that this means that agents have the knowhow to copy a sound that they hear, but it does not mean that they do it. In fact, as I will explain, agents never copy (never imitate) a sound that they have just heard. Thus they do not imitate each other, and we avoid the question ‘Why do they imitate each other?’ In the next chapter, I will describe how this type of translation, which I assume here, can be learnt by an agent.

The Artificial System Agent 1 produces a vocalization

79

Agent 2 produces a vocalization

Perceptual map

Perceptual map Cochlea model Sound Organ relation map

Organ relation map

Vocal tract model Muscle map

Muscle map

FIGURE 6.2. A general architecture of a full system involving the three representational spaces. In this chapter, I assume the capacity to pass from one representation to the other.

of neurons in the perceptual network. The third neural network would be composed of neural units coding for muscular activations, and receiving inputs from the activation of neurons in the network for relations between organs. After that, the activation of neurons in this last network would be used to control the speech organs, producing sounds. Technically, assuming that agents are capable of passing from one representation to another means that the connections between neural maps are such that when a sound activates the cochlea and the neurons in the perceptual network, then the neurons in the network of relations between organs which are the most highly activated have the preferred vector corresponding to the relation between organs which produced the sound. In addition, the neurons in the network of muscular activations which are most activated have the preferred vector corresponding to the muscular configuration which produced this relation between organs. Several papers have shown how these connections could be learnt during the course of infant babbling (Bailly et al., 1997; Oudeyer, 2002a). Because I am only using organ relation space, the neural units take inputs directly coded in terms of relations between organs. Producing a vocalization, programmed by a sequence of activations of several ni , as I explain below, is solely a matter of forming a continuous trajectory in organ relation space.

80

Self-Organization in the Evolution of Speech Agent 2 perceives this vocalization

Agent 1 produces a vocalization

Organ relation neural map

Inhibition/activation Signal GO

Units are sequentially activated and specify articulatory targets

The motor control system interpolates between these articulatory targets

Organ relation neural map Units are activated by the inputs provided by the temporal resolution filter

Perception

Trajectory in the space of organ relations

FIGURE 6.3. A simplified architecture. I use the representation level of relations between organs, because I assume that the agents are capable of passing from perceptual space to organ relation space, and from there to the space of muscular activations.

The agents only exchange trajectories. Figure 6.3 shows this simplification schematically. There is also a module called ‘inhibition/activation’ which can send a signal allowing a vocalization to be produced when the neural units are activated. We will call this signal the ‘GO signal’. This is to say that in the absence of this signal, the commands specified by the activation of the neural units are in effect not carried out. This means that the activation of some ni by an external sound does not directly provoke the reproduction of this sound. Copying requires the GO signal. In fact, the agents never use this GO signal when they have just heard a sound, but only at random instants, with the result that they do not copy what they have heard, and thus do not imitate each other. 6.1.3 Assumption 3: perception and plasticity In the last section, I explained that what an agent perceives in the vocalization of another agent is a continuous trajectory in organ relation space. Now I will explain what is done with this trajectory and how this changes the preferred vectors of neural units. At the outset, the agents are not able to detect high-level events in the continuous trajectory, which would allow them to find what points correspond


81

input space = organ relation space this input updates tuning functions of units at time t+1

preferred vectors of units in the network of one agent

t

this input updates tuning functions of units at time t

continuous trajectory perceived by one agent; it is segmented by the time resolution filter

time

FIGURE 6.4. The perception of dynamic vocalizations. Each agent obtains a continuous trajectory (in organ relation space) when it perceives the vocalization of another agent. It then uses a temporal resolution filter which segments this trajectory into a sequence of very short parts. For each of these parts, the average is calculated, and the result is a stimulus which is sent to the neural network, whose units are then activated. After reception of each stimulus and activation of the units, these are updated.

to the articulatory targets used by the agent who produced the vocalization. They segment the trajectory into lots of little slices, corresponding to the temporal resolution of perception (if we had used the three representations, this would correspond to the temporal resolution of the cochlea). Next, each little slice is averaged,6 giving a value in organ relation space, which is then sent to the neural units. These neural units are then activated according to the formula given in the ‘Neural Units’ section. Figure 6.4 shows this process schematically. The horizontal axis represents time, and the vertical axis organ relation space. Here organ relation space is one-dimensional. The continuous line represents a vocalization perceived by the agent. The little segments into which it is divided represent the averages of the little slices extracted by the temporal filter. These values are the inputs given sequentially to the neural units. The neural units of this agent are represented on the vertical axis: each point corresponds to a preferred vector. In effect, the values of these points are within organ relation space. Each of the neural units is activated by each of the averages. When they are activated, the neural units are modified. This means that their preferred vectors are changed. This change is a sensitization to the stimulus for those neurons which responded significantly. This implies that if the 6 If we are working in more than one dimension, each dimension is averaged.

82


same stimulus is given as input immediately afterwards, the response of the neural units will be a little bit stronger. Each such change is very slight, and weighted by the activations of each neural unit (a value between 0 and 1). Units which are very active change more than units which are not. Figure 6.5 illustrates the changing of preferred vectors. This figure represents the activation function of a neural unit before it receives any input, and after it has received and processed one: the preferred vector, i.e. the centre of the Gaussian, has been shifted. We see that from a geometric point of view the preferred vector is shifted in the direction of the stimulus. The mathematical formula of the new activation function is 1 1 2 2 e − 2 |vi,t+1 −s | /σ G i,t+1 (s ) = √ 2πσ

where s is the input, and vi,t+1 the preferred vector of neuron i after the processing of the stimulus s t perceived at time t: vi,t+1 = vi,t + 0.001.G i,t (s ).(s t − vi,t )

l (level of activation of unit ni)

tuning function of ni at time t tuning function of ni at time t+1

f preferred input vector of ni at time t

input at time t

(input space)

preferred input vector of ni at time t+1

FIGURE 6.5. The updating of each unit when activated by a stimulus is such that the preferred vector is changed so that the unit responds a little more if the same stimulus is presented again immediately afterwards. This is a sensitization of the units, which is stronger when the neurons are highly activated and weaker when they are less activated.


83

6.1.4 Assumption 4: production The production of a vocalization consists in choosing a sequence of articulatory targets and realizing them. These articulatory targets are specified by relations between organs, which are one-dimensional in the first simulations which will be carried out. To choose these articulatory targets, an agent sequentially and randomly activates units in its neural network, and at the same time sends the GO signal described above. This activation is a command implementing the concept of gesture in this work. A target is specified by the preferred vector of the activated neural unit. Then there is a control system which executes these commands by making the relations between organs move continuously and sequentially toward the target.7 If I had used the three representations, the control system would activate the muscles in such a way that the organs would be moved towards configurations satisfying the specifications of relations between organs. Here, the control system directly generates a continuous trajectory in organ relation space which passes through the targets. This is realized in this work by simple interpolation by splines, i.e. by polynomial interpolation (simple linear interpolations could also be used, which would not affect the running or the results of the simulations in any way). Figure 6.6 shows this process of production schematically. In this figure, the horizontal axis represents time and the vertical axis represents organ relation space. The preferred vectors of the neural units of an agent’s network are also represented on the vertical axis. Five of these units are activated sequentially, defining five articulatory targets. Then the control system (polynomial interpolation) generates a continuous trajectory passing all these points. This trajectory is the vocalization which will be perceived by the agents who hear it. The crucial point in this assumption is that the neural units ni are used both in the process of perception and in the process of production. Consequently, 7 Note that this way of producing vocalizations already contains elements of discreteness. I assume that vocalizations are specified by a sequence of targets. This is in fact in agreement with the literature on mammalian motor control (Kandel et al., 2000), which describes it as organized at two levels: a level of discrete commands (our articulatory targets), and a level concerned with their execution. Thus the element of discreteness seen in discrete commands should not be a trait which it is necessaery to explain in the context of research on the origins of speech, since it is already present in the motor control architecture of mammals. However, I do not assume that the articulatory targets are initially organized: the set of commands used to define the targets is a continuum, and there is no reuse of articulatory targets from one vocalization to another; discreteness and a form of primitive combinatoriality will emerge from the simulations. I also do not assume that there is initially any discreteness at the level of perception, in the sense that initially the agents are not capable of perceiving ‘events’ in the flux of sound. (However, at the end of a simulation it will be possible to identify the categories of articulatory targets used to produce vocalizations.)

84

Self-Organization in the Evolution of Speech target space = organ relation space preferred vectors of units in the network

target 1

target 3

target 5 continuous trajectory generated by the control system after specification of five targets

target 2

target 4 time

FIGURE 6.6. When an agent produces a vocalization, several articulatory targets are specified by random sequential activation of neural units. The preferred vectors of these neural units define the relations between organs to be reached at given times. These activations are commands called gestures in articulatory phonology. Next, a control system constructs a continuous trajectory which passes through all the articulatory targets.

the distribution of articulatory targets used in production is the same as that of the preferred vectors, which themselves change as a function of the vocalizations heard in the environment. So, when a vocalization is perceived by an agent, this increases the probability that the sounds that compose this vocalization will be reused by the agent in its future vocalizations. It is interesting to note that this phenomenon of phonological attunement is observed in very young babies (Vihman, 1996). Note that this does not come about by a process of imitation, but is in fact a side effect of the increase in sensitivity of the neurons, a quite generic mechanism of low-level neural dynamics (Kandel et al., 2000). 6.1.5 Assumption 5: initial distribution of preferred vectors The preferred vectors of the neural units are by default initially random in a uniform distribution, in the basic version of the system. A uniform distribution means that there are preferred vectors throughout the whole space at the same density everywhere. This means that at first the agents produce vocalizations composed of articulatory targets distributed uniformly through the space. This in turn implies that initially the whole continuum of possible


85

gestures is used (so there is no discreteness), and because there are many neurons portioned out in the whole space, the reuse of articulatory targets is very rare and due to chance (so there is no combinatoriality). Also, the overall activation of the neural map is of the same amplitude whatever the stimulus. This assumption will be modified in the ‘biased’ version of the system, in which the initial distribution of the preferred vectors will no longer be uniform. A biased distribution is not symmetrical: initially certain regions of organ relation space will contain more preferred vectors than other regions. Such a bias allows one to consider constraints due to the function mapping articulatory configurations to sounds. A uniform distribution comes down to assuming that this function is linear and symmetrical. A biased distribution takes into account the possible non-linearities. Looking at the human vocal tract, there are relations between organs for which a little change produces a little change in the sound, but there are also relations between organs for which a little change produces a large change in the sound. If I had used an architecture like that in Figure 6.2, in which the neural units of the network of relations between organs are activated by neurons of the perceptual network, this would mean that there are certain sounds which significantly activate neural units in the network of relations between organs whose preferred vectors are in an extended region of organ relation space, and other sounds which only activate the neurons whose preferred vectors are in a confined region of organ relation space. This makes the learning rules of the neurons have different outcomes in different parts of the space. In some regions of the space the preferred vectors will change faster than in others. This leads to a non-uniformity in the distribution of preferred vectors, with more neurons in the regions where small articulatory changes yield small changes in sound, and fewer neurons in regions where small articulatory changes give large changes in sound. In the next chapter, where I will implement a part of the architecture in Figure 6.2, this bias will be introduced by using a realistic articulatory synthesizer during the whole simulation. For the present, and to aid comprehension, I introduce this bias from the start by controlling and modifying directly the initial distributions of preferred vectors. Tweaking the initial distribution, in particular using a uniform distribution, allows us to see what outcomes are due or not due to the presence of nonlinearities in the function mapping from sounds to articulatory configurations. In particular, I will show that neither discreteness nor combinatoriality

86


requires non-linearities in order to be explained—a rather original conclusion given the existing theories in the literature (see Chapter 4). 6.1.6 Assumption 6: no coordinated interactions The agents are distributed randomly around their world. At random moments they produce a vocalization, which is heard by the nearby agents, and also by themselves. The choice of how many agents hear the vocalization produced by one of them does not affect the results that I will present: these are about the same whether one chooses one, two, three, or more agents. For the sake of generality, the simulations reported below only use one agent. From an algorithmic point of view, this is is equivalent to randomly choosing two agents from the population, and making one produce a vocalization which is heard and processed by both of them. 6.1.7 What is not assumed The agents are not playing a language game in the sense used in the literature (Cangelosi and Paris, 2002; Steels, 1997); in particular they do not play the imitation game which is used in de Boer (2001) and Oudeyer (2001b; 2002b). Their interactions are not structured; there are no roles or coordination. In fact they do not have any social dispositions at all. They do not distinguish between their own vocalizations and those of others. They do not communicate. Here, ‘communication’ denotes the emission of a signal by an individual with the intention of transferring some information which will modify the state of another agent, which is not what happens here. In effect, the agents do not even know that there are other agents around them, so it would be difficult to say that they communicate.

6.2 Dynamics 6.2.1 The case of uniform initial distribution I will now describe what happens to a population of agents which implement these assumptions. Organ relation space will here be one-dimensional, and the initial distribution will be uniform. In this version σ 2 = 0.001 and there are 150 neural units and 10 agents.

The Artificial System density

87

density

0.014

0.012

0.012

0.01

0.01

0.008

0.008 0.006 0.006

preferred vectors of Niunits

0.004

distribution of preferred vectors

0.004 0.002

0.002 0

0 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 1

1

organ relation space

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 2

1


FIGURE 6.7. Initial distribution of preferred vectors of two agents; organ relation space, on the horizontal axis, is one-dimensional here. The agents produce vocalizations specified by articulatory targets spread across the whole continuum (the vocalizations are holistic.)

Figure 6.7 illustrates the distribution of preferred vectors of two agents at the outset of the simulation. The horizontal axis represents organ relation space (e.g. place of articulation or lip-rounding), and the points shown inside it are the preferred vectors of the neural units of one agent. The vertical axis represents the density of preferred vectors; this allows us to see better how they are spread around, especially where many points are on top of each other. We see that they are distributed approximately uniformly. As the learning rule of the neural units makes the agents tend to produce the same distribution of sounds as they hear around them, and as initially all the agents produce roughly the same distribution of sounds, the initial situation is an equilibrium and is symmetrical. It is a situation in which each neural map is in an initial state analogous to the initial state of the ferromagnetic plates described in Chapter 3 (with the difference that here we have a population of neural maps that interact with each other). Because of the stochasticity in the mechanism, there will be fluctuations. Studying the evolution of the distributions shows that the initial equilibrium is unstable: some fluctuations are amplified and the state of the system changes. Figure 6.8 shows the distribution of preferred vectors of the same two agents 2,000 vocalizations later. It can be seen that now there are clusters, and these clusters are the same for both agents. The new distribution of their preferred vectors is multi-modal: symmetry has been broken. This means that the articulatory targets that they use to produce vocalizations are now taken from


88

density

density

0.04

0.04

clusters of preferred vectors

0.035 0.03 0.025

0.035 0.03 0.025

0.02

0.02

0.015

0.015

0.01

0.01

0.005

0.005

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 1

1


0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 2

1


FIGURE 6.8. The crystalized distribution of preferred vectors of neural maps of the same two agents, after 2000 vocalizations: they are multi-modal, which means that the articulatory targets used are taken from among a small number of clusters, and moreover these modes are the same in both agents (the speech code is shared and discrete). Due the the fact that there are few modes, they will automatically be systematically reused to construct vocalizations (so the code is combinatorial).

among one of the clusters or modes. The continuum of possible targets has been broken, and production of vocalizations is now discrete. In addition, the number of clusters appearing is small, which automatically results in the articulatory targets being systematically reused to produce vocalizations, which have become combinatorial. All the agents share the same speech code in the same simulation. By contrast, in two different simulations, the position and number of modes is different. This is true even when the same parameters are used in the simulation. It is due to the inherent stochasticity of the process. Figure 6.9 illustrates this diversity. Evolution stabilizes during the simulation. To show this, the degree of cluster formation and the similarity among the distributions of preferred vectors of agents were calculated at each time-step. This was done using the average entropy of the distributions and the Kullback–Leibler distance between two distributions (Duda et al., 2000).8 Figures 6.10 and 6.11 show the evolution 8 At first a model of the distributions of preferred vectors in each neural map is made. The ‘fuzzy binning’ technique (Duda et al., 2001) is used. This consists of approximating the distribution locally at a certain number of points spread out in the space to be modelled. Here, we take 100 points regularly spaced between 0 and 1. For each of these points v, an approximation of the local density of points is calculated with the formula ||v−vi || 150 1 1 − σ2 pv = e n 2π σ i =1

where the vi are the preferred vectors of the neurons of the neural map. σ is set so that the Gaussians have a width equivalent to 1/100. Once the distributions of the preferred vectors of the maps of all

The Artificial System 0.022

0.02

0.02

0.018

0.018

0.016

0.016

0.014

0.014

0.012

0.012

0.01

0.01

0.01

0.008

0.008

0.008

0.019 0.018 0.017 0.016 0.015 0.014 0.013 0.012 0.011 0.01 0.009 0.008

89

0.02 0.018 0.016

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.014 0.012

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.02 0.018 0.016 0.014 0.012 0.01 0.008 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.026 0.024 0.022 0.02 0.018 0.016 0.014 0.012 0.01 0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.019 0.018 0.017 0.016 0.015 0.014 0.013 0.012 0.011 0.01 0.009 0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.024 0.022 0.02 0.018 0.016 0.014 0.012 0.01 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

FIGURE 6.9. Examples of final crystallized distributions of preferred vectors in different simulations. All these results are obtained using the same parameters. The stochasticity of the system makes possible the generation of phoneme inventories of different sizes, and their spatial arrangements are also different.

agents have been modelled, the entropy of each one can be calculated (Duda et al., 2001). Entropy allows indirect evaluation of the degree of cluster formation, or organization, of the points in a distribution. Entropy is maximal for a completely uniform distribution, and minimal for a distribution of points or vectors which all have the same value (when there is a single point cluster). Entropy is defined by the formula entr opy = −

100

pv l n( pv )

i =1

Next, the average of all the entropies of all the distributions (one for each agent) is calculated, giving an evaluation of the average degree of cluster formation across the maps of all the agents. This is thus a measure of the degree of phonemic coding in the population of agents. To evaluate the degree of similarity between two distributions p and q of the preferred vectors of each agent, the Kullback– Leibler distance function is used, defined as follows: 1 pv qv di s tanc e( p, q ) = q v l og ( ) + pv l og 2 pv qv v

In this way all the pairwise distances between the distributions of all the agents are calculated, and the average is taken to determine at what point the agents have (or do not have) a shared organization of their space of commands.

90

Self-Organization in the Evolution of Speech 4.605 4.6 4.595 4.59 4.585 4.58 4.575 4.57 4.565

0

1000

2000

3000

4000

5000

6000

7000

FIGURE 6.10. To evaluate the temporal evolution of distributions of preferred vectors of neural units, their average entropy was calculated at each time-step; it can be seen that it decreases, corresponding to the formation of clusters or modes, and that it stabilizes, corresponding to a converged state (with several modes). 0.0035

0.003

0.0025

0.002 0.0015

0.001

0

1000

2000

3000

4000

5000

6000

7000

FIGURE 6.11. The average kl-distance between distributions of preferred vectors of the neural units of each agent; we see that it stays the same, meaning that the modes of all the agents are identical at the end of the simulation.

of these two measures in a simulation involving ten agents. On the one hand, entropy decreases, and then stabilizes, which shows crystallization, or cluster formation. On the other hand, the average distance between distributions of two agents does not increase (initially, they already have the same uniform


91

distribution), and even decreases, showing that the modes that emerge are the same for all agents. The reason why there is crystallization is that because of the natural stochasticity of the mechanism, from time to time certain sounds are produced more often than others by the population of agents (here again, ‘sound’ means a little slice of a vocalization). This creates deviations from the uniform distribution, which are sometimes amplified by the learning mechanism in a positive feedback loop. Then symmetry is broken. We see the same typical ingredients of self-organization phenomena that I described in Chapter 3. To be exact, however, what I am here calling the state of crystallization in which several modes emerge is not yet the equilibrium state of the system. In fact, if one let the simulation run for an an extremely long time, in all cases, whatever the parameters, the outcome would be a single cluster, a single mode. This is because the adaptation rule for preferred vectors is the only driving pressure, at each time-step pushing the preferred vectors toward the vector corresponding to the stimulus. As the stimuli are generated from these distributions of preferred vectors, they are always inside the zone defined by all the preferred vectors of all the agents in the community. And thus globally at each time-step this zone shrinks (even if locally the neurons may distance themselves from each other because of the non-linearity of the adaptive rule). But at the moment when different clusters are formed as I have just shown, the stimuli become concentrated right in the zones defined by the clusters. Imagine that there are two clusters, C1 and C2. That means that stimuli are statistically very concentrated around the centres of C1 and C2. Now take a stimulus corresponding to the centre of C1. It will simultaneously make the preferred vectors of C1 and C2 move closer to the vector defining it. For those of C1, this will result in making them move even closer to the centre of C1, and they will finish by hardly moving at all once they are there. For those of C2, there will be very little effect: the attractive force due to the Gaussian function is extremely weak at average and longer distances. For example, if one assumes σ 2 = 0.001, the default parameter of the simulations, then if C1 and C2 are separated by a distance of 0.2, the displacement of the vectors of C2 toward C1 with each perception of a stimulus from C1 is 5.36∗10−20 . This means that it would need of the order of 1018 time-steps in the simulation to see the two clusters come completely together, which would take a simulation time on the computer longer than the age of the universe, and thus much greater than the lifetimes of my agents or indeed of any organism. This is why I am justified in calling the state in which several modes like C1 and C2 appear ‘a state of convergence’. Thus

92


clusters are formed which can quickly merge with each other, and as soon as all the remaining clusters are far enough apart (about 0.1 when σ 2 = 0.001), the stage is reached when no more mergers happen for a very long time (this in fact never happens because the agents do not live long enough). This phenomenon of self-organized pattern formation during the passage of a dynamic system toward an equilibrium state is analogous to those discovered by Prigogine (Nicolis and Prigogine, 1977) in dissipative systems. Moreover, it can be noted that this kind of phenomenon was not studied or even conceptualized until the end of the twentieth century because the mathematical tools used by physicists only enabled them to calculate equilibrium states. It was the use of computer simulations that made it possible to observe the behaviour of dynamic systems before they reached their equilibrium states, and to discover that highly organized structures could be formed. In the same way, the results presented in this work necessitate computer simulations because to my knowledge there is no way of predicting them with existing pure mathematical techniques. Finally, if there is only one agent in a simulation, and it is allowed to produce and hear its own vocalizations, then its network of neural units will also selforganize into a state where several modes exist. This means that there are two separable results: discreteness and combinatoriality are explained by the coupling between production and perception with the ni , and can be obtained with just one agent; but when several agents are put together, then their repertoires of clusters, i.e. of commands and thus of gestures, converge (while if each developed its repertoire in its own corner, this would be particular to each agent).9 Studying variation in the σ 2 parameter The artificial system has a certain number of parameters: σ 2 , the number of agents, the number of neurons, and how many agents hear a vocalization. Only σ 2 has a crucial influence on the dynamics. In fact, for example, the number of neurons changes nothing at all in the results when it is large enough, i.e. when it allows a sufficiently dense initial coverage of the space. Experimentally, the number just has to be greater than 100 neurons to obtain the results we have presented. The influence of the number of agents was also tested by doing 9 If one agent with a uniform neural map is put into a population of agents which have already formed a speech code, this agent will learn that code. This means that the mechanism for learning a speech code is the same as that which enables a population to form one starting from nothing.


93

simulations with between 1 and 50 agents: convergence is obtained every time at the end of a number of interactions by each agent rising very little (between 150 and 500). The number of agents hearing vocalizations pronounced by one of them also has very little influence. I will thus devote this section to showing the influence of σ 2 . I varied this parameter in a range of values from 0.000001 to 0.1 (using the values 0.1, 0.05, 0.01, 0.005, 0.001, etc., 0.000001). Figure 6.12 shows some of the Gaussian functions associated with these parameters. It can be seen that all the relevant space is covered, i.e. that the possibilities go from the Gaussian with width equal to the width of the whole space down to the Gaussian with tiny width. For each of these parameter values, 10 simulations with 10 agents were run, and the average entropy of the distributions of agents was measured once convergence was reached (in the sense explained in the previous section). Entropy is a way of measuring the number of emerging modes (or the nonemergence of modes, when entropy is maximal). Figures 6.13 and 6.14 give the results. To represent σ 2 , which varies over several powers of 10, a logarithmic scale (base 10) was used. For example, σ 2 = 0.001 = 10−3 is represented by point 3 on the x axis in Figure 6.13. Three parts are distinguished, which could be called phases, as in the case of the ferromagnetic plaques discussed in chapter 3. The first phase involves

35

1/sqrt(2*3.14*0.05)*exp(-e*e/0.1) 1/sqrt(2*3.14*0.001)*exp(-e*e/0.001) 1/sqrt(2*3.14*0.001)*exp(-e*e/0.00003)

30 25 20 15 10 5 0

-0.4

-0.2 0 0.2 sigma^2 (in powers of 10)

0.4

FIGURE 6.12. Some Gaussian functions corresponding to the different values of σ 2 used in this work.

94

Self-Organization in the Evolution of Speech 4.62 4.6

entropy

4.58 4.56 4.54 4.52 4.5 4.48

0

1

2 3 4 5 sigma^2 (in powers of 10)

6

7

FIGURE 6.13. Variation in the average entropy of the distributions of preferred vectors produced by simulations using different values of σ 2 . To represent σ 2 a scale of powers of 10 is used: for example, s ig ma 2 = 0.001 = 10−3 is represented by point 3 on the x axis. Three phases can be seen: for low values, only one mode is formed; for high values, the distribution stays random and uniform; for intermediate values, several well-defined modes emerge.

all the values of σ 2 greater than 0.05: a single cluster forms. This is the maximal amount of order that can be obtained, corresponding to minimal entropy, and the greatest breaking of symmetry. At the other extremity of the space of values, when σ 2 is less than 10−5 , entropy is maximal: the initial symmetry corresponding to a random uniform distribution of preferred vectors is not broken. This state of maximal symmetry corresponds to a state of disorder. Figure 6.14 (bottom panel) gives an example of this. Between these two regions of possible values, there is a third phase corresponding to the formation of several well-defined clusters (between two and a dozen, beyond which there are no proper clusters and the agents are not coordinated). It is in this region that the σ 2 = 0.001 default parameter value of the preceding sections is found. This organization of preferred vectors into well-defined structures is a complex structure appearing at the boundary region between ‘order and chaos’, as it is often simply labelled in the literature (Kauffman, 1996). The transitions between these three phases are analogous to those that I described for Bénard cells or ferromagnetic plaques in Chapter 3. In addition, it should be noted that the region of parameter values in which repertoires shared between all the agents are formed is very large: between 0.00001 and 0.01, i.e. a

The Artificial System 0.06 0.055 0.05 0.045 0.04 0.035 0.03 0.025 0.02 0.015 0.01 0.005

sigma^2 = 10^-1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.035 0.03 0.025 0.02

sigma^2=10^-2

0.015 0.01 0.005 0.03 0.028 0.026 0.024 0.022 0.02 0.018 0.016 0.014 0.012 0.01 0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

sigma^2 = 10^-2.3 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.022 0.02 0.018 0.016

sigma^2 = 10^-3

0.014 0.012 0.01 0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.02 0.018 0.016

sigma^2=10^-3.3

0.014 0.012 0.01 0.008

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0.02 0.018 0.016 0.014

sigma^2 = 10^-4

0.012 0.01 0.008 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 0.0135 0.013 0.0125 0.012 0.0115 0.011 0.0105 0.01 0.0095 0.009 0.0085

sigma^2 = 10^-5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

FIGURE 6.14. Examples of systems generated for different values of σ 2 .

95

96


space covering several powers of 10. It can thus be said that the behaviour of the artificial system is very robust in the face of changes in parameter values.

6.2.2 The case where the initial distribution is non-uniform In the preceding section, I assumed that the initial distribution of preferred vectors was roughly uniform. This meant that the function mapping a sound to articulatory configurations was linear, and thus took no account of constraints due to the physical properties of the vocal tract. This was interesting, because it enabled us to show that no initial asymmetry is necessary to obtain discreteness (which is a property of asymmetry). In other words, this shows that there is no need to have discontinuities or non-linearities in the function mapping sounds to articulatory configurations to explain phonemic coding (this does not mean that the non-linearities do not help, just that they are not necessary). However, this function has a particular form in humans, which introduces a bias in speech sounds. I explained earlier that this bias could be modelled for the moment by manipulating the initial distribution of preferred vectors. I will work in this chapter with an abstract bias, which does not realistically reproduce the non-linearities of the human vocal tract, but whose generic nature will enable us to understand the consequences of the presence of nonlinearities. I am still dealing here with the case of a one-dimensional organ relation space. The initial density of preferred vectors increases linearly between 0 and 1 (it was constant in the case of a uniform distribution). Figure 6.15 shows the initial distributions of preferred vectors of two agents. It can be seen that there are fewer neurons with preferred vectors close to 0 than neurons with preferred vectors close to 1. This naturally leads to a statistical preference for modes or clusters located in the second part of the space in relation to clusters situated in the first part of the space. Figure 6.16 shows the same two agents 2,000 vocalizations later. The preference for modes in the second part of the space is, however, only statistical: it is possible for groups of agents to develop a system with just as many modes in the first part of the space. Figure 6.17 gives some examples of the diversity of systems obtained. This phenomenon is crucial for understanding both the presence of statistical structural regularities in the phoneme inventories of human languages and at the same time their great diversity. I will make a more detailed study of this aspect in the following chapter, in which we will use realistic constraints which


0.016

97

density

density

0.014

0.014

0.012

0.012

0.01

0.01

0.008

0.008 0.006

0.006

0.004

0.004

0.002

0.002 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

0

1

0


agent 1

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 2

1


FIGURE 6.15. Initial biased distribution of preferred vectors of neural units of two agents. This models constraints due to the non-linearity of the function mapping sounds to articulatory configurations. It can be seen here that there are more preferred vectors in the second part of the space. 0.04

density

density

0.04

0.035

0.035

0.03

0.03

0.025

0.025

0.02

0.02

0.015

0.015

0.01

0.01

0.005

0.005

0 -0.1 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 1

1

0 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1


agent 2


FIGURE 6.16. Distribution of the preferred vectors of the same two agents after 2,000 vocalizations. There is a preference for clusters, and therefore for articulatory targets, in the second part of the space.

will enable us to compare the results from the artificial system with the sounds of human languages.

6.3 Categorization and acoustic illusions In the work so far, the agents have had no mechanism for categorizing the articulatory targets that they use. That is, they were not able to collect similar


98 0.035

0.03

0.03

0.025

0.025

0.02

0.02 0.015 0.015 0.01

0.01

0.005

0.005 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0.04

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0.025

0.035 0.02

0.03 0.025

0.015

0.02 0.01

0.015 0.01

0.005

0.005 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

0.035

0.03

0.03

0.025

0.025

0.02

0.02 0.015 0.015 0.01

0.01

0.005

0.005 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

FIGURE 6.17. Some examples of systems obtained after 2,000 vocalizations. The preference for clusters located in the second half of the space is only statistical: it sometimes happens that the first part of the space contains the majority of clusters.

targets in the same bag. They had no way of seeing that when they activated two neurons from the same cluster (once clusters were formed), they were actually using this same cluster, and so the same phoneme. Thus in a certain way discreteness and combinatoriality were in the eye of the beholder,


99

but the agents had no knowledge of these properties. In the same way, when they perceived a vocalization and passed it through the temporal filter to decompose it into little slices thenceforth approximated by their averages, they had no way of organizing these sounds into different categories. This could have enabled them to retrieve the articulatory targets used to produce the vocalization. In this part of the work I will therefore extend the neural mechanisms so that the agents are capable of categorization. The way in which I define this capacity is as follows: the neural maps that compose the agent’s brain are considered as dynamic systems, and categorization comes down to getting the system into a stable state in which the activations of all the neural units remain fixed. Two stimuli will be categorized as the same if the state into which they move the system is the same, and different if this state is different. In the preceding sections, in a certain way, the agents had neural maps which directly reached a stable state after the perception of a stimulus: one input activated all the neurons, and this activation did not switch until a new input was provided. But in this way, two stimuli close to a cluster of preferred vectors, but slightly different, led to overall activity of the map which was similar, but not identical. So we need to add a mechanism which will make the neural units in such a case be led to exactly the same pattern of activations. For this, I will locate the neural map in its general context. In humans, this kind of neural map has often been used to model cortical maps which are, as their name indicates, devices that make models of their environment (there are auditory, visual, tactile maps, and so on), and whose information is possibly used by other parts of the brain. These other parts of the brain use the stored information thanks to a form of decoding which ideally recalculates the input stimulus which led to the current set of neural activations. A discovery in neuroscience, due to Georgopoulos et al. (1988), enables us to set up a model of the way in which this decoding is carried out. The method uses the concept of ‘population vector’. It involves the sum of all the preferred vectors of the units of the neural map (weighted by their activation) and normalized by the sum of all activations. If s is the stimulus, and N the number of neural units N

pop(s , t) = i =1 N

G i,t (s )vi

i =0

G i,t (s )

where pop(s , t) is the population vector which recalculates s starting from the set of activations G i,t of neural units and of their preferred vectors vi .

100


When there are many neural units and their preferred vectors are uniformly distributed through the space, then this method recalculates fairly precisely the input stimulus. However, if the distribution of preferred vectors is not uniform, some inexactness appears. Some researchers think that this inexactness is a defect of Georgopoulos’ model, and have tried to put more precise formulas in place (Salinas and Abbot, 1994). But I think, on the contrary, that this inexactness allows us to account for psychological phenomena like acoustic illusions. We will show that the perceptual magnet effect, described in Chapter 2, can be explained by this inexactness. Then we will show how a simple feedback loop between the neural map and the decoding system gives us a categorization mechanism. Recall briefly what the acoustic illusion called the ‘perceptual magnet effect’ consists of. Our perception of sounds is biased by our knowledge of the sounds of our own language. Kuhl et al. (1992) showed that when people have to judge the similarity of pairs of vowels, they tend to perceive vowels as closer than they really are in an objective physical space when they are both close to the same prototype vowel in their language, and to perceive vowels as further away from each other than they really are in physical space when they are near to different prototypes. In brief, there is a sort of perceptual deformation that ‘attracts’ sounds around each prototype in a language (from the point of view of a person’s sensation of it). As a side effect, the differences between vowels of different categories grow. This is an instance of a wider psychological phenomenon called ‘categorical perception’ and is defined thus: ‘Categorical perception occurs when the continuous, variable and confusable stimulation that reaches the eyes and ears is sorted out by the mind into discrete, distinct categories whose members somehow come to resemble one another more than they resemble members of other categories’ (Harnad, 1987). Evidently, as these illusions depend, in the case of sounds, on the sound prototypes in a given language, they are cultural phenomena. Now the process of decoding with the population vector is just right for modelling the phenomenon of a hearer’s sensation of a sound. It has moreover already been used by Guenther and Gjaja (1996) to account for the perceptual magnet effect. In this model, the authors use a neural map similar to the one used here, with the difference that they use a scalar product rather than a Gaussian as the activation function of the neural units. The adaptation rule for preferred vectors of neurons is also similar to ours. But a large difference with the work presented here is that they get an agent to learn a sound system which already exists (in this case a vowel system). They do not locate


101

themselves in the perspective of research on the origins of speech, and do not ask the question where the sound system they assume comes from. To illustrate this process, I will redo some simulations similar to those presented earlier, but using a two-dimensional organ relation space, which will enable us to visualize the perceptual magnet effect with the population vector decoding system. Figure 6.18 shows an example of the distributions of preferred vectors of the neural maps of two agents at the beginning of a simulation. After 2,000 vocalizations, the same two agents are found to have neural maps like those presented in the upper panel of Figure 6.20. Clusters shared by the agents are formed, but they are not very visible in the representation of preferred vectors because on the one hand most of the preferred vectors in the same cluster are represented by the same point (they have quasi-identical values), and on the other hand there are still a few isolated neurons themselves also represented by a single point. It is the lower panels, with arrows, that will let us see the distribution of preferred vectors that will be explained in the following paragraphs. I will now evaluate the way in which these two agents ‘sense’ sounds at the beginning and end of the simulation. To do this, I generate some artifical static sounds which serve as input stimuli. These stimuli are spread regularly throughout the space, according to a regular grid. For each of these stimuli, the activations of all the neural units are calculated, then the population vector is calculated, which gives a point in organ relation space, the result of the decoding. The set of these stimuli and their decodings from activations of the neural maps is represented by arrows: each arrow corresponds at its start to a stimulus

Neural map of agent 2 initially

Neural map of agent 1 initially 1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

FIGURE 6.18. The preferred vectors of two agents at the beginning of the simulation when the abstract organ relation space is two-dimensional.

102


Basins of attraction of agent 1's neural map initially 1

Basins of attraction of agent 2's neural map initially 1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

FIGURE 6.19. Perceptual warping representation of the way the agents sense sounds at the beginning of the simulation. The start of each arrow corresponds to a stimulus activating the neural map, and the ends of these arrows correspond to the reconstructions of the stimuli after decoding of the activations of the neurons by the population vector.

and at its head to the point decoded by the population vector. Figure 6.19 thus represents the way in which the agents ‘sense’ sounds at the beginning of a simulation. It can be seen that the decoded points correspond fairly well to the stimuli (despite some tiny inexactitudes). The lower panel of Figure 6.20 represents the same two agents’ way of sensing sounds 2,000 vocalizations later. It can be seen that now the decoding is no longer at all precise. On the contrary, the decoded points are shifted towards the centre of the closest cluster: there is a perceptual warping. Now this corresponds exactly to the perceptual magnet effect. In effect, the centres of clusters correspond to prototypes in their sound systems, and so the agents behave in the same way as Kuhl et al.’s (1992) subjects. In addition, if Figure 6.20 is seen as a surface viewed from above, whose slopes are represented by arrows (the arrows pointing downwards), it can be seen that a landscape with valleys appears. If a marble is dropped in one of these valleys, it will roll to the bottom and stop at the same place, from whatever position it is dropped. If it is dropped in another valley, again it will roll downwards, but stop at a different valley bottom. This way of seeing Figure 6.20 is in fact the basis of an extension of the system enabling it to categorize sounds, in the sense set out above. Indeed, as the point decoded by the population vector is specified in the same representational space as the input point, one can easily redeploy this vector to the input layer of the neural map as a stimulus, but generated by

The Artificial System 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

Neural map of agent 1 after 2000 vocalizations

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

Neural map of agent 1 after 2000 vocalizations

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

Corresponding basins of attraction of agent 2's neural map

Corresponding basins of attraction of agent 1's neural map 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

103

1

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

FIGURE 6.20. Example of the state of the neural maps of two agents after 2,000 vocalizations. The upper panels directly represent the preferred vectors. The lower panels represent the way in which the agents sense sounds at this time. It can be seen that there is an organization of the space into regions in which stimuli are perceived as closer to the centres of these regions than they objectively are. These regions are shared by both agents. They correspond to basins of attraction defined by categorization behaviour once a coding/decoding feedback loop is introduced into the neural map.

the agent itself, as Figure 6.21 shows. This idea gets close to the re-entrance systems described by Edelman (1993) in his theory of human brain functioning. Indeed, he describes the human brain not as a device in which information flows in one direction from the sensor to the control centres, but rather as a system in which the control system itself sends a lot of information toward the sensors, creating feedback loops. In the system, once the decoded point is redeployed as an input, it reactivates all the neural units, and a new decoding is effected. Then the process is iterated. Geometrically, the sequence of positions of these successive points follows exactly the trajectory of the marble rolling down the valley: after several iterations the decoded point is the same as the point given as input to the neural

104

Self-Organization in the Evolution of Speech Acoustic stimulus

Neural map activation (coding)

Population vector calculation (decoding)

FIGURE 6.21. The categorization mechanism. After the stimulus has activated the neural map, the population vector is used to reconstruct this stimulus and the result is fed back into the neural map. This process is repeated until the neural map activation is stabilized. The attractor corresponds to the recognized category of the stimulus.

map. The activations of the neural units stay the same with each iteration once this fixed point is reached. The system is then in a stable state, corresponding to a categorization of the stimulus given initially as input. To follow the trajectory of these successive points, simply start at the stimulus point and follow the arrows, for example in Figure 6.20. Each neural map defines a landscape of arrows, and thus of valleys and valley bottoms (in the language of dynamic systems these are called basins of attraction and attractors) which are peculiar to it. When the preferred vectors are distributed uniformly across the whole space, a certain number of valleys appear whose locations and shapes differ from one simulation to another. Figure 6.19 gives some examples. When clusters appear, then those which are big enough each define a valley and an attractor. Figure 6.22 gives some examples. Thanks to the smoothing property of the Gaussian activation function for neural units, the fact that there are neural units that do not belong to clusters (this results from the stochasticity of the mechanism), and which thus introduce a kind of ‘noise’ into the landscape of neural units, does not modify the global landscape of the valleys of a set of clusters. Thus, for example, the agents in Figure 6.20 do not have exactly the same neural maps, but share the same landscape of valleys, because they have the same clusters. They thus categorize sounds in the same way. They are now capable themselves of measuring the discreteness of their speech system. Each has a way of telling when two

The Artificial System 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

105

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

FIGURE 6.22. Examples of the systems of basins of attraction that can be obtained. It can be seen that their number, as well as their shapes and their locations, are varied.

sounds are different and when they are the same. They are now capable of segmenting the continuous trajectory of a vocalization into parts of which each little slice, the output of the temporal filter, is an input categorized in the same way.

7. Learning Perceptuo-motor Correspondences

In the previous chapter, I assumed that the agents were able to translate from the space of perceptual representations to the space of gestural and muscular representations. This allowed us to present a simple mechanism accounting for the self-organizational dynamics forming a discrete combinatorial speech code shared by a population of agents. I suggested, however, at the beginning of this work that the architectural components of the agents would be generic and not specific to speech. It remains to show how the capacity to pass from one space to another can be realized by neural structures that are neither prewired nor specific to speech. Indeed, while certain researchers propose that this capacity is innate (Mehler et al., 2000), I will show how it can be learnt. To do this, I will modify my previous assumptions. While I assumed earlier that agents were capable, given a sound stimulus, of retrieving the corresponding articulatory configuration, I will no longer assume this here. I also explained that there were two functions to learn: one function from perceptual space to organ relation space, and another mapping from relations between organs to the space of muscular activations. For the sake of simplicity, and because an effective and precise model of the function mapping relations between organs to muscular activations is not available, I will concentrate on learning the function from sounds to relations between organs. This means that I will have to work with two representations, a perceptual representation and a representation of the relations between organs. The agents have no longer one but two neural maps, each coding a different space (see Figure 7.1). The perceptual map is composed of neurons with properties identical to those described earlier, but these take as input values provided by a model of the ear. The motor network is composed of neurons taking as input activation values from the perceptual map. In addition, these motor neurons have output connections destined to send commands to the vocal tract’s control system. They have preferred vectors corresponding to these output signals: their value represents the relation between organs to be reached when they are activated at the same time as a GO signal is sent.

Learning Perceptuo-motor Correspondences Agent 1 produces a vocalization

107

Agent 2 hears a vocalization

Acoustic measures Perceptual neurons

Perceptual neurons Cochlea model sound

Hebbian connections Vocal tract model Motor neurons Motor commands

Motor neurons [i]z

[i]z

FIGURE 7.1. Agents’ architecture when learning the perceptuo-motor correspondences.

The two maps are fully connected; all the perceptual neurons are connected to all the motor neurons through connections which propagate activations from the perceptual map to the motor map. Each connection has a weight which initially has a random value close to zero. The preferred vectors of the perceptual neurons, representing the value that maximally activates each neuron, are also initially random, following a uniform distribution. The preferred vectors of the motor neurons, representing the relation between organs to be reached when they are activated, are also initially random and uniformly distributed. To produce a vocalization, the mechanism is the same as before: neurons in the motor network are sequentially and randomly activated. The activation of a neuron fixes an articulatory target, which is a relation between organs to be reached. Then a control system operates to reach each articulatory target in sequence, thereby generating a continuous articulatory trajectory in organ relation space (this is still a polynomial interpolation). In contrast with the previous chapter, I here use operational models of the vocal tract and of the cochlea, which are fixed and the same for all agents in a given simulation, and which map an acoustic representation to each configuration of the vocal tract. This acoustic representation will here be in terms of formants, as detailed below. The agents in this version exchange acoustic trajectories, no longer directly exchanging trajectories in organ relation space, as in the earlier version.

108


The perception of a sound is also similar to the earlier version in algorithmic terms. The perceived acoustic trajectory is transformed into a perceptual trajectory by a model of the cochlea. This perceptual trajectory will here be composed in two dimensions: the first formant and the effective second formant, as explained below. This perceptual trajectory is then passed to the temporal filter, which cuts it into little slices, corresponding to the temporal resolution of the cochlea. Each little slice is averaged, giving a point value which serves as a stimulus to the agent’s nervous system. This stimulus first activates perceptual neurons. Their preferred vector is then modified just as in the earlier version: they are modified in such a way that they are activated a little more if the same stimulus is presented again immediately afterwards, and this change is larger for more activated neurons. The mathematical formula for the new activation function is 1 1 2 2 e − 2 |vi,t+1 −s | /σ G i,t+1 (s ) = √ 2πσ

where s is the input, and vi,t+1 the preferred vector of neuron i after the processing of s : vi,t+1 = vi,t + 0.001.G i,t (s ).(s − vi,t ) After the perceptual neurons have been activated, two cases arise: • The agent hears the vocalization produced by another agent: then the activity of perceptual neurons is propagated to the motor neurons. Each neuron i in the perceptual map is connected unidirectionally to all the neurons j in the motor map. The connection between the perceptual neuron i and the motor neuron j is characterized by a weight wi, j , which is used to compute the activation of neuron j when a stimulus s has been presented to the perceptual map, with the formula N 1 2 G j,t (s ) = √ ∗ e − i =1 wi, j G i,t (s )/σ 2π σ where N is the number of perceptual neurons. • The agent hears its own vocalization: the motor neurons are already activated, because the vocalization was produced by the agent itself (this happens about 50 per cent of the time). There is no propagation of activation in this case, but a learning rule is applied to the weights wi, j connecting the perceptual and the motor neurons. These connection weights

Learning Perceptuo-motor Correspondences

109

are reinforced for connections between neurons whose activations are correlated, and weakened for connections between neurons whose activations are decorrelated. The weights wi, j are initially set to a small random value, and evolve so as to represent the correlation of activity between neurons. This is how agents will learn the perceptual/articulatory mapping. The learning rule is Hebbian (Sejnowsky, 1977): δwi, j = c 2 (G i,t − < G i >)(G j,t − < G j >) where G i,t denotes the activation of neuron i at time t and < G i > the mean activation of neuron i over a certain time interval (correlation rule). c 2 denotes a small constant. This learning rule allows to learn the perceptual/motor mapping through vocal babbling. The preferred vector of each neuron in the motor map is updated each time the motor neurons are activated (which happens both when the agent produces a vocalization and when it hears a vocalization produced by another agent, as I will explain below). This update is made in two steps: (1) one computes which neuron m is most activated and takes the value vm of its preferred vector; (2) the preferred vectors of all neurons are modified with the formula v j,t+1 = v j,t + 0.001.G j,t (s ).(v − v j,t ) where G j,t (s ) is the activation of neuron j at time t with the stimulus s (as I will detail later on) and v j,t denotes the value of v j at time t. This law of adaptation of the preferred vectors has the consequence that the more a particular neuron is activated, the more the agent will produce articulations which are similar to the one coded by this neuron. This is because geometrically, when vm is the preferred vector of the most active neuron, the preferred vectors of the neurons which are also highly activated are shifted a little bit towards vm . The initial value of all the preferred vectors of the motor neurons is random and uniformly distributed. In this chapter the motor neural map contains 500 neurons (above a certain number of neurons, which is about 150 in all the cases presented in this book, nothing changes if this number varies). When all the activations have been propagated, and both the preferred vectors and the weights modified, then the relaxing process for each of the neural networks begins. This process is the same as in the previous chapter, applied in parallel to the two networks. Starting from an initial activation pattern of each neural network in response to the sound stimulus, the population vector

110


is calculated, then redeployed as an input, producing a new activation pattern. This is iterated until the activation pattern stabilizes. This corresponds to categorization behaviour. In this new architecture, the crucial feature of the coupling of production and perception is retained: the distribution of preferred vectors of perceptual neurons evolves in parallel with the distribution of preferred vectors of motor neurons. If at a given moment all the sounds associated with articulatory configurations coded by the preferred vectors of motor neurons are produced, and these sounds are transformed by the bias of the model of the vocal tract and of the ear into a perceptual representation, then the distribution of points obtained is roughly the same as the distribution of preferred vectors in the perceptual neural map. Conversely, if the distribution of sounds that an agent hears changes, this will affect the distribution of preferred vectors both in their perceptual map and in the motor map. This coupling between the motor map and the perceptual map again has an important dynamic consequence: the agents will tend to produce more vocalizations composed of sounds that they have already heard. Put another way, when a vocalization is perceived by an agent, this increases the probability that the sounds that compose this vocalization will be reused by the agent in future vocalizations. Initially, as in the earlier version, the preferred vectors are all random and uniformly distributed. This means that the vocalizations produced are specified by articulatory targets spread uniformly throughout the continuous space of possible targets. The space is thus not yet discretized; there is no phonemic coding. In addition, as in the earlier version, the initial situation is in an equilibrium state, as all the agents produce sounds composed of targets following the same distribution and adapt themselves to approximate the distribution of sounds that they hear. We will see, however, that here again random fluctuations will break this symmetry and push the agents away from this equilibrium toward another equilibrium state (which will be stable). Using two neural maps not only illustrates how the articulatory function can be learnt but also, and above all, accounts more realistically than the preceding version for the constraints due to the non-linearities of this articulatory function. Indeed, in the earlier version I only used an abstract articulatory synthesizer to generate the initial distribution of preferred vectors. Here I will use an articulatory synthesizer throughout the whole simulation, and the bias will be directly applied by the synthesizer to the neural maps. Recall at first that there are articulatory configurations for which small changes produce


111

small changes in the sound produced, as well as articulatory configurations for which small changes produce large acoustic changes. When the neurons in the motor network have random uniformly distributed preferred vectors, this distribution will be quickly biased: the non-linearities cause the adaptation of neurons to happen in a heterogeneous way. For certain stimuli, many neurons will have their preferred vectors substantially modified, while for other stimuli, only a few neurons will have their preferred vectors modified substantially. This leads rapidly to non-uniformities in the distribution of the preferred vectors of the motor neurons, with more neurons in the regions where small articulatory changes produce small acoustic changes than in the regions where small articulatory changes produce large acoustic changes. Consequently, the distribution of the articulatory targets of vocalizations also becomes biased, and learning by the neurons in the perceptual network results in the preferred vectors of these neurons also being biased.

7.1 The articulatory synthesizer and a model of vowel perception In this version of the system I use a realistic model of part of the function mapping sounds to relations between organs, and of the function mapping percepts to sounds. This model corresponds to the subsystem of the human vocal tract that enables us to produce vowels. It was developed by de Boer (2001). The model uses a three-dimensional articulatory space, each dimension representing a relation between organs: tongue position (place of articulation), tongue height (manner), and lip-rounding. The position of each articulator has values between 0 and 1, and a triplet of values ar i = (r, h, p) defines an articulatory configuration. From each point in this space, following de Boer’s model, the first four formants can be calculated (these are the frequencies of peaks in the energy spectrum, or poles in the function transforming articulatory configurations into acoustic waves). This calculation was modelled by a polynomial function generated by interpolation between points of a database (Vallée, 1994) representing vowels with their articulatory configurations and the associated formants. The formula is as follows: F 1 = ((−392 + 392r )h 2 + (596 − 668r )h + (−146 + 166r )) p 2 +((348 − 348r )h 2 + (−494 + 606r )h + (141 − 175r )) p

+((340 − 72r )h 2 + (−796 + 108r )h + (708 − 38r ))

112


F 2 = ((−1200 + 1208r )h 2 + (1320 − 1328r )h + (118 − 158r )) p 2

+((1864 − 1488r )h 2 + (−2644 + 1510r )h + (−561 + 221r )) p

+((−670 + 490r )h 2 + (1355 − 697r )h + (1517 − 117r )) F 3 = ((604 − 604r )h 2 + (1038 − 1178r )h + (246 + 566r )) p 2

+((−1150 + 1262r )h 2 + (−1443 + 1313r )h + (−317 − 483r )) p

+((1130 − 836r )h 2 + (−315 + 44r )h + (2427 − 127r ))

F 4 = ((−1120 + 16r )h 2 + (1696 − 180r )h + (500 + 522r )) p 2

+((−140 + 240r )h 2 + (−578 + 214r )h + (−692 − 419r )) p

+((1480 − 602r )h 2 + (−1220 + 289r )h + (3678 − 178r ))

Next, I use a model of the cochlea summarizing the information that it sends to the brain when vowel-like sounds are heard. As in the previous chapter, vocalizations are still complex and dynamic. I use again the temporal resolution filter which splits the continuous acoustic four-dimensional (F 1 , ..., F 4 ) trajectories into a sequence of points; these are then fed into the cochlea model, which then computes a perceptual representation. This model, used by Boë et al. (1995) and de Boer (2001), calculates a two-dimensional representation from the first four formants. The first dimension is the first formant, and the second dimension is called the effective second formant and is a non-linear combination of formants F 2 , F 3 , and F 4 . The formula is  F 2 , if F 3 − F 2 > c      (2 − w1 )F 2 + w1 F 3   , if F 3 − F 2 ≤ c and F 4 − F 2 ≥ c    2 ′ F 2 = w2 F 2 + (2 − w2 )F 3   − 1, if F 4 − F 2 ≤ c and F 3 − F 2 ≤ F 4 − F 3   2      (2 + w2 )F 3 − w2 F 4  − 1, if F 4 − F 2 ≤ c and F 3 − F 2 ≥ F 4 − F 3 2 with c − (F 3 − F 2) w1 = c (F 4 − F 3 ) − (F 3 − F 2 ) w2 = F4 − F2 where c is a constant with value 3.5 Barks (the formants are also expressed in Barks, which are approximately a logarithmic transformation of measures in Hertz). These formulas model the fact that the human ear cannot discriminate narrow-band frequency peaks in the high frequencies (Carlson et al., 1970).


113

7.2 Dynamics: predicting human vowel systems The simulations which I will describe here have the same structure as the earlier simulations: a dozen agents wander randomly in a virtual space, and from time to time produce a vocalization, which is heard by themselves and by the nearest agent. Each of their neural maps has 500 neurons, and σ = 0.15 (this is the width of the Gaussian defining their activation function, and is equivalent to 15 per cent of the extent of each dimension.) To visualize the state of an agent’s neural system, the representation of their perceptual neural networks is used. In effect, these are of two dimensions (first formant and effective second formant) and contain the same information as the motor networks in terms of distribution (which is what interests us). I represent an agent’s perceptual network in two ways: by showing all the preferred vectors, and by showing the dynamic categorization landscape linked to the coding/decoding cycle by the population vector. Figure 7.2 shows the perceptual maps of two agents after 200 interactions. This allows us to visualize the bias due to the articulatory synthesizer, as explained above. The unit of measurement is the Bark: the horizontal axis represents the first formant, and the vertical axis the effective second formant. It can be seen that the distribution of preferred vectors is no longer uniform. It is contained within a triangle (the vowel triangle), which is itself covered non-uniformly. When the initial situation was an equilibrium due to the fact that all the agents had a uniform distribution of preferred vectors, this bias augmented the natural fluctuations in the system to create non-uniformities which are amplified by a positive feedback loop. The agents, as in the earlier version, ‘crystallize’ in a new situation where the preferred vectors of their neurons are grouped in clusters, which define phonemic categories. Figure 7.3 shows the two agents of the previous figure 2,000 vocalizations later. Figure 7.7 (below) shows the evolution of the similarity between the preferred vectors of the ten agents compared pairwise. This is roughly constant, which shows that they all have the same distributions of preferred vectors (as each other) over the course of the simulation, and in particular the same clusters after 2,000 interactions. The fact that the distributions converge proves that they have learned to master the articulatory function. This can be understood by noting that the condition which brings about this crystallization process is the presence of positive feedback loops. If agents had connections between the two neural maps which did not map each sound onto a roughly corresponding articulation, but onto another articulation producing a different sound, then the activation of a region of motor space in one agent


114

Acoustic neural map of agent 1 initially 15 14

[i]

13

13

[’e’]

12

12

11

11

[a]

10

10

9

9 8 7 2.5 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

Acoustic neural map of agent 2 initially

14

formant 2

3

3.5

4

4.5

5

5.5

formant 1 (in bark) 6 6.5 7

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

[’o’]

[u]

7 2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

Corresponding basins of attraction for agent 2


0

8

1

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

FIGURE 7.2. The perceptual neural maps of two agents 200 vocalizations after the start. The upper panels show the preferred vectors and the lower panels show the basins of attraction that they define. It can be seen that the constraints due to the use of an articulatory synthesizer very quickly bias the initial distribution of preferred vectors: they are spread non-uniformly in a triangle (the vowel triangle).

would not lead to the activation of the same region in other agents. It would follow that the production of a particular sound by one agent would not increase the probability of this sound being produced by other agents, and thus there would be no positive feedback loop. Just as in the previous version, each simulation produces a unique system of phonemic categories (here vowels). Figures 7.4, 7.5, and 7.6 give other detailed examples of the systems that can be obtained in the simulations. Figure 7.8 gives further examples of configurations of the basins of attraction of the acoustic maps obtained in different populations of agents. Now I have also explained that, at the same time as this diversity exists, statistical regularities characterizing the phoneme inventories can appear. As I am using an articulatory vowel synthesizer which copies that of humans, and as precise databases of statistical regularities are available characterizing the vowel inventories of

Learning Perceptuo-motor Correspondences Acoustic neural map of agent 1 after 2,000 vocalizations 15

115

Acoustic neural map of agent 2 after 2,000 vocalizations 12.5

formant 2

12

14

11.5

13

11

12

10.5 10

11

9.5

10

9

9

formant 1 (in bark)

8 2.5

3

3.5

4

4.5

5

5.5

8.5

6


8 3.5 4.5 3 4 5 5.5 Corresponding basins of attraction for agent 2

6

2.5

7

15

14

14

13

13

12

12 11

11

10

10

9

9

8

8

7 2.5

7 3

3.5

4

4.5

5

5.5

6

6.5

7

3

3.5

4

4.5

5

5.5

6

6.5

FIGURE 7.3. The neural maps of the two agents of the previous figure 2,000 vocalizations later. It can be seen that shared clusters have formed in both agents, defining a particular categorization system shared by both agents. The system shown in this figure is the system most often obtained in the simulations. It is also the most frequent in the languages of the world. It is the five-vowel /i,u,e,o,a/ system.

human languages, I have therefore proceeded to make a comparison between the vowel systems generated by my system and those of humans. The database of human languages which is used is UPSID317 (UCLA Phonological Segment Inventory Database) worked out by Maddieson (1984). It contains 317 vowel systems, belonging to twenty different language families, chosen for representativeness in terms of geography and population genetics (see Figure 7.9). Each vowel system consists of a list of vocalic segments said to be ‘representative’. In fact, in a single language, and even in a single person, a vowel can be pronounced differently depending on the preceding or following phonemes or on the rhythm of the utterance (these are co-articulation phenomena). The set of pronunciations of a phoneme is called the set of allophones of this phoneme. To represent a phoneme by a single point in UPSID, the most frequent allophone was chosen. Moreover, I will not directly


116 12.5 12

12.5

formant 2

12 11.5

11.5 11

11

10.5

10.5

10

10

9.5

9.5

9

9

8.5

formant 1 (in bark) 8.5

8 2.5

3

3.5

4

4.5

5

5.5

8 2.5

6

14

14

13

13

12

12

11

11

10

10

9

9

8

8

7

2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

7

3

2.5

3.5

3

3.5

4

4

4.5

4.5

5

5

5.5

5.5

6

6.5

6

7

FIGURE 7.4. Another example of a system that can be obtained. This is ‘an eight vowel system’. 13.5 formant 2 13 12.5 12 11.5 11 10.5 10 9.5 9 8.5 8 2.5 3

formant 1 3.5

4

4.5

5

5.5

13.5 13 12.5 12 11.5 11 10.5 10 9.5 9 8.5 8 2.5

14

14

13

13

12

12

11

11

10

10

9

9

8

8

7

2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

7

2.5

3

3

3.5

3.5

4

4.5

4

5

4.5

5.5

6

5

6.5

FIGURE 7.5. Another example ‘of a vowel system containing five vowels’.

7

Learning Perceptuo-motor Correspondences 14

14

formant 2

13

13

12

12

11

11

10

10

9

9 8

8 7

117

formant 1 2.5

3

4

3.5

5

4.5

5.5

7

6

14

14

13

13

12

12

11

11

10

10

9

9

8

8

7

7 2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

2.5

2.5

3

3

3.5

3.5

4

4

4.5

4.5

5

5

5.5

5.5

6

6.5

6

7

FIGURE 7.6. Another example ‘of a vowel system containing six vowels’. 7

entropy

8

6.8

kl-distance

7

6.6 6.4

6

6.2 6

5

5.8

4

5.6 3

5.4 5.2

2

5 4.8 0

200 400 600 800 1000 1200 1400 1600 1800 2000

time

1 0

200 400 600 800 1000 1200 1400 1600 1800 2000

time

FIGURE 7.7. Evolution of average entropy of agents’ distributions of preferred vectors in one simulation, and evolution of the average kl-distance between agents’ distributions compared pairwise. The entropy curve shows the formation of clusters. The kl-distance curve shows that all agents’ clusters are the same (it rises a little, but this variation is negligible, given that the kl-distance between two random distributions of five clusters is of the order of 105 ).

use the segments of this database, but rather the revised groupings made by (Schwartz et al.,1997b), enabling us to set out the regularities. In fact, no two languages exist with exactly the same vowel prototypes (e.g. two languages can have slightly different ways of pronouncing [e]). The method I used, which

118


14

14

13

13

12

12

11

11

10

10

9

9

8

8

7 2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

7 2.5

14

14

13

13

12

12

11

11

10

10

9

9

8

8

7 2.5

3

3.5

4

4.5

5

5.5

6

6.5

7

7 2.5

14

15

13

14

12

13

3

3.5

4

4.5

5

5.5

6

6.5

7

3

3.5

4

4.5

5

5.5

6

6.5

7

3

3.5

4

4.5

5

5.5

6

6.5

7

12

11

11

10

10 9

9

8 7 2.5

8 3

3.5

4

4.5

5

5.5

6

6.5

7

7 2.5

FIGURE 7.8. Further examples of vowel systems that can be obtained. It can be seen that both the number and the shapes of the basins of attraction are varied.

turned out to be efficient for setting out regularities, is similar to that used by Crothers (1978). It consists in grouping vowel systems according to the relative positions of vowels with respect to each other rather than according to their absolute positions. Also, vowels are represented in the acoustic space ′ (F 1 , F 2 ), in the same way as they are represented in the perceptual maps in the simulations. Figure 7.10 shows the set of possible patterns that I use in this

Learning Perceptuo-motor Correspondences 5 Sina liberian

1 Eskimo-Aleut 2 Indo-european

17 Amerind (North America)

4 Caucasian

1

6 Paleo-Siberian 6

7

1

2

2

3

3 Ourala allsique

4

3

17

5 15

10

12 14

16

15 Afro-Asiatic 16

7 Austro-Asiatic

2

15

14

17 16 Amerind (South America)

119

10 Austro-chai 10

8 Indo-Pacific

11 Dravidian 10 12 Nilo-Saharan 10

9

9 Australian

14 Niger-Kordo Fanian 13 Khoisan

FIGURE 7.9. The UPSID317 database contains 317 vowel systems belonging to 20 different language families. (Adapted from Escudier and Schwartz, 2000.)

[u] [’o’]

[a]

[’e’] [i]

FIGURE 7.10. The patterns with which I have identified each system generated are combinations of the locations shown in this figure. Note that I use here the same notation as Schwartz et al. (1997b), with the horizontal axis corresponding to the first formant and the vertical axis to the effective second formant, but with the high values at the bottom and the lower values at the top (the axis switches direction in relation to the figures given earlier).

120


classification: black circles on the vowel triangle represent possible phoneme locations. There are twelve possible locations. As I have said that I am interested in the relative positions of phonemes, I allow myself small shifts, rotations, and scalings in the process of matching particular vowel systems to generic patterns of vowel systems. Another abstraction from the UPSID database that I am using as a basis for comparison is the number of systems with two vowels, three vowels, four vowels, etc. This measure is inherently much more direct than going by the structures of the vowel systems. Five hundred simulations were run, all with the same parameters. Each time, the number of vowels in the inventory and the relative positions of the vowels were recorded. This second measure was taken by hand, which seemed not only the quickest method but also the most precise (indeed, it is not obvious how to classify this type of pattern automatically). De Boer (2001), moreover, used the same method to compare the vowel systems generated by his simulations with those of human languages. Here, to determine the number and location of vowels, I used the attractor landscape representation, i.e. the representation of the agents’ categorization behaviour. This is more efficient than looking at the distribution of preferred vectors in the perceptual maps, because clusters are sometimes disturbed by rare neurons with preferred vectors with values far from these clusters. In addition, the categorization representation directly gives a prototype for each category: the attractor point of the coding/decoding cycle of the population vector. The results are given in Figures 7.11 and 7.12. In Figure 7.11, it can be observed that the sizes of vowel systems in human languages and those of the populations of agents are very similar. In particular, they are characterized by a maximum of five vowels. Figure 7.12 shows the distribution of vowel system structures in human languages and in the systems generated by the agents. Only the two most frequent systems are represented in each case, up to eight vowels. It can be seen that despite the large space of possible systems, the two most frequent systems are the same for n = 3, 6, and the two most frequent systems in humans are in the three most frequent artificial systems for n = 4, 5, 7. Considering percentages, these are quite correlated. The prediction for the most frequent vowel system, /i,u,e,o,a/, is 25 per cent, against 28 per cent in UPSID. On the other hand, the human systems represented in the figure correspond to 59.5 per cent of the systems in UPSID, and the artificial systems in the figure are 75.6 per cent of the total of artificial systems generated. This shows that the relationship between frequent systems and more


121

frequency

distributions of sizes of vowels systems 50 45 40 35 30 25 20 15 10 5 0

emergent systems human systems

4

3

5 6 7 sizes of vowel systems

8

9

FIGURE 7.11. Distributions of sizes of vowel systems obtained in the simulations and in the UPSID database. Most frequent vowel systems in human languages and artificial systems 3 vowels u

0.6%

4 vowels

6 vowels

5 vowels 1.9%

1.9%

7 vowels

3.1%

1.6%

8 vowels 0.9%

a i

0.1%

2%

0.3%

3.2%

3.7%

3%

1.5%

28%

2.5%

0%

0%

9.1%

6%

9%

1.7%

0%

6.4%

6.3%

1.9%

1.9%

1.2%

0%

12%

: frequency in human languages

19%

: frequency in artificial systems

0%

1.6%

FIGURE 7.12. Distribution of vowel inventories obtained in the simulations and in the UPSID database. The notation is the same as in Figure 7.10.

eccentric is roughly followed. It also shows the diversity of the systems generated. By contrast, the predictions of the simulations deteriorate for systems of more than seven vowels, because these are not generated at all, while they do exist in humans (but are certainly less frequent).

122


Given the large number of possible vowel systems, the similarities between the human and artificial systems are striking. In particular, the performance of the system in prediction is comparable to that of de Boer (2001), with the difference that in de Boer’s simulations, vocalization repertoires of different sizes are obtained by varying a noise parameter, while here I obtain diversity of sizes with the same parameters. The differences between the predictions and the human systems should be interpreted in the light of the way the UPSID systems and the artificial systems were constructed. In fact, I have explained in previous chapters that my simulations dealt with the formation of a speech code before it was recruited for purposes of communication. That is to say, that the generated systems are prelinguistic systems which have not yet undergone cultural evolution under the functional pressure of communication. By contrast, the UPSID systems are vowel systems of contemporary languages. This is to say that they are vowel systems which have evolved under functional pressure of communication, and this evolution has already been going on for a very long time. To summarize, the speech systems of our simulations and those of UPSID correspond to very different eras in history (one before the appearance of complex communication, and the other long after the appearance of language). This is why the similarities and differences between the two systems are interesting but not crucial.

8. Strong Combinatoriality and Phonotactics

In previous chapters, the agents produced vocalizations whose articulatory targets were chosen randomly from the repertoire defined by the neural networks. We saw how this repertoire of articulatory targets could self-organize, passing from a quasi-continuum to a set of discrete phonemes. We also saw that, because the number of these discrete articulatory units (phonemes) was small compared to the number of vocalizations an agent could produce in its lifetime, there was necessarily systematic reuse of these phonemes in composing the vocalizations. By contrast, there was no organization to the specific ways in which these phonemes were reused—it was random. This means that all possible sequences of phonemes could be produced by the agents. Furthermore, the reuse of phonemes was systematic but not governed by rules or patterns of recombination. So, the form of combinatoriality which appeared was really primitive. Now it happens that the sound systems used by humans, as explained in Chapter 2, organize very strictly the ways in which phonemes can be combined. In particular, every language only allows certain sequences of phonemes, and not others. For example, in English spink is a possible word, while npink or ptink are impossible. In Tashliyt Berber, tgzmt and tkSmt are allowed, but they are impossible in French. Thus each language is not only a code defining a shared inventory of phonemes, but also a code defining a shared repertoire of possible combinations of phonemes. The basic unit of combination currently accepted in linguistics is the syllable, defined as a vocalization produced during one oscillation of the jaw (MacNeilage, 1998). The inventories of syllables permitted in the world’s languages are also structured. That is, there is structure in the set of permitted phoneme combinations. A syllable comprises a certain number of ‘places’, such as the onset, the nucleus, and the coda, in which only certain phonemes can occur. Thus, syllable repertoires can be summarized using patterns, and this kind of rule-based system of systematic phonemic reuse is what I will call ‘strong

124


combinatoriality’. For example, one can summarize the permitted phoneme sequences of Japanese syllables by the patterns CV/CVN/VN, where for example CV defines syllables composed of two slots, and in the first slot only the phonemes belonging to a group that is called ‘consonants’ are allowed, while in the second slot, only the phonemes belonging to the group that is called ‘vowels’ are allowed (N stands for ‘nasals’). Further, there are patterns which are statistically preferred over others in the world’s languages. For example, all languages allow syllables of the type CV, while many do not allow consonant clusters at the beginnings of syllables. In this chapter, I will show how an extension of the model presented in Chapter 6 enables not only shared discrete phoneme inventories to appear but also shared pattern-based rules of phonemic combinations, constituting a shared phonotactics and the formation of strong combinatoriality. I will also show how certain patterns can be statistically preferred when articulatory and energetic constraints are introduced.

8.1 Temporal neurons and their self-organized death Instead of selecting articulatory targets by an algorithm that randomly activates neurons in the ‘spatial’ map, neurons which I will call ‘temporal’ will carry out this task. That is, to the neural map modelling relations between organs (the spatial map), I add a neural map modelling sequences of articulatory targets (i.e. sequences of activations of neurons in the spatial map). These temporal neurons are each connected to several spatial neurons. They can both send and receive signals through these connections. These neurons are temporal because their activation function has a temporal dimension: their activation does not depend solely on the amplitude of activation of the spatial neurons to which they are connnected, but also on the order in which they are activated. Thus, when they receive signals from the spatial map, their activation is calculated from the temporal evolution of the activity of the spatial neurons,1 which is itself determined by the perceived acoustic trajectories. The mathematical formula for calculating the activation G Ti of a temporal neuron i when a vocalization is perceived and activates the spatial neural map (using

1 This temporal evolution arises because the activation of one neuron might not be the same at the beginning and at the end of the vocalization if the formants are not the same.

Strong Combinatoriality and Phonotactics

125

the same mechanism as in previous chapters) is G Ti =

N T t=0 j =1

1 1 2 2 G 2 /σ 2 √ · e |t−T j | /σ · √ .e j,t 2π σ 2πσ

with T denoting the duration of the perceived vocalization, N the number of spatial neurons to which it is connected (which will be here 2, and each temporal neuron is initially connected to 2 randomly chosen spatial neurons), T j a parameter which determines when the temporal neuron i is sensitive to the activation of the spatial neuron j , and G j,t the activation of the spatial neuron j at t. Here, the T j values are such that the temporal neuron that they characterize is maximally activated for a sequence of spatial neuron activations in which two neurons are never maximally activated at the same time and for which the maximal activation is always separated by a fixed time interval. In brief, this means that rhythm is not taken into account in this simulation: only the order is considered here. Mathematically, T1 = 0, T2 = τ, t3 = 2 · τ, . . . , TN = (N − 1) · τ where τ is a time constant. As stated in the first paragraph, the temporal neurons are also used to activate the spatial neurons. The internal activation of one temporal neuron, coupled with a GO signal, provokes the successive activations of the spatial neurons to which it is connected, in the order specified by the T j parameters. This implies that the temporal pattern is regular, and only one neuron is activated at the same time. In this chapter, each temporal neuron will be connected to only two spatial neurons, which means that a temporal neuron will code for a sequence of two articulatory targets (N = 2). This will allow us to represent easily the temporal neural map, but this is not crucial for the results. When an agent decides to produce a vocalization, which it does at random times, it activates one temporal neuron chosen randomly and sends a GO signal. Initially, a large number of temporal neurons are created (500, compared with 150 neurons in the spatial map), and these are connected randomly to the spatial map with random values of their internal parameters. Using many neurons means that basically all possible sequences of activations of spatial neurons are encoded in the initial temporal neural map. The plasticity of the temporal neurons is different from the plasticity of spatial neurons.2 2 Yet some recent experiments (not decribed in this book because they were not conducted with the same systematicity) indicate that it is possible to use for both neural maps the same neural dynamics

126


The parameters of temporal neurons stay fixed during the simulations, but the neurons can die. As a consequence, what changes in the temporal neural map is the number of surviving neurons. The neuronal death mechanism is inspired from apoptosis, or programmed cell death in the human brain (Ameisen, 2000), and fits with the theory of neural epigenesis developed by Changeux and Danchin (1976). The theory basically proposes that neural epigenesis consists of an initial massive generation of random neurons and connections, which are afterwards pruned and selected according to the level of neurotrophins they receive. Neurotrophins are provided to the neurons which are most often activated, and prevent them from automatic suicide (Ghosh, 1996). I apply this principle of generation and pruning to the temporal neurons in my system, depending on their mean activity level. The mean activity of a temporal neuron j is computed with the formula M A j,t =

M A j,t−1 · (wi ndow − 1) + G T j,t window

where window has the initial value 50 (the value of the window size influences the speed of convergence, but the system is rather robust in terms of end result if it is varied). The initial value M A j,0 is equal to 2 · vitalThreshold. The vitalThreshold constant defines the level of activity below which the neuron is pruned. This threshold remains the same for all neurons in the map. The value of this threshold is chosen so that there is not enough potential activity for all the neurons to stay alive: stability arises at the map level only after a certain number of neurons have been pruned.

8.2 The dynamic formation of phonotactics and patterns of combinations A population of ten agents interacts in the same way as in Chapter 6: they are located randomly in a virtual environment, and at random moments produce a sound by random activation of a neuron in their temporal network. The nearest agent, and the vocalizing agent itself, hear the vocalization and update the neurons of their two networks. I will make the same assumptions as in Chapter 6 concerning the nature of the spaces I am dealing with and their dimensionality. There, it was assumed and still obtain results similar to those I present. In these experiments, the common neural dynamics was the same as that used here for the temporal neural map.


1.2

second target in organ relation space

AGENT 1

127

AGENT 2 1.2

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0 -0.2

-0.2 0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

representation preferred of temporal vectors neurons and initial of activity level spatial neurons

1

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

first target in organ relation space

FIGURE 8.1. The two neural maps of two agents at the beginning of the simulation: the rectangle on the left represents the spatial map (whose preferred vectors are represented by points on the axes) and the temporal map (whose neurons are represented by little areas) of one agent, and the rectangle on the right represents the corresponding maps of another agent.

that the agents could pass from the acoustic space to organ relation space, and also between organ relation space and the space of muscular activations.3 So here I focus just on organ relation space. Moreover, as at the beginning of Chapter 6, a one-dimensional organ relation space will be used. This will make it possible to visualize the neurons in the temporal map: since they are only connected to two neurons in the spatial map, they are defined by two values (two one-dimensional articulatory targets), and thus by a point in a two-dimensional space. Figure 8.1 represents the networks of two agent at the beginning of a simulation. Both the horizontal and the vertical axes represent organ relation space. The points on the two axes are the preferred vectors of 3 Thus in this section we will suppose that there is a module somewhere that automatically carries out the translations between acoustic space and organ relation space on the one hand and between organ relation space and muscular space on the other. Thus we do not go through these spaces in the simulations in this chapter, but take a short cut: all vocalizations are produced and perceived in organ relation space here (though humans, of course, go through the three spaces). This short cut is possible because it was shown in Ch. 7 how these translations could be learnt in principle.


128


AGENT 1

AGENT 2

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

first target in organ relation space preferred representation of vectors temporal neurons of neurons with their activity level in spatial map

FIGURE 8.2. Illustration of the sort of temporal map obtained if the elimination mechanism for temporal neurons is not used.

the neurons of the spatial map. Thus the same points occur on both axes. Then the points in the body of the plotted space represent neurons of the temporal map. Each one has an X and a Y value at one of the preferred vectors of the spatial neurons: the value on the horizontal axis corresponds to the first articulatory target coded by the temporal neuron, and that on the vertical axis corresponds to the second articulatory target coded by the temporal neuron. It can be seen that the set of temporal neurons covers just about all the continuous space of possible combinations defining a vocalization. The little areas around each point represent the average activation level (here initial) of the temporal neurons. The larger the area, the higher the level of activation (it is initially the same for all neurons). Before I present the results yielded by the generation/elimination dynamic of the temporal neurons, Figure 8.2 shows what happens if no neurons at all are eliminated. It shows an agent’s map after 1,000 interactions in a population of ten agents. We observe, as in the simulations presented in the previous chapters, that a clustering appears in the spatial map. As far as the temporal neurons are concerned, we observe that they still cover all the mathematically possible combinations of two phonemes (whose categories are defined

Strong Combinatoriality and Phonotactics second target in organ relation space

AGENT 1

AGENT 2

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1

0.1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

129

1

first target in organ relation space

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

preferred representation of vectors temporal neurons of neurons with their activity level in spatial map

FIGURE 8.3. If the pruning mechanism on temporal neurons is used, this is the kind of pattern obtained at the end of a simulation. On the one hand, note that not all phoneme combinations are represented in the temporal map, and thus the agents only produce those combinations for which they have temporal neurons. On the other hand, the possible phoneme combinations are organized in patterns.

by belonging to clusters). There is thus no constraining principle affecting the combination of phonemes, thus no phonotactics, and even fewer patterns. Figure 8.3 now shows what is obtained if elimination of insufficiently activated neurons is implemented. Here I show the networks of two agents, taken randomly in a population of ten interacting agents, after 1,000 interactions. Note first that the neurons in the temporal map no longer cover all the possible combinations. And they cover the same combinations in both agents. This implies that the vocalizations they produce only exhibit certain phoneme combinations and not others, and that the rules of combination are shared: thus we observe the appearance of culturally shared phonotactic rules. Obviously, these rules are different for each simulation, and therefore diverse. Figure 8.4 gives another example of a result. A second observation could be made concerning the organization of temporal neurons surviving in the space of combinations that they encode. It can be seen that the combinations appearing after 1,000 interactions are not


130


AGENT 1

AGENT 2

1

1

0.9

0.9

0.8

0.8

0.7

0.7

0.6

0.6

0.5

0.5

0.4

0.4

0.3

0.3

0.2

0.2

0.1 0

p1 0

p2 p3

p4 p5

0.1 0.2 0.3 0.4 0.5

p6

p7

0.1

p8

0.6 0.7 0.8 0.9

1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

first target in organ relation space preferred representation of vectors temporal neurons of neurons with their activity level in spatial map

FIGURE 8.4. Another example of a phonotactic system generated by a population of agents using the pruning mechanism.

distributed randomly among those that are possible. They organize themselves into rows and columns. For example, in Figure 8.4 there are two columns and one row. If the phonemes associated with the eight clusters of the spatial map are labelled p1 , p2 , . . . , p8 , as indicated in the figure, then the syllable repertoires of these agents can be summarized as ( p6 , ∗), ( p8 , ∗), and (∗, p7 ), where ∗ is a ‘wild card’ meaning ‘any of the phonemes p1 , . . . , p8 ’. This implies that the system of vocalizations that the agents are producing are now strongly combinatorial: some phonemes are reused systematically and in a patternbased manner for the building of different complex vocalizations. Yet it should be noted that the types of pattern that appear are quite different from the types of pattern in real human languages, for example the CV/CVN/VN organization of syllables in Japanese. Indeed, in human languages, patterns define slots in which the set of phonemes that can appear are often disjunct and form what is called a phonological category: in particular, the set of consonants (C) and the set of vowels (V) have intrinsic properties which determine their valences and thus their privilege of occurrence in certain slots. To summarize, the systems that appear in the simulations are strongly combinatorial, but lack phonological categories. So the complexity of the patterns that form in the simulations has not yet reached that of contemporary human languages.


131

number of neurons in temporal map

500 450 400 350 300

number of surviving temporal neurons for agent 2

250


200 150

0

200

400

600

800

number of interactions

1000

1200

time

FIGURE 8.5. Evolution of the number of surviving temporal neurons in the simulation which generated the system in Figure 8.3. At first a phase is seen in which a certain number of neurons die, and then there is a stabilization phase in the system during which the remaining neurons succeed in surviving.

The states shown on Figures 8.3 and 8.4 are convergence states. Indeed, the states both of the spatial map and of the temporal neural map crystallize after a certain amount of time. In previous chapters, I explained in detail why the spatial map practically converged into a set of clusters. We are now going to visualize and explain how and why the temporal neural map also crystallizes. Figures 8.5 and 8.6 show the evolution of the number of surviving neurons within the temporal maps of two agents: we can see the separation between the phase of neuron death and the stabilization phase. The formation of shared repertoires, and on the other hand the formation of patterns, is the result of a dynamic of competition and cooperation among temporal neurons. In fact, the vital threshold for these neurons was chosen so that in the case of Figure 8.2, where there are temporal neurons for all combinations, the frequency with which each combination is produced would be too weak to raise them above the critical activation level. Imagine that the initial situation in the simulation is that in Figure 8.2, where the clusters have already formed in the spatial network. In other words, imagine that we disengage the dynamic of the spatial neurons from that of the temporal neurons. In that case, every neuron has an initial activation level (M A j,0 = 2.vitalThreshold) twice as high as its vital threshold level. This vital level was fixed to be higher than the average activation level generated by this initial configuration. As a

132


number of neurons in temporal map

500 450 400 350 300


250 200 150


100 50

0

200

400

600

800

number of interactions

1000

1200

time

FIGURE 8.6. Evolution of the number of surviving temporal neurons in the simulation which generated the system in Figure 8.4. We can observe that here the two agents do not possess exactly the same number of surviving neurons: this is due to the intrinsic stochasticity of the system. Nevertheless, as Figure 8.4 indicates, they share the same phonotactics and the same patterns.

consequence, the mean level of activity of all neurons is going to decrease at the beginning of a simulation. Because there is natural stochasticity in the system, due to the random choice of temporal neurons when vocalizations are produced and the not quite equal sharing of neurons in the clusters of the spatial maps, these activation levels will not all decrease at exactly the same rate. In particular, some neurons will get below their vital level before others, and therefore die. The survival of a neuron in a cluster of an agent’s temporal map depends in part on the number of neurons corresponding to the same combination in other agents, whose survival itself depends on the density of the cluster in question in the first agent. This creates positive feedback loops, with the result that when by chance a certain number of neurons die in an agent’s cluster, this facilitates the death of corresponding neurons in other agents. In the same way, healthy clusters (meaning that they have many neurons with high activation levels, which prolongs their effectiveness) will facilitate the survival of corresponding neurons in other agents. In this way, the interaction of forces of competition and cooperation takes advantage of fluctuations to lead the system into a stable state, in which each neuron ends up with an activation level higher than the vital level.


133

We just saw that cooperation can happen between neurons of clusters corresponding to the same phoneme combination in different agents. I will now explain how cooperation can also happen between clusters sharing only one phoneme at the same place in a vocalization. This is due to the mode of activation of temporal neurons, as detailed in the formula given in the previous section. For example, let us denote p1 , p2 , p3 , and p4 four distinct articulatory targets belonging to four distinct clusters. If the similarity of two vocalizations with the same sequence of phonemes is about 1, then the similarity between the vocalization coded by the sequence ( p1 , p2 ) and the vocalization coded by the sequence ( p1 , p3 ) is about 0.5, and the similarity between ( p1 , p2 ) and ( p3 , p4 ) is about 0. This means that the level of activity ‘provided’ to the temporal neurons of a cluster cl , thanks to two clusters of temporal neurons in other agents which share exactly one phoneme in the same location, is about the same as the level of activity provided to the neurons in cl , thanks to the cluster in other agents which corresponds to temporal neurons sharing all the phonemes in the right location with those in cl . As a consequence, groups of clusters reinforcing each other will form during the self-organization of the temporal neurons map. These are the lines and the columns that we observed on Figures 8.3 and 8.4, and this explains why we observe the formation of phonological patterns in the phonotactics developed by the agents. To summarize, the interactions between competition and cooperation among individual clusters explains the formation of shared and stable repertoires of allowed phoneme sequences, and the interaction between competition and cooperation among groups of clusters explains the formation of phonological patterns.

8.3 The impact of articulatory and energetic constraints With the mechanism presented in the previous sections, if a large number of simulations are run, there will be no statistical preferences for localization of either temporal or spatial neurons. This is similar to the simulation without articulatory bias in Chapter 6, in which the clusters had as much chance of appearing at one spot in organ relation space as at another. As in Chapter 6, we will see here how an articulatory bias can introduce preferences. This articulatory bias, modelling non-linearities in the function mapping sounds to articulatory configurations, will be modelled through the initial distribution of preferred vectors. I will, however, add another bias


134

introducing energetic constraints. In a human, for example, each vocalization calls for the displacement of a certain number of organs, and this displacement costs energy: some vocalizations are easier to pronounce than others from the point of view of the muscular energy expended. Moreover, several researchers (Lindblöm, 1992; Redford et al., 2001) have already proposed that energetic cost is an important constraint in the formation of the sound systems of human languages. This constraint will be modelled here by assigning each vocalization an energy cost, to be defined, which will influence the level of neurotrophins received by the neurons coding for them (thus, at equal frequency, a neuron coding for an easily pronounced syllable will receive more neurotrophins than a neuron coding for a syllable that is hard to pronounce, and thus will have a greater chance of survival). The combination of these two constraints will enable us to show how the interaction of two different biases can lead to the formation of repertoires of syllables with structures whose statistical properties are not deducible from any single bias on its own. The initial distribution of preferred vectors of the spatial map will be similar here to the initial distribution of preferred vectors in the simulation with articulatory bias of Chapter 6: the density of preferred vectors will increase between 0 and 1. This means that initially there will be a higher density of preferred vectors closer to 1, and the lowest density will be near 0. Figure 8.7 gives two examples of the initial distribution of preferred vectors.

0.016

density

density

0.014

0.014

0.012

0.012

0.01

0.01

0.008

0.008 0.006

0.006

0.004

0.004

0.002

0.002 0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 1


1

0

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

agent 2

1


FIGURE 8.7. Example of the initial biased distribution of preferred vectors in the spatial maps of two agents: there are more preferred vectors around 1 than around 0.


135

The energy cost associated with a vocalization is defined by the displacement from a position of rest fixed as coded by point 0 in organ relation space. The most distant position from this is coded by 1. To calculate the energy expended, the articulatory trajectory is discretized and the sum of all the distances between the points on this trajectory and the resting position is calculated. If p1 and p2 are two phonemes used as articulatory targets in a vocalization, and p1 , pi nt1 , pi nt2 , . . . , pi ntN−1 , p2 are the successive points in the trajectory generated after interpolation, then the energy expended is e( p1 , p2 ) = p12 + pi2nt1 + pi2nt2 + . . . + pi2ntN−1 + p22 This energy will influence the survival of the temporal neurons. Indeed, I explained earlier that the survival of temporal neurons depended on the level of neurotrophins that they received. A neuron could receive neurotrophin in proportion to its level of activation. The stress associated with the spending of energy can in reverse prevent the reception of neurotrophins (Ghosh, 1996). In particular, temporal neurons coding for vocalizations with targets close to 0 will be favoured by this constraint as compared to the temporal neurons coding for vocalizations with targets close to 1. I will denote Nti,t the level of neurotrophins received by the temporal neuron Ni at time t. Then we can compute Nti,t = M Ai,t − c 1 .e( p1,Ni , p2,Ni ) where c 1 is a normalizing constant so that both the terms of activation and of energy have the same ranges, and where p1,Ni and p2,Ni are respectively the first and second articulatory target encoded by temporal neuron Ni . Again, there is a constant vitalThreshold such that if the level of neurotrophins Nti,t becomes smaller, then the temporal neuron Ni is pruned. This constant is chosen so that not all temporal neurons can survive. Here, M Ai,0 = 0.06, vitalThreshold = 0.03, c 1 = 15, and there are 150 spatial neurons and 500 initial temporal neurons. Figure 8.8 gives a glimpse of the initial temporal map of two agents. Only 150 temporal neurons are represented here rather than all 500, so that one can easily visualize the disparity in densities. The segments on each neuron represent their initial level of neurotrophin. This initial value is not the same for all neurons because it takes into account the energy value which varies as a function of the articulatory configuration associated to each temporal neuron. A look at Figure 8.8 shows that there are more temporal neurons in the top right, but that those in the lower left have a lower energy cost, and

136

Self-Organization in the Evolution of Speech Agent 1

Agent 2

Second articulatory target 1.2

1.2

1

1

0.8

0.8

0.6

0.6

0.4

0.4

0.2

0.2

0

0

-0.2

0

0.1

0.2 0.3 0.4

0.5 0.6 0.7 0.8 0.9

1

First articulatory target Preferred vectors of spatial neurons

-0.2

Second articulatory target

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

First articulatory target

Temporal neurons with the associated level of neurotrophins

FIGURE 8.8. Example of biased initial temporal neural map: the segments associated with the temporal neurons show the initial level of neurotrophins of these temporal neurons. We observe that the temporal neurons close to (0, 0) have the largest initial level of neurotrophins, but that the temporal neurons close to (1, 1) are more numerous and so will be activated more often initially, which means that they will receive more neurotrophins than those close to (0, 0).

therefore receive more neurotrophins at the same level of activation. There are thus two forces, each favouring the survival of one corner of the space to the detriment of another. It can easily be seen that if there were no energy cost, as in Chapter 6, there would be a statistical preference for clusters of spatial neurons close to 1, and thus automatically a statistical preference for clusters of temporal neurons in the upper right quadrant. Likewise, if only neurons corresponding to vocalizations which do not use much energy were to survive, one would see a statistical preference for temporal clusters in the lower left. The combination of these two forces will give a different statistical preference. To evaluate this I ran 500 simulations, and for each one I plotted (Figure 8.9) all the temporal neurons of one agent. Figure 8.9 thus represents the set of surviving temporal neurons after 500 simulations. We observe that there is a clear statistical preference for vocalizations composed of targets located in the centre of the space, and not near 0, as the energetic constraint alone would


1.1

137

Second articulatory target

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7 0.8 0.9 1 First articulatory target

FIGURE 8.9. Distribution of surviving temporal neurons in 500 simulations.

produce, or near 1, as the non-linearity articulatory constraint alone would produce. This shows how crucial it is to understand in detail all the constraints influencing the formation of repertoires of vocalizations, as well as the interactions among these constraints, if one wants to understand why, for example, human languages prefer CV syllables to CCVC syllables. This result is positive in the sense that it illustrates the kind of dynamics that can give rise to apparently idiosyncratic phonotactic regularities. This helps us develop our intuition of the self-organizing processes that shape vocalization systems. But this result is also negative in the sense that it shows how far we are from being able to predict the statistical phonotactic preferences of humans. Indeed, our knowledge of the physiological, energetic, and representational dimensions of human speech is extremely low. There are few areas for which we have good models, such as the perception and production of vowels, which allow us to use realistic constraints in a predictive model of the statistical regularities of vowel systems. But, for example, we know very little about the energetic cost of vocalizations; and the existing models of the brain’s representations of speech signals, which is crucial for the understanding of the articulatory/perceptual non-linearities, are still very speculative. We are not even able to make a list

138


of all the possible constraints that might influence the process of creation of vocalizations. This also explains why, instead of building a system based on very speculative models of realistic constraints, I chose to build a system with completely abstract representations and constraints, which facilitates the understanding of the dynamics. Finally, it should be said that another constraint which would be very interesting to integrate is the functional constraint. Indeed, for reasons explained in the introduction, I developed a system free of functional pressure for efficient communication: the agents had no motivation for building a communication system with a large repertoire of distinctive vocalizations. I have shown in the last three chapters that even without this motivation, and with no repulsive force, still the system self-organized a shared repertoire of vocalizations which can be categorized distinctively and which possess phonotactics. Yet, if we imagine that this system actually describes a process that took place in the evolution of humans before they had language, it would have been recruited later on in order to communicate. This means that a functional pressure came in and added new constraints, such as the perceptual distinctiveness between similar vocalizations, which typically would disfavour sequences of identical phonemes like aaa or mmm. This case could possibly be studied by coupling the system described in this book with the imitation game invented by de Boer (2001) and extended to syllables in Oudeyer (2001c).

9. New Scenarios

In the three preceding chapters, I described operational mechanisms enabling us to simulate a population of agents developing a speech system with properties similar to the speech code of human languages. These simulations were built on a certain number of defined assumptions. The previous chapters showed what these assumptions led to in terms of dynamics and results. I will now discuss what is interesting in these assumptions. First, I will show that they are compatible with certain observations from neuroscience on the human brain. As was explained at the beginning of the book, this is not essential for the evaluation of an artificial system, but it is nevertheless of interest. Next I will study in what ways the simulation and its assumptions can illuminate the problem of the origin of speech. I will use the explanatory framework developed in Chapter 3 to show the interest in these assumptions. In particular, I will explain how the artificial system is compatible with functionalist explanations, and that it adds strength to them by giving an explanation of type 2, which was previously lacking in this approach: it will easily be seen that the complexity of the assumptions, of a quite different order from the speech codes that are generated, makes their discovery by natural selection much more understandable, if we assume a classical explanatory framework in which the origin of speech is attributed to its advantage in communication. On the other hand, the generic nature of the assumptions is such that we can propose several scenarios in which they could have appeared with no connection to the function of linguistic communication. In particular, I will use the concept of architectural constraint and propose that these assumptions could be side effects of the evolution of other structures whose function is not connected to language. In brief, this will allow us to suggest that the speech codes that we use today are perhaps exaptations, the first versions of which were possibly the outcome of self-organization of structures that had nothing to do with language.

140


9.1 Compatibility with neuroscience In Chapter 7 I showed similarities between vowel systems generated by artificial agents and those of human language. I will show here that the assumptions leading to these predictions are also compatible with what is known about the human brain. When I speak here of resemblances, once again I am not trying to establish an isomorphic correspondence between the components of the agents’ brains and those of human brains. I will only try to show that the general functioning principles of the two types of brain are compatible. First of all, the basic building blocks of my agents’ brains are neurons whose characteristic behaviour is to respond maximally to a particular stimulus; this response decreases as the input stimuli differ from the preferred stimulus, called here the preferred vector. As far as the spatial neurons (introduced in Chapter 6) are concerned, this model of neuron behaviour is widely used across all the modelling literature in neurosciences (Arbib, 2002). It is even difficult to find models using neurons following any other principle. Obviously, the way in which the response decreases can vary from one model to another. I chose to use a Gaussian, a very common choice in this same literature. Moreover this choice is not crucial, because any activation function giving a low value far from the preferred vector will lead to the same dynamics. As for the temporal neurons presented in Chapter 8, whose Gaussian function has a temporal component, they are less frequently used in modelling but are also well represented among the accepted conceptions of neuroscience: neurons used in the visual cortex have this type of behaviour and activation function (Dayan and Abbot, 2001). It is plausible that similar neurons are used in auditory processing, which, like vision, involves a temporal component in perception (Kandel et al., 2000). The dynamics of adaptation to stimuli which modifies the preferred vectors of neurons is also widely used in the literature. Reinforcement of the sensitivity of neurons to stimuli which makes them react strongly is at the heart of Kohonen’s self-organizing nets (Kohonen, 1982), themselves considered to be models of the cerebral cortex. This adaptive law is often formulated in terms of Hebbian reinforcement of connections between neurons (Arbib, 1995), which is a little different from the implementation in the present work. However, my model is mathematically equivalent, in terms of the dynamics of preferred

New Scenarios

141

vectors, to that of Guenther and Gjaja (1996), which is a recognized model of vowel perception using neural nets very similar to those used in Chapters 6 and 7. Neurons are organized in networks particular to each modality, corresponding to the way in which the cortex is conceived in brain theory manuals (Arbib, 1995). In addition, these nets are connected in such a way that the agents manage to master the mappings between one space and another: they can find articulatory configurations corresponding to a sound. This mastery over translation from one space to another is evidently present in humans, otherwise they would not be able to imitate each other and speech would not exist. I will approach the crucial question of the origin of this capacity in the next section. For the moment, we merely note that this capacity is present and very common in a general way in mammalian nervous systems: for example, an equivalent capacity for mastering the coordination between movement of the hands and sight of such movement exists in mammals which have hands. In contrast, the present state of knowledge in neuroscience does not give us a precise idea how this mastery is implemented in the brain (either of the correspondence between vision and manual movement or between sounds and articulatory configurations). Recent discoveries concerning mirror neurons (Rizzolati et al., 1996) have, however, stimulated much interest in the scientific community. These are neurons observed in monkeys but certainly also present in humans, which are normally activated by the animal when it makes a motor action (like picking up an apple), but which are also activated when it sees another monkey carrying out the same action. However, this discovery only reveals certain parts of the neural pathway, enabling translation from one space to another, and we do not even know whether these neurons are innately hard-wired or the result of learning. For this reason, the neural nets in the simulations were connected in as generic a way as possible, without assuming precise innate wiring and relying on connections for which the Hebbian adaptation rule makes it possible to learn the translation between the various motor and perceptual spaces. The preferred vectors of neurons as well as their connections are initially random, and adapt (and even survive or disappear in the case of temporal neurons) during the lifetime of an agent. This way of building a brain is in line with selectionist theories of neural epigenesis developed by Edelman (1993) and Changeux (Changeux and Danchin, 1976). These theories have

142


recently been reinforced by the discovery of the phenomenon of apoptosis, or programmed neuron death, showing that all neurons have an internal suicide programme which must be inhibited if they are to survive (Ameisen, 2000). This inhibition works on reception of neurotrophins, especially when the neurons are sufficiently stimulated electrically. This is how many researchers in neuroscience envisage the construction of the brain, as initial random generation of neuronal material, which is then shaped during the course of interaction with the environmment (Changeux and Danchin, 1976). I modelled the stimulus categorization behaviour of each neural network by means of dynamic relaxation of these nets of recurrent neurons, until they converged on an attractor (in this work a fixed point). The category assigned to a stimulus was the identity of the attractor. Seeing perceptual categories as attractors in brain activity considered as a dynamic system is an idea current in the scientific community, defended for example by Edelman (1993) and Kaneko and Tsuda (2000). Freeman (1978) made measurements of the dynamics of the electrical activity of the rabbit’s olfactory bulb, confirming the plausibility of this hypothesis. The agents’ production of vocalizations in the simulations consisted in defining a certain number of articulatory targets which were then reached by a continuous displacement of the articulatory organs. This way of conceiving of motor control is quite widely accepted (Kandel et al., 2000). Generally speaking, this conception analyses motor control as organized at two levels: first there is the production of a motor program specifying the targets to be reached (in the case of speech this is the specification of the gestural score); then there is the execution of this program carried out by lower-level neural devices, whose task is to move the organs in keeping with the constraints applying at the moment. In this book, I have concentrated on the level of motor programs, modelling the level at which they are executed by a naive mechanism (polynomial interpolation). This means that I have not studied co-articulation phenomena and the influence these may have on the construction of phoneme inventories. This could be an interesting extension of this work. Further, as the architecture I have used is not specific to the vocal apparatus, the work presented here could also be extended to account for the formation of the motor primitives often assumed in the literature on motor control (Mataric et al., 1998). The coupling of perceptual and motor networks, as well as the adaptive dynamic of the preferred vectors of neurons, make the perception of a sound

New Scenarios

143

augment the probability of producing it in the future. This corresponds exactly to the phenomenon of phonological attunement described by Vihman (1996) for phonological development in babies.

9.2 Contribution to scenarios of the origins of speech As their name indicates, I have until now considered that the assumptions of the artificial system were given a priori. I have shown that they made it possible to generate in a population of agents speech codes sharing a certain number of fundamental properties with human speech codes. I will now question the possible provenance of the biological components characterized by these assumptions. The existence of perceptual and motor systems independent of each other is not very hard to explain. All mammals with ears necessarily have sets of neurons dedicated to processing sound, and others dedicated to the control of the organs of their mouth and vocal tract. As has been seen, the way motor control works in this study is very general, and common to all motor activity. In addition, the way in which the networks adapt in sensitizing themselves to the stimuli which activate them is characteristic of systems that try to represent the outside world in the brain. It seems that the real discussion applies rather to the connection between auditory perceptual structures and phonatory motor structures. The structure I have assumed enables agents to learn to retrieve the motor commands corresponding to the vocalization of another agent. Now, put like that, we should wonder how this capacity could have appeared. 9.2.1 An adaptationist scenario: an origin linked to the evolutionary advantage of linguistic communication systems First of all, it is very possible that the neural structure connecting auditory and phonatory neural systems appeared in the course of a Darwinian process of genetic evolution under pressure for a linguistic system that allows individuals to communicate verbally about a large number of things, and thus would have favoured the natural selection of neural structures enabling the development of a discrete and combinatorial speech code. The simulations I have

144


presented are completely compatible with this functionalist scenario. In this case, moreover, the simulations take on a crucial interest. On the one hand they show that the formation of an effective speech code in a population of agents does not require an explicit pressure for distinctive and shared vocalization systems, exercised on the cultural level, as in de Boer’s (2001) or Browman and Goldstein’s (2004) simulations. In these simulations, this cultural pressure is encoded by the fact that the agents have an inventory of sounds that they try to keep distinct from each other; this is not the case in my simulations, in which no ‘repulsive force’ is present, and where, moreover, the very concept of an inventory is not pre-programmed but emerges as an outcome of the simulations (in the movement from holistic vocalizations to discrete combinatorial vocalizations). De Boer also has a mechanism forcing inventories of sounds to grow in size, with the bias toward random invention of new sounds. In my simulations, no bias of this type is present. Yet more crucially, the artificial system of this book enables us to get an idea of how to solve the chicken-and-egg problem set out in Chapter 2. This problem has until the present yielded no answer, not even a very speculative one. Recall that it is a question of knowing how a convention like that of the speech code, which is necessary for the establishment of any linguistic communication, could be formed when no such communicative convention already existed. Indeed, earlier work, like that of de Boer (2001), but also that of Steels (1997), about other domains of language, showed how linguistic convention could be formed, based on the initial assumption that the agents could interact according to ‘language games’. Now such language games, whether it is an imitation game or a naming game, for example, are precisely already conventionalized systems of a complexity at least equal to that of the speech code, as detailed in Chapter 4. They involve the use of signals to indicate who is the speaker and who is the hearer, or knowledge of who should do what at which moment, as in the rules of a social game. These rules are complex, syntactic, and partly arbitrary (many equivalent variants can be imagined for each game). They are thus already (pre)linguistic forms of communication. So it is difficult to imagine how such games could be played without a system of forms (visual or acoustic and thus speech-like) enabling the transmission of information from one individual to another, regulating their interactions. The work I have presented in this book provides a possible answer to this question. Indeed, the artificial system that I built does not appeal to any language game, or any sort of social coordination. Thus, I showed how a conventional

New Scenarios

145

code (here of speech) could bootstrap itself without the need to assume any capacity for cultural interaction equivalent in complexity from an evolutionary point of view. This enables us to imagine how natural selection, in an environment favouring the reproduction of individuals capable of linguistic communication, could find biological structures paving the way for the bases of efficient linguistic communication systems. By contrast, the works of Steels (1997), de Boer (2001), and Kirby (1998)—even if they have shown very convincingly how languages could form and become more complex, given quasilinguistic capacities—did not make it easy to see how the assumed components of their agents could have appeared. This is why, in Chapter 4, I defended the idea that these works illuminate questions of the formation and evolution of particular languages, whereas the artificial system of the present work makes possible clearer hypothesis about the origin of language itself. In addition, the nature of the system’s component casts a very different light from the ideas proposed by the school of cognitive nativism of Pinker and Bloom (1990), for example. These authors propose that universal rules describing the phonetics and phonology of languages are hard-wired in a precise manner and pre-programmed in the genome. This means that the organization of sounds into a discrete system is pre-programmed into the maturation of the brain. Now this is very different from what happens in the simulations presented in this book: the neural circuits absolutely do not pre-programme a discrete shared speech code, and even less the preference for certain sounds or sound combinations. In fact, the power of the artificial system resides in the fact that, even if seen in a functionalist scenario in which the components have been selected by pressure of communication, it is very simple and general in nature. Indeed, especially for the connections between the perceptual and motor systems, there is no precise innate wiring, but rather a totally random innate wiring whose organization responds to a Hebbian law of adaptation, a law of a very general nature. The initial preferred vectors of the perceptual and motor nets are also themselves random. I showed that, despite these poorly organized circuits in the initial brains of the agents, a spontaneous structure, a speech code, could form in a population of agents. The idea thus given credence is that it was perhaps after all not so difficult for natural selection to find a genetic programme enabling agents to develop a speech code, in the case where this capacity has adaptive value. The simulation shows that it is not necessary to explore the immense space of complicated genetic programmes which would generate complex innate

146


neural structures like those proposed by the cognitive nativists. On the contrary, small manipulations of very simple cerebral structures, like the random connections between acoustic perceptual networks and phonatory networks, by adaptive changes following a Hebbian law, are sufficient. But the ease with which a discrete shared speech code can be generated in a population of agents enables us not only to set up the existing functionalist scenario, but also to generate new ones. I will describe two such new scenarios in the following sections.

9.2.2 Another adaptationist scenario, with the exaptation of discreteness, shared categorization, and combinatoriality It is possible that the structure connecting the perceptual and motor networks is the result of genetic evolution under a functional pressure for imitation. By imitation, I mean ‘the capacity of an individual to reproduce the behaviour of another individual when it perceives this behaviour’. Especially for sounds, it is a matter of the capacity to reproduce the sounds produced by other agents. It is interesting to remark that the capacity for imitation could have appeared in humans for reasons other than linguistic communication. For example, it could have developed for reasons of social cohesion, in which copying the behaviour of another agent could be a mark of clan membership, as is the case in many species of birds (Konishi, 1989). This is a form of primitive communication which has nothing to do with the linguistic communication of modern humans. In particular, this kind of communication in no way requires the presence of a shared phonemically coded repertoire, with units that can be systematically reused and recombined. Neither does it require a system for categorizing the space of sounds. In this kind of imitation behaviour, there is either copying of sounds or there is not, but there is no attempt to discriminate sounds taken in pairs from a repertoire. There is no need to have a system of categories or global differences if the only useful task is to judge the similarity between a sound produced by oneself and someone else at a given moment. Now the simplicity of the neural architecture used in the artificial system is such that it is difficult to imagine anything even simpler fulfilling the needs of this ‘social’ imitation task. It corresponds to an architecture that could have evolved by a Darwinian genetic process for this task. What is original is that the simulation shows that this neural structure self-organizes and produces ‘for

New Scenarios

147

free’ a discrete combinatorial speech code, in which phonemes are systematically reused according to precise phonotactic rules, and with a system of categories shared by all the agents in the same community. In fact, we have the wherewithal for speech without the need for speech. This speech code could obviously, however, be recruited once the linguistic function had appeared in the community of agents. In this scenario, the discreteness and combinatoriality of the speech code, which initially appeared as a side effect of a genetic evolution for imitation, would be used for a new function which played initially no role in its evolutionary formation: what we are considering here is an exaptationist scenario for the origins of discreteness, shared categorization, and combinatoriality, as suggested in Chapter 3. 9.2.3 An exaptationist scenario in which the origin of the whole speech system results from architectural side effects There is another possibility for the formation of this neural connection structure which makes it possible to learn to translate from an acoustic representation to a motor representation. I repeat that in order for the system to function, it is only necessary at the start to have perceptual neurons, each sensitive to random sounds, motor neurons whose activation produces a random articulatory configuration particular to each neuron, and random connections between these two sets of neurons with a Hebbian adaptive dynamic. Thereafter, the random uniform activation of motor neurons produces movements in the articulatory vocal apparatus, which produces sounds, which in return activates neurons in the perceptual network, and then the connections between the two networks self-organize in such a way that at a certain point translation between the two representations has been learnt. This is what is called babbling. This architecture requires no precise hard-wiring during ontogenesis, which is programmed by the genes. As a consequence, the connections of a very general nature which are necessary to the system, between the perceptual and motor networks, could also have emerged as a side effect of general architectural constraints applying to the development of the brain. Indeed, it is obvious that this type of connection between certain modalities like vision and motor control of the arms is very useful for the survival of the individual and has existed for a long time in mammals. It is probable that the most efficient and robust means of establishing these connections which are useful to the individual is massively to connect all modalities without distinction, and only afterwards to eliminate certain unused connections,

148


rather than to connect certain modalities precisely during ontogenesis. This is the phenomenon of architectural constraint developed in Chapter 3. This construction method could be more effective because it requires fewer genetic specifications for the growth process, and could therefore be more robust, and the advantage of this robustness could be greater than the metabolic cost of the a priori useless connections. Moreover, this way of conceiving the development of the brain, as was explained in Chapter 8 in justifying the dynamic of temporal neurons, is in the spirit of the selectionist theories of Changeux or Edelman. According to these theories, there would at first be massive generation of neurons and connections with a large random component, and then a shaping phase in which the inactivated neurons die off. However, if this scenario is right, there would seem to be no reason why other mammals, and in particular monkeys and apes, should not also have a brain whose development produces connections between acoustic perceptual networks and motor networks for the vocal tract. So why don’t they have a speech system like humans? Why aren’t their vocalizations organized like ours? It is in fact probable that they have connections that form during the growth of their brain. The answer can be found elsewhere: it is babbling that makes the difference. One of the assumptions of my artificial system that is realized in humans and not in other mammals is precisely this capacity to activate frequently, spontaneously, and randomly their vocal motor network. This means that the agents spontaneously try many articulatory configurations and systematically repeat these attempts. In other words, they ‘train themselves’. To my knowledge, no other mammal trains itself in this way in a motor activity. For example, once they have thrown a projectile at a target, monkeys and apes never try to repeat this activity spontaneously ‘in a vacuum’, making random variations by motor babbling (Coppens and Picq, 2001). It seems that this capacity to train oneself and to indulge in motor babbling is a fundamental evolutionary change contributing to the emergence of humans (Coppens and Picq, 2001). It obviously makes it possible to develop motor and perceptual talents which are very useful to the individual. Primitive humans must have engaged in all sorts of motor babbling. There must have been some motivation to explore the motor activities available to the body. This motivation, moreover, still exists, and is perfectly illustrated by the ‘body-babbling’ of infants. This motivation also drives individuals to explore their vocal space, and makes them babble vocally. This has the precise out-

New Scenarios

149

come of activating the connections between the acoustic perceptual networks and the motor networks for the vocal tract, and thus letting them survive in the sculpting phase of brain development. While these connections are certainly eliminated in large part in other mammals, they are preserved in humans. And from there we arrive exactly at the point of departure for the simulation presented in this work, which shows that a self-organization phenomenon is produced and generates ‘for free’ a system of sounds coded phonemically and shared by agents in the same community. This scenario thus involves two mechanisms for the creation of forms which are distinct from the ‘naive’ mechanism of ‘random variation of a trait’ and ‘selection by a functional criterion’ often invoked to explain the forms of living things (see Chapter 3). The first mechanism is that of side effect due to architectural constraints, promoted by Gould. Connections between acoustic perceptual networks and vocal motor networks are a side effect of the general development of the brain, in the same way as the space inside the shells of Gould’s snails (see Chapter 3) is a side effect of the growth mechanism of their shells. In both cases, structures at first have no function, and are only recruited later as a basis for speech or as a shelter for eggs. The second form-creating mechanism is what is characterized by the selforganization of neural structures woven into a speech code shared by a whole population. It is the same kind of mechanism as makes Bénard liquids organize into symmetrical cells or ferromagnetic plates magnetize themselves. In these examples, an ordered symmetrical structure (among agents or among cells) emerges as an effect of the local non-linear interaction of components which were not selected for the form which they together cause to appear. Indeed, just as Bénard cells appear without obviously having any adaptive value for a better replication of the liquid containing them, speech codes could be formed without initially having an adaptive value for a better replication of the organisms which generated them. This is even compatible with classical neoDarwinian theory, provided that one can show (as has been shown above) that each of the components that interacted to form speech codes could have appeared for adaptive reasons independent of speech.

10. Constructing for Understanding

The artificial system I have built showed how speech codes, sharing crucial properties with those of humans, could be formed in a population of agents in which the codes and their properties had not been pre-programmed. The agents are not even endowed with any social capacity. If communication is defined as the sending of a signal by an agent intended to modify the internal state or behaviour of another agent, then obviously the agents in the artificial system do not communicate. Nor is there any explicit pressure for linguistic communication which would push the agents to form repertoires of sounds contrasting with each other. Nor do they imitate each other, because they do not immediately reproduce the vocalizations that they hear and do not store them explicitly in memory so as to be able to reproduce them later. Nevertheless, thanks to the self-organizing properties of the complex system formed by the neural coupling between perceptual and motor modalities in each agent, and by the association among agents due to the simple fact that they inhabit an environment where they hear each other, the simulations showed that an organized system of vocalizations emerged spontaneously. While at the start they only produce anarchic, holistic, and inarticulate vocalizations, after several hundred interactions they produce discrete combinatorial vocalizations, with phonotactic rules, and conventionalized (all the agents in the same simulation share the same system of vocalizations at the end, and agents in different simulations generate different systems). Agents are even subject to phenomena of acoustic illusion formed and acquired culturally, as are humans. Finally, a statistical study of the repertoires of vocalizations allows us to find the same regularities as those of human languages, in particular as regards the vowels (and when morpho-perceptual constraints similar to those of humans are used). It can really be said that in the artificial system, speech is self-organized. Indeed, the components with which the agents are initially endowed are of a lower level of complexity than those of the speech codes that are generated. The properties characterizing these components, as well as their local

Constructing for Understanding

151

interactions, are qualitatively different from those characterizing the global structure formed by the speech code. The system’s dynamics illustrates how the same mechanism can form a whole complex set of structures constituting the speech code, starting with assumptions of a lower order of complexity. This enables us to imagine how natural selection could have ‘discovered’ the speech code, by showing that it was not necessary to find genomes which would code precisely for each of the complex structures of the speech code that we have set out, but only genomes coding for the fairly generic and very roughly pre-wired neural architecture of the agents. Indeed, if we put ourselves in the explanatory framework detailed in Chapter 3, the artificial system I have presented suggests an explanation for the speech code analogous to D’Arcy Thompson’s explanation of the hexagonal form of honeybees’ wax cells. I argued that, even if the emergence of speech was the result of a pressure for linguistic communication, i.e. is of evolution in an environment favouring the reproduction of humans who could linguistically communicate, merely noting that such a structure is very useful for communication is not a sufficient explanation. The discovery of this useful structure by the naive mechanism of ‘generation of random variants plus selection’ can certainly be far from trivial, and even impossible unless the space of forms is not constrained by self-organizational phenomena. This is why it is also crucial to explain how these structures could have been discovered, in particular by showing that the work of natural selection is reduced to finding a structure less complex than that of speech, but whose self-organization does the work of generating this complexity of speech. There already existed a number of proposed explanations for the structures of speech described here which relied on classical neo-Darwinian functionalist argumentation. Lindblöm (1992), for example, proposed that the statistical regularities in vowel inventories could be explained in terms of optimal perceptual distinctiveness, and thus in terms of their effectiveness in communication. Studdert-Kennedy (1998) proposed that phonemic coding enables transmission of information at a rate that drastically increases the power of communication, and thus that it is the result of an adaptation for communicating more effectively. These explanations are certainly right and important, but they are not sufficient. They do not say how these optimally organized systems could have been found. De Boer (2001) developed an artificial system making it possible to imagine how a population of agents, pre-programmed to try to construct

152


a shared repertoire of distinct sounds, could, in a cultural and decentralized manner, make a vowel system evolve adapted to this communication task. However, beside the fact that only two properties of the speech code are handled by this artificial system (the sharing of phonemes and the statistical regularities of their inventories), de Boer’s agents are already endowed at the start with very complex capacities. They are pre-programmed to try to imitate each other, to correct each other when they fail, and to maintain a repertoire of distinctive speech sounds. Their interactions are structured by the rules of the imitation game, which provides turn-taking in the game, varying roles (speaker/hearer), and signals for coordination and conveying to others whether imitation succeeds or not. Now, as I explained in detail, such capacities for social interaction already necessitate the existence of shared codes whose complexity could not be less than that characterizing an inventory of a few vowels. Thus vowel systems emerge in a population of agents who already have complex pre-linguistic and social capacities of at least the same order of complexity as the vowel systems themselves. De Boer’s artificial system is thus attacking the problem of the formation of particular languages. Moreover, de Boer (2001) presented crucial results about sound changes in the course of the cultural history of populations of agents. However, this system does not really attack the problem of the origin of the linguistic capacities given to the agents, i.e. of the origin of language. The linguistic capacities of these agents are complex, and it is difficult to see how they could have been formed by natural or cultural selection. In contrast, the artificial system I have presented in this book is based on assumptions which are of a notably lower order of evolutionary complexity than the speech code. It enables us to visualize how primitive forms of speech code could have appeared without the prior existence of structures for communication. So it is attacking the problem of the origin of language (to be precise, the origin of speech codes which are prerequisites of language). This artificial system is moreover completely complementary to de Boer’s. It is useful to take the view that the artificial system I developed shows how the first speech codes could have emerged at a time when there was not yet any linguistic communication, and that de Boer’s system shows how these codes, once they had been recruited for communication, could have been modified and shaped to adapt to the constraints of communication. My artificial system stands in the tradition inaugurated by Thompson in explaining the formation of hexagonal wax cells in beehives. He showed that it is not necessary that the bees have a precise plan in their heads, or the equivalent of a compass and rulers to realize

Constructing for Understanding

153

these shapes. It was sufficient that evolution happened upon a genome which made the bees pile up roughly circular cells, not too distorted, and of about the same size, for the pattern of hexagonal cells to self-organize all by itself.1 Likewise, this artificial system enables us to imagine how the pattern of the speech code could have organized itself, starting from the interaction of less complex biological components. The relative simplicity of the assumptions of the artificial system also lets us put forward other, original and surprising, hypotheses. In particular, we have seen that it is possible to explain the origin of each component independently of any linguistic communicative function. I explained that the crucial structure of connections between the auditory and vocal neural nets could be the result of a biological evolution under pressure for imitation, which can happen without pressure for linguistic communication. I also proposed an alternative scenario in which these connections might be a side effect due to architectural constraints imposed by the general construction of the brain. The combination of this side effect with the specifically human capacity to explore the motor space systematically and repetitively by babbling, biased by an innate motivation of very general curiosity, would have allowed these connective structures to survive during epigenesis,2 unlike in monkeys and apes, for example. Then the self-organization phenomenon described in this book would have unfolded, and speech codes would have appeared without there being any need for them. Only later would they have been recruited for communication. They are thus perhaps exaptations rather than adaptations. I have tried in this book to advance the structuration of the research on the origin of speech by relocating the problem in the wider context of the origin of forms in biology, and by constructing an artificial system exploring some of the complex dynamics which could take place during these formative processes. The relationship between self-organization and natural selection set out in Chapter 3 allows a better structuring of the research space of explanatory theories of speech, and in particular puts works like those of Lindblöm, Studdert-Kennedy, Browman and Goldstein, de Boer, and myself in perspective in the same frame of reference. On the other hand, I have shown how useful the construction of an artificial system can be in the process of de1 This does not rule out the possibility that a Baldwinian process later led to the incorporation in the genome of more precise coding for innate cognitive structures guiding the formation of hexagons even more exactly, as von Frisch (1974) proposed. 2 ‘Epigenesis’ is used here in the sense of the development of the organism as a consequence of the interactions between the gene instructions and the influences of the environment.

154


veloping explanatory theories. Indeed, the dynamics of my artificial system are such that it seems hard to conceive by purely verbal speculation how its premises could lead to the generation of speech codes. As in many other domains where complex dynamic systems are involved (see Chapter 5), the construction of such artificial systems is a necessity for developing our intuitions on the functioning of self-organizing mechanisms. Using a logic of abduction (Peirce, 1958[1931–5]), this can moreover be done at an early stage at the price of using speculative premises, even false premises, as long as they are useful. The premises of the artificial system presented here are obviously speculative; they are also plausible, but we do not know whether they are true or false. I have tried to show that they are useful because they enabled us to imagine plausible and original answers to crucial questions about the origin of speech which have remained until the present relatively unexplored. Construction of the system allowed us to fill out the existing functionalist theories, but also to open up new spaces for research and thinking by drawing the outline of an exaptationist theory. Research on the origins of speech is only just beginning, and there remains an enormous work of populating, organizing, and selecting in this space of theories.

Bibliography

AMEISEN, J. C. (2000) La sculpture du vivant: le suicide cellulaire ou la mort créatrice. Paris: Editions du Seuil. ANDRY, P., P. GAUSSIER, S. MOGA, J. P. BANQUET, and J. NADEL (2001) ‘Learning and communication in imitation: an autonomous robot perspective’, IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 31(5), pp. 431–44. ARBIB, M. (1995) The Handbook of Brain Theory and Neural Networks. Cambridge, Mass.: MIT Press. —— (ed.) (2000) The Handbook of Brain Theory and Neural Networks, 2nd edn. Cambridge, Mass.: MIT Press. BACHELARD, G. (1865) La formation de l’esprit scientifique. Paris: Vrin. ´ (1997) ‘Learning to speak: speech BAILLY, G., R. LABOISSI E` RE, and A. GALV AN production and sensory-motor representations’. In P. Morasso and V. Sanguineti (eds.), Self-Organization, Computational Maps and Motor Control, pp. 593–615. Amsterdam: Elsevier. BALDWIN, J. (1896) ‘A new factor in evolution’, American Naturalist, 30, pp. 441–51. BALL, P. (2001) The Self-Made Tapestry: Pattern Formation in Nature. Oxford: Oxford University Press. BATALI, J. (1998) ‘Computational simulations of the emergence of grammar’. In Hurford et al. (1998: 405–26). BERNARD, C. (1945) Introduction a` l’étude de la médecine expérimentale. ´ Geneva: Editions du cheval ailé. BICKFORD, J. A. and D. TUGGY (2002) ‘Electronic glossary of linguistic terms (with equivalent terms in Spanish)’, http://www.sil.org/mexico/ling/glosario/ E005ai-Glossary.htm. BINMORE, K. (1992) Fun and Games: A Text on Game Theory. Lexington, Mass.: D.C. Heath. B OE, L. J., J. L. SCHWARTZ, and N. VALL E´ E (1995) ‘The prediction of vowel systems: perceptual contrast and stability’. In E. Keller (ed.), Fundamentals of Speech Synthesis and Recognition, pp. 185–213. Chichester: John Wiley. DE B OER , B. (2001) The Origins of Vowel Systems. Oxford: Oxford University Press.

156

Bibliography

BROWMAN, C. P. and L. GOLDSTEIN (1986) ‘Towards an articulatory phonology’. In C. Ewan and J. Anderson (eds.), Phonology Yearbook 3, pp. 219–52. Cambridge: Cambridge University Press —— —— (2000) ‘Competing constraints on intergestural coordination and selforganization of phonological structures’, Bulletin de la communication parlée, 5, pp. 25–34. CAMAZINE, S., J. L. DENEUBOURG, N. R. FRANKS, J. SNEYD, G. THERAULAZ, and E. B ONABEAU (2001) Self-Organization in Biological Systems. Princeton, NJ: Princeton University Press. CANGELOSI, A., and D. PARISI (eds.) (2002) Simulating the Evolution of Language. London: Springer. CARLSON, R., GRANSTROM, B., and FANT, G. (1970) ‘Some studies concerning perception of isolated vowels’, Speech Transmission Laboratory Quarterly Progress and Status Report, Department of Speech Communication and Music Acoustics, KTH, Stockholm, 2/3, pp. 19–35. CHALMERS, A. (1991) La fabrication de la science. Paris: La Découverte. CHANGEUX, J., and A. DANCHIN (1976) ‘The selective stabilization of developing synapses: a plausible mechanism for the specification of neuronal networks’, Nature, 264, p. 705. CHAUVET, G. (1995) La vie dans la matiére: le rôle de l’espcase en biologie. Paris: Flammarion. CHOMSKY, N. (1975) Reflections on Language. New York: Pantheon. —— and M. HALLE (1968) The Sound Pattern of English. New York: Harper and Row. COPPENS, Y., and P. PICQ (2001) Aux origines de l’humanité, i: De l’apparition de la vie a` l’homme moderne. Paris: Fayard. CROTHERS, J. (1978) ‘Typology and universals of vowels systems’. In J. H. Greenberg, C. A. Ferguson, and E. Moravcsik (eds.), Universals in human language, ii: Phonology, pp. 93–152. Stanford, Calif.: Stanford University Press. DARWIN, C. (1999[1859]) On the Origin of Species. New York: Signet. DAWKINS, R. (1982) The Extended Phenotype. Oxford: Oxford University Press. DAYAN, P., and L. F. ABBOT (2001) Theoretical Neuroscience: Computational and Mathematical Modeling of Neural Systems. Cambridge, Mass.: MIT Press. DUDA, R., P. HART, and D. STORK (2001) Pattern Classification. Chichester: Wiley. EDELMAN, G. M. (1993) ‘Neural Darwinism: selection and reentrant signaling in higher brain function’, Neuron, 10, pp. 15–125. EINSTEIN, A. (1955[1922]) The Meaning of Relativity. Princeton, NJ: Princeton University Press. —— and L. INFELD (1967[1938]) The Evolution of Physics. New York: Simon and Schuster. ELDREDGE, N., and S. J. GOULD (1972) ‘Punctuated equilibria: an alternative to phyletic gradualism’. In T. J. M. Schopf (ed.), Models in Paleobiology, pp. 82–115. San Francisco: Freeman.

Bibliography

157

ESCUDIER, P., and J.-L. SCHWARTZ (eds.) (2000) La parole. Paris: Hermès Sciences. FEYERABEND, P. (1979) Contre la méthode: esquisse d’une théorie anarchiste de la connaissance. Paris: Seuil. FREEMAN, W. J. (1978) ‘Spatial properties of an EEG event in the olfactory bulb and cortex’, Electroencephalography and Clinical Neurophysiology, 44, pp. 586–605. FRENCH, R. M., and A. MESSINGER (1994), ‘Genes, phenes and the Baldwin effect: learning and evolution in a simulated population’. In R. A. Brooks and P. Maes (eds.), Artificial Life IV: Proceedings of the Fourth International Workshop on the Synthesis and Simulation of Living Systems. Cambridge, Mass.: MIT Press, pp. 277–82. VON F RISCH , K. (1974) Animal Architecture. London: Hutchinson. GEORGOPOULOS, A. P., R. E. KETTNER, and A. B. SCHWARTZ (1988) ‘Primate motor cortex and free arm movements to visual targets in three-dimensional space. II: Coding of the direction of movement by a neuronal population’, Journal of Neuroscience, 8, pp. 2928–37. GHOSH, A. (1996) ‘Cortical development: with an eye on neurotrophins’, Current Biology, 6, pp. 130–3. GINTIS, H., E. ALDEN, and S. B OWLES (2001) ‘Costly signaling and cooperation’, Journal of Theoretical Biology, 213, pp. 103–19. GLASERSFELD, E. (2001) ‘The radical constructivist view of science’. In A. Riegler (ed.), Foundations of Sciences, 6 (1–3), pp. 31–43. GOLD, E. (1967) ‘Language identification in the limit’, Information and Control, 10, pp. 447–74. GOLDSTEIN, L. (2003) ‘Development of phonology’, http://www.ling.yale. edu:16080/ling165/. GOULD, S. J. (1982) Le pouce du panda. Paris: Grasset. —— (1997) ‘The exaptive excellence of spandrels as a term and prototype’, Proceedings of the National Academy of Science USA, 94, pp. 10750–5. —— and E. S. VRBA (1982) ‘Exaptation: a missing term in the science of form’, Paleobiology, 8, pp. 4–15. GUENTHER, F. H., and M. N. GJAJA (1996) ‘The perceptual magnet effect as an emergent property of neural map formation’, Journal of the Acoustical Society of America, 100, pp. 1111–21. GUILLAUME, P. (1925) L’imitation chez l’enfant. Paris: Alcan. HARNAD, S (1987) ‘Psychophysical and cognitive aspects of categorical perception: a critical overview’. In S. Harnd (ed.), Categorical Perception: The Groundwork of Cognition, pp. 1–52. Cambridge: Cambridge University Press. HINTON, G. E., and S. J. NOWLAN (1987) ‘How learning can guide evolution’, Complex Systems, 1, pp. 495–502. HURFORD, J., M. STUDDERT-KENNEDY, and C. KNIGHT (1998) Approaches to the Evolution of Language: Social and Cognitive Bases. Cambridge: Cambridge University Press.

158

Bibliography

JOHNSON, M. H. (to appear) ‘Developmental cognitive neuroscience: relating brain development to cognitive and behavioral change’. In D. Messer and S. Millar (eds.), Developmental Psychology. London: Edward Arnold. KANDEL, E., J. SCHWARTZ, and T. JESSELL (2000) Principles of Neural Science, 4th edn. New York: McGraw-Hill. KANEKO, K., and I. TSUDA (2000) Complex Systems: Chaos and Beyond—–a Constructive Approach with Applications in Life Sciences. New York: Springer. KAPLAN, F. (2001) La naissance d’ une langue chez les robots. Paris: Hermès Science. KAUFFMAN, S. (1996) At Home in the Universe: The Search for Laws of SelfOrganization and Complexity. Oxford: Oxford University Press. KEEFE, A., and J. SZOSTAK (2001) ‘Functional proteins from a randomsequence library’, Nature, 410, pp. 715–18. KIRBY, S. (1998) ‘Fitness and the selective adaptation of language’. In Hurford et al. (1998: 359–83). KITANO, H. (2002) ‘Systems biology: a brief overview’, Science, 295, pp. 1662–4. —— (2004) ‘Cancer as a robust system: implications for anticancer therapy’, Nature Reviews Cancer, 4(3), pp. 227–35. KOBAYASHI, T., and T. KURODA (1987) Morphology of Crystals. Tokyo: Terra Scientific. KOHONEN, T. (1982) ‘Self-organized formation of topologically correct feature maps’, Biological Cybernetics, 43, pp. 59–69. KOLB, B., and I. Q. WISHAW (2001) An Introduction to Brain and Behaviour. New York: Worth. KONISHI, M. (1989). ‘Birdsong for neurobiologists’, Neuron, 3, pp. 541–9. KREBS, J. R., R. ASHCROFT, and M. WEBER (1978) ‘Song repertoires and territory defense in the great tit’, Nature, 271, pp. 539–42. ¨ KUHL, P. K., K. A. WILLIAMS, F. LACERDA, K. N. STEVENS, and B. LINDBL OM (1992) ‘Linguistic experience alters phonetic perception in infants by 6 months of age’, Science, 255, pp. 606–8. KUHN, T. S. (1970) The Structure of Scientific Revolutions. Chicago: University of Chicago Press. KUPIEC, J.-J., and SONIGO, P. (2000) Ni Dieu ni géne pour une autre théorie de l’hérédité. Paris: Seuil. LABOV, W. (1994) Principles of Linguistic Change, i: Internal Factors. Oxford: Basil Blackwell. LADEFOGED, P., and I. MADDIESON (1996) The Sounds of the World’s Languages. Oxford: Blackwell. LANGTON, C. (1995) Artificial Life: An Overview. Cambridge, Mass.: MIT Press. LIBERMAN, A. M., and I. G. MATTINGLY (1985) ‘The motor theory of speech perception revised’, Cognition, 21, pp. 1–36.

Bibliography

159

¨ , (1972) ‘Numerical simulations of vowel LILJENCRANTS, L., and B. LINDBL OM quality systems: the role of perceptual contrast’, Language, 48, pp. 839–62. ¨ , B. (1992) ‘Phonological units as adaptive emergents of lexical LINDBL OM development’. In C. Ferguson, L. Menn, and C. Stoel-Gammon (eds.), Phonological Development: Models, Research, Implications. Timonnium, Md.: York Press, pp. 565–604. MACNEILAGE, P. F. (1998) ‘The frame/content theory of evolution of speech production’, Behavioral and Brain Sciences, 21, pp. 499–548. MADDIESON, I. (1984) Patterns of Sound. Cambridge: Cambridge University Press. MATARIC, M., M. WILLIAMSON, J. DEMIRIS, and A. MOHAN (1998) ‘Behaviorbased primitives for articulated control’. In Proceedings of the Fifth International Conference of the Society for Adaptive Behavior, pp. 165–70. Cambridge, Mass.: MIT Press. MEHLER, J., A. CHRISTOPHE, and F. RAMUS (2000) ‘What we know about the initial state for language’. In A. Marantz, Y. Miyashita, and W. O’Neil (eds.), Image, Language, Brain: Papers from the First Mind–Brain Articulation Project Symposium, pp. 51–75. Cambridge, Mass.: MIT Press. MORGAN, C. L. (1896) ‘On modification and variation’, Science, 4, pp. 733–40. MRAYATHI, M., R. CARR E´ , and B. GU E´ RIN (1988). ‘Distinctive region and modes: a new theory of speech production’, Speech communication, 7, pp. 257–86. NAKANISHI, A. (ed.) (1980) Writing Systems of the World: Alphabets, Syllabaries, Pictograms. Rutland, Vt.: Tuttle. NICOLIS, G., and I. PRIGOGINE (1977) Self-Organization in Nonequilibrium Systems: From Dissipative Structures to Order through Fluctuations. New York: Wiley. VAN O OYEN , A., J. VAN PELT, M. A. C ORNER , and S. B. KATER (2003) ‘Activitydependent neurite outgrowth: implications for network development and neuronal morphology’. In A. van Ooyen (ed.), Modeling Neural Development. Cambridge, Mass.: MIT Press. OUDEYER, P.-Y. (2001a) ‘Coupled neural maps for the origins of vowel systems’. In G. Dorffner, H. Bischof, and K. Hornik (eds.), Proceedings of the International Conference on Artificial Neural Networks 2001, pp. 1171–6. Berlin: Springer. —— (2001b) ‘The epigenesis of syllable systems: a computational model’. In Proceedings of ORAGE 2001, Paris: L’Harmattan. —— (2001c) ‘The origins of syllable systems: an operational model’. In J. Moore and K. Stenning (eds.), Proceedings of the 23rd Annual Conference of the Cognitive Science Society, pp. 744–9. Hillsdale, NJ: Laurence Erlbaum. —— (2001d) ‘Origins and learnability of syllable systems: a cultural evolutionary model’. In P. Collet, C. Fonlupt, J. K. Hao, E. Lutton, and M. Schoenauer (eds.), Artificial Evolution, pp. 143–55. Berlin: Springer. —— (2002a) ‘Phonemic coding might be a result of sensory-motor coupling dynamics’. In B. Hallam, D. Floreano, J. Hallam, G. Hayes, and J-A. Meyer (eds.),

160

Bibliography

Proceedings of the 7th International Conference on the Simulation of Adaptive Behavior, pp. 406–16. Cambridge, Mass.: MIT Press. —— (2002b) ‘A unified model for the origins of phonemically coded syllables systems’. In W. Gray and C. Schunn (eds.), Proceedings of the 24th Annual Conference of the Cognitive Science Society, pp. 738–43. Hillsdale, NJ: Laurence Erlbaum. —— (2003) ‘The social formation of acoustic codes with “something simpler” ’. In K. Dautenham and C. Nehaniv (eds.), Proceedings of the Second International Conference on Imitation in Animals and Artefacts. Aberystwyth: University of Wales. —— (2005a) ‘From holistic to discrete speech sounds: the blind snow-flake maker hypothesis’. In M. Tallerman (ed.), Language Origins: Perspectives on Evolution, pp. 68–99. Oxford: Oxford University Press. —— (2005b) ‘Aux sources du langage: l’auto-organisation de la parole’, Cahiers romans de sciences cognitives, cuadernos romances de ciencias cognitivas, Cadernos românicos em ciências cognitivas, Quaderni romanzi di scienze cognitive, In cognito, 2(2), pp. 1–24. —— (2005c) ‘The self-organization of speech sounds’, Journal of Theoretical Biology, 233(3), pp. 435–49. —— (2005d) ‘The self-organisation of combinatoriality and phonotactics in vocalisation systems’, Connection Science, 17(3–4): 325–41. —— (2005e) ‘How phonological structures can be culturally selected for learnability’, Adaptive Behavior, 13(4), pp. 269–280. PEIRCE, C. S. (1958[1931–5]) Collected papers. Vols. i–vi, eds. H. Charles and P. Weiss; Vols. vii, viii, ed. A. Burks. Cambridge, Mass.: Harvard University Press. PINKER, S., and P. BLOOM (1990), ‘Natural language and natural selection’, Brain and Behavioral Sciences, 13, pp. 707–84. POPPER, K. (1984[1959]) La logique de la découverte scientifique. Paris: Payot. REDFORD, M. A., C. C. CHEN, and R. MIIKKULAINEN (2001) ‘Constrained emergence of universals and variation in syllable systems’, Language and Speech, 44, pp. 27–56. RIZZOLATTI, G., and M. ARBIB (1998) ‘Language within our grasp’, Trends in Neuroscience, 21, pp. 188–94. —— L. FADIGA, V. GALLESE, and L. FOGASSI (1996) ‘Premotor cortex and the recognition of motor actions’, Cognitive Brain Research, 3, pp. 131–41. SALINAS, E., and L. F. ABBOTT (1994) ‘Vector reconstruction from firing rates’, Journal of Computational Neuroscience, 1, pp. 89–107. SCHWARTZ, J. L., L. J. B OE, N. VALL E´ E, and C. ABRY (1997a) ‘The dispersion/ focalization theory of vowel systems’, Journal of Phonetics, 25, pp. 255–86. —— —— —— —— (1997b) ‘Major trends in vowel systems inventories’, Journal of Phonetics, 25, pp. 233–53.

Bibliography

161

SEJNOWSKY, T. (1977) ‘Storing covariance with non-linearly interacting neurons’, Journal of Mathematical Biology, 4, pp. 303–12. SEKULER, R, and R. BLAKE (1994) Perception. New York: McGraw-Hill. SHANNON, C. (1948) ‘A mathematical theory of communication’, Bell System Technical Journal, 27, pp. 379–423, 623–56. STEELS, L. (1997) ‘The synthetic modeling of language origins’, Evolution of Communication, 1(1), pp. 1–35. —— (2001) ‘The methodology of the artificial’, Behavioral and Brain Sciences, 24(6), pp. 1077–8. STEVENS, K.N. (1972) ‘The quantal nature of speech: evidence from articulatoryacoustic data’. In D. David (ed.), Human Communication: A Unified View, pp. 51–66. New York: McGraw-Hill. STUDDERT-KENNEDY, M. (1998) ‘The particulate origins of language generativity: from syllable to gesture’. In Hurford et al. (1998: 202–21). —— (2005) ‘How did language go discrete?’ In M. Tallerman (ed.), Language Origins: Perspectics on Evolution, pp. 48–67. Oxford: Oxford University Press. —— and L. GOLDSTEIN (2003) ‘Launching language: the gestural origin of discrete infinity’. In M. Christiansen and S. Kirby (eds.), Language Evolution: The States of the Art, pp. 235–54. Oxford: Oxford University Press. THOMPSON, D. (1961[1917]) On Growth and Form. Cambridge: Cambridge University Press. TRITTON, D. J. (1988) Physical Fluid Dynamics. Oxford: Oxford University Press. VALL E´ E, N. (1994) ‘Systèmes vocaliques: de la typologie aux prédictions’. Doctorat en sciences du langage, Université Stendhal, Grenoble, France. —— L.-J. B OE, J.-L. SCHWARTZ, P. BADIN, and C. ABRY (2002) ‘The weight of phonetic substance in the structure of sound inventories’, ZAS Papers in Linguistics, 28, pp. 145–68. VIHMAN, M. (1996) Phonological Development: The Origins of Language in the Child. Cambridge, Mass.: Blackwell. WALDROP, M. (1990) ‘Spontaneous order, evolution, and life’, Science, 247, pp. 1543–5. WILLIAMS, G. (1996) Adaptation and Natural Selection: A Critique of some Current Evolutionary Thought. Princeton, NJ: Princeton University Press. WOLFRAM, S. (2002) A New Kind of Science. Canada: Wolfram Media.


Index

Figures and notes are indexed as f and n in bold. Where two figures appear on one page, they are indexed consecutively, e.g. 37f(a) or 37f(b). σ 77, 78f, 86–87, 91–94, 94f, 95f, 96 Abbott, L. F. 100, 140 abduction 69, 154 Abry, C. 25, 57, 117, 119 acoustics 10, 21, 24, 54, 65, 111, 146, 148, 149 continuum 22 forms 14 illusions 25, 28, 97–105 modifications 54 representations 21f, 76, 107, 147 signals 77–8 space 65, 77, 118–19, 127 trajectories 24, 30, 77–8, 107, 108, 112, 124 activations 79–80, 84f, 99–101, 102f, 104, 104f, 107, 108, 109–10, 114, 124, 125–6, 128, 129, 131–4, 136, 141, 147, 149; see also muscles, activations function 76–7, 78f, 79, 82, 83, 100, 104, 106, 107, 108, 113, 124, 125, 140

adaptationism 11, 11n, 13, 56, 56n, 91, 100, 109, 111, 140–1, 142–7, 149, 151, 153 Alden, E. 51 allophones 24, 25, 115–16 alphabets 22 Ameisen, J.-C. 126, 142 American 26f Andry, P. 72 apoptosis 126, 142 Arbib, M. 140, 141 architecture 106, 147–9, 151 constraints 48–9, 50–2, 67, 139, 147–8, 149, 153 articulations 10, 12, 16–21, 22, 24, 25, 30, 55, 56, 64, 75, 85, 87, 109, 110–11, 113–14; see also speech bias 110, 114f, 133–4 configurations 53–4, 85–6, 96, 97f(a) 106, 110–12, 135, 141, 147, 148 constraints 124, 133–8 continuum 22, 30, 63–4 costs 57 ease 63

164

Index

articulations (continued) organs 54–5, 142 phonology 16–17, 20–1, 21f, 75, 84f space 54, 59–60, 111 synthesizer 110, 111–12, 113–14, 114f, 115 targets 17, 24, 30, 65, 75, 76n, 78, 78f, 81, 82, 83, 84, 84f, 84–5, 87, 87f, 88, 88f, 97, 97f(b), 98, 99, 107, 111, 123, 124, 125, 127, 128, 133, 135, 136f, 137f, 142 trajectories 30, 53, 80, 107, 135 articulatory/perceptual non-linearities 137–8 artificial systems 9–10, 11–12, 30, 36, 59, 66, 67, 68, 69–70, 73n, 74, 75–9f, 80–105, 120–1, 121f(b), 122, 139, 143, 144–6, 148, 150, 151–154 construction 67–74 Ashby, W. R. 1 assumptions 9, 10, 11, 12, 54, 62–3, 65, 72–4, 76, 83, 86–7, 106, 126, 139, 140, 143, 144, 148, 152, 153 atoms 35–6, 37f(a) attractors 32, 35, 104, 142 landscape 41, 102, 120 attunement game 63–4 auditory neural systems 143, 153 babbling 79, 108, 109, 148–9, 153 Bachelard, G. 68 Badin, P. 25

Bailly, G. 76, 78n, 79 Ball, P. 3, 5, 33, 48 Baldwin, J. 48n, 73 Baldwin effect 48n, 73–4n, 153n Ball, P. 38, 44 Bamquet, J. P. 72 Barks 112, 113 basilar membrane 15, 15f, 17f basins of attraction 41, 42, 102, 103f, 104, 105f, 114f, 118f Batali, J. 10, 68 beehives 45–7, 47f, 52, 67, 152–3 Beloutzov–Zabotinsky reaction 48, 49f Bénard cells 32, 32n, 40–2, 44, 94 temperature 35, 35f, 40–1 Bénard Convection 32 Bénard liquids 34f, 149 Bernard, C. 68 biased distribution 85, 97f(a) Bickford, J. A. 19 bifurcation 37f(b), 38 biology 3, 5, 38, 50, 68, 73–4, 145 components 143, 153 forms 40, 43–4, 153 bipedalism 29, 42–3 birds 50–1, 146 Blake, R. 16 Bloom, P. 53, 62, 145 Boë, L.-J. 25, 57, 117, 119 Bonabeau, E. 3 Bowles, S. 51 brains 5, 6, 7, 8, 20, 21, 21f, 57, 62, 65, 99, 103, 112, 126, 137–8, 139, 140, 141–3, 145, 147, 148–9, 153

Index bridges 50 Browman, C. 16, 55, 56, 63, 64, 65, 66, 75, 144, 153 Camazine, S. 3, 38 Cangelosi, A. 10n, 68, 86 Carlson, R. 112 Carré, R. 54 categorical perception 100 categorization 10, 97–105, 102, 103, 104, 104f, 105, 110, 113, 115f, 120, 138, 142, 146–7 causal explanations 55 cautionary tale 73 cells 41–2, 44, 47, 71, 149, 153; see also hexagonal cells; square cells; stem cells division 48, 49f Chalmers, A. 68 Changeux, J.-P. 126, 141, 142, 148 Chauvet, G. 5 Chen, C. C. 57, 134 Chinese 29 Christophe, A. 106 classification system 24–5 clusters 87–8, 88f, 89f, 90, 90f(a), 91, 96, 97(b), 98, 99, 100, 102, 104, 113, 115f, 117(b), 120, 124, 128, 129, 130–3, 137 see also modes cochlea 14, 15, 16f, 21, 53, 63, 78, 79, 81, 96, 107–8, 112 codes see speech, codes coding 44, 78, 79, 106, 109, 134, 135, 151, 153n see also decoding

165

coding/decoding 103f, 113, 120 cognitive nativism 53, 145–6 combinatoriality 10, 12–13, 22, 24, 30, 54, 55, 63, 66, 85–6, 88, 92, 98–9, 146–7, 150; see also speech, codes strong 123–38 commands see gestures communication 14, 29, 43, 51, 56, 58–9, 61, 64–7, 86, 122, 138, 139, 143, 145, 150, 151–3; see also linguistic communication complexity 1, 9, 30, 30n, 36, 42n, 44, 52, 60, 130, 139, 145, 150–2; see also self-organization evolutionary 66, 66n sciences of 1–2, 5, 9 systems 3, 144, 150 theory 5, 6 computational theories 71–2 computers; see also simulations modelling 9–13 programs 75 connection weights 108–9 consonants 18–19, 22, 23f(b), 25, 26f, 27f(a), 28–9, 58, 124, 130 constrictions 17, 19, 19f, 20–1, 22, 28, 30, 55, 63–4 variables 17, 18, 18f constructivism 68, 70 control systems 83, 84f, 103, 106

166

Index

convection currents 33, 33f, 34f coordinated interactions 64, 86; see also interactions Coppens, Y. 148 coupling: perceptual and motor modalities 150 production and perception 92, 110, 138 Crothers, J. 118 crystallization 1, 2f, 3f, 87, 88f, 89f, 90, 91, 113 culture 5–6, 10, 100, 144, 150, 152 evolution 9, 122 selection 62, 66 Danchin, A. 126, 141, 142 Darwin, C. (1809–1882) 6, 73 Darwinian reasons 48, 143, 146–7 Dawkins, R. 1, 46n, 51 Dayan, P. 140 de Boer, B. 10, 29, 59, 60, 61, 62, 63, 64, 66, 66n, 68, 77, 86, 111, 112, 120, 122, 138, 144, 145, 151, 152, 153 decoding 8, 99, 100–2, 102f, 103, 103f, see also coding; population vector decomposition 15–16, 16f, 17f Demiris, J. 142 Deneubourg, J. L. 3, 38 development 29, 44 disciplines 8 discreteness 22, 54, 55, 63–4, 85–6, 92, 96, 98–9, 146–7;

see also speech, codes systems 10, 22, 63, 92, 145, 146, 150 vocalizations 66, 88, 150 Dispersion-Focalization theory 57 dissipative systems 32, 34, 92 Distinctive Region Model 54 distinctive speech sounds 152; see also speech, sounds distributions 87, 88, 88f, 88n, 89f, 89n, 90, 90f(a), 90f(b), 91, 93, 94f, 97f(b), 101, 110, 113, 114f, 117f(b), 120, 121f(a), 121f(b), 133, 134, 134f see also uniform distributions diversity 30, 88, 96, 114, 121, 122 DNA 41–2 Duda, R. 62, 88, 88n, 89n dynamic systems 9–10, 32, 35, 38, 41–2, 44, 68, 92, 99, 104, 106, 142, 154 dynamic vocalizations 81; see also vocalizations dynamics 86–97, 113–22, 137–8, 139, 140–2, 142–3, 147, 148, 151, 153–4 ears 8, 14, 15, 106, 110, 112, 143 ecosystem 5 Edelman, G. 103, 141, 142, 148 edge of chaos 36, 94 Einstein, A. 70, 71 Eldredge, N. 41 energetic constraints 124, 133–8 energy 56, 61, 111 costs 57–8, 134–5, 137–8

Index

167

flow 32, 34, 47 vowel systems 57 English 25, 28, 29, 32n, 54, 123 entropy 88, 89n, 90, 90f(a), 93, 94f, 117f(b) epigenesis see neural epigenesist equilibrium states 33, 34, 35, 36, 41, 87, 91, 92, 110, 113 Escudier, P. 22, 27, 119 evolution 7, 145, 146–7, 148, 151, 153 explanations 42–8, 51, 74 theory 7, 41, 42–8 exaptations 11, 11n, 48–51, 67, 139, 146, 147–9, 153–4

creation 12, 39, 149 living 12, 31, 51–2, 149 organisms 40, 41, 42 phenotypes 40, 42 repertoires 1, 10, 14, 61 Fourier analysis 15, 16f Franks, N. R. 3, 38 Freeman, W. 142 French 123 French, R. M. 72 function mapping 85–6, 97(a), 106, 111–12, 133 functionalism 42, 50, 55, 139, 145, 146, 147, 151, 154 speech sounds 56–8 fuzzy binning technique 88n

Fadiga, L. 141 Fant, G. 112 feedback loops 100, 103, 103f; see also positive feedback loops ferro-magnetization 35–7f(a), 37f(b), 38, 41, 87, 93–4, 149 Feyerabend, P. 68 fishes 44, 46 fixed point see attractors Fogassi, L. 141 formants 57, 57n, 59–60, 78, 107, 108, 111–13, 116f(a), 116f(b), 117f(a), 119f(b) formation 29, 48, 49, 51, 55, 59, 60, 122, 124, 131, 133, 134, 137, 144, 147, 152–3 languages 61, 65–66, 145, 152 forms 32, 36, 42, 44, 48, 53, 58, 144, 151, 152 biology 40, 43–4, 153

Gallese, V. 141 Galván, A. 76, 78n, 79 Gaussier, P. 72, 82 Gaussian functions 77, 78f, 88n, 91, 93, 93f, 94, 100, 104, 113, 140 genes 39–40, 41, 44, 48, 51, 67, 73, 147 genetics 7, 9, 73 genomes 7, 8–9, 40, 41, 42, 44, 52, 53, 73, 145, 151, 153 genotypes 73–4 space 40, 43–4, 47 Georgopoulos, A. P. 99, 100 gestures 16–17, 18, 18f, 19, 20, 21, 22, 24, 25, 28, 30, 55, 63, 64, 65, 75, 76, 83, 84f, 85, 92 see also discreteness; relations between organs commands 65

168

Index

gestures (Continued) continuum 24 forms 14 scores 19, 20, 20f(a), 20f(b), 21, 22 space 24 representations 21, 21f, 24, 106 tracks 54, 54f, 55–6 Ghosh, A. 126, 135 Gintis, H. 51 Gjaja, M. N. 100, 141 Glasersfeld, E. 68, 70 glottis 14, 15, 15f, 19 GO signal 80, 83, 106, 125 Gold, E. 62 Goldstein, L. 15, 16, 18, 20, 21, 54, 55, 63, 64, 65, 66, 75, 144, 153 Gould, S. J. 41, 48, 49, 50, 149 Granstrom, B. 112 Guenther, F. 100, 141 Guérin, B. 54 Guillaume, P. 72 Harnad, S. 100 harmonics 15, 16f, 17f Hart, P. 62, 88, 88n, 89n Hebbian rule 109, 140, 141, 145–7 hexagonal cells 45–7, 48, 67, 151, 152–3 hexagonal shapes 47, 52 hexagonal tessellation 47, 47f Hinton, G. 73, 73n humans 3, 9, 65, 137, 138, 140, 141, 148–9, 150, 151, 153;

see also brains; speech, codes languages 10–11, 12, 28, 53, 59, 96–7, 115, 120, 121f(b), 130, 134, 137, 139, 140, 150 systems 11, 12, 74, 113, 120–3 imitation 55, 59–60, 72, 84, 146–7, 150, 152, 153 games 59, 61, 63, 64, 66, 86, 138, 152, 144, 152 implementations 53, 69 Infeld, L. 71 information 14, 43, 51, 54, 60, 86, 99, 103, 112, 113, 144, 151 insect societies 3–4, 38, 51; see also self-organization, termite nests interactions 72f, 86, 93, 113, 129–30, 131f, 132f, 133, 142, 144–5, 149, 150, 152, 153 interdisciplinarity 7–9 interpolations 83, 111, 135; see also polynomial interpolations iron plates 35–6, 41 Japanese 25, 26f, 28, 54, 124, 130 jaw 17, 123 Jessell, T. 75, 83n, 84, 140, 142 Kandel, E. 75, 83, 84, 140, 142 Kaneko, K. 142 Kaplan, F. 10, 29, 59, 61, 68 Kauffman, S. 1, 3, 94 Keefe, A. 40 Kirby, S. 10, 59, 61, 68, 145 Kitano, H. 5

Index Kohonen, T. 140 Kolb, B. 17 Konishi, M. 146 Kuhl, P. K. 25, 26f, 100, 102 Kuhn, T. S. 68, 70 Kullback–Leibler distance 88, 89n Kupiec, J.-J. 5 Laboissière, R. 76, 78n, 79 Lacerda, F. 25, 26f, 100, 102 Lamarckism 73 landscape see attractors, landscape Langton, C. 68 Language Acquisition Device 53 languages 5–7, 9, 14, 21, 53, 54, 62, 68, 122, 123, 124, 138, 139, 144, 145; see also formation, language; humans, languages evolution 6, 7, 10, 11, 29, 62 faculty 7–8 families 115, 119f games 59–60, 61, 66, 86, 144–5 origins 6–7, 9, 10, 11, 61, 64, 67, 68, 145, 152 larynx 8, 14–15, 18, 19f, 22 learnability 61 learning 61, 62, 72, 73, 85, 87, 109 Liberman, A. M. 21 Liljencrants, L. 57 Lindblöm, B. 25, 26f, 43, 57, 58, 59, 61, 63, 100, 102, 134, 151, 153 linear mapping 40, 96 linguistic communication 11, 13, 24–5, 52, 67, 139,

169

143–6, 150, 151, 152, 153; see also communication linguistics 68, 123 lips 14, 16–18, 19, 19f, 22, 54, 75, 87, 111 liquids 42 heating 32–3, 33f temperature 33, 33f, 34, 34f, 40–1 living forms 12, 31, 51–2 living organisms 31, 39–40, 41, 44 MacNeilage, P. 123 Maddieson, I. 22, 54, 115 magnetization 37f(b) Mataric, M. 142 mathematical evaluations 8, 9 Mattingly, I. G. 21 mechanisms 32, 38–9, 42n, 47, 51, 59, 66, 69, 70, 71, 74, 75–6, 84, 87, 91, 97–9, 104, 106, 107, 125, 133, 144, 149, 151 ; see also neural mechanisms constant 44, 45f pruning 130f Mehler, J. 106 Messinger, A. 73 methodology of the artificial 68 Miikkulainen, R. 57, 134 mirror neurons 141; see also neurons modalities 55, 141, 147–8, 150 modes 69, 88, 88f, 90f(a), 90f(b), 91–2, 93, 96; see also clusters

170

Index

Moga, S. 72 Mohan, A. 142 molluscs 44, 46f, 48, 49f Morgan, C. L. 73 morpho-perceptual nativism 53 morpho-perceptual organs 62 morphology 54 constraints 12, 66 correlates 55 motor commands 24–5, 83, 142, 143 motor control 142, 143, 147 motor maps 17, 109, 110 motor networks 106, 111, 113, 142–3, 145, 146, 148, 149 motor neurons 106, 107, 108, 109, 110, 111, 147; see also neurons; perceptual neurons motor spaces 113–14, 141, 153 motor systems 143, 145 Motor Theory of Speech Perception 21 Mrayati, M. 54 muscles 18, 18f, 21, 21f, 24, 30, 76, 77, 106, 134 activations 78, 79, 80f, 106, 127 Nadel, J. 72 Nakanish, A. 22 narratives 69, 70, 71–2 nativist cognition 62 natural sciences 7, 68–74 natural selection 1, 3, 12, 13, 30, 32, 38–9, 39n, 40, 42, 42n, 43, 43n, 44, 47, 51–2, 56.

66, 67, 139, 143, 145, 151, 153 see also self-organization natural systems 3, 32, 35 nature 8, 9, 31, 39 neo-Darwinism 7, 11, 39, 39n, 40, 43, 47, 51, 52, 56, 67, 149, 151; see also Darwin, C. networks 143 neural architecture 12, 78, 146 neural epigenesis 126, 141–2, 153 neural maps; see also temporal neural maps 76, 79, 85, 87, 88f, 88n, 89n, 92n, 99, 100–101, 101f, 102, 102f, 103, 103f, 104, 104f, 106, 107, 100, 110–11, 113, 115f, 124, 127f neural nets 65, 76, 141 neural networks 78–9, 81f, 83, 92, 109–10, 123, 129, 141, 142 neural structures 13, 53, 143, 145–7, 149 neural systems 113 neural units 76, 80–1, 81f, 82, 82f, 83–7, 89f, 90f(a), 90f(b), 97f(a), 99–101, 104 neurons 76, 76n, 77, 78f, 79, 81–2, 82f, 84–5, 91, 92, 96, 98, 99, 100, 101, 102f, 103, 106, 107, 109, 111, 113, 120, 124, 125–7, 129,

Index 131, 132f, 133–4, 140–3, 148 neuroscience 7, 8, 66n, 99, 139–43 neurotrophins 126, 134–6, 136f, 142 Newton, Sir I. (1642–1727) 1, 3 Newtonian physics 70 Nicolis, G. 92 noise 40, 63, 64, 65, 77, 104, 122 non-linearities 43n, 53, 54, 64, 85–6, 91, 96, 97f(a), 110, 111, 112, 133, 137, 149 transition 38 non-uniform distributions 96, 113, 114f see also distributions; uniform distributions norms 29, 59, 66 Nowlan, S. 73, 73n On Growth and Form (Thompson) 44, 45n ontogenetics 9, 30 ontogenesis 44, 147, 148 operational mechanisms 139 operational models 9, 107 operational scenarios 58 optimality 52, 56–7, 58 order and chaos 94 organisms 40–4, 46n, 48–50, 51, 91, 149 see also living organisms organizational properties 1, 32 organs 14, 16–18, 20, 24, 55, 75, 143; see also relations between organs

171

relation space 77n, 78–80, 80f, 81, 81f, 83, 85, 86–7, 87f, 96, 97f(a), 97f(b), 101, 101f, 106, 107, 127, 127f, 128, 130f, 128f, 133–4, 134f speech 21, 54–5, 79 trajectories 21, 24 origins; see also languages, origins of forms 43, 48 living 52 of speech 12, 13, 29, 70, 74, 101, 139, 143–9, 153–4 Oudeyer, P.–Y. 10, 61, 62, 63, 64, 66, 68, 76, 79, 86, 138, 153 panda 49 Parisi, D. 10n, 86 parallel stripes 33f, 34, 34f, 41 parameters 41, 44, 64, 91, 92–3, 94–5 120, 122, 125, 126 Parisi, D. 10n, 68 patterns, 1, 34, 34f, 38, 44, 48, 49f, 92, 118, 119f(b), 120, 123–4, 129, 129f, 130, 131, 153; see also speech, sounds hexagonal 41 of combinations 126, 130 organized 32 sounds 11, 62 temporal 125 Peirce, C. S. 69, 154 perception 54, 60, 75, 80–2, 83–4, 99 perception/production coupling 83

172

Index

perceptual categories see attractors perceptual distinctiveness 56, 57, 58, 63, 138, 151 perceptual magnet effect 25, 26f, 100–2 perceptual maps 106–7, 110, 113, 118–19, 120 perceptual motor mapping 109 perceptual networks 111, 113, 142–53 perceptual neural maps 110, 114f perceptual neural networks 78, 79, 85, 113 perceptual neurons 107–8, 147 perceptual non-linearities 54; see also non-linearities perceptual representations 106, 110, 112 perceptual salience 56, 57 perceptual spaces 77, 80f, 106, 141 perceptual systems 143, 145 perceptual trajectories 108 perceptual warping 25, 102, 102f perceptual/articulatory mapping 109 perceptuo-motor correspondences 77–80, 106, 107f, 108–22 phases 35, 41, 93–4, 94f phenotypes 73–4 forms 40, 44 space 40, 42, 43–4, 51, 52 phonatory motor structures 143 phonatory networks 146 phonatory neural systems 143 phonemes 19, 20f(b), 22, 24, 25–8, 30, 54, 56, 57–8, 61, 63, 66,

98, 114, 115–16, 120, 123–4, 128–9, 129, 129f, 130, 133, 135, 138, 147, 152 inventories 26, 28, 55, 58, 89f, 96–7, 123–4, 142 phonemics 22, 25, 113 coding 54, 55–6, 59, 96, 110, 146, 149, 151 phonetics 19, 145 phonology 22, 28, 130, 133, 145 attunement 64, 84, 143 phonotactics 28, 58, 63, 66, 123–30, 130f, 131–3, 137, 138, 147, 150 Picq, P. 148 Pinker, S. 53, 62, 145 plasticity 77, 80–2, 125–6 polynomial interpolations 83, 107, 111, 142; see also interpolations Popper, K. 68 populations of agents 11, 12, 29, 91, 92n, 106, 114, 130f, 139, 143, 144, 145, 146, 150, 151–2 population vectors 99–102, 102, 102f, 103, 104f, 106, 109–10, 113, 120 see also decoding positive feedback loops 26, 32, 113, 114, 132; see also feedback loops poverty of the stimulus 62 preferred vectors 76, 79, 80, 81–2, 82f, 83, 84, 84f, 85–6, 87, 87f, 88, 88n, 89f, 90f(a), 90f(b), 91, 94, 94f, 96,

Index 97f(a), 97f(b), 99–101, 101f, 104, 106, 107–11, 113, 114f, 117f(b), 120, 127–8, 133–4, 134f, 140–1, 142–3, 145 Prigogine, I. 1, 92 production 56, 83–4, 137, 142 vowels 137 programmed cell death 126 programmed neuron death 142 pronunciation 117–18, 134 property 32, 41 psychology 55, 58, 71 punctuated equilibrium 41

quantal theory of speech 53–4

Ramus, F. 106 random mutations 39 randomness 38, 86, 107, 109, 110, 123, 125, 126, 130, 132, 141, 142, 144–6, 147, 148, 151 Rayleigh–Bénard convection 32, 32n, 33–5 re-entrance systems 103 reality 70–2 Redford, M. A. 57, 58, 61, 134 reductionism 1, 3, 53–6, 59 regularities 25, 30, 114–15, 117, 118, 150, 151, 152 relation space 20, 76, 78 relations between organs 65, 75, 78–9, 80f, 83, 84f, 85, 106, 107, 111, 124;

173

see also organs trajectories 76, 76n, 77–8, 78f replication 40, 51, 149 representations 7, 70, 72, 75–6, 78, 79f, 81, 83, 101, 102–3, 106, 112, 113, 137–8, 147 reproduction 43–4, 51, 145, 151 Rizzolati, G. 141 robots 10, 59, 72–3, 75 Rotokas 29 salient sounds 58 Salinas, E. 100 scalar products 100 scenarios 139–49 Schwartz, J. 75, 83n, 84, 140, 142 Schwartz, J.-L. 23, 25, 27, 57, 112, 117, 119 science 68–72; see also natural sciences search space 40–2 Sejnowsky, T. 109 Sekuler, R. 16 self-organization 38, 40, 41, 42, 123, 137, 138, 139, 140, 145–7, 149, 150–1, 153, 154 see also natural selection bubbles 1, 32 constraints 138 dunes 1, 2f ice crystals 1, 2f, 3f, 4, 32, 71, 72f, 71, 72f mountains 1, 2f shells 39–40, 49f, 50–1, 149 termite nests 3–4, 4f

174

Index

sensitization 81f sentences 7–8 sequence of gestures see relations between organs Shannon, C. 54 shapes 38, 39, 43, 43n, 45, 47, 52, 62, 71, 72f, 104, 153 organisms 40, 42, 48 side effects 48–51, 52, 67, 100, 147–9, 153 sigma see σ sign languages 8, 10–11, 14, 29, 29n signals 144, 150, 152 simulations; see also computers 59, 61, 62, 63–6, 71–3, 75, 83, 86, 87, 88, 89f, 90, 90f(b), 91–3, 101, 101f, 102, 102f, 107, 110–11, 113, 114, 115f, 118–19, 120, 121, 121f(a), 122, 125, 126, 127f, 128, 129, 129f, 130, 131f, 132, 133–4, 136, 137f, 139, 141, 142, 143–4, 145–7, 149, 150 Sneyd, J. 3, 38 Sonigo, P. 5 Sonority Hierarchy 28, 61 sounds 15, 21f, 24–5, 30, 43, 53, 54, 59, 60, 62, 64–5, 66, 76, 77–9, 80, 84, 85–6, 87, 91, 96, 97, 99, 100–2, 102f, 104–6, 109–12, 113–14, 141, 143, 144–5, 146, 147, 150, 152

see also patterns, speech; speech, sounds classification 28, 29 continuum 26f inventories 59, 144 patterns 11, 62 perception 15, 20, 21, 30, 75, 142–3 production 60, 75 salient 58 space 25, 146 systems 30, 59, 101–2, 123, 134, 145, 149 waves 14, 15, 15f, 17 spatial maps 124, 125, 127–8, 130, 131, 134, 134f neural 124–5 spatial networks 131 spatial neurons 124–6, 131, 133, 137, 140 spatial organization 1, 32 spectrum of amplitude 25 speech 6–8, 10–11, 11n, 14, 14n, 24, 28, 30, 53–6, 65, 67, 68, 69, 106, 108, 141, 147, 149, 150–1, 153; see also origins, of speech; patterns, sound; speech, organs; vocal tracts codes 10–11, 12, 13, 14, 24, 29, 44n, 52, 58–9, 67, 88, 88f, 92n, 122, 123, 139, 143, 144–5, 147, 149, 150–4 combinatorial 22–4, 88f, 106, 143, 147; discrete 12, 22–4, 30, 64–5, 106, 143, 145, 146, 147; diversity 28–9;

Index human 14–31, 137, 143; origins 30–1, 30n, 55, 56–7; universals 21–8 instruments 14–16 inventories 25 production 20–1, 21f, 58, 75 properties 24, 55 quantal theory 53–4 regularities 25 shared 24 signals 137–8 sounds 4, 7–8, 10, 14, 16, 54, 55, 96, 152 properties 53, 56–8; syntax 10, 14, 123 systems 28, 67, 104–5, 122, 139, 147–149 universals 21 square cells 41; see also cells; hexagonal cells; stem cells static vowels 61; see also vowels statistical phonotactic preferences 133, 137 Steels, L. 10, 29, 59, 61, 68, 86, 144, 145 stem cells 40–1, 44; see also cells; hexagonal cells; square cells Stevens, K. N. 25, 26f, 53, 54, 100, 102 stochasticity 61, 87, 88, 89f, 91, 104, 132, 132f stops 18 Stork, D. 62, 88, 88n, 89n

175

strong combinatoriality see combinatoriality, strong structures 40, 43, 43n, 44–5, 48, 49–50, 51, 52, 56, 66–7, 92, 94, 96, 106, 113, 120, 123, 134, 139, 145–6, 149, 150–3 Studdert-Kennedy, M. 54, 55, 56, 64, 151, 153 syllabaries 22 syllables 22, 24, 28, 30, 57–8, 61–3, 68, 123–4, 130, 134, 138 symbolic representations 11 symmetry 33, 36, 87, 110, 149 breaking 32, 35, 36, 94 liquids 33–4 syntactic rules 30 systemic biology see biology, integrative systematic re-use see combinatoriality Szostak, J. 40 talk see speech Tashliyt Berber 28, 123 temporal evolution 90f(a), 124n, temporal filter 81, 99, 105, 108 resolution 112 temporal maps 127, 127f, 128, 129, 129f, 131, 132, 135 temporal networks 126 temporal neural maps 125–6, 131, 136f temporal neurons 124–7, 127f, 128, 129, 129f, 130, 131, 131f, 132, 132f, 133, 135–6, 136f, 137f, 140, 141, 148 temporal resolutions 108

176

Index

thawing 1, 32, 139 theories 69, 70, 71–2, 74, 141–2, 148, 153–4 theory of mind 72–3 Theraulez, G. 3, 38 Thom, R. 1 Thompson, D. 44, 45, 45f, 45n, 46, 47, 48n, 67, 151, 152 tones 29 tongues 15, 16–17, 18, 19, 54, 111 trachea 14 traits 40, 42, 43, 51, 73 organisms 48, 50 trajectories 18, 21 Tritton, D. J. 34 Tsuda, I. 142 Tuggy, D. 19 tundra 1, 32 turn-taking 60, 152 uniform distribution 86–8, 89n, 90–1, 94, 100, 104, 107, 110–11 universal rules 145 universals 21–8 UPSID 22, 25, 27f(a), 27f(b), 28, 115–16, 119f, 120–1, 121f(a), 122 utility 52 utterances 20, 54–6, 60, 115 Vallée, N. 25, 57, 111, 112, 117, 119 valences 130 Varela, F. 1 variation 40, 51 vectors see population vectors; preferred vectors

velum 14, 15, 18, 19, 54 verbal theories 68–9, 71–2 Vihman, M. 84, 143 virtual worlds 75 vocal neural nets 153 vocal tracts 8, 12, 14, 15, 15f, 16, 17, 18, 21, 24, 53, 54–5, 63, 85, 96, 106, 107, 110, 111, 143, 148 vocalizations 12, 14, 19, 21, 22, 65, 66, 75, 79–80; 80f, 81, 83, 83n, 84, 84f, 85–6, 87, 87f, 88, 88f, 92, 93, 96, 97f(b), 98f, 99, 101–2, 103f, 105, 107, 107f, 108, 109, 110, 111–13, 114f, 115f, 122, 123, 124–5, 126, 127n, 128, 129, 132–8, 142, 143, 144, 148, 150 see also sounds Von Foerster, H. 1 Von Frisch, K. 48n, 153n vowels 12, 19, 22, 23f(a), 25, 26–7 26f, 28–9, 54, 58–62, 100–1, 111–12, 114, 120, 124, 130 constriction 18–19 inventories 114–15, 120, 121f(b), 151 perception 111–12, 137, 141 prototypes 117–18, 120 repertoires 59 systems 27, 27f(b), 43, 58, 59, 60–1, 63, 66, 115, 115f, 116f(a), 116f(b), 117f(a), 118, 118f, 119f(a), 119f(b),

Index 120, 121f(a), 121f(b), 122, 140, 152 energy 57; human 113–22; shared 60, 68 triangle 113, 114f, 120 Vrba, E. S. 50 water molecules 1, 2f, 4, 32, 71 Waldrop, M. 38 whirlpools 1, 4, 32

177

wild cards 73 Williams, K. A. 25, 26f, 100, 102 Williamson, M. 142 Wishaw, I. Q. 17 Wolfram, S. 71 words 7, 19, 20f, 21, 28, 57–8 writing systems 14, 22 zones of stability 54

Self-Organization in the Evolution of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Evolution of Morphology (Studies in the Evolution of Language)

Language Evolution (Studies in the Evolution of Language)

Language Evolution (Studies in the Evolution of Language)

The Cradle of Language (Studies in the Evolution of Language)

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Origins of Meaning (Studies in the Evolution of Language)

The seeds of speech: language origin and evolution

Why We Talk: The Evolutionary Origins of Language (Studies in the Evolution of Language)

Why We Talk: The Evolutionary Origins of Language (Studies in the Evolution of Language)

Grooming, Gossip, and the Evolution of Language

The Evolution of God

The Evolution of Parliament

The Evolution of God

The Evolution of Plants

The Evolution of God

The Evolution of Cooperation

The Evolution of Anisogamy

The Evolution of God

The Evolution of Desire

The Evolution Of Morphology

The Evolution of Plants

The Evolution of Reason

Self-Organization in the Evolution of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Origin of Speech (Studies in the Evolution of Language)

The Evolution of Morphology (Studies in the Evolution of Language)

Language Evolution (Studies in the Evolution of Language)

Language Evolution (Studies in the Evolution of Language)

The Cradle of Language (Studies in the Evolution of Language)

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Evolution of Language

The Origins of Meaning (Studies in the Evolution of Language)

The seeds of speech: language origin and evolution

Why We Talk: The Evolutionary Origins of Language (Studies in the Evolution of Language)

Why We Talk: The Evolutionary Origins of Language (Studies in the Evolution of Language)

Grooming, Gossip, and the Evolution of Language

The Evolution of God

The Evolution of Parliament

The Evolution of God

The Evolution of Plants

The Evolution of God

The Evolution of Cooperation

The Evolution of Anisogamy

The Evolution of God

The Evolution of Desire

The Evolution Of Morphology

The Evolution of Plants

The Evolution of Reason

Recommend Documents