Metaphor and Gesture (Gesture Studies, Volume 3)

Metaphor and Gesture

Gesture Studies (GS) Gesture Studies aims to publish book-length publications on all aspects of gesture. These include, for instance, the relationship between gesture and speech; the role gesture may play in social interaction; gesture and cognition; the development of gesture in children; the processes by which spontaneously created gestures may become transformed into codified forms; the relationship between gesture and sign; biological studies of gesture, including the place of gesture in language evolution; and gesture in human-machine interaction. Volumes in this peer-reviewed series may be collected volumes, monographs, or reference books, in the English language.

Editors Adam Kendon

University of Pennsylvania, Philadelphia

Cornelia Müller

European University Viadrina, Frankfurt/Oder

Volume 3 Metaphor and Gesture Edited by Alan Cienki and Cornelia Müller

Metaphor and Gesture

Edited by

Alan Cienki Vrije Universiteit Amsterdam

Cornelia Müller Europa-Universität Viadrina

John Benjamins Publishing Company Amsterdam / Philadelphia

8

TM

The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences – Permanence of Paper for Printed Library Materials, ansi z39.48-1984.

Library of Congress Cataloging-in-Publication Data Metaphor and gesture / edited by Alan Cienki and Cornelia Müller.        p. cm. (Gesture Studies, issn 1874-6829 ; v. 3) Includes bibliographical references and index. 1.  Gesture. 2.  Metaphor.  I. Cienki, Alan J. II. Müller, Cornelia. P117.M48    2008 808.5--dc22

2008008202

isbn 978 90 272 2843 7 (Hb; alk. paper) © 2008 – John Benjamins B.V. No part of this book may be reproduced in any form, by print, photoprint, microfilm, or any other means, without written permission from the publisher. John Benjamins Publishing Co. · P.O. Box 36224 · 1020 me Amsterdam · The Netherlands John Benjamins North America · P.O. Box 27519 · Philadelphia pa 19118-0519 · usa

Table of contents Contributors Acknowledgements

vii ix

Introduction Alan Cienki & Cornelia Müller

1

Why study metaphor and gesture? Alan Cienki

5

From left to right... : Coverbal gestures and their symbolic use of space Geneviève Calbris

27

Gesture as a conceptual mapping tool Robert F. Williams

55

A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor Rafael Núñez

93

Peircean semiotics meets conceptual metaphor: Iconic modes in gestural representations of grammar Irene Mittelberg

115

Unexpected metaphors David McNeill Catchment, growth point, and spatial metaphor: Analyzing Derrida’s oral discourse on deconstruction Jacques Montredon, Abderrahim Amrani, Marie-Paule Benoit-Barnet, Emmanuelle Chan You, Régine Llorca, & Nancy Peuteuil

155

171



Metaphor and Gesture

Form, meaning, and convention: A comparison of a metaphoric gesture with an emblem Fey Parrill What gestures reveal about the nature of metaphor Cornelia Müller

195

219

Commentaries on the value of studying metaphor and gesture from the perspectives of different disciplines Metaphoric gesture and cognitive linguistics Ronald W. Langacker

249

Metaphoric gestures and cultural analysis Naomi Quinn

253

Metaphor and gesture: A view from the microanalysis of interaction Jürgen Streeck

259

Implications of cognitive metaphor and gesture studies for Ethnomethodology and Conversation Analysis and vice versa Anders R. Hougaard & Gitte R. Hougaard Sign and gesture: Towards a new paradigm Sherman Wilcox The study of metaphor and gesture: A critique from the perspective of semiotics Paul Bouissac

265

273

277

The neuroscience of metaphoric gestures: Why they exist George Lakoff

283

Metaphor and gesture: Some implications for psychology Raymond W. Gibbs, Jr.

291

Index

303

Contributors Alan Cienki Vrije Universiteit Amsterdam Cornelia Müller Europa-Universität Viadrina Geneviève Calbris CNRS – Université Paris V / ENS Lettres et Sciences Humaines de Lyon Robert F. Williams Lawrence University Rafael Núñez University of California, San Diego Irene Mittelberg Vrije Universiteit Amsterdam David McNeill University of Chicago Jacques Montredon Abderrahim Amrani Marie-Paule Benoit-Barnet Emmanuelle Chan You Régine Llorca Nancy Peuteuil Université de Franche-Comté Fey Parrill Case Western Reserve University Ronald W. Langacker University of California, San Diego Naomi Quinn Duke University

 Metaphor and Gesture

Jürgen Streeck The University of Texas at Austin Anders R. Hougaard Gitte R. Hougaard University of Southern Denmark Sherman Wilcox University of New Mexico, Albuquerque Paul Bouissac University of Toronto George Lakoff University of California, Berkeley Raymond W. Gibbs, Jr. University of California, Santa Cruz

Acknowledgements We wish to express our appreciation to a number of individuals who helped in various ways to make this volume possible. We are grateful to the other presenters at the theme session on metaphor and gesture, organized at the Eighth International Cognitive Linguistics Conference, which gave rise to the idea for this volume: Scott K. Liddell, David McNeill, Irene Mittelberg, Fey Parrill, and Eve Sweetser, as well as Susan Duncan, who provided valuable comments in her role as discussant. Bertie Kaal promoted the publication of an edited volume on this topic from the beginning, and we thank her for her faith in us. We appreciate the guidance of Anke de Looper at John Benjamins in overseeing the production of the book, and the advice of Adam Kendon as an editor of Benjamins’ series on Gesture Studies. We owe thanks to Mary Copple for her translation work in the volume and to Ulrike Bohle and Elisabeth von der Ohe for assistance with copy editing. Finally we wish to thank all of the authors who contributed chapters and commentaries to this volume for their work and their patience. Alan Cienki and Cornelia Müller

Introduction Alan Cienki & Cornelia Müller

The German physiologist and psychologist Wilhelm Wundt (1832–1920) was perhaps the first to note that gestures could be used metaphorically. In his Völkerpsychologie (1922), Wundt discusses the “symbolic gesture,” which transfers the concepts being expressed from the perspective of one domain to that of another.1 He cites the examples of time as space, and the transfer of the abstract to the perceptual. However, the interrelation of the topics of metaphor and gesture really only began to receive close attention at the end of the 20th century, after the boom in metaphor studies beginning in the 1980s, and the expansion of gesture studies starting in the 1990s. (See Cienki, this volume, for details.) The present volume is an overview of the current state of research on metaphor and gesture. Though it is a selective snapshot, we feel it is a representative one, at least of contemporary U.S. American, French, and German work at the intersection of these fields of study. There are many approaches to the study of metaphor, and also many to the study of gesture. Consequently, the authors here consider each of them from a number of perspectives. Metaphor is viewed both in conceptual and semiotic terms. In this volume metaphor is sometimes discussed in terms of systematic cross-domain mappings, as is characteristic of some work in conceptual metaphor theory. But metaphor is also related to idiosyncratic, imagistic ways of thinking in-the-moment which may sometimes be reflected in gesture. The semiotic analyses focus on the metaphoric use of different forms of iconic structures in gesture. Gesture is itself also analyzed from different points of view. On one side, gesture is studied as a window onto the thought processes of the speaker/gesturer. It is argued that gesture can reveal metaphoric thinking, which may be taking place with or without accompanying speech. On the other side, we see how gesture can be used as a tool by the hearer/viewer in order to construct the speaker’s intended meaning. Collectively the papers show how gesture can provide a valuable source of insight for metaphor scholars. 1. “… daß sie die auszudrückenden Vorstellungen aus einem Anschauungsgebiet in ein anderes überträgt…” (Wundt, 1922, p. 165).



Alan Cienki & Cornelia Müller

The volume is structured in the following way. It begins with Alan Cienki’s overview of findings from existing research on metaphor and gesture, including a consideration of the implications and questions that the study of each brings to the other. The next paper introduces some basic distinctions between the metaphoric and non-metaphoric in the context of a specific study: Geneviève Calbris explores how metaphoric notions are rooted in non-metaphoric, physical actions, through a detailed look at a set of interviews with a politician. The three chapters which follow involve analyses of gesture as used in pedagogic contexts. While Robert Williams continues the theme of how abstract thought is grounded in physical action, he includes the role of object manipulation, and analyzes how it may serve to anchor a metaphoric blend in the process of teaching children to tell time with a clock. Rafael Núñez examines the specific domain of mathematics and how gestures by lecturers on the topic provide additional evidence of metaphoric conceptualization. Irene Mittelberg’s study of lectures by linguistics professors shows the importance of different iconic modes in their metaphoric gestures, bringing together Peircean semiotic theory with conceptual metaphor theory in her analysis. The following three contributions build specifically on the study of gesture as it has been developed by the psychologist David McNeill. In his own paper, McNeill considers gestures which at first might not be considered metaphoric, but which are revealed to function this way when considered in the larger discourse context. Jacques Montredon and colleagues apply McNeill’s ideas of gestural “catchment” and “growth point” to analyze the metaphoric conceptualization of a core theoretical notion in postmodernism, that of deconstruction. Fey Parrill tests a claim about the idiosyncratic nature of spontaneous coverbal gesture through an experimental comparison of the degree to which people rate canonical versus noncanonical forms of two distinct gestures as natural. In the final main chapter, Cornelia Müller points out that the study of metaphoric gesture highlights properties of metaphor which have seldom been addressed in the literature, leading her to argue for a dynamic approach to metaphor as a cognitive activity. As this overview of the contents shows, the works here engage different kinds of data as well as various research methods. Some involve elicited data, while in others, the data are naturalistic. In the former category are responses recorded from participants who were invited to produce a narrative or engage in a conversation, or to respond to gestures which they were shown as stimuli. The latter group includes televised broadcasts of interviews with well-known speakers, and recordings of educators in the act of teaching (both in a primary school and at universities). Some studies concentrate on a substantial quantity of video recording of an individual speaker, and others look at a variety of speakers in different



Introduction

contexts. A number of methodological approaches were taken to analyze the data. These include: statistical analysis; interaction analysis; blending analysis; semiotic analysis; linguistic analysis, i.e., closely tied to the lexical semantics of the accompanying speech; and types of hermeneutic analysis between the levels of spoken word and gesture. The volume, therefore, gives a sense of the variety of ways in which this topic can be researched. A note on terminology is in order. “Metaphoric” is the adjectival form used in the chapters in this volume when discussing gestures as expressions of metaphor, so: metaphoric gestures. As discussed in Cienki’s chapter, McNeill’s (1992) examples focussing on one category of gestures as metaphoric—those which may reflect a discourse topic or genre as if an object in the hand—led some subsequent scholars to limit their use of the term ‘metaphoric’ to these types of gestures. However as the studies in this volume show, there are many kinds of metaphor which may be expressed in gesture, and many kinds of gestures which can express metaphors. The research presented here shows that there are multiple answers to the question of what can constitute a metaphoric gesture. In addition, the chapters reflect two ways of listing conceptual metaphors that are currently common in the literature: either using all small capital letters (good is up) or capitalizing the first letter of each main word (Good is Up); no theoretical distinction is intended in the chapters by the use of one system or the other. The idea for this publication arose out of a theme session on metaphor and gesture which we organized for the Eighth International Cognitive Linguistics Conference, held in Logroño, Spain in July 2003. Some of the papers from that theme session provided the starting point for the current collection, and others were solicited. In the open discussion at the end of the theme session, several members of the audience pointed out the value of studying the intersection of these fields for their particular discipline, and how this was something they had not considered before. Following up on this, we sought commentaries from experts in different fields on that very question: What potential importance could the study of metaphor and gesture have for your field? The volume ends with these short pieces, which may inspire readers to pursue this area of research in new directions. For those who are not acquainted with work on metaphor and gesture, this volume gives it a face; it makes this a recognizable topic of research. For those who are conversant with research on either metaphor or gesture, we hope that this book will provide useful insights on a familiar field from a new point of view.





Alan Cienki & Cornelia Müller

References McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Wundt, W. (1922). Völkerpsychologie. Eine Untersuchung der Entwicklungsgesetze von Sprache, Mythus und Sitte. Vol. 2: Die Sprache. Leipzig: Alfred Kröner Verlag.

Why study metaphor and gesture? Alan Cienki There are numerous ways to research gestures which represent abstract notions, and this paper begins with an overview of some of them which are represented in the current volume – from various semiotic approaches to experimental psychological studies. Then particular attention is given to metaphoric gestures studied as expressions of conceptual metaphors. This line of research has shown some of the similarities and differences between verbal and gestural metaphoric expression. The paper surveys some of the evidence provided from gesture studies which supports the view of metaphor as a cognitive phenomenon, and the notion that thought, even for abstract topics, is grounded in embodied experience. However, the study of gesture also raises some questions for research on conceptual metaphors and how it is conducted. Topics discussed include how one identifies metaphoric expressions, what counts as evidence of conceptual metaphors, how one labels them, and how gesture highlights the graded nature of metaphoricity.

1. Introduction If we consider the study of metaphor in the tradition of works such as Lakoff and Johnson (1980, 1999) and Lakoff (1993), one of the basic principles is that metaphor stems from (at least potential) conceptual mappings between domains. If metaphor has as its basis cross-domain mappings in the conceptual system, then words should offer just one form in which they may appear. One should be able to find metaphoric expressions in various forms of human behavior, and not exclusively in language. And indeed, there has been some research on conceptual metaphor and its expression in visual media, music, and various forms of cultural practices and rituals (as examples see Fernandez, 1991; Forceville, 1996; Zbikowski, 2002). Since the 1980s, there has been an increasing amount of research which shows that spontaneous gestures during language production, especially gestures of the hands and forearms, can also constitute metaphoric expressions. This paper offers an overview of some of the findings to date of work published in this area, as well as an introduction to some pertinent questions raised by this



Alan Cienki

research. A recurring theme involves the challenges one confronts in the application of conceptual metaphor theory to non-verbal or paralinguistic data. Ultimately, the study of gesture raises concerns for some aspects of the theory itself in its current form. 2. Background 2.1

What is a gesture?

In the broad sense, “gesture” can refer to any willful bodily movement, however the focus in this paper, as in this volume, is on gestures of the hands. While a prototypical gesture passes through three phases – the preparation, stroke, and retraction – it is the stroke phase which is considered to minimally constitute a gesture (Kendon, 1980, 2004). Rudolf Laban, the analyst and transcriber of dance movements, notes, “Since it is absolutely impossible to take account of each infinitesimal part of movement we are obliged to express the multitude of situations by some selected ‘peaks’ within the trace-form which have a special quality” (Laban, 1966, p. 28). Following in his metaphorical footsteps, McNeill (1992, p. 376) characterizes the gesture stroke as “the phase carried out with the quality of ‘effort,’ a concept developed for dance notation.” The stroke phase is thus the kernel of a gesture in kinesic terms. It is also the part of the gesture which is of primary interest for determining the function of the gesture as a whole. McNeill (1992, p. 375) adds, “Semantically, it is the content-bearing part of the gesture.” Therefore the stroke phase is of central interest in this and the other papers in this volume. Also note that the papers in this volume are concerned with voluntary movements other than those that function as “adaptors” (Eckman & Friesen, 1969), such as the self-adaptor of adjusting the position of one’s eyeglasses with one’s hand. Gestures differ in the degree of conventionality of their forms and their functions. We can speak of a gradient of gestures: from those which have developed fixed meanings in the culture in which they are used to gestures which are produced spontaneously and often unwittingly, and the meaning of the latter (if we can even talk about them having meanings) is highly dependent on the context. The former type, the formulaic, quotable gestures, are sometimes called emblems, a term first proposed by Efron (1941) and developed into its currently accepted sense in Ekman and Friesen (1969). Examples include the American “OK” gesture, made with thumb and forefinger forming a ring shape by touching the fingertips together (a gesture discussed by Parrill, this volume); or the “thumbs up” gesture, indicating a positive evaluation with the thumb extended upward vertically and the remaining fingers curled closed. In contrast to emblems, spontaneous gestures



Why study metaphor and gesture

which are often produced unwittingly while speaking (ones Kendon, 1988 classifies as gesticulation) do not have pre-determined meanings, and so they can provide important insight into the processes of formulating thoughts while producing language which are normally beyond conscious awareness. (See, for example, McNeill & Duncan, 2000. See also McNeill, 2005, pp. 5–12 for distinctions of other gesture continua.) 2.2

What is a metaphoric gesture?

While research from various theoretical backgrounds has touched on the possibility that gestures could be metaphoric (such as Calbris, 1990), among the first works to specifically apply Lakoff and Johnson’s (1980) notion of metaphor to the study of gesture are McNeill and Levy (1982) and McNeill (1992). Later work (e.g., Cienki, 1998a) has explicated the interaction between, and independent production of, metaphoric language and metaphoric gestures, to be considered further below. Webb (1996) discusses the possible bases for components of metaphoric gestures. Sweetser (1998) considers some metaphors for speech and thought as expressed gesturally. But what constitutes a metaphoric gesture? In McNeill and Levy (1982) and McNeill (1992), metaphorics are included as one of four types of spontaneous gesture with speech. These four types, which are not intended as a mutually exclusive classification system, are: – beats (rhythmic gestures which mark words or phrases as significant for their discourse/pragmatic content), – deictics (point at concrete entities or at particular spaces), – iconics (depictive of the form or movement of physical entities, or the physical relation between them), and – metaphorics (whose pictorial content presents an abstract idea). However, the metaphoric gestures which received the most attention in this work were ones in which the hands were shaped in the air as if holding or loosely supporting an object (often palm up, with a loosely open hand), and were used in contexts to simply refer to an abstract notion, such as the narrative genre under discussion by participants in the experiments. McNeill argues that these gestures constitute expressions of the conduit metaphoric model, first discussed by Reddy (1979). This model characterizes the common practice of thinking and talking about ideas as if they were objects, and about communication as if it were a simple transfer of the ideas via a container (of words or texts) from one person to another. Furthermore, because McNeill’s (1992) book was the first widely disseminated book in English which included a discussion of metaphoric gesture, many came to





Alan Cienki

consider conduit gestures as the prime examples of metaphoric gestures. However, as the papers in this volume show, it is just one category among many. Another classification, which presents an account of metaphoric gestures from a different perspective, is that of Müller (1998). This classification focuses on the function of various types of gestures, namely on the following categories. – discourse gestures, which structure an utterance (such as making beats for emphasis, or counting out the logical points one is making on the fingers of one hand), – performative gestures, which enact speech acts (such as dismissing an offer or idea with a motion sweeping away, or requesting something with a hand held out open, palm up), and – referential gestures, which can refer to something concrete or to the abstract. In the case of concrete referential gestures, they can indicate their referent iconically in several ways. For example, one’s hand(s) can draw, model, or represent entities. Thus two hands with thumb and forefingers extended at right angles, the other fingers folded, can represent two corners of a square picture frame. However, if we take abstract referential gestures, they cannot represent the abstract referent itself iconically, since what is being referrred to usually lacks a structure which can inherently be depicted with the hands. Müller (1998) shows that McNeill’s (1992) iconic and metaphoric gestures are in fact both equally iconic signs, but what distinguishes them is whether they are depicting the referent itself – concrete reference to an entity, action, or relation – or whether the referent is another entity, action, or relation in terms of which the topic is being characterized (in which case we have abstract reference). We can think of the hands making the same framing gesture, described above, but used when the speaker is talking about the organization of a theory. In this case, the speaker is making abstract, gestural reference to the theory’s organization as a physical structure, namely, a framework. Note that “emblem” gestures are not excluded from this system. So the “thumbs up” gesture points upward for a reason: it invokes reference to the abstract idea of good things as being up, versus the bad as being down. The distinction between concrete or abstract reference entails a description of gestures in terms of their function. However, as Fricke (2004) makes clear, there is not a necessary connection between abstract referential gestures and metaphoricity. Indeed, she points out that some abstract referential gestures are not metaphoric, while some concrete referential gestures are. One can be talking about a triangle in the context of a geometry lesson and represent it with the fingers of two hands. But this abstract referential gesture is not functioning as a metaphor in this context. It is a representation of a mathematical construct. (It may be seen as metonymic, however, by showing a particular form of triangle to stand for the general



Why study metaphor and gesture

class of such shapes.) By contrast, we might find the expression of a metaphor in a concrete referential gesture. Fricke (2004, p. 180) notes that one could be talking about another person derogatorily and refer to him as “This ass!” (in the sense of “donkey,” in German: “Dieser Esel!”) while holding one’s hands up at the sides of one’s head, imitating a donkey’s ears. Here we have concrete reference with the gesture serving the metaphoric function of characterizing a person as an ass. The key point here is that the description of a gesture as metaphoric involves the interpretation of a mapping between two domains. (Of course ascertaining whether we are dealing with one or two domains is itself a matter of interpretation in context, as Croft [1993] makes clear.) Nevertheless, while it seems intuitively correct that most abstract referential gestures can be further classified as metaphoric, and concrete referential gesture as non-metaphoric, it remains to be seen through empirical study whether this is in fact the case, and to what degree this may differ across cultures and situations of use. 2.3

On metaphoricity in words and in gesture

Building on Müller (2004/in press), we can note three parameters that are useful to bear in mind when considering conceptual metaphors and metaphoric expressions: – the degree of conventionality of a conceptual metaphor in the given culture (from conventional to novel); – the degree of conventionality of a metaphoric expression in the given culture (from conventional to novel); – the degree to which a metaphoric expression is highlighted in a given instance of use (making it cognitively less or more salient). The interrelations between these parameters as they relate to the process(es) of speaking can be considered as follows. In the study of language, we can question what it means to say that a verbal expression is metaphoric. First we can note that conceptual metaphors fall along a scale of conventionality to creativity. For example, from existing research on the dominant languages in Europe and America, one can conclude that in these cultural spheres it is much more conventional to conceptualize life as a journey rather than to think of life as a banana. (I will follow the convention used in the literature of writing posited conceptual metaphors in the form “target is [or as] source domain” in small capital letters.) Innovative conceptual metaphors necessarily require novel means of expression (in this case, perhaps it could be: “Life is a banana: you should peel it carefully and enjoy every bite”). For more conventional conceptual mappings, there is a scale from more conventional to more novel forms of expression. “I feel like my life is going nowhere” strikes this native





Alan Cienki

speaker of English as a more conventional way of expressing life is a journey than “He skateboarded his way through life.” Some (like Kyratzis, 1997 and Müller, 2004/in press) argue that the potential for activation of metaphoricity is scalar, such that the underlying conceptual metaphors can be more or less frozen or defrosted, or more or less asleep or awake (depending on the metaphor one wants to use for metaphoricity itself). Contextual factors influence the degree of salience of an expression, and so the potential for realization of its metaphoricity. The salience of words being spoken can be increased by the use of marked prosody (Pierrehumbert & Hirschberg, 1990), the use of lexical tuning devices (such as “so to speak” before or after a metaphoric expression, which draws attention to it) (Cameron & Deignan, 2003; Goatly, 1997; Goddard, 2004), and also through co-verbal gesturing that is more expansive than normal for the speaker (Müller, 2003). Some combination of these behaviors would likely make the metaphoricity of the co-occurring verbal expression even more salient. The same scale of conventionality for metaphoric expression applies to spontaneous gesture with speech. This ranges from the use of a conventional metaphor with a conventionalized form of expression (namely an emblem, like the “thumbs up” gesture) to novel expressions used once in a given context and maybe never again. And just as the use of a metaphoric expressions in words (conventional or novel) can be highlighted in context, making its metaphoricity more salient, the same process of highlighting can happen with a metaphoric expression in gesture. While this salience might be expected with a creative use of gesture, it may also potentially occur with the use of a more conventional gestural form. As an example, think of the common conduit type mentioned above, perhaps embodied in the form of a loosely cupped hand, held palm up, and used when the speaker is presenting a new idea to the listener. Even this kind of gesture can be highlighted not only by more expansive use of the gesture space (perhaps through an exaggerated movement leading to presentation the hand shape in its final form and position), but also by the speaker directing his/her gaze at the gesture, or by the use of marked prosody (emphatic stress, lengthening, or extreme pitch contour) in the speech accompanying the gesture. Finally, expression of the same source domain for a given target domain in both words and gesture at the same time can highlight the metaphoric mapping between target and source, as discussed in section 3.1 on “commonalities” below. The issue of frozen or sleeping metaphors relates to language in an historical perspective just as it relates to the historical dimension of the use of gesture within a culture. So when many speakers of English use the word evolution (let alone revolution), they are probably unaware of its historical connection to Latin volvere ‘to roll’ even though the related English word revolve clearly uses this physical meaning of rotation. However, the semantic connection could be made salient by



Why study metaphor and gesture

the speaker, especially for poetic or humorous effect. An example from a written text comes from a search of the LexisNexis Academic newspaper database, namely a letter to the editor of the Tulsa World newspaper (Nov. 24, 2005, p. A32). The author is contesting the teaching of Darwin’s theory of evolution in schools because of her religious convictions. The letter contains the word “evolution” six times and “evolving” once in its 211-word text, but bears the title “Confusion revolves” (emphasis added – AC). In terms of gesture, Calbris (1990, pp. 196–198) discusses gestures which reflect the etymologies of the words with which they are used, such as a rotating gesture being used with a word like evolution. The point is not that speakers know and are enacting these etymologies. Rather the pattern of thinking of that abstract domain in terms of the given physical manifestation is prevalent in the language and culture in other ways, and so reappears with the use of this word, even unwittingly to the speaker at that moment. So there may be a common metaphoric mapping in the culture from the notion of a moving object (such as a round object rolling) onto processes of various kinds, including abstract (complex) ones, like evolution. This prevalent pattern may appear in either verbal expressions or in gestures. We can extend this point by noting that a speaker may simply be employing a cultural convention when enacting a gesture, and be quite unaware of its metaphoric connections. So a speaker may give the “thumbs up” to indicate something is good because that is the convention s/he is familiar with from the surrounding culture, not because the speaker was aware at the moment of the conceptual metaphor good as up. Therefore one must be clear about what one is claiming when saying that a given gesture is metaphoric, just as one should consider this in relation to claims about the metaphoricity of verbal expressions. In the following sections we consider in further detail what else is the same, and what is different, about the expression of metaphor in words and in gestures. 3. Metaphors expressed in words and gestures 3.1

Commonalities in the expression of metaphor in words and in gestures

Sometimes we can see the same conceptual metaphor expressed in gesture and in speech at the same time. The examples below are from conversations which were elicited from students on how they take exams at their university, and what they consider honest behavior to be in this context. All were native speakers of American English. In (1), the speaker talks about dishonest behaviors using a blend of metaphorical notions. The transcription of speech in examples (1)-(3) follows many of the





Alan Cienki

conventions of DuBois et al. (1993), with adaptations here for the gesture transcription. Each line indicates an intonation unit, ^ = primary stress accent, ` = secondary stress accent, = an equal sign (=) represents = lengthening of the preceding vowel or consonant, a comma (, ) indicates an intonation unit with a terminal pitch that signals continuation, a period/full stop (. ) marks a final intonation unit falling to a low pitch at the end, and surrounds speech at a rapid pace (allegro). Speech accompanied by gesture is underlined, with square brackets [ ] placed around the words uttered during the main stroke phase of the gesture. In the gesture transcription in italics underneath, the description of the stroke phase also appears in square brackets, 2H = both hands, LH = left hand, RH = right hand, PU = palm up, PD = palm down, OH = open hand. (1) there’s `such different ^levels, 2H PUOH at the same level G1: [LH PU moves downward while RH PD on top of it moves upward], leaving a large space between the open hands

Figure 1.  Dishonest behaviors as different vertical levels

The mention of “levels” reifies the behaviors, possibly as horizontal layers. The quality of how serious the different behaviors are thought to be is mentioned as a vertical scale, according to which more serious is higher up. Verbally, we see this in the expression “all the way up to.” Similarly, in her gesture (G1 = gesture 1, shown in Figure 1), the speaker uses her hands as flat surfaces, showing the reified behaviors, and holds them at different vertical heights, showing the different levels (rendering degree as distance).1

1.

I am grateful to Mathias Roloff for providing the drawings.



Why study metaphor and gesture

We can also find gestures expressing conceptual metaphors which are shared in the culture and are familiar from expressions in verbal form, even if they are not being used at the moment in the co-occurring speech. We see this in example (2). In the transcription of speech, short and long pauses are indicated by .. and ... respectively. In the gesture transcription, PC = palm facing the center space. (2)

Like dis^honest suggests.. like..., um, not `truthful, like, the [^truth] is what.. like, G2: LH PCOH and flat moves up and then [downward] to just above left leg

Figure 2.  Gesture used with “truth”

The speaker refers to “the truth” without any verbal metaphoric expressions, but while uttering this word he makes a small chopping gesture in the air with his left hand flat in the vertical plane, and then briefly holding this position (G2, shown in Figure 2). This corresponds to other expressions in English in which the truth is characterized as straight, and telling the truth is metaphorically having one’s words (as if objects) move in a straight direction, as in the expression “tell it to me straight” (see Cienki, 1998b). 3.2

Differences in the expression of metaphor in words and in gestures

While example (2) points out something that words and gestures can have in common – that the same metaphoric mapping can appear in either form of expression – it also points out that words and gestures can serve different expressive functions at the same time. In this example, the words spoken make the target domain explicit





Alan Cienki

(truth), while the gesture illustrates a metaphorical way of thinking about this target by depicting the source domain (a straight object). Conversely, we can have metaphoric expressions in language which are not accompanied by metaphoric gestures. We often speak with little or no use of additional spontaneous gesture. Metaphoric expressions in language can also be accompanied by gestures which are not metaphoric or which have “low metaphoricity.” An example would be simple beat gestures made for emphasis, which at most could be argued to express an ontological metaphor (q.v. below), reifying the idea being talked about without further elaborating on its character through any more specific gestural form or motion. We find a more extreme example of difference when two conceptual metaphors involving the same target domain, but different source domains, are expressed in words and co-occurring gestures. In (3) the speaker is saying that “an abstract thought of honesty,” devoid of practical considerations, is clearly demarcated, “black or white,” and she makes a chopping gesture while saying that, as if dividing space. In the transcription of speech, a hyphen (-) indicates a truncated word, and surrounds words spoken at an increasingly accelerated pace. The caret (^) in the gesture transcription marks the occurrence of the gesture beats with the co-occurring words. (3) y’know, ther- there [`is] no gradations. 2H together, R flat on top of L, [RH sweeps to right side] ^ ^ ^ ^ G3: LH PUOH, RH flat [makes beats with right edge against palm of LH]

Figure 3.  Gesture referring to absolute categories

So in speech we have a “color” metaphor, such that the stark difference between two kinds of moral behavior is likened to the visual difference between black and white. But in gesture we see a spatial metaphor, whereby the vertical hand shows



Why study metaphor and gesture

the distinction between the moral categories of right and wrong as if it were a clear division separating two distinct spaces (G3 in Figure 3). (Compare a similar example in Cienki, 1998a, in which a different speaker talks about traditional morality as being “black and white,” whereas the ambiguity of contemporary moral standards is “gray.” In the former case she also makes a gesture as if dividing up the space in front of her with hands flat in the vertical plane, but for “gray” she uses a loosely held, claw-shaped hand, as if to show the unclear demarcation between the different moral spaces.) This gestural behavior makes sense, given the difficulty of trying to use hand shapes or movements to depict the variety of colors we can discern visually. But this also means more generally that we can have metaphors expressed in words which cannot be expressed be in gestures, namely if the source domain cannot be iconically represented in terms of shape, position, and/or movement. This contrasts with sign languages, which have conventional forms to symbolically refer to colors or black and white. In addition, we can see metaphoric expressions in gesture which lack equivalents in the speaker’s language. For example, Calbris (1990, p. 87) observes the following about the location of gestures for the temporal ordering of events, “In European cultures, the future is in the direction of walking or writing, that is forward or to the right.” Conversely, the past is often gestured as being behind or to the left. (See Núñez & Sweetser, 2006, for a contrasting case in the South American language Aymara.) Calbris (1985) provides many examples, one of which is given in (4). The arrows indicate the directions in which the speaker pointed with his extended right index finger. (4) (from Calbris, 1985, p. 51) < – avant ou – > après le second tour (des élections) ‘before or after the second round (of elections)’ Whereas we do find verbal expressions in European languages for the future as being located ahead of us, it is rare in the extreme to say “*I did X to the right of Y” as a synonym for “I did X after Y.” But the left-to-right time-line does play a role in other contexts, such as the convention of drawing a time line from left to right – something which provides supporting evidence for some level of cognitive reality of the metaphors past is left and future is right in these cultures. (See additional examples of left/right temporal metaphors in gesture in Calbris, this volume, and Cienki, 1998a.) We can see with these examples that gestural data do not just replicate what we already know about conceptual metaphor based on verbal data. They provide additional possible evidence of cross-domain mappings which may be the source of the gestures. The following sections outline some other ways in which gestural





Alan Cienki

data illuminate the study of metaphor, and how research focussed on metaphor can benefit gesture studies. 4. Some solutions that the study of gesture brings to the study of metaphor 4.1

A solution to the criticism of circularity

One important contribution that the study of gesture can make to conceptual metaphor theory is that it provides a solution to one of the criticisms of it, that it relies on circular logic (Murphy, 1996). One concern is that many of us metaphor researchers are implicitly arguing “that verbal metaphoric expressions are evidence of conceptual metaphors, and then saying that we know that because we see conceptual metaphors expressed in language” (Cienki, 1998a, p. 190). Gestural data provide an independent source of evidence with which to argue for the psychological reality of conceptual metaphors. However one needs to bear in mind whether one is making claims about the individual at the moment of speaking, or about the larger cultural group of which s/he is a member – the “supraindividual,” the level at which the claims of conceptual metaphor theory make more sense, as some have argued (Gibbs, 1999; Steen, 1994). 4.2

Gesture and embodied cognition

Gesture provides evidence for the embodied basis of thought. Gesture can provide an important locus for cognitive linguistic research on metaphor because it physically manifests the tenet that (many) metaphors are grounded in embodied action. Gestures can depict in space elements from the source domain of a metaphor, something which is not possible for metaphoric expressions in spoken languages. (See Gibbs & Berg, 2002, and the responses to their article, on the broader issue of mental imagery and embodied activity.) Of course this physical depiction of metaphoric source domains is already known to play an important role in signed languages (Taub, 2001; Wilcox, 2000). 4.3

Ontological metaphors

On a certain level, one can make an argument that any time a gesture is made when there is not a concrete referent in the given context, the gesture is metaphoric simply by virtue of representing an ontological metaphor, showing something abstract as concrete. Here we can draw on Fauconnier’s (1985, 1994) theory of mental spaces. Mental spaces have been described as “small conceptual packets constructed as we



Why study metaphor and gesture 

think and talk, for purposes of local understanding and action” (Fauconnier & Turner, 2002, p. 40). As Lakoff (1987, p. 281) adds, “any fixed or ongoing state of affairs as we conceptualize it is represented by a mental space.” Discourse-structuring gestures which highlight different parts of a logical argument can be seen as representing the speaker’s mental spaces (about the organization of the argument) in the form of physical spaces (see Sweetser, 2007). It can be seen as expressing a form of ontological metaphoric mapping of abstract as concrete in which the source domain is no more specific than that of a distinct space. It also represents a gesture that is metaphoric not on the level of the semantic referents, but on the pragmatic or metanarrative level in that the target domain is the discourse structure (such as the distinction between two different ideas, or between what is known information and some new information). (Compare McNeill, Cassell, & Levy’s [1993] discussion of abstract deixis.). This “setting up” of different parts of one’s argument by locating them in different spaces gesturally is reflected in the verbal formula that sometimes accompanies it, namely “on the one hand,... on the other hand.” We thus find that some of the different ways in which metaphoricity functions in thought (making reference to something or someone within a narrative, or referring on the pragmatic level to the narrative structure itself) can appear as metaphoric gestures which serve different functions. 4.4

The function of metaphoric gestures for the speaker, and the addressee

With any gesture, we can consider the role that it plays from the perspective of the gesturer or from that of the observer. Many in gesture studies have shown the important function that gestures can have for speakers/gesturers as they formulate their ideas “on the fly.” Because of the huge potential range of forms they can take, spontaneous gestures can provide insight into thinking for speaking (McNeill & Duncan, 2000; Slobin, 1987). Metaphoric gestures can help us understand more about the specific roles of metaphor in this process. However, as the speaker lays out the ideas and/or discourse structures of his/ her turn at talk, this can also serve a communicative function and benefit the addressee in that it can make the logical organization of the ideas in the argument physically clear. In an extended example in Cienki (1998a, pp. 197–198), one student contrasts the process of preparing for a test with the act of taking a test, and keeps track of which process he is referring to in his narrative by pointing back and forth to spaces on his left and right sides, with test-preparation on the left, and test-taking on the right (which is also consistent with the left/right distinction for past and future times, discussed above). While the speaker may be doing this to help himself track his referents as he is formulating his argument, his gestures apparently help his addressee as well, as she can be seen looking at his hand while



Alan Cienki

he is doing this gesturing. Thus the addressee’s gaze at the speaker’s gestures can serve as one indicator for the researcher of when gestures, metaphoric or otherwise, are serving a communicative function. 5. Some questions that the study of gesture raises for conceptual metaphor theory Several problematic issues in research in the framework of conceptual metaphor theory come to the fore once we move beyond lexical data. 5.1

Evidence of conceptual metaphors

Particularly in the early research on conceptual metaphors, the evidence cited consisted mostly of verbal expressions that were intuitively plausible, but which were often made-up sentences. They usually followed prescriptive grammar rules for syntactic completeness (having at least a noun phrase and a verb phrase). Constructed examples, such as “He has a lofty position” and “She has high standards” would be cited as evidence of metaphoric expressions which involve the source domain up. In some ways, this is a holdover from the Chomskyan tradition in American linguistics in which such sentences, conforming to the native-speaker/author’s intuitions for well-formedness, provided the only legitimate object of study. However, research on metaphor in spontaneous gesture by and large has not followed that path. It has looked at naturally-produced data. This is partly a result of the ease with which interactional data can be captured today with video technology. (Indeed, as video technology keeps changing, so will the kinds of research which we can do.) But the bottom line is that if we are testing the hypothesis that conceptual metaphors are instantiated in human behaviors, we need to look at human behavior not just selectively, piecemeal, but as it occurs in various natural contexts, in situ. 5.2

Identifying metaphoric expressions

Another issue raised by research on metaphor and gesture has to do with how metaphoric expressions are coded. This is an issue which still is often not discussed explicitly in metaphor research, on any kind of data. One group of researchers which has devised a reliable procedure for the identification of metaphorically used words in texts is the Pragglejaz group, named after the first initials of the ten group members (Pragglejaz Group, 2007). One of the most useful parts of such a project is simply the documentation of the methodological difficulties which must



Why study metaphor and gesture 

be overcome to achieve reasonable agreement among analysts when working on a given text. Perhaps a similar research group should be formed to develop reliable procedures or guidelines for the identification of the metaphoric use of gestures. One thing that is important to bear in mind in any research on metaphoric expressions is that what is coded as metaphoric will depend on the goals of one’s project. Is the goal to find anything that is potentially metaphoric in nature? Or to find instances of expressions (be they verbal, gestural, or other forms) which may represent conscious awareness of metaphoric mappings? Or... something else? 5.3

Labelling of conceptual metaphors

A further question concerns how we traditionally describe conceptual metaphors, that is: in terms of verbal statements, like good is up. This descriptive device, even if we just consider it a heuristic, carries a lot of theoretical assumptions with it which are usually not acknowledged, but which become more salient when studying non-verbal data. One problem is that describing the source domain of a metaphor with words affects the nature of how one (as a metaphor researcher) understands the source domain. For example, we can roughly describe the “conduit” metaphor for communication as it appears in gestures by talking about one of the relevant source domains as container (an “expected metaphor” which McNeill discusses in this volume). But the gesture shows us more than that: we often see it expressed with a palm up, and with a loosely cupped hand. How do we incorporate those qualitative elements into our description of the source domain? Source domains of metaphors often draw on the embodied nature of our experience, but some of this experience can only be captured partially or inadequately in words. Much of our physical experience is better described in terms of an image or a movement. Given the saying in English that “a picture is worth a thousand words,” perhaps we should follow the route of cognitive grammar (e.g. Langacker, 1987, 1991) and rely on diagrams as a heuristic when appropriate for describing imagebased source domains of conceptual metaphors. One additional complication in labelling of metaphors can be found in groups of gestures that share commonalities of form and/or motion in a family-resemblance way, and which have the potential to be used with related metaphoric meanings. So in studying gestural expressions of Russians relating to the notion of chestnost’ (‘honesty’), we see different forms related to straightness reported in Cienki (1999) such as flat hand shape and loose hand moving straight out from face. How should one pick out which aspect of the source domain is significantly represented: the flatness of the hand shape, the straight manner of motion, or the “solid” nature of both the hand shape and the manner of motion? Is just calling it straight



Alan Cienki

(as I did at the time) adequate? This is, therefore, a further complication which work with gestural data makes us aware of. 5.4

Highlighting of metaphoricity

As discussed earlier, several works on metaphor in language have tackled the issue that metaphoricity is better characterized as a property which can be in the background or the foreground to varying degrees, rather than being a black and white matter of whether certain words uttered were metaphoric or not. My impression is that this issue is even more readily apparent with the study of gesture because of the salience (or not) of gesture as used in a contextual scene. We are left with the question: How can we best handle this issue in our analyses of conceptual metaphors? Müller (2004/in press, and this volume) suggests some points to be taken into consideration, including simultaneous reinforcement of the same metaphor in words and gesture, the size of the gesture in the speaker’s gesture space, and the speaker’s gaze at the gesture. 5.5

The time course of metaphor use

Though the production and reception of both written and spoken language are inherently tied to the temporal dimension, as Chafe (1994) makes clear, each involves a different kind of flow of experience. We know from research on spoken language that verbal metaphors cluster according to certain patterns in dialogs (Cameron & Stelma, 2004), a consequence of the interactional nature of this context of language use. The study of metaphor in gesture with speech can complement such research on the use of metaphoric expressions over time during speaking (see, for example, the dynamic approach to metaphor advocated in Müller, 2004/in press, and this volume). As of yet, little is known about the organization, timing, and different possible functions of metaphoric gestures over the course of conversations. 5.6

The relation of metaphor to the modality in which it is expressed

One of the fundamental claims of Lakoff and Johnson (1980) is that metaphor is a general cognitive phenomenon, not limited to language; the fact that metaphor can be manifested in various modalities supports this notion (see discussion in Cienki & Müller, in press; Müller 2004/in press; Müller & Cienki, in press). But saying that a general cognitive principle is involved in the creation of metaphors does not necessarily mean that all metaphors are generally of the same nature. To what degree is the nature of the metaphors we use structured by the media we have



Why study metaphor and gesture 

at our disposal to express them? For example, there is a lot of variability across individuals in terms of both the quality and quantity of their gestures when they talk, as one can quickly see from looking at video recordings of a number of different speakers. Different metaphors are likely expressed by those who employ discourse-structuring gestures more frequently than by those who make greater use of referential gestures. The research on thinking for speaking (Slobin, 1987) and thinking for speaking and gesturing (McNeill & Duncan, 2000; Cienki & Müller, 2006) would lead us to believe that this is the case. However, the question of whether different types of metaphor are being expressed more in gesture than in words, and whether different categories of metaphor are being expressed more in certain types of gestures than in others, are topics which await empirical study. 6. What the study of metaphor can bring to gesture studies 6.1

Complement to research on concrete referential gestures (often called “iconic”)

There is a substantial amount of research on gestures of the concrete, that is, gestures for which one can say with some certainty (based on context) that they have referents which are physical objects, their actions, or the relations between them. In McNeill’s system these are called iconic gestures (although as mentioned above, this terminology can be confusing, since metaphoric gestures also represent the source domain of a metaphor iconically to some degree). One reason they have been studied so extensively has to do with a tradition in gesture studies of videorecording participants as they retell the plot of a movie they have seen, often a cartoon which involves elaborate motion events. This research method facilitates the production of concrete referential gestures, but reduces the amount of thought and talk on abstract topics, which might be accompanied by metaphoric gestures. More research on talk about abstract topics will likely provide more data with metaphoric gestures. 6.2

The conduit is not the only metaphoric gesture

A type of gesture of the abstract which the research on film narration does lend itself to is one which simply represents the fact of an idea itself, or the film or story genre, as an object in the air which the speaker is holding in his or her hands. This is largely an artefact of the method of this research – retelling a narrative. McNeill (1992, p. 189) states that “Metaphorics appear at the metanarrative level, where the content consists of the story structure itself viewed as an object or space.” But this



Alan Cienki

is a factor of focussing on the conduits as the only kind of metaphoric gestures. As the papers in this volume show, other situations of talk afford the production of other types of metaphoric gestures, many (or even most) of which function on the narrative level. There is still much that remains to be known about the variety of forms of metaphoric gestures, and about how and when they are used. 6.3

Metaphor is hidden in some existing gesture research

There is a growing body of research about the “meanings” of various gestures, yet the metaphoric nature of these gestures is often not acknowledged. For example, Brookes (2001) describes a gesture (an emblem) in South Africa which is used to express various nuances of the meaning “clever.” It is made by pointing up to one’s eyes with the index and pinkie fingers, with the other fingers curled to the palm, as the palm faces oneself. She contrasts this with a gesture made to indicate “stupid,” in which the right hand is held flat, palm towards the face, and drawn to the right across one’s eyes. Though it is not part of that author’s argument, one can see possible metaphoric motivations behind the forms of these gestures, whereby knowing/understanding is seeing and not knowing is not seeing. It is possible that pursuing the role of metaphor here could provide insight to connections with broader cultural/metaphoric models which help motivate the use of these gestures. 7. Conclusion Overall, the interrelation of the topics of metaphor and gesture can prove fruitful from several perspectives. From the point of view of gesture studies, we know far more about gestures which make concrete reference than we do about those which refer to the abstract. The study of such gestures which are metaphoric on the semantic or pragmatic level not only confirms what we know about how gestures function in general, it also raises new issues for the field of gesture studies. From another perspective, the study of gesture has ramifications for research on metaphor, providing confirmation of some previous theoretical claims, but also raising some new concerns on the levels of theory and method. The study of metaphor in gesture is in line with the increasing attention in cognitive linguistics to metaphor as a multimodal phenomenon (e.g., Forceville, 2005; Forceville & Urios-Parisi, in press). In contrast to a view of metaphor as a disembodied property of written words on a page, research on gesture adds to what we know about how metaphor is expressed and used through multiple modalities at the same time, not only orally/aurally via the words of speech, but also spatio-motorically and visually. Once one is aware of the fact that there can be



Why study metaphor and gesture 

metaphors in gesture and other media of expression, one cannot help but notice their frequency and the varied contexts of their occurrence. Acknowledgement I am grateful to the Institute for Comparative and International Studies of Emory University (Atlanta, USA), for the faculty international travel grant which allowed me to present the original version of this research at the Eighth International Cognitive Linguistics Conference in Logroño, Spain in July of 2003.

References Brookes, H. J. (2001). O clever “He’s streetwise.” When gestures become quotable: The case of the clever gesture. Gesture, 1, 167–184. Calbris, G. (1985). Espace–temps: Expression gestuelle du temps. Semiotica, 55, 43–73. Calbris, G. (1990). The semiotics of French gestures. Bloomington, IN: Indiana University Press. Cameron L. & A. Deignan (2003). Combining large and small corpora to investigate tuning devices around metaphor in spoken discourse. Metaphor and Symbol, 18, 149–160. Cameron, L. & J. Stelma (2004). Metaphor clusters in discourse. Journal of Applied Linguistics, 1, 107–136. Chafe, W. (1994). Discourse, consciousness, and time: The flow and displacement of conscious experience in speaking and writing. Chicago: University of Chicago Press. Cienki, A. (1998a). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford, CA: Center for the Study of Language and Information. Cienki, A. (1998b). Straight: An image schema and its metaphorical extensions. Cognitive Linguistics, 9, 107–149. Cienki, A. (1999). Metaphors and cultural models as profiles and bases. In R. W. Gibbs, Jr. & G. J. Steen (Eds.), Metaphor in cognitive linguistics (pp. 189–203). Amsterdam: John Benjamins. Cienki, A. & C. Müller (2006). Verbal to gestural, and gestural to verbal, metaphoric expression. In the panel session “The dynamics of metaphoricity in spoken interaction” at the conference “Researching and Applying Metaphor 6,” Leeds, UK, April 2006. Cienki, A. & C. Müller (In press). Metaphor, gesture, and thought. In R. W. Gibbs, Jr., (Ed.), The Cambridge handbook of metaphor and thought. Cambridge: Cambridge University Press. Croft, W. (1993). The role of domains in the interpretation of metaphors and metonymies. Cognitive Linguistics, 4, 335–370. DuBois, J. W., S. Schuetz-Coburn, S. Cumming, & D. Paolino (1993). Outline of discourse transcription. In J. A. Edwards & M. D. Lampert (Eds.), Talking data: Transcription and coding in discourse research (pp. 45–87). Hillsdale, NJ: Lawrence Erlbaum Associates. Efron, D. (1941). Gesture and environment. Morningside Heights, NY: King’s Crown Press. Ekman, P. & W. V. Friesen (1969). The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica, 1, 49–98.



Alan Cienki Fauconnier, G. (1985/1994). Mental spaces. Cambridge: Cambridge University Press. Fauconnier, G. & M. Turner (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York, NY: Basic Books. Fernandez, J. W. (Ed.) (1991). Beyond metaphor: The theory of tropes in anthropology. Stanford, CA: Stanford University Press. Forceville, C. (1996). Pictorial metaphor in advertising. London/New York: Routledge. Forceville, C. (2005). Cognitive linguistics and multimodal metaphor. In K. Sachs-Hombach (Ed.), Bildwissenschaft: Zwischen Reflektion und Anwendung (pp. 264–284). Cologne: Von Halem. Forceville, C. & E. Urios-Aparisi (Eds.) (In press). Multimodal metaphor. Berlin/New York: Mouton de Gruyter. Fricke, E. (2004). Origo, Geste und Raum: Lokaldeixis im Deutschen. PhD dissertation, Technische Universität Berlin, Germany. Gibbs, R. W., Jr. (1999). Taking metaphor out of our heads and putting it into the cultural world. In R. W. Gibbs, Jr. & G. J. Steen (Eds.), Metaphor in cognitive linguistics (pp. 145–166). Amsterdam: John Benjamins. Gibbs, R. W., Jr. & E. A. Berg (2002). Mental imagery and embodied activity. Journal of Mental Imagery, 26, 1–30. Goatly, A. (1997). The language of metaphors. New York/London: Routledge. Goddard, C. (2004). The ethnopragmatics and semantics of “active metaphors.” Journal of Pragmatics, 36, 1211–1230. Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relation between verbal and nonverbal communication (pp. 207–227). The Hague: Mouton. Kendon, A. (1988). How gestures can become like words. In F. Poyatos (Ed.), Cross-cultural perspectives in nonverbal communication (pp. 131–141). Toronto: Hogrefe. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Kyratzis, A. (1997). Metaphorically speaking: Sex, politics, and the Greeks. Ph.D. dissertation, Lancaster University, UK. Laban, R. (1966). The language of movement: A guidebook to choreutics. Boston, MA: Plays, Inc. Lakoff, G. (1987). Women, fire, and dangerous things: What categories reveal about the mind. Chicago: University of Chicago Press. Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (pp. 202–251). Cambridge: Cambridge University Press. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books. Langacker, R. W. (1987). Foundations of cognitive grammar, vol. I: Theoretical prerequisites. Stanford, CA: Stanford University Press. Langacker, R. W. (1991). Foundations of cognitive grammar, vol. II: Descriptive applications. Stanford, CA: Stanford University Press. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. McNeill, D., J. Cassell, & E. T. Levy (1993). Abstract deixis. Semiotica, 95, 5–19.



Why study metaphor and gesture 

McNeill, D. & S. Duncan (2000). Growth points in thinking-for-speaking. In D. McNeill (Ed.), Language and gesture (pp. 141–161). Cambridge: Cambridge University Press. McNeill, D. & E. Levy (1982). Conceptual representations in language activity and gesture. In R. J. Jarvella & W. Klein (Eds.), Speech, place, and action (pp. 271–295). New York: John Wiley & Sons. Müller, C. (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag A. Spitz. Müller, C. (2003). What gestures reveal about consciousness of metaphoricity. Presented at the 8th International Cognitive Linguistics Conference, La Rioja, Spain, July 2003. Müller, C. (2004/in press). Metaphors. Dead and alive, sleeping and waking. Habilitationsschrift, Freie Universität Berlin, Germany. In press with the University of Chicago Press. Müller, C. (This volume). What gestures reveal about the nature of metaphor. Müller, C. & A. Cienki (In press). Words, gestures and beyond: Forms of multimodal metaphor in the use of spoken language. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor. Berlin/New York: Mouton de Gruyter. Murphy, G. L. (1996). On metaphoric representation. Cognition, 60, 173–204. Núñez, R. & E. Sweetser (2006). With the future behind them: Convergent evidence from language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401–450. Parrill, F. (This volume). Form, meaning, and convention: A comparison of a metaphoric gesture with an emblem. Pierrehumbert, J. & J. Hirschberg (1990). The meaning of intonational contours in the interpretation of discourse. In P. R. Cohen, J. Morgan, & M. E. Pollack (Eds.), Intentions in communication (pp. 271–311). Boston, MA: MIT Press. Pragglejaz Group (2007). MIP: A method for identifying metaphorically used words in discourse. Metaphor and Symbol, 22, 1–39. Reddy, M. (1979). The conduit metaphor: A case of frame conflict in our language about langauge. In A. Ortony (Ed.), Metaphor and thought (pp. 284–297). Cambridge: Cambridge University Press. Slobin, D. (1987). Thinking for speaking. Proceedings of the thirteenth annual meeting of the Berkeley Linguistics Society (pp. 435–445). Berkeley, CA: Berkeley Linguistics Society. Steen, G. (1994). Understanding metaphor in literature. New York/London: Longman. Sweetser, E. (1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier. Sweetser, E. (2007). Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics. In M. Gonzalez-Marquez et al. (Eds.), Methods in cognitive linguistics (pp. 201–224). Amsterdam: John Benjamins. Taub, S. (2001). Language from the body: Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press. Webb, R. (1996). Linguistic features of metaphoric gestures. Ph.D. dissertation, University of Rochester, NY. Wilcox, P. P. (2000). Metaphor in American Sign Language. Washington, DC: Gallaudet University Press. Zbikowski, L. (2002). Conceptualizing music: Cognitive structure, theory, and analysis. Oxford: Oxford University Press.

From left to right... Coverbal gestures and their symbolic use of space Geneviève Calbris This study overlaps with Johnson’s (1987) ideas about the image schemas underlying thought, but rather than grasping them through the deductive analysis of verbal metaphors, they are approached here through the analysis of the symbolic components of gestures which, accompanying speech, refer to abstract entities. The corpus analyzed was compiled from six television interviews with the French Prime Minister, Lionel Jospin. The interviews averaged thirty minutes and took place over a one-year period. Only symbolic variations along the transverse axis, which is both symmetrical and oriented, are studied. The transverse axis appears to encompass several symbolic axes: a value axis, a spatio-temporal axis, and a logico-temporal axis, themselves derived from the axes of physical progression, which are growth, walking, and writing. A fourth, two-directional symbolic axis depicting an evolving process is added to the above. Successive translations of the upward and forward axes on to the rightward axis, achieved for contextual or ergonomic reasons, are mentally possible, since all three are axes of physical progression. This projection on to the side-toside axis accounts for the semiologically motivated complexity of the symbolic system represented on this major axis.

The semiological analysis of coverbal gesture referring to the abstract, called metaphoric gesture by McNeill (1992), confirms the Metaphors we live by (Lakoff & Johnson, 1980) and the incarnation of cognition analyzed by Johnson in The body in the mind (1987), and later in Philosophy in the flesh (1999): the perceptual experience of the body, incorporated and mentally assimilated, will serve as the basic model for our conception of the abstract. The semantic study of gesture ‘provides insight’ into this intellectual and fundamentally metaphorical step of attaining the abstract. The nature of the gestural sign, physical and analogical, predisposes it to account for this perceptual anchorage of the mental world. It would be surprising not to see it actualize verbal spatial metaphors. One can expect the gestural sign to



Geneviève Calbris

express other kinds of metaphors that are less easily verbalized, and one can expect that analyzing it will enable one to extricate the associative network of metaphors operating on lower levels of consciousness. The coverbal gesture can express a spatial metaphor that is otherwise verbalized. It is neither an illustration of the literal sense nor a conscious illustration of the verbal metaphor. In general, it re-actualizes in a non-conscious way our physically experienced, intelligent, and non-conscious process of attaining the abstract. It depicts the intermediate stage between the concrete and the abstract and depicts a preconceptual schema. As gesture intervenes spontaneously on a less conscious level than speech, it should enable us to reach further back into the genesis of the metaphorical process. To have more impact, this paper only deals with the simple differentiated use of the right hand and the left hand using examples taken from a previous study (Calbris, 2003). Noting the co-occurrences between the use of one hand or the other and the sense of the gesture, partly deduced from the verbal context, teaches us that the alternation of the right hand and left hand is neither arbitrary, nor simply contrastive, and that the often non-conscious choice of the hand used to express oneself is semantically motivated. Equally, the challenge is to demonstrate that physical manifestation is fundamentally ambiguous and that it corresponds to different physical characteristics that may be pertinent. In the change of hand, should one see the principle of the change (to something “other”), the location designated by the change (on the right or on the left along the transverse axis), or the direction of a path deduced from the points of departure and arrival which are indicated by the change of hand (from left to right or from right to left along the transverse axis). Confronting the diverse data enables one to identify the contextual sense of the gesture. Here, the latter depends on a double confrontation: with the utterance on the one hand, and with the preceding gesture on the other. This paper therefore deals uniquely with the symbolic use of the left and the right along the transverse axis, seeking to explore the combinatorial possibilities of the percepts that underlie it. For example, the transverse axis corresponds to the frontal plane; it imposes the image of the symmetry of all bodies, human, animal or vegetal, viewed from the front. This axis is favorable for visually tracking the displacement of a moving body; it is also the axis of reading and writing, pointing toward the right in the Western world. We shall see the two respective schemas derived from it, that of symmetry and that of the logical sequence of events, interacting in the gestural presentation of a dichotomous enumeration. This paper, which traces evidence back to its origin, is divided into two major parts: the qualitative analysis of utterances showing the existence of analogical signs (A), and then the interpretative hypotheses on the origin of these analogical signs



From left to right 

(B). One deals with the evidence; the other attempts to explain it. With respect to metaphor, one could say that the first part deals with the metaphorical link between the physical and the semantic levels of the sign (A), and that the second part conjectures about the metaphorical process that, originating in sensory-motor experience, has enabled the constitution of the sign (B). The two corresponding parts, A and B, follow the same hierarchic structure determined by the fact that the transverse axis is the only one along which the symmetry of the body (1.), resulting from a symmetrical development (2.), is expressed, and that it is also a directional axis (3.). 1. Symmetry 2. Symmetrical development 3. Progression along the transverse axis Data and methods The corpus used in this study consists of television interviews with the French Prime Minister Lionel Jospin (LJ), conducted by journalists. It includes six interviews averaging thirty minutes each, broadcast on three different French television stations between July 1997 and April 1998. LJ sat facing the journalists during five of the six interviews and more to their right during the sixth. The gestures were coded in terms of the different types of features that can describe a gesture, i.e., the choice of the left or right hand, the hand configuration, its orientation, and its movement. In line with the aim stated above, a code was assigned to the use of the left hand, the right hand, movements, and leftward and rightward orientations. For the semantic analysis of “left” and “right,” it is important to know, first of all, whether the speaker is left- or right-handed. A quantitative analysis (of the five programs in which he is facing the journalists) showed that LJ is an ambidextrous right-hander (35.5% of all gestures were made with the right hand, 43.5% with both hands, and 21% with the left hand). A few examples taken from other corpora will be given to justify certain points as these are raised. Now let us go on to the qualitative analysis, which requires us to take certain precautions because although it is visible, gesture is not obviously understandable. Although it is analogical, the gestural sign is ambiguous, and although its meaning depends on the utterance, it is not a duplicate of the verbal information. The gestural sign forms part of a coherent system of analogical signs. The analysis of analogical connections between gestures and their contextual meanings requires a fairly large corpus, and it is carried out in several stages: 1. Coding gestural components;



Geneviève Calbris

2. Listing examples according to a gestural component (1. movement; 2. configuration); 3. Comparing examples on each level, the common physical element being presumed to represent the common semantic element analogically; 4. Cross-checking to verify each analogical connection. In the present case, it is the large number of hand changes encountered in the corpus and their comparison that has enabled the ascertainment of both the variety of the signs and the coherence of the system underlying the symbolic use of the left and the right. Qualitative analysis The coverbal signs studied in the examples are regrouped according to their meanings, which are metaphorical representations originating from the same fundamental schema explicated in the interpretative second part of the paper above p. 29.

1

Symmetry

The ordered presentation of the examples under a given heading goes from those posited as involving the most conscious motivation (that which is denominated the “left” is located on the left) to the least. The heading regroups the gestural coverbal signs whose motivation is explained by a metaphorical use of the left and the right sides of the body, characterized by an axis of symmetry that seems to cut it vertically into two equal parts. The interpretive part of the paper will make a distinction between the symmetrical body, standing upright and seeking to maintain its balance, or walking progressively along an axis. Symmetry is knowledge inherent to the body; the right and the left hands function separately (two different autonomous entities), they are physically similar (two equivalent entities, X or Y) and often reunited to act efficiently (two complementary entities, X and Y). Because of this, they have a latent capacity to metaphorically represent two other entities: abstract, different, equivalent, complementary, or even more opposed. The axis of symmetry determines the permanent game of shifting weight as the body seeks to maintain equilibrium both in a static position and during walking: one step to one side, a second to the other (see 1.3, Two-item enumeration, enumeration of a series), and eventually coming to a halt and returning to the resting position with the feet side by side (1.4 Three-item enumeration).



From left to right

1.1

“The Left” in politics

Present in the human, animal, vegetal, and mineral realms, the principle of symmetry is perceptually so pregnant that man reproduces it in almost all of his creations, for example, a parliamentary semicircle (metaphorical transfer?). The French refer to the Socialists, those who occupy the left half of the semicircle, by saying ‘the Left’. LJ therefore verbally refers to the Left (a political entity physically located on the left) while gesturing with the left hand (a bodily entity physically located on the left). Almost everything referring to the political “Left,” whether legislative bodies, major left-wing issues, measures taken by the Cabinet, or reminders of goals, is designated or depicted by the left hand. Here is an example1 among the 26 occurrences:

(1) [open L pointing forward at head height, moves away] That must be the main axis, [L in fist is lowered] that must be the course [to hold] (see illustration below).

That must be the main axis ça doit être l’axe essential,

that must be the course [to hold] Ça doit être le cap

Figure 1.  That must be the main axis

Figure 2.  That must be the course

LJ is referring to the objective of the left-wing government. It is thus with the left hand that he depicts this objective (designated route toward a distant goal) as well as the effort required to realize it (hand clenched in a fist). The diverse components of the gesture (laterality and differentiated configuration) each express a metaphor. 1. For the sake of legibility in the examples given in the rest of this article, gestures are described in square brackets at the point in the utterance where they appeared. The utterance segment corresponding to the duration of the described gesture is shown in italics. The following codes were used to represent gestures: left hand [L], right hand [R], both hands [LR], rightward movement [.r], leftward movement [.l].





Geneviève Calbris

The left-wing government is mentally located on the left. A last example, apparently anodyne, confirms this: LJ generally counts out the elements of a long enumerative series on his right hand; the only exception to this is when he enumerates the various left-wing ministers on the fingers of his left hand (Calbris, 2003, p. 76). 1.2

Distinction between oneself and others

As a right-handed person, LJ refers to himself with the right hand and gestures with the right hand when expressing himself as a private individual.  Since the hands are naturally disposed to represent two equivalent and distinct entities, the reference to others is signified gesturally by a change of hand. ONESELF. LJ gestures with his right hand whenever he speaks as a private individual. His gestures expressing a profession of faith or self-defense, with his fingers or palm(s) on his chest, are always produced with the right hand and never with the left alone. However sometimes, for more emphasis, he gestures with both hands placed against the chest:

(2) [R] It wasn’t up to me, as a politician, to decide (see illustration 3)



(3) But I don’t see why [LR] it concerned me alone

It wasn’t up to me, a politician, to decide Il ne me revenait pas à moi, homme politique, d’en trancher

Figure 3.  It wasn’t up to me... to decide High involvement: Self-defense

No longer expressing himself as the person in charge of a left-wing government (mentally located on the left as indicated by the co-occurring use of the left hand), but as an individual (LJ is right-handed) in the process of reasoning, LJ quite naturally uses his right hand. His references to precision are almost always made with the right hand, whether to signal a detail via a raised index finger (94% with the right hand), or to represent extreme precision via the fingernails of the thumb and



From left to right 

index finger touching to form a circle (90% with the right hand), whenever accuracy and rigor are at stake. One might think that in an analogous situation, a left-hander manipulating ideas as a private individual would express himself with the left hand. Yet we are provided with a nuance by a left-hander forced to use his right hand, the philosopher Michel Serres, and who clearly states during a television interview2: “I would practice fencing with my left hand. But I write with my right hand; in a way, I talk with my right hand; all my cultural activities are on the right.” As a result of academic education, he has established a dichotomous division between his practical and sport activities, which he has continued to perform with his left hand, and his academic activities, which he has learned to perform with his right hand.

(4) And then, [R] as for me, I discuss it on site as a representative, a national representative but also as a [L] local representative, which I am (see illustration)

Again below, the reasoning man speaks with his right hand and then abruptly changes to the other hand as an actor for the Left:

(5) [R] So I think this crisis will be healthy. If on the contrary /.../. But [R] it is undeniably the first hypothesis I support. In any case, [L] this is the way I’d continue, not simply by speaking to the French people when you ask me to, but by acting.

As for me, I discuss it on site... Moi, je discute sur le terrain...

local representative, which I am élu local comme je le suis

Figure 4.  As for me, I discuss it on site

Figure 5.  Local representative, which I am

The individual

The representative of the Left Acting as two persons

2. Vidéos. Mots et politique (6), Le mot droite: Les mots ont la parole. http://www.cerimes. education.fr



Geneviève Calbris

Some people may think that the speaker changes hand simply to create a contrasting effect. According to this hypothesis, he should change hands in order to refer to two distinct persons. But this is not the case if we consider Example 7: LJ evokes the President and himself with the same hand, the left. The confrontation of the examples contradicts the hypothesis in question and confirms the coherence of the symbolic system of metaphors that are going to emerge from the ensemble of examples. LJ refers to everything that concerns the Left, including himself at the head of a left-wing government, with the left hand, but as a right-handed individual he designates himself as an individual and expresses a personal train of thought, as a private man, with the right hand. Through semantic and not stylistic contrast, we are going to see him below placing others in opposition to himself by changing hands: OTHERS. When LJ employs the left hand to report the speech of President Chirac, and the right hand for his own discourse, he establishes a dichotomy between others and himself:

(6) The President told me very clearly: [L] Me, I’m committed to the stability pact. [R] Me, I haven’t changed my mind about the stability pact,

The notion of “oneself opposed to others” appears to be expandable, going from the individual to the group, the country, and the continent, each contrasted in turn to the other individual, group, country, or continent situated on the left. LJ, as a right-hander, positions himself on the right and thus puts the “other” entity, Chirac, on his left. LJ is French, so France is on the right and Germany is on the left. France, being part of Europe, is placed on the right, in opposition to the U.S. located on the left. Table 1.  Others and Oneself OTHERS Left Hand

ONESELF Right Hand

Him: Chirac Him: a protestor Them Luxembourg Foreign country Germany USA

Me Me Us France France France Europe

OTHERS AND THE LEFT. LJ’s gestural system is coherent. He expresses himself as a private man with the right hand, but uses the left hand as the person in charge of the left-wing political party. For a right-hander like him, others, who are experienced



From left to right 

as the other self or the opposing other, are signified with the left hand. The unexpected occurrence of these two distinctions within the same utterance explains why two successive references, one to others and one to himself as the prime minister of a left-wing cabinet, are both accompanied by a left-hand gesture:

(7) It is together that the [L] President of the Republic, head of the delegation in the summit conferences of the European Council, [L] and the Prime Minister that I am

The gesture could have had a simply enumerative function, by locating X on the left and Y on the right, or even a deictic function by locating others on the left and oneself on the right. This is not the case. The gesture does not serve the purposes of enumeration but of qualifying the individuals named: the gesture specifies that the President is the other person in charge and that the Prime Minister is the person in charge of the Left. LJ replaces the enumerative contrast with a qualifying localization. The gestural information provided complements the utterance: the denomination is verbal and the qualification of the persons named is gestural. In summary, the comparison of the symbolic representations leads us to the following interpretive hypothesis. LJ, a right-handed individual, uses his right hand to express himself as an individual: the choice of the right hand corresponds, for him and probably for all right-handers, to the expression of the intimate self. The reference to others, other similar and different individuals, implies by analogy the intervention of the other hand: otherness is expressed by changing sides. This representation of two similar elements supposes the underlying and pregnant experiential schema of symmetry inherent to all living bodies: the reference to two autonomous and similar entities is expressed by the differentiated use of the two hands, autonomous and similar bodily entities, the dominant hand being reserved for the expression of ‘myself ’. 1.3

Two-item enumeration

The idea of separateness is expressed in enumerations of elements to be distinguished from one another. The 34 occurrences of this type of dichotomous enumeration can be reduced to three types of gestures, each representing a different cognitive attitude (Calbris, 2003, pp. 78–85). The gesturer finds himself mentally in front of two entities (say, flowers and fruit), or in front of two distinctive characteristics of the same type of entity (white and red flowers). This kind of internal differentiation can lead to the creation of two new entities (a white bouquet and a red bouquet). Abstract entities are analogically signified by (human) bodily entities. So, by metaphor, when LJ is facing two distinct entities, he accounts for their separateness by using both hands alternately, first the left then the right [L, R].



Geneviève Calbris

Accordingly, in Example 8 in Table 2, he is talking about striking a balance between what is demanded from capital (left hand) and labor (right hand), and then from direct taxes (left hand) and indirect taxes (right hand). The constituents of each pair must be “balanced.” When he wants to distinguish a distinctive quality within the same set, he does so by making a differentiating movement to the left and then to the right [.l,.r], either with one hand or with both hands together to form a single configuration. Example 9 illustrates such a single entity, “companies” (presented with a left-hand gesture), which includes very large companies (on the left) and medium-sized companies (on the right). He may also want to create two new entities by differentiation. In this case, he communicates the act of separating by a symmetrically opposed movement of each hand, the left hand moving leftwards and the right hand moving rightwards [L.l, R.r]: see Example 10 below. While in the first case (8) he is describing a separation [L, R], in the third (10), he is making a separation [L.l, R.r]. Table 2.  Two-item enumeration ex.

gesture left

right

gesture principle expressed

Hand [L], movement [.l] Hand [R], movement [.r] (8) 1 [L] 3 [L]

budget that strikes a balance between what is demanded from capital, from labor, [R] 2 from direct taxes, from indirect taxes [R] 4

1 [L.l]

the Matignon agreements between a certain number of companies, very large ones but also medium-sized ones

[L.r] 2

qualities

problems, I’ve said this, regarding both identity, and strategy

[R.r] 2

Entity creation

(9)

(10) 1 [L.l]

Opposable entities Opposable

Whether it is a matter of differentiating substances or qualities, the principle of distinction is inspired by the axis of symmetry that separates every living body into two parts, right and left (8, 9, 10). This distinctive separation is expressed by a change of hand (the right and the left) if it is about entities (8, 10), and by a change of direction (one movement to the right, another to the left) if it is about qualities (9, 10). In other words, we are dealing with two metaphors that can be cumulative. The first concerns distinctive separation, inspired by the perceptual separation of a whole divided into two parts, right and left, along the axis of symmetry (8, 9, 10). The second concerns identificatory delimitation, an abstract entity being represented



From left to right 

by a concrete, physically located entity (8, 10). We find them cumulated and both involved when new entities are being created by subdivision (10). Moreover, note that the gestural representation is itself a metaphoric depiction of the metaphors in question. The gesture re-motivates the metaphor whose abs-traction out of the concrete it reveals. For all such occurrences in the corpus, the gesture starts either with the left hand followed by the right hand, or with a leftward movement followed by a rightward movement. The first element uttered is on the left, the second is on the right. In other words, with a few rare exceptions due to semantic considerations – such as the idea of an opposition being translated by the use of the opposite direction to the norm – the normal progression goes from left to right. What does this signify? Why is the left hand used first here? The explanation will be given later, in 3.3. 1.4

Three-item enumeration

The same holds true for three-item enumerations. The first element in the enumeration is represented on the left, the second on the right, and the third and last in the middle, generally with both hands: (11) [parallel palms facing each other as if to frame something] sums, [entire configuration is moved to left side] for farmers who raise, [then moved to right side] or who grow, [then back to center] or who develop. (12) LJ is talking about an important meeting that took place in New Caledonia:... between [left palm on table] Jean-Marie Tjibaou, [right palm on table] Jacques Lafleur, [then re-centered, palms face to face] and Michel Rocard. This re-centered ending corresponds to our embodied walking schema: one imbalanced step with one foot, a second step with the other foot, and then stopping in a stable position, with both feet brought together at the end. Bipedal and bimanual, man transfers this movement to the hands. He also transfers the idea of forward progression from the back-to-front axis to the left-to-right transverse axis, since the gesture begins on the left. See section 3.3. The two concerted experiences, equilibrium via the symmetry of the body situated in a transverse plane and the forward progression of walking, quite naturally interfere. How can one harmonize the representation of symmetry with that of progression? I propose that the back-to-front axis of progression is projected onto the transverse axis. The latter not only accounts for symmetry (left/right) but is also an axis of progression going from left to right, in conformity with the direction of writing in the Western world. Thus, symmetrical progression goes from left to right. Now, this transverse axis metaphorically represents the progression of



Geneviève Calbris

observed events; it represents logico-temporal succession: the first term in the enumerative series starts off on the left (Two-item, Three-item, N-item enumeration). 1.5.

N-item enumeration

Example 13 offers a synthesis of the three perceptual schemas evoked: firstly, the axis of symmetry (two symmetrically situated elements), then the directional axis of walking (personal spatio-temporal progression from the back to the front), and finally, the directional axis of occidental writing (progression of events from left to right). The first pair of equivalent terms is presented from left to right; the second and new pair (also from left to right) is presented along another line located further forward, a step forward on the spatio-temporal axis of the speaker-actor. Abstract3, metaphoric4 deixis emerges from our sensory-motor experience of the body; it is physically motivated. (13) One addresses the problems of security: [right index finger lowered on to the table] in college, [displaced to the right] at school [returns to the left] in the neighborhood [moves forward] on the bus [displaced again to the right] on the subway or metro In the above example, by successively displacing his right index finger on the table, LJ locates the security problems inside a square: at the bottom of the square he opposes the establishments, enumerated from left to right, to the means of transport located at the top of the square, also enumerated from left to right. Transport:

Left bus (4)

Right (5) subway/metro

Establishment:

college (1) neighborhood (3)

(2) school

To thoroughly examine this issue, I think it would be useful here to give a selection of examples produced by other speakers who this time accompany their enumeration with a lateral tilting of the head (Calbris, 1983). The lateral tilting of the head does not necessarily begin on the left; sometimes it begins on the right. Does that contradict the metaphorical principles mentioned earlier? The comparison of the following examples, both with regard to their physical expression and to their semantic content, particularizes the nuances of the metaphorical game (14–16).

3.

McNeill, D., J. Cassell, & E. T. Levy (1993). Abstract deixis. Semiotica, 95, 5–19.

4. Sweetser, E. (1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier.



From left to right 

The enumeration of a series divided up into dichotomous pairs (organized vs. unordered, sacred vs. profane) does indeed begin on the left (14). Likewise, the enumeration of a series of elements whose order of presentation follows a logical progression (from the most important administrative zone to the least important one) also begins on the left (15). By contrast, the enumeration of a series of terms that are not opposable, that cannot be arranged into an order, but are equivalent because they are synonyms, can begin on the right (16). (14) … and on the other hand, differentiated into euh [head.l] organized space, [.r] unordered space, [.l] sacred space, [.r] profane space (a lexicologist, 1980) (15) During the presidential campaign F. Mitterrand announces: more liberty [head.l] for the regions, [.r] for the departments, [.l] for the communes, [.r] for the associations (05.05.81) (16) An actor interviewed about a detective film talks about various: [head.r] secret, [.l] special, [.r] marginal, [.l] parallel services (1981) It ensues that the principles of dichotomous symmetry (one side, the other side), of logico-temporal ordering (from left to right) are indeed followed in the enumerative series. One also observes that the lateral ‘balancing’ movement of the head presents the advantage of metaphorically representing an acoustic balancing (14: repetition of the same term) or a semantic balancing (16: series of synonyms). A final observation, the spontaneous choice of the body segment (the head instead of the hand) to perform the alternating movement is therefore not arbitrary, but justified. We are dealing with a ‘physical metaphor’ (feet -> hands -> head) at the core of the gestural representation, which is itself metaphoric. 1.6. Balance and counterpart “One or the other” – this may be the expression of a choice or a hesitation. One compares the value, the symbolic weight of X and Y, one weighs up the pros and cons before choosing, unless both entities are interdependent and require a certain equilibrium between them in order to function. An increase causing more weight on one side (growth is greater) will be compensated by a slight increase on the other side (a slight rise in prices). The idea of the counterpart is expressed by a change of hand: (17) Well [hands framing the goal, palms pointing forward] this, this battle we must wage, [right hand draws a rising diagonal line] and I think that if the growth is greater, [left-hand finger pincer] there’ll necessarily be a, a slight rise in prices.



Geneviève Calbris

Presented verbally as a consequence of growth “if..., there’ll necessarily be...,” the rise in prices is presented gesturally, as its negative counterpart, by a change from the right hand to the left hand. The metaphoric gesture completes the utterance. 1.7

Signalling a parenthetical remark

In considering the examples below, it appears that a parenthetical remark – shown between dashes – is regarded as a side issue that momentarily interrupts the current line of reasoning: (18) If this crisis shakes each of us up, [R in fist] and makes us pay more attention to the unemployed in every one of our decisions – [L points to addressee who represents the people] including both men and women – [R again] then I think this crisis will be healthy (19) And what we are currently doing, [R, palm facing down tapping on table as if to mark off] the Minister of the Interior, but the Minister of Economy and Finance – [L raised and held up during side remark] you saw the surveys from the tax inspection office – [R again] but the Minister of Agriculture, In a very appropriate way, LJ switches hands for exactly the duration of the parenthetical remark, the side issues of the right-hand topic being on the left, and vice versa. In (18), LJ changes hands, as a cut-in, in order to involve his interlocutor, whom he designates as the representative of the people he is addressing. In (19), the parenthetical remark “you saw...” is signified by a change to the left hand, with the same configuration, palm down but held until the resumption of the list of ministers, marked off by taps on the table with the right hand. The process of utterance production is experienced by the speaker as a path towards a goal whose central line is the topic. One always mentally places oneself facing one’s center of interest, with the topic to be verbally developed in front of oneself, like the path to be travelled. The accessory element that comes to mind is thought/experienced/represented as a side issue. The speaker stops momentarily en route (hand suspended for the duration of the parenthesis) in order to envisage or present a side issue pertaining to the question (depicted by switching hands) before continuing along the path, which he indicates by reusing the initial hand. 2. Symmetrical development The normal left-to-right progression observed in the enumeration examples presented above expresses a temporal or logico-temporal course. Here, a symmetri-



From left to right 

cally progressing movement corresponds to an evolutive process, as shown in the example below. Two distinct processes are at stake here. As such, they are represented with one hand followed by the other, but the hands do not move forward. If the right hand moves rightwards, the left moves leftwards, even though the progressions in question will take place in the future: (20) To raise the question of self-determination: [L.l] Was New Caledonia going to remain French [R.r] or was it going to choose independence? One finds oneself confronted with a question which is resolved by taking into account the existing system of signs and by comparing similar contexts. The gesture does not represent the logico-temporal progression of events because, in that case, the left hand would go to the right. But it moves in the opposite direction, to the left. Why don’t the two hands, each representing a possible evolution in the future, move forwards, one after the other? Then they would have represented two equivalent progressions! But LJ is talking about different and contrasting evolutions. Couldn’t he have accounted for this idea via two diverging lines moving forwards? That is not what he represents: one line goes to the left, and the other to the right. One approaches the explanatory hypothesis by comparing analogical cases, i.e. other utterances accompanied by a leftward movement of the left hand and which have in common the presentation of the idea of an evolution: (21) Because we decided [L.l] immediately to actively take political action to create jobs (22) Our 1998 budget is [L.l] progressing with respect to the ’97 budget [.l] because we are reaping the benefits of growth (23) We, [L.l] we promoted growth (24) [L.l] let’s continue talking The idea of a “process,” present in the decision regarding which policy to implement (21) as well as in the evolution of the budget, the reaping of the benefits of growth (22), the growth process to promote (23), and even simple continuation (24), is signified by a leftward movement if produced with the left hand.

L.l ← • → R.r, L.l ← • (→ R.r)

The gestural representation performed by the right hand would create confusion, because the logico-temporal course is conceived and expressed in Western cultures as going from left to right. It is also logical from the speaker’s viewpoint to spontaneously resort to the other possibility, a centrifugal movement of the left hand in order to avoid confusing two processes, one which is logical, and the other natural.



Geneviève Calbris

Thus, in this leftward movement it is not the movement’s end point that is pertinent but its point of departure. Linked to the idea of evolution common to both utterances, the centrifugal movement (of the left hand toward the left) expresses the symmetrical expansion, the symmetrical development of all living entities, and which is constantly observed in nature. The gesture in (20) reveals the metaphor linking the natural evolution of an entity (New Caledonia) with that of every living entity whose process of maturation is characterized by a symmetrical development. The symmetrical extension in width appears more pertinent than enlargement, a more general trait retained for depicting an elevation or an increase, as a stage of growth. The symmetrical development is depicted on the transverse axis by two centrifugal lines. The stylization that is representative of the phenomenon via its projection on to a plane, on to a line, creates ambiguity: is it a question of two contradictory orientations or the natural symmetrical evolution of a living entity? Having examined the utterance, it ensues that the gesture expresses the comparison (left hand/right hand) of two possible natural evolutions of the same entity. How can the choice and the symmetrical development be expressed simultaneously? Via metonymic decomposition, the pertinent evolution of one half represents that of the whole. The first evolution (left hand) is represented by that of the left half (centrifugal movement to the left), and the second evolution (right hand) is represented by that of the right half (centrifugal movement to the right). The gestural representation of the result of the maturation process undergone by a concrete referent (increase in volume) would ‘sculpt’ the object to be visualized (Müller, 1998, p. 117): the representation expresses the visual effect. By contrast, the more abstract gestural representation  of the maturation process itself, characterized by a visible phenomenon of symmetrical expansion, implies a centrifugal representation, the left hand going to the left and/or the right hand going to the right. 3. Progression along the transverse axis Due to its left-to-right orientation, the transverse axis can be advantageously used to oppose notions (symmetry of the body). Because of its capability to represent opposition, we shall see that this axis supports two other axes of physical progression: the growth axis (upward), and the walking axis or paths in space or time (forward). The scale of values on this axis, whose initial symbolic direction is bottom-totop, is transposed to back-to-front (walking direction) and left-to-right (writing direction), with the negative end of the scale on the left and the positive on the right. Let us now look at the different ways of locating entities on the transverse axis.



From left to right 

3.1

Value axis

For example, to represent a range of values, the low numbers are on the left, as are ideas like regression, reduction, and deficit: (25) [thumb and index finger forming a point, right finger pincer appears to be moving a cursor to the right] we slightly increase public expenditures, [then left finger pincer does likewise to the left] we continue to reduce the deficits (see illustration) We slightly increase public expenditures Nous augmentons légèrement les dépenses

We continue to reduce the deficits

Figure 6.  We slightly increase public expenditures

Figure 7.  We continue to reduce the deficits

Nous continuons à réduire les déficits

Value axis 3.2

Spatio-temporal axis

A path in space or time is depicted by a left-to-right movement. But given that body symmetry allows this axis to account for splitting in two as well as two-entity oppositions, it can be used to oppose past and future, or precedence and succession, by locating the past on the left side and the future on the right side. Here is a representative sample from the 20 occurrences noted in the corpus: (26) and so I can clearly see the beginnings of a tendency to say: [a look at left side of table indicating in advance where left index finger will touch table] Ah he did say that one day, [then right index finger touches a point farther away on right side of table] is he going to do it?



Geneviève Calbris

3.3

Logico-temporal axis

Given that causes precede effects, the cause or the condition of a phenomenon will be located on the left, and the effect or the consequence will be on the right. In the following example, the gestural location of the ideas expressed indicates the logico-temporal relationship between them: income on the left must precede subsequent expenditures, situated on the right: (27) a certain number of priorities [right hand points left] because there, we’re talking about income, [then points right, a gaze at addressee, with body leaning forward] should perhaps talk about what, about what it’s used for, namely, expenditures The following example concerning the distribution and production of goods has the advantage of presenting two schemas that have been studied: the separate entity and the first condition. Both actions are conceived as two objectives whose realization requires the logico-temporal relationship to be respected: produce first in order to be able to distribute later. On the transverse axis representing logico-temporal succession, production will therefore be located on the left and distribution on the right. Although the logical relation is given by displacing the hand, the representation of the objectives conceived as two different entities requires both hands. (28) And so I’d like to tell you [L.r] in order to distribute, [L.l] must produce. Therefore, our action, it’s going to be guided in the long term, by the objective [L] to produce | [R] and to distribute5 Interpretative hypotheses Some of the proposed hypotheses are connected; others overlap. The symmetrical development of all living entities (2 below) determines the pregnant image, constantly observed around us, of an entity which is clearly symmetrical and divisible into two parts that cannot be dissociated (1). This symmetry obliges us to master equilibrium in order to be able to act and to move along the sagittal axis. The schemas of progression (3) and symmetry (1) interfere, resulting in a symbolic asymmetry already partly initiated by the dominant hand (4).

5. For clarity’s sake, the manual configurations, other elements of metaphoric representation, have not been mentioned.



From left to right 

1. Symmetry Let us set aside references to left-wing politics, named for the side of the semicircle (Chambre des Députés) where left-wing politicians sit in the French parliament. This disposition to the left, physical and objective, results from the external world. Its expression alone is transposed on to the body. All other lateralized gestures referring to an abstract entity are rooted in perceptual experiences. Right-handed, LJ uses his right hand to evoke anything that is truly personal. He uses his left hand to refer to others, “another” of his kind, and even the very notion of otherness. One can assume the opposite for left-handers, an hypothesis worth verifying. This symbolization, which divides the body in two to evoke two persons, and even creates a duality between the private individual and the social one – the righthanded person on one side and the left-wing party leader on the other – can be accounted for in terms of our self-perception of our bodies as apparently divisible into two parts. Body parts come in pairs, not only the arms and legs which are movable entities that can be visually disconnected, but also elements that cannot be separated from the main body such as the eyes and ears. Hands and legs are continuously experienced as synergistic and complementary: an arm in a sling is particularly crippling, and upright stance and locomotion require symmetry. The central axis that separates these body pairs is drawn by the spinal column, and marked off on the ventral side by the nose, mouth, neck, navel, and sex organs. This image of symmetrical elements spread along a central axis or a potential separation line is highly salient. Every manufactured object is designed in our image: it has two sides, with the “ventral” side being visible, functional, and always symmetrical. The physical, natural characteristic is metaphorically transferred on to cultural production. This correspondence responds less to the demands of utility than to a need for a conceptual order, for an imposed norm of variable form but always abided by. This scheme of pairs and symmetrical separations6 is found in the “otherness” examples, and in the two- or three-item enumerations. In short, we are dealing with a Siamese entity of corporal elements whose action as a binomial requires the duality to be balanced. In other words, we permanently have the image of the 2 in 1, of the 1 in 2, and we already have the game of relations between these halves: the complementarity 1+1, the opposition 1/1, and the equivalence 1=1. These relations are gesturally represented in the enumeration (1+1), the dichotomous enumeration (1/1), and the balanced enumeration (1=1). 6. Its fundamental character was noted by Jousse (1940): for him, ‘bilateralism’ alongside ‘mimicry’ (Fr. ‘mimisme’) and ‘formalism’ (tendency toward the stereotype) is one of the three fundamental laws governing our mode of knowledge.



Geneviève Calbris

The proprioceptive mobilization of the bodily elements in symmetry enables the equilibrium of the body to be maintained or re-established, whether we are seated, standing upright or walking. The basic balance schema consisting of force vectors and some point or axis or plane in relation to which those forces are distributed (Johnson, 1987, p. 85) allows for many metaphorical projections, one of which is compensation (see 1.6). Left

References to others

() /||\ /\

Right

References to oneself (for a right-hander)

__||__ Separation I Secondary side

I Central axis

Dominant side

Figure 8.  Symmetry

So there is an asymmetry in terms of strength (Secondary – Central – Dominant), already noted in Russian by Cienki (1999). In Western culture, the right-hander is ‘skilful’ and the one who is ‘clumsy’ (Fr. ‘maladroit’) is left (Fr. ‘gauche’), in his physical as well as his social activity. Let us not forget that the French word sinistre (n. ‘catastrophe’, adj. ‘lugubre’, meaning ‘lugubrious’) comes from the Latin sinister, ‘left’. Another argument taken from practical life: a right-hander prefers to break his left arm rather than his right one; he feels his left arm is less important. The common experience of lateralization favors the right side where the dominant hand is situated, insofar as the majority of the human population is right-handed. The asymmetry to the advantage of the right becomes cultural; society adopts the point of view of the majority. The positive value accorded to the right, which is already explained by a majority lateralization to the right, will be reinforced by other perceptual experiences expressed on the diverse axes of progression, and re-grouped on the transverse axis.



From left to right 

2. Symmetrical development Cells divide in two to form an embryo. Accordingly, all maturation processes observed in nature come about through the symmetrical development of volume, characterized in two-dimensional vision as a gain in height and a symmetrical extension in width. Once mature, the body reaches its maximum height, volume, and weight. These physical parameters are tightly linked to each other in our mental representations, where physical strength determines social position and power. Inversely, a desire for self-affirmation or domination is expressed symbolically by greater height or greater breadth.

Figure 9.  Symmetrical development

The natural evolution of a living entity whose maturation process is characterized by a symmetrical expansion is represented by a symmetrical extension from the center toward the left and toward the right. 3. Progressions along the transverse axis How can we explain the fact that the transverse axis is the carrier of three progression axes? Let us consider these axes one by one before looking at the transfer process. Vertical axis pointing upward, as in growth. Physical growth is manifested by an increase in height, and no matter which kingdom (animal or vegetable) it belongs to, the mature adult is stronger, in two senses of ‘strong’. Strength is marked symbolically by greater height, as exemplified by the race among multinational corporations to build the tallest skyscraper. The vertical axis pointing upward represents the social hierarchy and becomes a scale of values, with the negative or inferior at the bottom of the scale and the positive or superior at the top (Axis 1). The ascending scale of values progresses in fact toward greater strength or intensity. The underlying schema would therefore be that of a push toward self-realization and by metaphorical extension, toward the realization of any entity with regard to its essence/specificity. This impression of ascending movement is seen in diverse aspects of life. Already experienced in our efforts to struggle against gravity at the beginning of our lives, this push is felt in the energizing emotions of anger, joy, and desire, physical or sublimated. Sagittal axis pointing forward, as in walking. With their sensory organs on the ventral side and their feet pointed forward, no bipeds walk sideways like a crab;



Geneviève Calbris

they walk forward. This oriented line becomes the schema for spatio-temporal travel toward a goal (Axis 2). The steps just taken are behind oneself (immediate past) and those to be taken are in front (future). This point of view appears to be specific to Western societies, for in other cultures, for example that of the Aymara, who situate themselves in a static way on the time line (Núñez & Sweetser, 2006), the past is located in front of oneself because it is known (Klein, 1987). Moreover, when human beings move toward a goal they have set for themselves, they feel their entire body in motion, in addition to seeing the physical environment changing constantly as they move. Perceptually, the sagittal axis is the axis of change, and hence of action; it is the only axis along which the whole body is engaged in the action, and where the person senses him/herself to be a genuine actor. It is indeed along this axis that the gesturing speaker situates himself as an actor and depicts personal commitment. Transverse axis pointing toward the right, as in writing. The direction of writing in the Western world is from left to right. Unlike the walking axis where human beings sense their whole body moving, on the transverse or writing axis (Axis 3) they can see their hands drawing a line, and can therefore look at the action from a distance. The perceptual experience is different on these two axes, as revealed above by the differentiated gestures found for temporal localization. Time is represented on the sagittal axis (walking) for situating something with respect to the current moment (past-present-future), but on the transverse axis for situating something with respect to any other moment7 (precedence-succession). The notion of logico-temporal succession, always seen from a distance, is represented on this axis, with the cause situated on the left, and the effect on the right. In fact, logico-temporal succession is never gestured on the back-to-front axis. Transfer of the vertical axis (Axis 1) to the sagittal axis (Axis 2). To depict two types of physical progression, growth and walking, the vertical and sagittal axes are combined on the sagittal axis, which thereby acts as a back-to-front scale of values. Axis 2 thus represents not only personal spatio-temporal travel but also the notion of growth. Furthermore, L. Ekberg has highlighted this transfer on the lexical level, hence the title of her 1997 article, “The mental manipulation of the vertical axis: How to go from ‘up’ to ‘out’, or from ‘above’ to ‘behind.’” Transfer of the sagittal axis (Axis 2) to the transverse axis (Axis 3). The second step in the set of symbolic translations is the collapsing of the two physical progression axes, walking (Axis 2) and left-to-right writing (Axis 3). Accordingly, the transverse axis pointing to the right represents a self-removed view of a spatio7.

Calbris G. (1985). Espace-Temps: Expression gestuelle du temps. Semiotica, 55-1/2, 43–73.

Calbris, G. & J. Montredon (1986). Des gestes et des mots pour le dire. Paris CLE International, pp. 140–141.



From left to right 

temporal course, with the advantage (given the symmetry of the body) of being able to easily oppose two notions, like precedence and succession, in a single movement that goes from one side to the other. We thus have three types of physical progression – (1) natural growth, (2) the walking direction, and (3) the writing direction – and we transfer axes for ergonomic reasons by going successively from bottom-to-top (growth axis), to backto-front (walking direction), and to left-to-right (writing direction). The upward axis (Axis 1) is laid down in front of us (Axis 2), and then by lateral rotation, becomes the sideways axis (Axis 3). Things which should be located gesturally under the table or behind oneself, being inaccessible, are transferred to the front of the body, either to behind the hand or behind some central point. This transfer is mentally possible insofar as the mind draws analogies between the various types of physical progression. In replacing the other two axes, the transverse axis (Axis 3) receives each of their specific meanings: spatio-temporal travel (Axis 2) and a scale of values (Axis 1).

Left

Right --------------------•--------------------> 1. Growth, value scale Negative value Positive value 2. Walking, spatio-temporal progression Past, precedence Succession, future 3. Writing, logico-temporal succession Cause Effect

Figure 10.  Transfer of the progression axes to the transverse axis

One must not forget the schema of symmetrical growth toward maturity mentioned above, which accounts for any developmental process and is equivalent to a sort of fourth axis. It is represented by a symmetrical spreading movement. The transverse axis thus carries four symbolic axes, namely, symmetrical development (4), logico-temporal succession (3), and, by proxy, spatio-temporal progression (2) and a scale of values (1).

1. G rowth, value scale 2. W alking, spatio-temporal progression 3. W riting, logico-temporal succession 4. S ymmetrical development

---------------------•------------------> ---------------------•------------------> ---------------------•------------------>

Figure 11.  Representations collapsed on to the transverse axis



Geneviève Calbris

Better still, the same physical axis can even carry symbolic axes that point in opposite directions, at least for left-handers8, who make a rightward left-hand movement for a spatio-temporal or logico-temporal progression [L.r] and a leftward lefthand movement for the evolution of a process [L.l]. When LJ’s right hand is not available (for example, because the journalists are on his right or he is holding a sheet of paper in his right hand), he acts like a left-hander, as we have seen: his left hand moves to the right to mark time and to the left to depict an evolving process. 4. Progressions and symmetry on the transverse axis The symbolic asymmetry between the left and right sides indicated above is explained by the transfer of the diverse progressions and scales of corresponding values onto the transverse axis, which is the only axis employed by the body divided along an axis of symmetry into two parts, left and right. The superimposition of progressions on a bisected axis culminates in the opposition of the extremes in symmetrical pairs. The negative values accumulate on the left, and the positive values on the right. Thus, one has values that are at the same time contrary and symbolically oriented, toward the right. Ultimately, thanks to the schema of opposition, every value determines its opposite. And in the new duality envisaged, the depreciated minor element will be mentally located on the left. This symbolic dualism found in many very different cultures manifests itself in all aspects of daily life (Needham, 1973), opposing man and woman, child/adult, girl/boy, weakness/strength, obscurity/clarity, etc... (A)Symmetry 1. Growth 2. Walking 3. Writing

LEFT clumsy others ___ | ___ accessory negative, inferior last, second old

RIGHT skilful oneself important

------•------> superior, positive ------•------> first (arrived) ------•------> new

negative values

positive values

Moreover, we have noted a functional and symbolic asymmetry to the advantage of the right hand. The right-hander favors the right hand, his dominant hand, and what he favors will mentally be on his right-hand side: oneself on the right, others on the 8. See Patrick Poivre d’Arvor, left-handed journalist, interviewing LJ. Examples are reported in a forthcoming book on the symbolic space of the French.



From left to right

left. By extension, what is important to him will be mentally located on his good side, on the right. Taking into account the perceptual schema that generates oppositions, what he considers to be accessory will be located on his left. The table above summarizes the whole data set: one sees how the concatenation of similar schemas (diverse progressions) and their combination with others (opposing symmetry) produce new, synthetic ones (opposite values symbolically oriented). Conclusion Let us now recapitulate the evidence, the gesturally expressed metaphors, as well as their likely origins in progressively layered experiential schemas retracing, so to speak, the history of the human body: from its symmetrical development, the acquisition of upright stance by resisting gravity, the bipedal body maintaining equilibrium, to the beginnings of alternate locomotion by moving one foot after the other, enabling continuous action along the sagittal axis. Changing hands expresses the existence of two distinct entities (the one/the other, oneself/others), even more so their dichotomous opposition (enumeration opposing capital to work, direct taxes to indirect taxes, etc.). Changing hands also expresses the existence of two distinct processes, the movement itself of the hand being responsible for expressing the process in question. The speaker opposes, for example, an increase in X and a reduction in Y, or one particular evolution to another possible evolution of X. Changing hands expresses a relation of equivalence between two terms linked to the notion of equilibrium, just as the idea of counterpart is linked to that of regaining balance. Finally, changing hands expresses parenthesis, the side thought annexed to the central question. The underlying schema here is no longer inspired by an ensemble, by a body divisible into two halves, nor even by a symmetrical body to be kept in balance during locomotion; it becomes that of a central axis and its sides, the symmetrical body confronting a path conceived in its own image, a central path and its asides (schema metaphor). The choice of hand (left or right) implicitly refers to the place it occupies on the transverse axis pointing from left to right. The simple fact of using the left hand (situated on the left of the transverse axis constituted by the body) localizes a notion, a value, an event at the starting point of the axis. Yet this physical axis corresponds to several symbolic axes: – The hierarchical axis of values going from left to right; the change of hand will oppose, for example, the bottom-of-the-range products (left hand, therefore on the left) to the top-of-the-range products (right hand, therefore on the right).





Geneviève Calbris

– The spatio-temporal axis of past-present-future going from left to right; the change of hand opposes a preceding event (left hand) to a subsequent event (right hand). – The logico-temporal axis accounting for the logical course of actions along the time line; the change of hand evokes their relation by opposing the preceding event, i.e. the cause (situated on the left by employing the left hand), to its consequence, the subsequent event (situated on the right by employing the right hand). In short, the judicious use of one hand then the other indicates the chrono-logical order of events to be respected: revenue first, expenditure later. Likewise, produce first in order to be able to distribute later. The principles of symbolization interfere with each other. Taking into account the left-right orientation of logico-temporal succession, all enumerative sequences of two, three, or N items, including the enumeration of a dichotomous pair, begin on the left: the axis of symmetry is oriented. And inversely, the oriented axis is symmetrical which, as we have seen above, enables one to oppose terms situated at the beginning and at the other end of the scale. The manual laterality transforms the a priori symmetrical axis into a dissymmetrical axis: the dominant hand is the right one in the majority of the population. The two halves or entities represented by each of the hands therefore do not have the same symbolic weight, nor do they have the same value. In fact, via metonymy, the right hand represents the subject of the action, the personal viewpoint and the notion of oneself. Via dichotomous opposition, the left hand represents others, similar entities, and not having priority. The last interference, the right side finds itself simultaneously enhanced in a fourfold way: by the manual laterality of the majority, and by the three axes of progression going from left to right. The metaphorical process induced by perceptual experience therefore appears like a permanent cognitive activity that maintains itself by exercising itself in all domains, including at the core of gestural representation. Along the way, we have effectively observed operations that substitute segments, transfer axes, project threedimensional space on to a simple line, and restore the symmetrical development of volume in a centrifugal (symmetrical) movement on the axis of symmetry. The semiological analysis of coverbal gestures confirms the existence of mental image schemas and enables one to envisage the process of their imbrication, creating new schemas that will in turn tend to organize themselves into systems.



From left to right 

References Calbris, G. (1983). Contribution à une analyse sémiologique de la mimique faciale et gestuelle française dans ses rapports avec la communication verbale (1. Expérimentation 2. Taxinomie 3. Synthèse 4. Annexe illustrative). Lille: Université de Lille III [1983, Thèse de Doctorat ès lettres, Paris III]. Calbris, G. (1985). Espace-Temps: Expression gestuelle du temps. Semiotica, 55 (1/2), 43–73. Calbris, G. (2003). L’expression gestuelle de la pensée d’un homme politique. Paris: CNRS Éditions, collection CNRS Communication. Calbris, G. & J. Montredon (1986). Des gestes et des mots pour le dire. Paris: CLE International. Cienki, A. (1999). The strengths and weaknesses of the left/right polarity in Russian: Diachronic and synchronic semantic analyses. In L. de Stadler & C. Eyrich (Eds.), Issues in cognitive linguistics: 1993 proceedings of the International Cognitive Linguistics Conference (pp. 299–329). Berlin: Mouton de Gruyter. Ekberg, L. (1997). The mental manipulation of the vertical axis: How to go from ‘up’ to ‘out’, or from ‘above’ to behind’. In M. Verspoor, K. D. Lee, & E. Sweetser (Eds.), Lexical and syntactical constructions and the construction of meaning (pp. 69–88). Amsterdam: John Benjamins. Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination and reason. Chicago: University of Chicago Press. Jousse, M. (1940). Le bilatéralisme humain et l’anthropologie du langage. Revue anthropologique, n° 4–6, 1940. Klein, H. (1987). The future precedes the past: time in Toba. Word, 38, 173–185. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. McNeill, D., J. Cassell, & E. T. Levy (1993). Abstract deixis. Semiotica, 95, 5–19. Müller, C. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Verlag Arno Spitz. Needham, R. (Ed.) (1973). Right and left: Essays on dual symbolic classification. Chicago: University of Chicago Press. Núñez, R. & E. Sweetser (2006). Looking ahead to the past: Convergent evidence from Aymara language and gesture in the crosslinguisticc comparison of spatial construals of time. Cognitive Science, 30, 1–49. Sweetser, E. (1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier.

Gesture as a conceptual mapping tool Robert F. Williams Metaphoric understandings emerge when conceptual mappings link elements of one domain with elements of another. Cross-domain mappings seem powerful and creative, but even mundane meanings involve mappings that link elements with one another. Instructional interactions bring the process of conceptual mapping into the open as one person strives to shape the understanding of another. Using data from a study of time-telling instruction, this chapter describes two important functions of gestures during instruction: (1) guiding mappings that link conceptual models with structures in the environment, and (2) adding image-schematic structure to the conceptualization. Both help form the understanding needed to perform successfully. These critical functions of gesture are apparent only when discourse is treated as multimodal and contextual – as involving the interrelation of talk, gesture, and action with the setting and purpose of activity.

1. Introduction Metaphor is an exercise in conceptual mapping: linking elements of one domain with elements of another. For conventional metaphors, these cross-domain mappings are well entrenched and appear as regular patterns in language and thought. The study of conceptual metaphor is largely about describing these patterns. In actual discourse, meanings are negotiated in the context of activity; here conceptual mappings need to be dynamically established. How do conceptual mappings get set up in discourse? What role, if any, does gesture play? These are the questions that guide the research described in this chapter. This chapter explores the conceptual mapping process as it unfolds in real time in instructional discourse. In instructional interactions, an experienced participant acts overtly to guide the conceptualizations of a novice. Here, gesture appears to play a key role, both in guiding the mapping of conceptual elements to structures in the environment and in adding structure to the ongoing conceptualization. These functions will be described and illustrated in what follows. First, a few words about the approach of the study are in order.



Robert F. Williams

The functions of gesture highlighted in this work emerge from the interrelation of gesture with talk and the material setting of activity. Talk, gesture, other actions, and the material context must all be included in the unit of analysis. Uncovering the functions of these elements of discourse also requires that we seek relationships between the observable and unobservable: analyzing how talk, gesture, and other actions relate to the conceptual operations that are fundamental to meaning-making. The study described here is innovative in two ways: it tests cognitive linguistic theory in the world of actual discourse, and it pushes the analysis of situated activity beyond what we can see. The goal of such research is to deepen our understanding of cognition as a process that spans the internal and external, illuminating how these couple with one another in the performance of complex activities. The research described in this chapter approaches this goal by integrating two distinct methodologies: (1) cognitive ethnography, which uses participant-observation, interviews, artifact analysis, and recordings of situated activity to analyze how cognitive activities are accomplished in real-world settings, and (2) a form of cognitive linguistic analysis which analyzes patterns in linguistic as well as paralinguistic data (including graphics and gestures) to reveal aspects of conceptual structure and meaning-making operations like conceptual metaphor and conceptual blending. The episodes of instructional discourse analyzed in this chapter come from a cognitive ethnography of time-telling instruction (Williams, 2004). Cognitive linguistic analysis of these episodes reveals how each utterance, gesture, and action guides the construction of time readings from the clock display. In these micro-analyses, the mapping function of gesture is revealed. The chapter begins by discussing conceptual mapping and the role it plays in conceptual metaphor and conceptual blending. It then introduces the key idea that the material world is used to anchor conceptual entities during discourse and problem-solving. Next, the chapter presents data from the time-telling study with step-by-step analyses of meaning construction, highlighting the role of gesture in guiding conceptual mappings. Finally, the chapter concludes with some general discussion of gesture as a conceptual mapping tool. 2. The importance of conceptual mapping Conceptual mapping is the fundamental mechanism underlying two related processes of meaning construction: conceptual metaphor and conceptual blending. Conceptual metaphors are patterns of cross-domain mappings that appear in thought and language. Conceptual blends are novel meanings that emerge from online integrations of conceptual content. Metaphoric blends are conceptual blends that integrate content from distinct domains linked by metaphoric mappings.



Gesture as a conceptual mapping tool 

Conceptual metaphor is, in the simplest sense, understanding one thing in terms of another. When we hear, “Tomorrow is a big day,” we understand that tomorrow’s events will be important. This is an instance of the conceptual metaphor Important Is Big (Lakoff & Johnson, 1980, 1999). The metaphor involves two domains: the size of physical objects and the subjective importance of events. The importance of events is the target domain, what we are thinking and talking about. The size of physical objects is the source domain, a resource used to structure our thinking and communicating about the target. We make use of that resource through conceptual mapping and the projection of inferential structure. Elements of the source domain are mapped to counterparts in the target domain. Once the domains have been linked, relations among elements in the source domain are used to draw conclusions about relations among counterparts in the target domain. If we hear, “The day after is even bigger,” we know that the next day’s events will be even more important, and so on. By projecting inferential structure from source to target, we use a familiar domain of experience to support reasoning in a domain that may be less clearly defined. This is the power of conceptual metaphor, a power rooted in the mechanism of cross-domain conceptual mapping. Conceptual mapping is equally fundamental to conceptual blending. Conceptual blending is the process of integrating disparate conceptual content into meaningful wholes (Fauconnier & Turner, 2002). According to conceptual blending theory, language prompts the building of mental spaces – packets of conceptual content – as well as cognitive operations that act on these spaces. Chief among these are conceptual mapping and blending. Conceptual mapping links elements in one mental space with elements in another. Conceptual blending compresses these relations to form new, blended spaces – integrated scenes with emergent structure that supports novel inferences. Research by Fauconnier and Turner (1998, 2002) describes the principles that govern these operations to produce different kinds of conceptual integration networks. Conceptual mapping provides the webbing that holds these networks together. As an illustration, consider the simple case of the Regatta blend (Fauconnier & Turner, 2002). An article in a sailing magazine compares the progress of a catamaran sailing from San Francisco to Boston in 1993 to that of a clipper ship sailing the same route more than a century earlier. The article states: “The crew of the Great America II is barely maintaining a 4.5-day lead over the ghost of the clipper Northern Light.” Interpreting this sentence involves creating a conceptual integration network that includes a blended space and two distinct mental space inputs to the blend. One input is the journey of the Great America II in 1993 while the other is the journey of the Northern Light in 1853. There are many differences between these two events – the era, the type of ship, the crew, the exact path traveled, the weather and sailing conditions – but there is similar event structure: rapid sailing



Robert F. Williams

from San Francisco to Boston. The shared structure provides a basis for conceptual mapping across the input spaces. Mappings link counterparts: ship to ship, route to route, and so on. These are selectively projected into the blended space. The ships appear together and the route is fused into one. Time is compressed, bringing a ghost of the clipper from the past into the present, but leaving the particulars of its journey behind. The blended space presents an integrated scene of two ships sailing the same route as rapidly as possible. In the blend, these ships can be construed as competing against one another. Framing the scene as a sailing race (a regatta) supports the inference that one ship is maintaining a lead over the other. Notice that this inference derives from emergent structure in the blended space, not from inferential structure projected from source to target. In this respect, conceptual blending differs from conceptual metaphor, yet both share the critical mechanism of conceptual mapping. The previous example of conceptual blending is fairly mundane since the input spaces are so similar. The creative power of conceptual blending is revealed when the inputs come from different domains with distinct event structures. One such example is the Boxing CEOs blend (Fauconnier & Turner, 2002). Here two corporate leaders (chief executive officers or CEOs) of rival corporations are depicted as competitors in a boxing match. Understanding the depiction involves constructing a conceptual integration network that blends business with boxing. A familiar conceptual metaphor, Competition Is Physical Struggle, provides the basis for conceptual mapping: leaders of corporations involved in business competition (metonymically standing for the companies they lead) are linked to boxers engaged in regulated physical combat. Other mappings follow: the marketplace maps to the boxing ring, government regulation maps to the referee officiating over the match, advertising and price-cutting map to punching and blocking, and so on. In the blended space, one CEO can deliver a knockout punch, rendering the other unconscious; in the business input (the focus of the discourse), this equates to driving the competitor out of business. The novel Boxing CEOs blend shows the creative power of conceptual blending, a power that depends on conceptual mapping to link the disparate inputs. As the last example suggests, conceptual metaphor and conceptual blending are closely related. Just as conceptual metaphors project inferential structure from source to target, some blends “borrow compression” by using the tightly integrated structure of one input space to frame diffuse elements from another (Fauconnier & Turner, 2002). In metaphoric blends, elements from the source and target domains are integrated in the blended space (Grady, Oakley, & Coulson, 1999). Whether something is best analyzed as a conceptual metaphor or a conceptual blend is not always clear; the choice depends on such factors as the directionality of the mapping (from source to target vs. across mental spaces), the origin of



Gesture as a conceptual mapping tool 

inferential structure (imported from the source vs. emerging in the blend), and the extent to which the domains are kept distinct or blended. Typically, conceptual metaphor analyses focus on entrenched patterns of cross-domain mappings, while conceptual blending analyses focus on the integration networks that produce novel meanings. It may be that conceptual metaphor is both a distinct form of conceptual integration and, once a metaphor has become entrenched and conventional, a source for cross-domain mappings as in the blending examples described above (more discussion of the relationship between metaphor and blending can be found in Grady, Oakley, & Coulson, 1999 and Fauconnier & Turner, to appear). What conceptual metaphor and conceptual blending clearly share is conceptual mapping as a fundamental mechanism. In this chapter, our focus is on the processes of conceptual mapping that form the core of metaphor and blending and how these are realized in discourse. Because the analysis focuses on online meaning construction, the terminology of conceptual integration will be used to describe the various inputs, cross-space mappings, and blends, including metaphoric blends. Emphasis will be placed on the role that gesture plays in linking counterparts in different input spaces. To explicate that role, we need to consider how gestures couple with objects in the course of activity. 3. Conceptual mapping in a material world Conceptual blends support the generation of inferences. Some inferences depend upon complex sets of relations or the precise way that elements are arranged. Such things are difficult to keep track of. One way to support such reasoning is to use the material world to anchor blended spaces (Hutchins, 2005). Consider the cultural practice of queuing or standing in line. Here the problem is determining the order of service, and the guiding principle is that people should receive service in the order in which they arrive (“first come, first served”). Standing in line is a way of encoding order of arrival. The arrangement of bodies anchors a conceptual blend that is used to compute whose turn it is to receive service. The anchored blend is shown in Figure 1.

 Robert F. Williams

1

Arrangement of Bodies

12

3…

2

3

… …

Model of Service Emergent property: The arrangement of bodies encodes the order of service.

Queuing or Standing in Line

Figure 1.  The Queuing or Standing in Line blend. The arrangement of bodies anchors the conceptual blend, stabilizing the representation of conceptual relations. (Adapted from Hutchins, 2005.)

One input to the blend is the perceptual scene as shown in the upper left of the figure. Here the circle represents a mental space (perception of people standing in a particular arrangement) while the box behind the circle marks the space as anchored by structure in the environment (the actual physical bodies in the world). The other input, shown in the upper right of the figure, is a familiar conceptual model of service.1 The model has slots filled according to order of arrival; these slots progress toward the experience of receiving service. Two related conceptual metaphors, Purposes Are Destinations and Progress is Forward Motion (Lakoff & Johnson, 1999) motivate the conceptual mapping. The path of motion of the bodies toward their destination maps to the path of progress toward service, linking each body to a slot in the service model. The blended space, shown at the bottom of Figure 1, has a key emergent property: the arrangement of bodies encodes the order of service. This materially anchored conceptual blend enables us to use spatial configuration to reason about temporal relations. We can infer, for example, that any specific person will receive service after everyone in front of that person and before everyone behind that person; that person #2 will be served just after person #1 and just before person #3; and so on. The environment maintains the configuration of elements so we can reason about who is to receive service when without 1. This conceptual model could also be called a “cultural model” because it is intersubjectively shared by members of a cultural group, or a “frame” because it is used to frame the situation (to structure the roles and relations). For the purposes of this chapter, I will use the generic term “conceptual model.”



Gesture as a conceptual mapping tool 

losing track of the relations. Because the blend is anchored, we can act on the world to generate inferences. We might, for example, shift our gaze from person to person while muttering counting labels (one, two, three…) to determine our position in the queue. This anchored conceptual blend is a cognitive artifact: a tool for computing information with important consequences in activity. Examples such as this demonstrate how conceptual models and material anchors work in concert to support reasoning (Williams, 2004). Conceptual models link elements relationally while material anchors fix the form of those relations in particular instances. Conceptual models are tools for reasoning about the world. They relate effect to cause, part to whole, and so on, but these relations are idealized and topological. The exact configuration of the relations depends upon the particular circumstance; this gives conceptual models the power to apply in many different circumstances. The material world, on the other hand, provides physical structure we can see and interact with – structure that has configuration but no intrinsic meaning. When this structure anchors a blended space, it becomes imbued with significance. Material anchors sustain meaningful relations as we reason, even through multi-step operations. Useful anchors become specialized artifacts, making it possible to dissolve blended spaces and reconstruct them later. Over time, these artifacts and the blends they anchor become highly conventionalized. A clock face, for example, supports conventional sets of blends used to generate different kinds of time readings, some of which will be illustrated in this chapter. Through the symbiotic relationship of conceptual models with material anchors, we reason about the world while we use the world to support reasoning. 4. Uncovering the conceptual mapping process How do we marry conceptual models to material structures in particular circumstances? From the perspective of the theory we are developing here, this is a conceptual mapping problem: a problem of linking counterparts. For an expert performer, the process seems to proceed automatically and invisibly. For a novice, it might not proceed at all, necessitating some form of instruction, social interaction through which an expert guides the novice through the process of setting up the conceptual mappings needed to perform successfully. Consider again the everyday problem of reading the current time. An expert time-teller can look at the configuration of hands on an analog clock and read the time rapidly and invisibly. Some times may be immediately recognized (such as “three o’clock”) while others are likely to be constructed (such as “three forty-two”). A time reading is constructed using anchored blends to generate components such as hours and minutes. Different kinds of time readings (say “five forty-five” versus



Robert F. Williams

“a quarter to six”) involve different sets of mappings from different conceptual inputs (see Williams, 2004, for a full account). The anchored conceptual blends used to read the time are familiar to the expert and instantiated with little noticeable effort. For a novice with a basic understanding of shape, number, and the division of the day into hours and minutes, there is still much to learn in order to read a clock: which structures on the clock to attend to, which conceptual models to apply, how to map conceptual elements onto the clock face, and how to use the resulting blends to generate time components. Thankfully for the continuation of time-telling as a human practice, these aspects of time reading need not be rediscovered by each new generation of time-tellers. Instead, they can be developed through direct, active instruction from more-experienced time-tellers. This discussion of novice/expert differences tells us where to look for conceptual mapping in situ. Conceptual mapping is likely to reveal itself when one person instructs another in how to interpret some state of the world. Instruction in how to use a cognitive artifact – how to read a clock, for example – is likely to bring conceptual mapping out into the open as a socially mediated process that can be directly observed. These considerations form the basis of the study described here: an investigation of time-telling instruction to uncover the conceptual mapping process and examine the role that gesture plays. 5. Methods of the study The data presented in this chapter come from a cognitive ethnography of timetelling instruction (Williams, 2004). The study involved participant-observation during mathematics lessons in four classes (1st through 3rd grade) in two elementary schools, a multi-ethnic church school in an urban center and a private school in an affluent coastal community, both in southern California. Data were gathered over the course of a school year by observing mathematics instruction, interacting with students as they worked on assignments, recording lessons related to timetelling, collecting related artifacts, and interviewing students and teachers about time-telling content and methods of instruction. The primary data set for the current investigation consists of recordings of teachers delivering instruction in clockreading. The ethnographic component of the study is used to provide warrants for specific interpretations of this video data. Episodes of explicit instruction in clock-reading have been transcribed using a multimodal format based on Goodwin (2003). Speech is depicted in text form with indications of vocal emphasis, pause lengths, etc., in keeping with the conventions of conversation analysis (a full list of conventions is provided in the appendix). Gesture and manipulation of objects are depicted in annotated still images



Gesture as a conceptual mapping tool 

from the video. These images are linked to boxes around speech that co-occurred with the gesture or action. Annotated images were used rather than coded textual descriptions for two primary reasons: (1) the images depict not only gestures, for which some coding schemes have been developed, but also manipulations of objects, for which no conventional coding schemes are available; and (2) the analyses depend crucially upon how the gestures couple with the objects, so that a visual depiction provides better support for the analysis than would a written description or code. The annotated images focus on depicting the most significant portion of the gesture, i.e., the gesture stroke; no attempt was made to explicitly notate other gesture phases, although these may appear in the annotations when the hand begins or ends at a position that has other significance. In the transcribed episodes of explicit instruction, the teacher does nearly all of the talking, gesturing, and manipulation of objects. The analysis follows the transcript line by line, diagramming how each of the teacher’s utterances, gestures, and actions affect the mental space inputs, cross-space mappings, and conceptual blends, including which elements are profiled in the discourse. The result of this analysis is a series of diagrams that depict the construction of meaning step by step. The analyses are informed and constrained by the ethnography and by juxtaposing multiple episodes in search of common patterns. A more detailed description of methods can be found in Williams (2006), including discussion of issues related to using cognitive ethnography to study instruction. In examining the diagrams of meaning construction, special attention is paid to the functions of gesture and how gesture is coordinated with the accompanying speech and coupled with specific structures in the environment – usually, but not exclusively, structures on the clock face. 6. The functions of gesture during instruction In what follows, we examine two excerpts from a single 1st-grade time-telling lesson, chosen for its richness in representing a variety of characteristics found in the study. This lesson took place about midway through the school year. In the first excerpt, the focus is on reading times as “a quarter past…,” a relative form of time reading referenced to the previous clock hour. In the second, the focus is on reading similar times as “…fifteen,” an absolute form of time reading expressed as an explicit number of hours and minutes. Each of these forms calls for a different set of conceptual mappings to generate the time components. We thus see different conceptual blends being constructed for the same clock display. Just prior to the first excerpt, the teacher does several things to prepare her students for the clock-reading instruction she is about to deliver. She places a circle



Robert F. Williams

made of felt on a display board, covers it with two vertically oriented half-circles, and then covers these with four quarter-circles; as she does this the students called out “whole,” “half,” “one fourth,” and “two fourths,” recalling a recent lesson on dividing a circle into fractional parts. Next, the teacher pulls the upper right quartercircle away from the others and tells the students that another way to say one fourth is “a quarter.” She refers to money (another 1st grade topic), saying that four quarters make a dollar, so one quarter is one fourth. Grasping the upper-right quartercircle again, she says, “It’s a quarter of the circle,” and places it back with the others. Without fully analyzing these actions, we simply note that they lay the groundwork for what follows: reminding students of the activity of dividing a circle into parts (activating relevant conceptual models and associated vocabulary) and introducing the term “a quarter,” which the teacher explicitly links to the upper right quarter-circle, the one she will focus on in the lesson. Now the teacher is ready to begin instruction in reading “quarter past” times on the clock. She sits in front of the students and reaches for her teaching clock, an object that looks like a clock face with movable hands. These hands are linked by gears, so that moving the long hand causes the short hand to move in a clock-like way. At this point, the clock-reading portion of the lesson begins. That portion is detailed in Transcript 1, which can be found in the appendix at the end of the chapter. The excerpt is analyzed below. 6.1

The Clock Quarters conceptual blend

In the brief episode of instructional discourse (33 seconds) depicted in Transcript 1, the teacher guides students through the construction of the Clock Quarters conceptual blend. Let us first describe this blend using the typical form of a blending analysis. Then we will examine how the blend is actually constructed in the discourse. In a typical blending analysis, the analyst identifies the inputs and cross-space mappings, describes how relations get compressed in the blended space, examines emergent structure, and discusses how the blend and its associated conceptual integration network are used to generate inferences relevant to the situation at hand. Applying this methodology to the discourse produces the diagram of the Clock Quarters blend shown in Figure 2.



Gesture as a conceptual mapping tool 

Analogy

11

12

1

10

2

9

3 8 7

Divided Circle

6

5

4

Clock 11

12

1

10

2

9

3 8 7

6

5

4

Emergent property: Canonical clock quarters support reading times as quarter past and quarter till

Clock Quarters Figure 2.  The Clock Quarters blend. The divided circle is mapped onto the clock face, producing the canonical clock quarters

One input to the blend is the visual perception of the face of the teaching clock, represented by the circle in the upper right of the figure. This circle has a square behind it, indicating that the mental space is anchored by structure in the environment, i.e., by the object the teacher is holding. The other input is the conceptual model of a divided circle, represented by the circle in the upper left of the figure. The idea of dividing a circle into halves and fourths was developed in a previous lesson and reinforced at the start of the current lesson. The divided circle is prototypical: the dividing lines are vertical and horizontal, placing the quarters in a canonical arrangement. This arrangement results from a process of successive halving: dividing the circle first from top to bottom, producing halves, and then from left to right, producing quarters. In the blended space, represented by the circle in the lower part of the figure, the vertical dividing line is anchored by the clock hands while the horizontal dividing line is anchored by a pointing stick placed across the middle of the clock face. Conceptual mappings link the bounded regions on the clock face with the canonical quarters in the divided circle. These Analogy mappings are compressed into Identity in the blended space, producing canonical clock quarters. This emergent structure in the blended space has an important consequence: it supports a new way to name times, as a quarter past the hour. The blend is anchored by material structures, namely the teaching clock and pointing stick, and these material anchors temporarily maintain the part-whole relations of the clock quarters as they are used to generate a “quarter past” time reading. This typical blending analysis provides important information about the inputs, cross-space mappings, compressions, and emergent structure in the blended space, as well as additional evidence of material anchors for conceptual blends. In



Robert F. Williams

other words, it tells us about the structure of the conceptual integration network built in the discourse. What it leaves out, however, is the process through which the network is built. How are the conceptual mappings set up? What roles do talk, gesture, and other actions in the setting play? To find out, we need to examine the discourse bit by bit, diagramming the developing conceptual integration network at each step of the process. 6.2

The process of conceptual mapping

Let us begin our analysis of the process of meaning construction by noting some distinctive features of the discourse. Ordinary conversation is characterized by a fairly even balance of contributions from participants, by false starts and overlaps as turns are negotiated, and by breakdowns and repairs as participants negotiate meaning. In stark contrast to conversation, the discourse transcribed here is onesided and follows a patterned structure. The teacher talks while the students listen. The teacher controls turns, cuing both the timing and form of student responses. She checks for agreement (“right?” [lines 14, 25]) or prompts for a specific word (“the…” [lines 20–21]). Students reply with simple affirmations (“yeah” [lines 15–16, 26]) or single words (“nine” [line 22]), depending on the form of the cue. The teacher’s talk also exhibits a well-ordered structure: a series of single wellformed phrases or clauses, each ending with emphasis on a word or pair of words of conceptual importance, followed by a pause of a half-second or more. The effect is one of introducing one piece of information at a time and waiting for that information to be processed. We see that although the teacher controls the discourse, the form of the discourse is constrained by the memory and processing capacities of the listeners, a group of six- and seven-year-old children. It is the children who must follow every step in the construction of the blend without getting lost or confused along the way. With this background in mind, let us now examine each step in the construction of the Clock Quarters blend, noting how the mappings are set up. In line 2, the teacher says “if I take my clock” while she picks up the teaching clock. This initiates the conceptual mapping shown in Figure 3.



Gesture as a conceptual mapping tool 

11

12

11

1

9 4 7

6

1

2

9

3 8

12

10

2

10

3 8

5

7

Actual Clock

6

5

4

Teaching Clock 11

12

1

10

2

9

3 8 7

6

5

4

“ Clock”

if I: take my clock picking up teaching clock

Figure 3.  The Clock blend. The teaching artifact is seen as a clock

The word “clock” activates the domain of clock knowledge, which for the students is still rather limited. They know that clocks are used to tell time, and they have some experience reading the hour and half-hour times. The co-timing of the teacher’s utterance and action associates the label “clock” with the object the teacher picks up, which is, of course, not a clock – it does not keep time – but an artifact specially crafted to support instruction in time-telling. Relating utterance to action maps ‘clock’ onto the teaching clock, a mapping facilitated by the close resemblance between the two. In the anchored Clock blend, the visible portion of the teaching artifact is interpreted as the face of a clock. Students who fail to grasp this will miss the point of the lesson. Next, the teacher begins constructing the anchored blend that will be used to read “a quarter past.” The rest of the transcript depicts this process of mapping the divided circle onto the clock face. The sequence of actions exactly parallels the process of dividing a circle as the students have experienced it in class: first draw a circle, then draw a dividing line down the middle, and then draw a second dividing line across from left to right. In the transcribed excerpt, the teacher uses gestures to “draw” these elements over structures on the clock face. The first mapping occurs in line 6 when the teacher says “it’s the same circle shape” while she traces a circle from the top of the teaching clock slowly around its perimeter clockwise (Figure 4).



Robert F. Williams

11

12

1

10

2

9

3 8 7

Circle

6

5

4

“ Clock” 11

12

1

10

2

9

3 8 7

6

5

4

Clock -Circle

it’s~the same circle shape(1.7) tracing circle

Figure 4.  Building the Clock Quarters blend (1). Mapping a circle onto the clock face and enacting a path of clock hand motion

The trace continues after the utterance has ended and is released when the teacher’s finger reaches the top of the clock again. A trace such as this has both indexical and iconic components (Goodwin, 2003). Indexically, a trace highlights structure in the material environment, while iconically, it outlines a conceptual entity. The teacher’s trace highlights the darkly colored band around the perimeter of the teaching clock while simultaneously outlining the shape of a circle. The net effect of the trace is to superimpose a conceptual entity onto a material structure, mapping a conceptual element to a material anchor. The conceptual mapping results not from the gesture alone but from the relation between the utterance, the gesture, and the material structure. The utterance activates the conceptual model of a circle, profiles the shape of the clock face (i.e., brings it into the foreground as the locus of attention), and sets up an analogical relation between the teaching clock and the circle the teacher displayed on the felt board just before the start of the excerpt. The co-timing of the gesture stroke with this portion of the utterance (“same circle shape”), the form of the gesture (a point carried through a circular path of motion), and the coupling of the gesture with the artifact (tracing over the clock band) provide a set of constraints that guide the mapping of the profiled element (the conceptual circle) to its material anchor (the clock band), producing the Clock Circle blend shown in Figure 4. In addition to this mapping function, the gesture in Figure 4 does something more: it depicts a path of motion relevant to time telling. Conceptually, a path of



Gesture as a conceptual mapping tool 

motion has source-path-goal image-schematic structure: a starting point (source), a moving object (trajector), a trajectory of motion (path), and a destination or endpoint (goal).2 A circle has no beginning or end, but when a circle is drawn, there is a starting point and endpoint to the drawing operation. The starting point and endpoint need not coincide: the drawing can overlap so long as the shape that remains has the appearance of a circle. When tracing a circle on the clock face, the trace can originate anywhere around the perimeter of the clock, it can proceed clockwise or counterclockwise at any speed, and it can end at any point beyond where it started, even making multiple rotations. Indeed, all of these were observed in other data. In the present excerpt, the trace begins at the top of the teaching clock, a considerable distance from the initial hand position (not shown in the transcript) in which the hand grasps the teaching clock at the 8. The trace proceeds slowly and steadily around the dial clockwise. It continues in steady motion after the utterance has ended and is released exactly when it reaches the top of the dial again. These aspects of the gesture support the interpretation that it reflects more than a simple outline of the clock shape. The gesture appears to enact a path of motion related to the system of time measurement, namely the movement of the long hand through a single clock hour. Nothing about this conceptual content is evident in the teacher’s speech or in the artifact she is holding; this source-pathgoal structure appears only in her gesture. The next part of the transcript (lines 8–10) shows another instance of a mapping gesture linking a conceptual element with a material anchor, but here we see the material anchor being prepared just before the mapping gesture is executed. In line 8, the teacher says “and I divide it” as she manipulates the clock hands into a decidedly unclocklike state: short hand straight up and long hand straight down (Figure 5). Why risk disrupting the Clock blend established in line 2? What the new configuration of the hands does is set up a ready anchor for the next conceptual element to be mapped to the clock face: the vertical dividing line. The teacher superimposes the vertical dividing line over the clock-hand anchor by tracing up and down over the newly aligned hands while saying “up and down here” (line 10, Figure 5). During the last downward stroke, she says “divide it into halves, right?” (line 14). This utterance calls attention to the regions carved out by the dividing line, bringing the halves of the blended clock-circle into profile and producing the Clock Halves blend (Figure 6).

2. Image schemas are recurring patterns in embodied experience that are important structuring elements to conceptualization (Johnson, 1987). A brief overview of source-path-goal and other common image schemas can be found in Lakoff and Johnson (1999).



Robert F. Williams

12

11

1

2

10 9

3 8 7

12

11

1

4

5

6

2

10 9

3 8 7

4

5

6

and I divi:de it(h) (1.2) up~and~down he:re(1.0) moving clock hand

tracing vertical line

Figure 5.  Building the Clock Quarters blend (2). Preparing a material anchor and mapping the vertical dividing line

11

½

12

1

10

½

2

9

3 8 7

Divided Circle

6

5

4

“ Clock” 11

12

1

10

½

9 8

7

2

½ 6

5

3 4

Clock Halves divide it into ha:lves, ri:ght?=

tracing vertical line

Figure 6.  Building the Clock Quarters blend (3). Profiling the regions bounded by the conceptual circle and dividing line



Gesture as a conceptual mapping tool 

The teaching clock with its hands arranged vertically anchors the part-whole configuration of halves of a circle. In the blend, the moving clock hands have vanished and been replaced by a single stationary divider. In this example, modifying the state of the artifact just before executing the trace prepared the object to couple with the mapping gesture. The trace then highlighted the relevant material structure while outlining the conceptual entity profiled in speech, again superimposing a conceptual element over its material anchor. The next action (lines 18–26) follows a similar pattern of preparing the material environment to anchor a conceptual element and then using a tracing gesture with speech to link the conceptual and material counterparts. Here there is another variation: the trace is done with the eyes rather than the hands.3 In this segment, the teacher uses a pointing stick as an anchoring structure. She picks up the stick while saying “now if I wanted to divide it into quarters.” On the word “quarters,” she places the stick across the face of the teaching clock (Figure 7).

¼

¼

¼

¼

11

12

1

10

2

9

3 8 7

11

12

1

10

6

5

4

2

9

3 8 7

6

5

4

now if I wanted to divide it into: (0.5) quar:ters : placing pointing stick

Figure 7.  Building the Clock Quarters blend (4). Preparing a material anchor for the horizontal dividing line

Once the stick is in place, the teacher’s hands are occupied, so she executes the mapping gesture with her eyes. While holding the stick, she says “we go from the nine to the three.” On the word “nine,” she fixes her gaze where the pointing stick 3. Gestures are generally regarded as movements of the hands and arms, but we can point with our bodies in many ways, especially when our hands are occupied or when the social situation demands subtlety (such as that the point be understood by compatriots but not by others). Although less prototypical than a finger-point, an eye-point or eye-trace can function in a similar way. In this respect, I classify them together.



Robert F. Williams

crosses the 9, and then she shifts her gaze along the pointing stick to where it crosses the 3, saying “to the three” (Figure 8).

11

12

1

10

2

9

3 8 7

11

12

1

10

5

6

4

2

9

3

8 7

5

6

4

we go from: the:: (0.5) ni:ne to the three:,=ri:ght?= scanning along stick

Figure 8.  Building the Clock Quarters blend (5). Mapping the horizontal dividing line onto the pointing stick anchor

11

12

1

10

2

9

3 8 7

11

12

1

10

6

5

4

2

3

9 8 7

6

5

4

from~nine~to~three~we~have fou:r pa:rts holding

Figure 9.  Building the Clock Quarters blend (6). Profiling the regions bounded by the conceptual circle and two dividing lines



Gesture as a conceptual mapping tool 

The motion description (go from X to Y) and eye-gesture (a smooth gaze shift as if tracking a moving object) trace a path along the pointing stick that accomplishes two things simultaneously: (1) defining the segment of the pointing stick that is to act as a material anchor, and (2) mapping the conceptual element, a horizontal dividing line, onto that anchor. The speech also calls attention to the 9 and the 3 as landmarks; these will be important for telling time at the quarter-hour marks. As she did moments before, the teacher then refers to the bounded regions, saying “we have four parts” (line 30). In the blend, this utterance profiles the clock-circle quarters whose configuration is anchored by the combination of objects the teacher is holding (Figure 9 above). The next utterance, “four equal parts” (line 31) profiles the relationship of the parts: the parts are equivalent in size and shape, a property of canonical quarters (Figure 10). Here again the teacher executes an eye-gesture: as she says “four equal parts,” she glances at each part in turn, performing a sequence of subtle eye-points while her hands remain occupied. The teacher’s final statement, “on our clock” (line 33), reactivates the clock input from line 2, so that the quarters are seen not in terms of the artifacts the teacher is holding but as quarters of the clock (Figure 11). The Clock Quarters blend is thus a nested blend with both clock and divided circle conceptual inputs. The blend is held together by the configuration of material artifacts – clock face, clock hands, and pointing stick – that anchors the part-whole relations.

10

11 12 1

2 3

9

8

11

10

1

8

2

7

12

6

1

4

2

3

4

5

7

6

5

4

fou:r equal pa:rts looking at each part

Figure 10.  Building the Clock Quarters blend (7). Profiling the equivalence of the parts while pointing to each part with the eyes



Robert F. Williams

10 9 8

Divided Circle 10 9 8

11 12 1

6

5

2 3

7

6

5

4

“ Clock” 2 3

7

11 12 1

4

Emergent property: Canonical clock quarters

Clock Quarters on our clock. holding

Figure 11.  Building the Clock Quarters blend (8). Profiling the clock input. The clock face is metaphorically construed as a divided circle

The Clock Quarters blend is not only an anchored conceptual blend; it is also a metaphoric blend. Notice the distinctness of the domains and the directionality of the mapping: from the source domain of geometric shapes (and their fractional parts) to the target domain of clocks, specifically the visible structure of the clock face as it relates to time-telling. The focus is on reading the time (the target), not reasoning about circles (the source). Once clock-reading becomes familiar, it can be used as the source domain for other metaphoric blends, as when describing spatial directions relative to the observer as “two o’clock” (ahead and to the right) or “seven o’clock” (behind and slightly to the left). It might seem curious that the metaphor in the Clock Quarters blend maps from the abstract (idealized geometric shapes) to the concrete (the clock face). After all, conceptual metaphor is argued to be the primary mechanism through which bodily experience in the physical world grounds reasoning in abstract domains such as logic and mathematics (Lakoff & Johnson, 1999; Lakoff & Núñez, 2000). Consider, however, that the divided-circle conceptual model – an idealized cognitive model – was developed through interactions with real objects: cutting paper circles into parts, drawing lines on printed circles, and arranging shapes on a felt board display. Once the conceptual model has been developed, it can be applied (with some help and practice) to other circumstances. Students can find geometric shapes in the visible structure of objects, including seeing quarter-circles in the clock face. The meta-



Gesture as a conceptual mapping tool 

phoric mappings project inferential structure (the part-whole relations of quarter circles) onto the clock face, enabling quarter-hour relative time reading. Now that the teacher has finished constructing the Clock Quarters blend, she is ready to introduce the reading of quarter-past times. She does this in the next segment of the lesson by associating the label “a quarter past” with the upper-right clock quarter, the one bounded by the long hand pointing at the 3. That portion of the lesson is not included here; further discussion can be found in Williams (2004, 2008). 6.3

Creating new conceptual mappings for the same clock state

Moments later in the same lesson, after the students have practiced reading several quarter-past times set by the teacher, another episode of explicit instruction occurs. Here the teacher shows the students how to read the same times as “…fifteen,” in other words, how to shift from reading quarter-hour relative time to reading absolute time when the long hand points at the 3. This excerpt of instruction (31 seconds) is detailed in Transcript 2 in the appendix. The episode in Transcript 2 begins with the students reading “a quarter past three” (line 2), a continuation of practice using the Clock Quarters blend. Line 4 marks the shift to the construction of a new blend, the Clock Counting blend, which the teacher will use to generate the equivalent absolute time reading. The teacher’s speech (“now another way that we say it”) cues the building of a new mental space for this different conceptualization (Figure 12).

10 9 8

11 12 1

2 3

7

6

5

4

“Clock”

New Space

now another way that we sa:y it moving clock hand

Figure 12.  Building the Clock Counting blend (1). Introducing a new space into the discourse while setting the teaching clock to the starting state for the counting process



Robert F. Williams

As the teacher cues this new space, she moves the long hand counterclockwise to the 12. Why? The reason becomes clear in what follows. The teacher is setting the artifact to a source state: the initial state of a process that will generate the new time reading. The goal state for that process is the clock display the students have just seen and are about to see differently. The way of getting from source to goal produces the new interpretation of the clock state. Underlying all of this is the conceptual metaphor A Process Is Motion Along a Path From Source To Goal, which is closely related to the Purposes are Destinations and Progress is Forward Motion metaphors discussed above. In order for students to experience the process and arrive at the appropriate interpretation of the goal state, the teacher must first map each element of a new conceptual input – the conceptual model for counting by fives – onto the relevant structures of the clock face. Here again, gesture is used both as a mapping tool and as a representational tool to add structure to the conceptualization. Line 6 introduces the counting model. When the teacher says “we count by fives,” she refers to an activity familiar to the 1st-grade students, who have practiced counting by fives throughout the school year. Counting is a ritualized activity in which a string of verbal labels is brought into coordination with a series of objects through some repetitive action. The string of labels is learned and recited as a chant (one, two, three…). Different strings are associated with different kinds of counting (one, two, three… vs. five, ten, fifteen… vs. ten, twenty, thirty…). The objects to be counted are treated as singular entities when counting by ones and as sets with a consistent number of elements when counting by fives, tens, or some other number. The repetitive action is usually sequential touching, pointing, or gaze-fixing, the latter two being derivative forms of touching (touching at a distance). The verbal label uttered vocally or subvocally when the final object is touched represents the total count; this is the total number of objects (when counting by ones) or the total number of elements in the object-sets (when counting by some other number) in the collection.4 In order for counting to produce the label that corresponds to the actual number of objects or elements present, the counting action must follow a path with the following properties: it must incorporate every object or object-set in the collection, and it must count each object or object-set only once. The path of the counting action is structured by a source-path-goal schema. The source (or starting point) leads to the first object counted, the path continues through each of the other objects (encountering each only once), and the goal coincides with the last object counted. The next counting label in the sequence is uttered as each object is intersected. Again we see how a gesture co-timed with speech and coupling with the environment serves a conceptual mapping function. 4. Other forms of counting involve different anchored blends. For more discussion, see Williams (2007), “Counting and conceptual blending”.



Gesture as a conceptual mapping tool 

In the case of counting, the gesture maps order in a sequence onto a non-ordered collection of objects. When the purpose of counting is to determine the total quantity of objects rather than to assign order, the mapping of label to object is quickly forgotten. In line 6 the teacher says “we count by fives.” This utterance activates both the conceptual model for counting and the string of verbal labels to be used (Figure 13).

“five, ten, fifteen ,…”

10 9 8

11

12

1

2 3

7

6

5

4

“Clock”

Counting by Fives

Counting A string ofverbal labels (a chant) is brought into coordination with a series of objects (singular objects or object-sets) through a repetitive action (sequential touching, pointing, or looking).

is we count by fi:ves touching clock hand

Figure 13.  Building the Clock Counting blend (2). Activating the conceptual model for counting and prompting the string of verbal labels to be used

The next portion of discourse (lines 8–10) sets up a series of conceptual mappings that make counting on the clock possible. In line 8, the teacher says “when we move this” while holding her pointing finger in contact with the long hand on the teaching clock (Figure 14). The word “this” prompts a search for a referent, which the touch-point satisfies. Together, the words (“when we move this”) and the gesture (pointing at/touching the long hand) set up the first direct mapping between the counting model and the clock face: they map the counting action to moving the clock hand. This establishes a metaphoric mapping in which the long hand is construed as the finger that touches each object while counting. Notice that the pointing of this metaphorical hand is markedly different from the pointing normally associated with a clock hand. In time-telling, the clock hand points to a tick mark or number along the outer dial. In counting, the human hand points to each object as it is counted. In time-telling, the clock hand is driven by an actuating mechanism steadily around the dial, in



Robert F. Williams

11

“five, ten, fifteen,…”

12

1

10

2

9

3 8 7

6

5

4

when we move this, touching clock hand

Figure 14.  Building the Clock Counting blend (3). Mapping the counting action onto moving the clock hand, metaphorically construing the clock hand as a human hand used to touch objects when counting

effect continuously pointing. In counting, the human hand is moved intentionally from object to object, with a clear start and finish to each movement, pointing only when it comes to rest on each object along the way. Now that the counting labels and counting action have been established, what remains is to identify the objects to be counted and to specify a path for the count. The objects are identified in the next part of the teacher’s utterance, “from number to number” (line 9), which is accompanied by series of points. As the teacher speaks, she slides her finger to the tip of the long hand and then jumps her finger to the 1, the 2, and so on around the dial (Figure 15). The teacher’s words identify numbers as the objects to be counted, while her series of touch-points picks out the numerals on the clock face, guiding the mapping of conceptual elements to specific environmental structures. This mapping elaborates the metaphor in which the clock hand is a counting hand, turning the numerals on the clock face into countable objects. What about the counting path? The teacher’s utterance uses a from/to construction with repetition of the word “number,” implying a form of iterative motion that corresponds with the typical action of counting, but the actual path of the count is not described in speech. Instead, the path is defined by gesture. The gesture starts at the top of the clock and moves clockwise around the dial, touching



Gesture as a conceptual mapping tool 

S

“five, ten, fifteen,…”

11 10

12 1

2

9

S

3 8 7

6

5

4

G?

G

source-path-goal schema

from number to number= touching numbers

Figure 15.  Building the Clock Counting blend (4). Mapping the objects-to-be-counted onto the clock face, creating metaphorical number-objects, and defining a path for the count

several numerals in sequence before falling away. The starting position and path of the count are clearly enacted in the gesture, but the endpoint remains unclear – the gesture is simply dropped in the middle of the next utterance (lines 9–10) after the teacher touches the 4. Evidence that the gesture defines the counting path comes in the form of the gesture motion. Bouncing from object to object is not the motion of a clock hand; it is, however, the canonical motion of counting, reinforcing the metaphoric construal. In this counting gesture, as in the clock-circle tracing gesture in the previous excerpt, the gesture embodies image-schematic structure that is not described in speech and not apparent in the environment. The gesture reveals an element of the speaker’s conceptualization and makes that element available to be apprehended by others. In both cases, that element is a source-path-goal schema that structures a path of motion. In the clock-circle example, the path structure is incidental: relevant to time-telling but not part of the blend being constructed. In the counting example, it is crucial. The count cannot begin with just any number on the clock face, nor can it proceed along just any path that happens to encompass the other numbers. The count must begin at the top of the clock and proceed clockwise, touching each number in sequence, for the correct time component to be generated. The path structure provided solely by gesture must be picked up by the learner



Robert F. Williams

– must be part of the learner’s conceptualization – for the activity to succeed.5 This has to happen even while the gesture is performing the mapping function, linking conceptual elements to their material anchors, which in this case also entails linking source domain elements to their target domain counterparts in the metaphoric blend. As the teacher continues gesturing from number to number, she says “there’s five minutes between each number” (lines 9–10), introducing a new conceptual input: the system of time measurement (Figure 16). S

“five, ten, fifteen,…”

11 10

12 1

9 8 7

Counting by Fives

6

5

“Clock”

...

2 3 4

days...

... hours ... ... minutes ...

G?

Time Measurement

there’s five minutes between each number touching numbers

Figure 16.  Building the Clock Counting blend (5). Mapping from the system of time measurement to the clock face and implicitly to the counting model. The mappings create metaphorical number-object-sets with five minute-elements

The teacher’s utterance reconstrues her mapping gesture so that it now associates intervals of time with spaces on the clock face, namely the spaces between adjacent numbers (tick marks being entirely ignored in the discourse).6 This mapping is undergirded by the Time As Space family of metaphors described in chapter 10 of Lakoff and Johnson (1999). Time is commonly conceptualized in terms of motion 5. See Cienki (2005) for experimental evidence that discourse participants are sensitive to image-schematic structure represented in gesture. 6. The use of a singular object with between (“between each number”) merits further discussion. Simply put, between each Xi implies between X1 and X2, between X2 and X3, and so on. Another example would be to say “there’s ten minutes between each class” to describe the passing period between each pair of adjacent classes in a university schedule.



Gesture as a conceptual mapping tool 

through space. In the case of the clock, the movement of the long hand from number to number is seen both as occurring in an interval of five minutes and as defining an interval of five minutes. In the metaphoric mapping, a “minute” is seen both as a spatial and temporal unit. The explicit mapping from the system of time measurement to the clock face also sets up an implicit mapping: from the system of time measurement back to the counting model. Recall that the numbers on the clock face have been metaphorically construed as objects-to-be-counted. In the conceptual model for counting by fives, each countable object is a set with five elements. The implicit mapping from time measurement to counting defines these elements as minutes. With mappings from two distinct inputs, the counting model and the system of time measurement, the numbers on the clock face become countable object-sets, each with five minute-elements. The single gesture that coincides with lines 9–10 of the transcript accomplishes several things at once: it maps from two distinct conceptual inputs while also adding image-schematic structure to the conceptualization. The first part of the gesture, which co-occurs with “from number to number,” maps from the counting model to the clock face while enacting and thus defining the counting path. The continuation of the gesture into “there’s five minutes between each number” maps from the system of time measurement to the clock face and implicitly back to the counting model. Although speech and gesture function together to establish mappings, we see here that individual utterances and gestures do not necessarily align in discrete units. In this instance, a single gesture spans two utterances, each cuing a mapping from a different conceptual model. What does seem important for a mapping gesture to succeed is an awareness of the element being mapped. This awareness is facilitated by the co-timing of speech and gesture. Once the mapping is clear, the gesture can be abandoned. Notice that in line 10, the gesture is dropped before the end of the utterance (“between each number”), the referent for “number” having already been established. Now that all of the mappings for the Clock Counting blend are in place, the teacher is ready to “run the blend,” i.e., to use it to generate a time component. In the discourse, this shift to running the blend is cued by the utterance “so if we were going to count by fives it would be” (line 12), which specifies how the actions the teacher is about to take are to be construed, namely in terms of the conceptual integration network she has set up (Figure 17).



Robert F. Williams

S 11 12

“five, ten, fifteen,…”

1

10

...days...

2

9

... hours ...

3 8 7

5

6

4

... minutes ...

G?

so if we were going to count by fives it would be:

Figure 17.  Running the Clock Counting blend (1). Cuing the transition to running the blend to generate a time component

The teacher runs the blend by grasping the long hand and moving it incrementally to the 1, the 2, and the 3, pausing at each number to look at the students and utter the corresponding counting label: “five,” “ten,” “fifteen” (lines 13–21, Figure 18). S 11 12

“five, ten, fifteen,…”

10

1

9

3 7

10

* “five” 12 5 1 2

9

3 8 7

6

5

4

6

... hours ...

G

4

8

11

...days...

2

5

... minutes ...

*

...

11 10

12 5 1

*

5

“ten” 2

9

3 8 7

6

5

4

...

11 10

12 5 1

9 8 7

6

5

5

2 5 3 “fifteen” 4

Clock Counting

(0.5) fi:ve moving clock hand

(0.6) te:n

(0.4) fiftee:n=

moving clock hand

moving clock hand

Figure 18.  Running the Clock Counting blend (2). Executing the counting process to generate the minute portion of the absolute time reading



Gesture as a conceptual mapping tool 

Here the teacher enacts the counting process she was preparing for when she moved the clock hand to the source state near the start of the excerpt just before executing the series of conceptual mappings. The goal of the count finally becomes apparent when the teacher releases the long hand after counting the number-object 3 as “fifteen.” To a 1st-grader, pointing at the symbol 3 and calling it “fifteen” could seem nonsensical, but it makes perfect sense in the blended space the teacher has constructed. In the Clock Counting metaphoric blend, the long hand is the pointingfinger proxy that touches each number-object as its associated label is uttered. Each of these movements spans an interval of five minutes, so that a total of fifteen minutes have been spanned when the long hand reaches the 3. This motivates the reading of the resulting clock state as “…fifteen.” The logic of this, namely that three times five is fifteen, is likely to elude the students for some time, but the practice of counting by fives on the clock face while pointing at the numerals, when coordinated correctly, will produce the correct minute portion of an absolute time reading, even in the absence of this deeper understanding. The teacher’s final statement, “so this is quarter past three but it is also three fifteen” (lines 22–27) equates the relative and absolute time readings (Figure 19). 11 12 1 2 10 9 3 8 4 7 6 5

10 9 8

11 12

*

1

7 6 5

2

3 4

Clock Quarters

=

10

“five ”

11 12 5 1

5

5

“ten ”

3

9 8

2

7

6

5

“fifteen ”

4

Clock Counting

so this is quarter past three but~it~is also(0.4) three::(0.4) fiftee::n lifting

Figure 19.  The same anchor for different blends. Establishing the equivalence of the different ways of reading the time

Here a single configuration of material structure anchors two completely different blends, each involving a unique conceptual integration network used to generate a



Robert F. Williams

particular kind of time reading.7 Compare the image in lines 24–27 with the image at the start of the excerpt in line 2. They are nearly identical.  Within a span of thirty seconds, the same clock state appears twice yet is seen differently each time it appears. This is a vivid illustration of the situated nature of seeing. Seeing is bound up with the context and purpose of activity. What is seen depends upon the conceptual models and mappings used to generate inferences relevant to the task at hand. The divided-circle and counting models support different inferences related to different ways of reading the time. In the instructional situation, the students are learning to “see” the clock in ways that support proficient time-telling. Here, Understanding Is Seeing is both metaphorical and literal. 7. Making meaning in activity In the examples presented in this chapter, two distinct ways of seeing the clock – two distinct metaphoric blends – arise from different sets of conceptual mappings set up in the course of a single clock-reading lesson. The mappings are cued by the orchestration of talk with action-in-the-setting and constrained by the nature and purpose of activity.8 In the time-telling lesson, picking up the teaching tool while saying “take my clock” maps “clock” onto the artifact the teacher is holding. Running a finger along the clock band while saying “same circle shape” maps a conceptual circle onto the clock band. Scanning along the pointing stick while saying “go from the nine to the three” maps a conceptual dividing line onto a segment of the stick (and makes the rest of the stick irrelevant). Pointing at the long hand while saying “when we move this” makes the clock hand a human-hand proxy for touching objects when counting. Touching the sequence of numerals on the clock face while saying “from number to number” establishes numbers as the objects-to-becounted while defining the counting path. Continuing the gesture while saying “there’s five minutes between each number” maps intervals of time onto segments of the clock face, implicitly mapping minutes to elements of the object-sets being counted. In each case, gesture is a tool for mapping conceptual elements to material

7. I here ignore how the hour portion of the time reading is constructed since this was not mentioned in the lesson. Reading the hour also relies upon anchored blends, and these, too, differ for relative and absolute time readings. For a complete discussion, see Williams (2004). 8. See Hutchins and Palen (1997) for similar claims about how meaning is constructed from space, gesture, and speech. Hutchins and Palen talk about the construction of complex, multilayered representations in the communication among members of an airline cockpit crew. The present chapter is similar in spirit but uses a different analytical framework, that of conceptual mapping and blending.



Gesture as a conceptual mapping tool 

anchors, for binding conceptual models to the world. When the mappings are metaphoric, gesture unites the source and target domain counterparts. A scan over the mapping gestures listed above reveals a distinct difference between those that appear in the first excerpt and those that appear in the second. In the first excerpt, the mapping gestures are traces; in the second, they are points. This difference can be resolved if we remind ourselves that a trace consists of a point plus an iconic component (Goodwin, 2003). In both excerpts, the pointing component of the gesture picks out the material structure that participates in the mapping. This may be a single structure (the long hand), several structures (the numbers on the clock face), or a portion of structure (a segment of the pointing stick). As the point picks out material structure, speech cues the conceptual element or elements to be mapped. In the tracing gestures, an iconic component added to the point outlines the conceptual element profiled in speech, superimposing it on the anchor. Note that a purely iconic gesture for a circle could simply draw the circle in the air, but an iconic/indexical trace draws the circle on top of an environmental structure. Iconic components are lacking in the gestures in the second excerpt most likely because the counting model lacks the geometric character of the divided-circle model and so is less fundamentally visual-spatial. Iconicity does appear in the enactment of the counting motion. Here the iconic component highlights the information provided solely in gesture, namely the counting path. We ought to expect that mapping gestures which link conceptual entities to the material environment will be indexical; the extent to which they are also iconic seems to depend on the nature of the relevant conceptual model and the unfolding activity. That gesture is used as a conceptual mapping tool is also evident in the ways the teacher prepares the environment to receive mapping gestures. Most obviously, she places a pointing stick across the clock face to anchor the horizontal dividing line. Before that, she presses the hands of the teaching clock apart – overriding the gearing that keeps them in clocklike alignment – to create an anchor for the vertical dividing line. Later, she moves the long hand back to the 12 (the starting count position) just before setting up the metaphoric mapping that makes it a proxy for the counting hand. In each of these examples, the teacher brings the world into alignment with the conceptual model before executing the gesture that maps element to anchor. If mapping gestures are used to construct anchored blends, are they obligatory to the process? Perhaps the mappings could have been set up solely in speech with sufficient prompting and feedback, but the properties of gesture make it an efficient medium for indexing structures in the environment and superimposing outlines of conceptual entities directly over their relevant counterparts. Gesture is also wellsuited to depicting paths and manner of motion. In short, speech, gesture, and material objects are different representational media with different properties.



Robert F. Williams

Speech is sequential and symbolic; gesture is visual-spatial, motional, and enactive. Speech readily cues other times and epistemic states (hypotheticality, for example), to which gesture may then also refer; gesture couples readily with structure in the environment. Both are transitory: speech loses its representational state as rapidly as it appears, while gesture can be sustained for a brief duration. In contrast to speech and gesture, the material environment is durable, but it takes more work to develop representations there. Mapping gestures are one way of coordinating speech with the setting, rapidly assigning representational states to bits of the world that anchor those states as we reason and communicate. How are mappings sustained through time? Mappings set up in discourse are likely to be forgotten as soon as the activity is over. Instruction must be followed by practice, and often more instruction, until the blends that support the activity become habitual. When innovative mappings do arise in discourse, they may be picked up and reiterated by other participants. If the ensuing blends prove helpful, they may be re-used and shared. Useful blends create pressures to modify the world in their support, leading to the crafting of artifacts that support particular practices and, ultimately, to instruction in the use of those artifacts. This seems to have occurred in the development of time-telling. The earliest anchors for timetelling were such things as shadows moving across landmarks. To these, scales were added, leading to the crafting of sundials and eventually mechanical (and later electronic and atomic) clocks, devices that anchor the system of time measurement. Because clocks change state in a systematic way, they stand as ready anchors for the blends used to generate time readings. This marriage of material artifacts and conceptual models, a union realized through conceptual mapping, enables us to do something we could never do in our heads: tell the time. That is no small achievement. Instructional discourse – including mapping gestures – sustains this sophisticated activity across generations. That is no small achievement either. 8. Conclusion In instructional discourse in time-telling, gestures are used to map conceptual elements profiled in speech to specific structures on the clock face (and other objects). When these mappings cross distinct domains, they can produce metaphoric construals of structure in the world, such as seeing a clock hand as a pointing finger. Further research is needed to illuminate the relationship between the mapping gestures described here and the metaphoric gestures described by McNeill (1992), in which the gesturing hand itself anchors a conceptual element. In addition to guiding conceptual mappings, the gestures analyzed in this chapter also



Gesture as a conceptual mapping tool 

enact paths of motion that are variously incidental or crucial structuring elements of the task-specific conceptualization. Mapping gestures like those described here seem to be a particularly useful tool for instruction, a form of social interaction in which one participant overtly guides the conceptualization of another. The extent to which mapping gestures occur in other forms of discourse remains to be investigated. It may be that instruction, because it involves such overt guidance, is more likely to bring conceptual mapping out into the open and into the gestures of participants. We should be careful, however, not to underestimate how often instruction surfaces in ordinary discourse in a variety of settings, even if only for a few fleeting moments. It does not take long to construct an anchored blend. Even in a formal school setting with young learners, the teacher in our examples took only about thirty seconds to set up all of the necessary mappings. Finally, it is worth noting that the mapping function of gestures described in this chapter would not be apparent if we restricted our definition of discourse to the talk that participants produce. Nor could it be understood if we treated speech and gesture as distinct channels of communication. The analysis of mapping, blending, and metaphoric blending presented here rests on the assumption that discourse is inherently multimodal – that it encompasses talk, gesture, and action coupled to the setting and purpose of activity. It is only in the coordination and interrelation of these elements that meaning emerges. Acknowledgments The research described in this chapter was supported in part by a dissertation fellowship from the Spencer Foundation. Thanks to Gilles Fauconnier and Edwin Hutchins for advising the research this chapter draws upon, and to Charles Goodwin, Ronald Langacker, and Rafael Núñez for comments on that work.

References Cienki, A. (2005). Image schemas and gesture. In B. Hampe (Ed.), From perception to meaning: Image schemas in cognitive linguistics (pp. 421–441). Berlin: Mouton de Gruyter. Fauconnier, G. & M. Turner (1998). Conceptual integration networks. Cognitive Science, 22, 133–187. Fauconnier, G. & M. Turner, (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books. Fauconnier, G. & M. Turner (To appear). Rethinking metaphor. In R. W. Gibbs, Jr. (Ed.), Cambridge handbook of metaphor and thought. Cambridge: Cambridge University Press.



Robert F. Williams Goodwin, C. (2003). Pointing as situated practice. In S. Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 217–242). Mahwah, NJ: Lawrence Erlbaum. Grady, J., T. Oakley, & S. Coulson (1999). Blending and metaphor. In R. W. Gibbs, Jr. & G. J. Steen (Eds.), Metaphor in cognitive linguistics (pp. 101–124). Amsterdam/Philadelphia: John Benjamins. Hutchins, E. (2005). Material anchors for conceptual blends. Journal of Pragmatics, 37, 1555–1577. Hutchins, E. & L. Palen (1997). Constructing meaning from space, gesture, and speech. In L. B. Resnick, R. Säljö, C. Pontecorvo, & B. Burge (Eds.), Discourse, tools, and reasoning: Essays on situated cognition, (pp. 23–40). New York: Springer-Verlag. Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books. Lakoff, G. & R. Núñez (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York: Basic Books. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Williams, R. F. (2004). Making meaning from a clock: Material artifacts and conceptual blending in time-telling instruction. Ph.D. dissertation, University of California, San Diego. Williams, R. F. (2006). Using cognitive ethnography to study instruction. Proceedings of the 2006 International Conference of the Learning Sciences. Mahwah, NJ: Lawrence Erlbaum Associates. Williams, R. F. (2007). Counting and conceptual blending. Paper presented at the 10th International Cognitive Linguistics Conference. Williams, R. F. (2008). Guided conceptualization: Mental spaces in instructional discourse. In T. Oakley & A. Hougaard (Eds.), Mental spaces in discourse and interaction, (pp. 209–234). Amsterdam/Philadelphia: John Benjamins.



Gesture as a conceptual mapping tool 

Appendix Transcript conventions: bold italics italics ~ .. o: r:: s::: (h) = [ CAPS ° ! ? . , (0.5) [boxed speech] (?) (( inaudible ))

vocal emphasis slight emphasis rapid speech slow speech lengthening of sound aspiration abrupt cut-off conjoined speech overlapped speech loud speech soft speech excited speech rising intonation falling intonation rising-falling intonation pause length in seconds co-occurrence with gesture/action in linked image uncertain transcription transcriber’s comment

 Robert F. Williams

Transcript 1



Gesture as a conceptual mapping tool 

Transcript 2



Robert F. Williams

A fresh look at the foundations of mathematics Gesture and the psychological reality of conceptual metaphor Rafael Núñez The study of speech-gesture-thought co-production serves multiple purposes, providing deep insight into many areas of investigation that go from psycholinguistics and cognitive linguistics to discourse analysis, and to the neuroscience of motor action and language. In this piece, building on my previous work (with George Lakoff) on the cognitive science of mathematics (Lakoff & Núñez, 1997, 2000; Núñez & Lakoff, 1998, 2005), I focus on the study of gesture production in order to address the question of the nature of mathematics and its foundations. I analyze (gestural) convergent evidence of the psychological reality of fundamental conceptual metaphors that we claim make infinitesimal calculus possible (in particular, what concerns limits and continuity). These conceptual metaphors, which are analyzed in detail in Where mathematics comes from (Lakoff & Núñez, 2000), emerge from fundamental mechanisms of everyday human imagination, language, and cognition, and structure the inferential organization of mathematical concepts and ideas. In this chapter, I show how the study of the gesture production of professional mathematicians turns out to be crucial in characterizing (in real-time) fundamental metaphorical contents that, while making the very mathematical ideas possible, are not captured by the standard well-accepted formalisms that are taken to “define” what mathematical concepts really are.

1. Introduction All over the world people gesture when speaking. Despite its universality and ubiquity, the natural phenomenon of gesture is poorly understood. The reason for this is relatively simple: With a few rare exceptions1, the investigation of human gesture has been, for a long time, left out of the study of the human mind. It is not 1. See, for instance, the creative, but largely unknown, work of the French Jesuit anthropologist Marcel Jousse (1886-1961) (Jousse, 1974, 1975).



Rafael Núñez

an exaggeration to say that gesture constitutes, inexcusably, a fundamental forgotten dimension of thought and language. One of the main problems has been that in western academic circles (especially in the United States), gesture has not been visible because it falls between the cracks of the well-established academic disciplines. In Chomskian linguistics, for instance – where language has been seen mainly in terms of abstract grammar, formalisms, and combinatorics – there was simply no room for “bodily production” such as gesture. In mainstream experimental psychology gesture was ignored, among other reasons, because, on the one hand, the field took language processing as the study of word-by-word sentence treatment (written or spoken) focusing on lexical items, phonology, and so on. On the other hand, being produced in a spontaneous manner, gesture was seen as very difficult to operationalize and hard to grasp experimentally. In mainstream cognitive science, which in its origins was strongly influenced by classic artificial intelligence, there was simply no room for gestures either. Cognitive science and artificial intelligence, being originally shaped by body-free abstract rule-driven digital computational paradigms, simply left no room for even considering the possibility that gesture may matter in the study of human cognition. In all these cases, gestures were completely ignored and left out of the picture that defined, for decades to come, what constituted genuine subject matters for the study of the mind. At best gestures were considered as a kind of epiphenomenon, secondary to other more important and better-defined phenomena. But in the last twenty years or so, this scenario has changed in a radical way with the pioneering work of A. Kendon (1980, 1982, 2004), D. McNeill (1985, 1992, 2000, 2005), S. Goldin-Meadow (2003), and many others. Research in a large variety of areas, from child development, to neuropsychology, to linguistics, and to anthropology, has shown the intimate link between oral production, gestural production, and thought. Finding after finding has shown, for instance, that gestures are often produced in astonishing synchronicity with speech, that they develop in close relation with speech, and that brain injuries affecting speech production also affect gesture production. There are several sources of evidence supporting (1) the view that speech and gesture are in reality two facets of the same cognitive linguistic reality, and (2) an embodied approach for understanding language, conceptual systems, and high-level cognition. Some of these sources are the following: 1. Universality: Speech-accompanying gesture is a cross-cultural universal (Iverson & Thelen, 1999; Kita & Essegbey, 2001; McNeill, 1992; Núñez & Sweetser, 2001, 2006). 2. Largely unconscious production: Gestures are less monitored than speech, and they are to a great extent unconscious. Speakers are often unaware that they are gesturing at all (McNeill, 1992).



A fresh look at the foundations of mathematics 

3. Speech-gesture synchronicity: Gestures are very often co-produced with speech, in co-timing patterns that are specific to a given language (McNeill, 1992). 4. Gesture production with no visible interlocutor: Gestures can be produced without the presence of interlocutors; for example, people gesture while talking on the telephone, and in monologues; congenitally blind individuals gesture as well (Iverson & Goldin-Meadow, 1998). 5. Speech–gesture co-processing: Stutterers stutter in gesture, too, and impeding hand gestures interrupts speech production (Mayberry & Jaques, 2000). 6. Speech–gesture development: Gesture and speech development are closely linked (Bates & Dick, 2002; Goldin-Meadow, 2003; Iverson & Thelen, 1999). 7. Speech–gesture complementarity: Gesture can provide complementary (as well as overlapping) content to speech content. Speakers synthesize and subsequently cannot distinguish information taken from the two channels (Kendon, 2000). 8. Gestures and abstract metaphorical thinking: Linguistic metaphorical mappings are paralleled systematically in gesture (Cienki, 1998; Cornejo, Simonetti, Ibáñez, Aldunate, Ceric, López & Núñez, submitted; McNeill, 1992; Núñez & Sweetser, 2001, 2006; Sweetser, 1998; see also contributions in this volume, which is especially dedicated to this subject). As a result, the study of gesture is, in contemporary academia, a rich field that informs many areas of investigation that go from psycholinguistics and cognitive linguistics to discourse analysis, to the neuroscience of motor action and language, and even to applied areas such as computer software and video game development, and human-computer interaction. The scholarly work published in the relatively new journal Gesture gives an excellent picture of the variety of domains that benefit from the study of gesture. In this chapter, I want to show yet another interesting application of the study of gesture, speech, and thought, especially gesture and metaphorical thought: the study of the inferential organization of abstract conceptual systems – in this case mathematics – through the analysis of gesture production. Building on my previous work with George Lakoff on the cognitive nature of mathematics (Lakoff & Núñez, 1997, 2000; Núñez & Lakoff, 1998, 2005), I want to address the question of the (cognitive) foundations of this discipline, providing empirical evidence for the psychological reality2 of crucial conceptual metaphors that make mathematics what it is. In order to make my argument as clear as 2. By psychological reality I mean (largely unconscious) observable real-time mind-related phenomena that inform us about the operational and functional reality of the metaphorical mappings involved. Gesture production is one such phenomenon. The study of this kind of phenomenon is radically different from, say, a purely etymological analysis of metaphorical expressions, which couldn’t inform us about the actual psychological processes involved in people’s minds at the moment they are using such expressions.



Rafael Núñez

possible, I will focus on a particular case study, namely, limits and continuity in calculus. It will be important to keep in mind that within this framework gestures are not investigated as mere aids for teaching and learning mathematics, but rather – when produced by professional mathematicians – as manifesting important dimensions of the actual inferential organization that constitutes mathematical concepts. In other words, in this chapter I will show how gesture studies can inform deep questions about the nature of mathematics itself. 2. What is mathematics? And what makes it so special? What is the nature of mathematics? What makes it possible? What is the cornerstone of such a fabulous objective and precise logical edifice? These are questions that have been treated extensively in the realm of the philosophy of mathematics, becoming, in the 20th century, specific subject matters for the rather technical fields of formal logic and metamathematics. Ever since, the foundations of mathematics have taken to be intra-mathematical (i.e., inside mathematics proper), as if the tools of formal logic alone are to provide the ultimate answers about the nature of mathematics3. But can the foundations of mathematics be themselves, mathematical entities? Or do they lie outside of mathematics? And if yes, where do these entities come from? I will argue that they come from the human imagination and everyday cognition, and that therefore one must address the above questions taking into consideration advances made in the scientific study of the mind, especially what concerns the mechanisms that make abstraction possible. Indeed, an essential characteristic of mathematics is that, despite being a highly technical domain, the very entities that constitute what Mathematics is are idealized mental abstractions. These entities cannot be perceived directly through the senses. Even, say, a point, which is the simplest entity in Euclidean geometry cannot be actually perceived. A point, as defined by Euclid is a dimensionless entity, an entity that has only location but no extension! No super-microscope will ever be able to allow us to actually perceive a point. A point, with its precision and clear identity, is an idealized abstract entity. The imaginary nature of mathematics becomes more evident when the entities in question are related to infinity where, because of the finite nature of our bodies and brains, no direct experience can exist with the infinite itself. Yet, infinity in mathematics is essential. It lies at the very core of many fundamental concepts such as limits, least upper bounds, topology, 3. Interestingly, this view later gave the basic assumptions of the early days of artificial intelligence and Cognitive Science: the idea that formal logic and the syntax of computational languages alone could provide the ultimate answers about the nature of the human mind.



A fresh look at the foundations of mathematics 

mathematical induction, infinite sets, points at infinity in projective geometry, to mention only a few. Moreover, mathematics has a unique collection of features. It is (extremely) precise, objective, rigorous, generalizable, stable, and, of course, applicable to the real world. Any attempt to address the nature of mathematics must explain these features. So, the intriguing question that comes to mind then is: If mathematics is the product of human imagination and abstraction – human ideas, how can we explain the nature of mathematics with this collection of unique features? It is easy to see that such a question doesn’t represent a real problem for approaches inspired in Platonic philosophies (very influential in philosophy of mathematics), which rely on the existence of transcendental worlds of ideas beyond human existence. The world of ideals exists outside, and independently, of human beings. But this view doesn’t have any support based on scientific findings and doesn’t provide any link to current empirical work on human ideas and conceptual systems (although it is supported, as a matter of faith by many Platonist scientists and mathematicians). The question doesn’t pose major problems to purely formalist philosophies either (also very influential), because in that worldview mathematics is seen as a manipulation of meaningless symbols. The question of the origin of the meaning of mathematical ideas doesn’t even emerge in the formalist arena. For those studying the human mind scientifically, however (e.g., cognitive scientists), the question of the nature of mathematics is indeed a real challenge, especially for those who endorse an embodied oriented approach to cognition. How can an embodied view of the mind give an account of an abstract, idealized, precise, sophisticated and powerful domain of ideas if direct bodily experience with the subject matter is not possible? In Where mathematics comes from, George Lakoff and I give some preliminary answers to the question of the cognitive origin of mathematical ideas (Lakoff & Núñez, 2000). Building on findings in mathematical cognition, and using mainly methods from cognitive linguistics, a branch of cognitive science, we suggest that most of the idealized abstract technical entities in mathematics are created via human cognitive mechanisms that extend the structure of bodily experience while preserving inferential organization. Such mechanisms are, among others, conceptual metaphors (Lakoff, 1993; Lakoff & Johnson, 1980; Lakoff & Núñez, 1997; Núñez, 2000; Núñez & Lakoff, 1998, 2005; Sweetser, 1990), conceptual blends (Fauconnier & Turner, 1998, 2002; Núñez, 2005), conceptual metonymy (Lakoff & Johnson, 1980), fictive motion and dynamic schemas (Talmy, 1988, 2003). Lakoff and I analyzed many areas in mathematics, from set theory to infinitesimal calculus, to transfinite arithmetic, to Boolean algebra, and showed how, via everyday human embodied mechanisms such as conceptual metaphor and conceptual blending, the inferential patterns drawn from direct bodily experience in the real



Rafael Núñez

world get extended in very specific and precise ways to give rise to a new emergent inferential organization in purely imaginary domains4. An important component of the argument Lakoff and I provide rests on the role conceptual metaphor plays in structuring the inferential organization of mathematical concepts and ideas. Because conceptual metaphor theory, originally based mainly on purely linguistic grounds, has made important claims about human cognition, abstraction, and mental phenomena, some psychologists have raised and addressed the question of empirical evidence to support the psychological reality of conceptual metaphor (Gibbs, 1994, 2007; Núñez, 2007). How do we know, for instance, that some of the metaphors we observe in linguistic expressions are not mere “dead metaphors,” expressions that were metaphorical in the past but which have become “lexicalized” in today’s language? How do we know that these metaphors are the actual result of real-time cognitive activity? And how can we find out the answers to such questions? As we will see, one answer is provided by the study of gesture-speech co-production. We are now in a position to turn into our case study. 3. Limits and continuity In the spirit of how the semantics of everyday human language is studied, let us see how mathematical ideas are actually presented, described, characterized, and even formally defined in mathematics books, academic journals, and college textbooks. A careful analysis of technical books and articles in mathematics (through the study of specific wordings, technical notations, figures, diagrams, etc.) provides very good insights into the question of how the inferential organization of human everyday ideas has been used to create mathematical concepts. So, let us take a look, for instance, at the classic book What is mathematics by R. Courant & H. Robbins (1978), and see what it says about limits in infinite series, and continuity of functions. a) Limits of infinite series In characterizing limits of infinite series, Courant & Robbins write: “We describe the behavior of sn by saying that the sum sn approaches the limit 1 as n tends to infinity, and by writing 1 = 1/2 + 1/22 + 1/23 + 1/24 + …” (p. 64, my emphasis)

4. The details of how conceptual metaphor and conceptual blending work go beyond the scope of this piece. For a general introduction to these concepts see Lakoff & Núñez (2000, chapters 1-3), Fauconnier and Turner (2002, chapter 3), and the references given therein.



A fresh look at the foundations of mathematics 

We can see that, strictly speaking, this statement refers to a sequence of discrete and motion-less partial sums of sn (real numbers), corresponding to increasing discrete and motion-less values taken by n in the expression 1/2n where n is a natural number. If we examine this statement closely we can see that it describes some facts about numbers, and about the result of discrete operations with numbers, but that there is no motion whatsoever involved. No entity is actually approaching or tending to anything. So why then did Courant and Robbins (or mathematicians in general, for that matter) use dynamic language to express static properties of static entities? And what does it mean to say that the “sum sn approaches,” when in fact a sum is simply a fixed number, a result of an operation of addition? b) Continuity Later in the book, the authors analyze cases of continuity and discontinuity of trigonometric functions in the real plane. Referring to the function f(x) = sin 1/x (whose graph is shown in Figure 1) they say: “… since the denominators of these fractions increase without limit, the values of x for which the function sin(1/x) has the values 1, -1, 0, will cluster nearer and nearer to the point x = 0. Between any such point and the origin there will be still an infinite number of oscillations of the function” (p. 283, my emphasis).

Figure 1.  The graph of the function f(x) = sin 1/x

Once again, if, as pure formal mathematics says, a function is a mapping between elements of a set (coordinate values on the x-axis) with one and only one of the elements of another set (coordinate values on the y-axis), all what we have is a static correspondence between points on the x-axis with points on the y-axis. How then can the authors (or mathematicians in general) speak of “oscillations of the function,” let alone an infinite number of them?

 Rafael Núñez

These two simple examples illustrate how ideas and concepts are described, defined, illustrated, and analyzed in mathematics books. You can pick your favorite mathematics books and you will see similar patterns. In both examples, static numerical structures are involved, such as partial sums, and mappings between coordinates on one axis with coordinates on another. Strictly speaking, however, absolutely no motion or dynamic entities are involved in the formal definitions of these terms. So, if no entities are really moving, why do mathematicians speak of “approaching,” “tending to,” “going farther and farther,” and “oscillating”? Where is this motion coming from? What does dynamism mean in these cases? What role is it playing (if any) in these statements about mathematics facts? Do formal definitions in mathematics provide any help in answering these questions? We will first look at pure mathematics to see what it says about these questions. 3.1

Looking at pure mathematics

How does pure mathematics characterize what continuity is? What is, mathematically, continuity taken to be? Mathematics textbooks define continuity for functions as follows: – A function f is continuous at a number a if the following three conditions are satisfied: 1. f is defined on an open interval containing a, 2. limx→a f(x) exists, and 3. limx→a f(x) = f(a). Where by limx→a f(x) what is meant is the following: Let a function f be defined on an open interval containing a, except possibly at a itself, and let L be a real number. The statement limx→a f(x) = L means that ∀ ε > 0, ∃ δ > 0, such that if 0 < x – a< δ, then f(x) – L < ε. As we can see, pure formal mathematics defines continuity in terms of limits (limx→a), and limits in terms of static universal and existential quantifiers ranging over static numbers (e.g., ∀ε > 0, ∃δ > 0), and the satisfaction of certain conditions which are described in terms of motion-less arithmetic difference (e.g., f(x) – L) and static smaller-than relations (e.g., 0 < x – a< δ). That’s it. The moral here is that these formal definitions – especially concocted by mathematicians in the late 19th century to avoid vagueness (for a discussion see Núñez



A fresh look at the foundations of mathematics 

& Lakoff, 1998), which are meant to be precise and rigorous and taken to be what continuity ultimately really is, don’t tell us anything about our questions. Nowhere in those formalisms we can see why a sum “approaches” a number, or why a number can “tend to” infinity, or why a function “oscillates” between values (let alone doing it infinitely many times, as in the function f(x) = sin 1/x). But this shouldn’t be a surprise. What Lakoff and I had suggested using techniques from cognitive linguistics (Lakoff & Núñez, 2000) showed what some wellknown contemporary mathematicians had already pointed out in more general terms (Hersh, 1997; Thurston, 2006): – The structure of human mathematical ideas, and its inferential organization, is richer and more detailed than the inferential structure provided by formal definitions and axiomatic methods. Formal definitions and axioms neither fully formalize nor generalize human concepts. We can see this with a relatively simple example (taken from Lakoff & Núñez, 2000). Consider the function f(x) = x sin 1/x whose graph is depicted in Figure 2. x sin 1/x for x ≠ 0 f(x) = 0 for x = 0

Figure 2.  The graph of the function f(x) = x sin 1/x

According to the ε – δ definition of continuity given above, this function is continuous at every point. For all x, it will always be possible to find the specified ε’s and δ’s necessaries to satisfy the conditions for preservation of closeness. However, according to the everyday notion of continuity – natural continuity (Núñez & Lakoff, 1998) – as it was used by Kepler, Euler, and the inventors of infinitesimal

 Rafael Núñez

calculus in the 17th Century, Newton and Leibniz, this function is not continuous. According to the inferential organization of natural continuity, certain conditions have to be met. For instance, in a naturally continuous line we are supposed to be able to tell how long the line is between to points. We are also supposed to be able to describe essential components of the motion of a point along that line. With the function f(x) = x sin 1/x we can’t do that. Since the function “oscillates” infinitely many times as it “approaches” the point (0, 0) we can’t really tell how long the line is between two points located on the left and right sides of the plane. Moreover, as the function “approaches” the origin (0, 0) we can’t tell whether it will “cross” from the right plane to the left plane “going down” or “going up.” This function violates these properties of natural continuity and therefore it is not continuous. The point is that the formal ε – δ definition of continuity doesn’t capture the inferential organization of the human everyday notion of continuity, and it doesn’t generalize the notion of continuity either. The function f(x) = x sin 1/x is ε – δ continuous but it is not naturally continuous. The moral here is that what is characterized formally in mathematics leaves out a huge amount of inferential organization of the human ideas that constitute mathematics. As we will see, this is precisely what happens with the dynamic aspects of the expressions we saw before, such as “approaching,” “tending to,” “going farther and farther,” “oscillating,” and so on. Motion, in those examples, is a genuine and constitutive manifestation of the nature of mathematical ideas. In pure mathematics, however, motion is not captured by formalisms and axiomatic systems. 4. Embodied cognition We can now look, from the perspective of embodied cognition, at the questions we asked earlier regarding the origin of motion in the above mathematical ideas. In the case of limits of infinite series, motion in “the sum sn approaches the limit 1 as n tends to infinity” emerges metaphorically from the successive values taken by n in the sequences as a whole. It is beyond the scope of this chapter to go into the details of the mappings involved in the various underlying conceptual metaphors that provide the required dynamic inferential organization (for details see Lakoff & Núñez, 2000). One can note, however, that there are many conceptual metaphors and metonymies involved. 4.1

Conceptual metaphor and conceptual metonymy

Conceptual metaphor and conceptual metonymy play a fundamental role in providing the inferential organization to the most basic pieces of our puzzle. For



A fresh look at the foundations of mathematics 

example, conceptual metonymies occur in cases where a partial sum is conceptualized as standing for the entire infinite sum; there are conceptual metaphors in cases where the sequence of these metonymical sums is conceptualized as a single trajector moving in space (the single case is indicated by the conjugation of the sum sn approaches in 3rd person singular); there are conceptual metaphors for conceiving infinity as a single location in space such that a metonymical n (standing for the entire sequence of values) can “tend to;” there are conceptual metaphors for conceiving 1 as the result of the infinite sum (i.e., not as a mere natural number but as an infinitely precise real number); and so on. Notice that none of these ideas and expressions can be literal. The facts described in these sentences don’t exist in any real perceivable world. They are metaphorical in nature. It is important to understand that these conceptual metaphors and metonymies are not simply embellishments added on top of formalisms, or “aids” to understand these formalisms. They are in fact constitutive of the very bodily-grounded forms of sense-making that make mathematical ideas possible. It is the inferential organization provided by our embodied understanding of “approaching” and “tending to” that is at the core of these mathematical ideas. In what concerns our “oscillating” function example, the moving object is again one single holistic object (the trigonometric function in the Real plane) constructed metaphorically from infinitely many discrete real values for x, which are progressively smaller in absolute terms. In this case motion takes place in a specific manner, towards the origin from two opposite sides (i.e., for negative and positive values of x) and always between the values y = 1 and y = -1. As we saw, a variation of this function, f(x) = x sin(x), reveals deep cognitive incompatibilities between the dynamic notion of continuity implicit in the semantics of the notion of oscillation and the static ε-δ definition of continuity coined in the second half of the 19th century (based on quantifiers and discrete Real numbers). These deep cognitive incompatibilities between dynamic-wholistic entities and static-discrete ones may explain important dimensions of the difficulties encountered by students all over the world when learning the modern technical version of the notions of limits and continuity (Núñez, Edwards, & Matos, 1999). 4.2

Fictive motion

Now that we are aware of the metaphorical (and metonymical nature) of the mathematical ideas mentioned above, let us analyze more in detail the dynamic component of these ideas. From where do these ideas get motion? What cognitive mechanism is allowing us to conceive static entities in dynamic terms? The answer is fictive motion. Fictive motion is a fundamental embodied cognitive mechanism through which we unconsciously (and effortlessly) conceptualize static entities in dynamic

 Rafael Núñez

terms, as when we say the road goes along the coast. The road itself doesn’t actually move anywhere. It is simply standing still. But we may conceive it as moving “along the coast.” Fictive motion was first studied by Len Talmy (1996), via the analysis of linguistic expressions taken from everyday language in which static scenes are described in dynamic terms. The following are linguistic examples of fictive motion: – – – – – –

The Equator passes through many countries The border between Switzerland and Germany runs along the Rhine. The California coast goes all the way down to San Diego After Corvisart, line 6 reaches Place d’Italie. The fence stops right after the tree. Unlike Tokyo, in Paris there is no metro line that goes around the city.

Motion, in all these cases, is fictive, imaginary, not real in any literal sense. Not only these expressions use verbs of action, but they provide precise descriptions of the quality, manner, and form of motion. In all cases of fictive motion there is a trajector (the moving agent) and a landscape (the space in which the trajector moves). Sometimes the trajector may be a real object (e.g., the road; the fence), and sometimes it is imaginary (e.g., the Equator; the border). In fictive motion, real world trajectors don’t move but they have the potential to move or the potential to enact movement (e.g., a car moving along that road). In mathematics proper, however, the trajector has always a metaphorical component. That is, the trajector as such can’t be literally capable or incapable of enacting movement, because the very nature of the trajectory is imagined via metaphor (Núñez, 2003). For example, a point in the Cartesian Plane is an entity that has location (determined by its coordinates) but has no extension. So when we say “point P moves from A to B” we are ascribing motion to a metaphorically entity that only has location. First, as we saw earlier, entities which have only location (i.e., points) don’t exist in the real world, so as such, they don’t have the potential to move or not to move in any literal sense. They simply don’t exist in the real world. They are metaphorical entities. Second, literally speaking, in the Cartesian Plane point A and point B are distinct locations, and no point can change location while preserving its identity. That is, the trajector (point P, uniquely determined by its coordinates) cannot preserve its identity throughout the process of motion from A to B, since that would mean that it is changing the very properties that are defining it, namely, its coordinates. With this basic understanding of how conceptual metaphor and fictive motion work together, we are in a position to see the embodied cognitive mechanisms underlying the mathematical expressions we saw earlier. Here we have similar expressions: – sin 1/x oscillates more and more as x approaches zero – g(x) never goes beyond 1



A fresh look at the foundations of mathematics 

– If there exists a number L with the property that f(x) gets closer and closer to L as x gets larger and larger; limx→∞ f(x) = L. In these examples fictive motion operates on a network of precise conceptual metaphors, such as numbers are locations in space (which allows us to conceive numbers in terms of spatial positions), to provide the inferential structure required to conceive mathematical functions as having motion and directionality. Conceptual metaphor generates a purely imaginary entity in a metaphorical space, and fictive motion makes it a moving trajector in this metaphorical space. Thus, the progressively smaller numerical values taken by x which determine numerical values of sin 1/x, are via the conceptual metaphor numbers are locations in space conceptualized as spatial locations. The now metaphorical spatial locus of the function (i.e., the “line” drawn in the plane) now becomes available for fictive motion to act upon. The progressively smaller numerical values taken by x (now metaphorically conceptualized as locations progressively closer to the origin) determine corresponding metaphorical locations in space for sin 1/x. In this imaginary space, via conceptual metaphor and fictive motion now sin 1/x can “oscillate” more and more as x “approaches” zero. 4.3

Dead metaphors?

Up to now, we have analyzed some mathematical ideas through classic methods in cognitive linguistics, such as conceptual metaphor and fictive motion. We have studied the inferential organization modeling linguistic expressions. But so far no much have been said of actual people speaking, writing, explaining, learning, or gesturing when doing mathematics. The analysis has been almost exclusively at the level of written and oral linguistic expressions. The remaining task now is to show that these linguistic expressions are not, as some scholars have suggested, mere instances of so-called dead metaphors, that is, expressions that once in the past had a metaphorical dimension but that now, after centuries of usage, have lost their metaphorical component becoming “dead.” The question that remains open is, could it be the case that the mathematical expressions we have seen, once were metaphorical expressions but now have become literal expressions whose meaningful origins speakers (mathematicians) don’t know anymore? If that is indeed the case, these expressions would be very much like those Latin or Greek etymologies underlying many English words which may have been known by speakers at a certain point in history, but whose original meaning is now lost. Is this what is happening to cases such as “approaching limits” and “oscillating functions”? Maybe all what we have in the mathematical expressions we have examined, is simply a story of dead metaphors, with no psychological reality whatsoever. As we will see, the study of human

 Rafael Núñez

gesture provides embodied convergent evidence showing that this is not the case at all. Via a detailed investigation of bodily motion (mainly hands and arms) and speech co-production, gesture analysis show that the conceptual metaphors and fictive motion involved in the mathematical ideas analyzed above, far from being dead, do have an embodied psychological reality that unfolds in real-time. 5. Enter gesture We are now in a position to analyze mathematical expressions like the ones we saw before, but this time focusing on the gesture production of the speaker. For the purposes of this chapter, an important distinction we need to make concerns the gestures that refer to real objects in the real world, and gestures that refer to some abstract idea that in itself doesn’t exist in the real world. An example of the first group is shown in Figure 3, which shows renowned physicist Professor Richard Feyn­man giving a lecture on physics of particles at Cornell University many years ago5. In this sequence he is talking about particles moving in all directions at very high speeds (Figure 3, a through e), and a few milliseconds later he completes his utterance by saying once in a while hit (Figure 3f).



(a) 00:04:17

(b) 00:04:24

(c) 00:05:07



(d) 00:05:18

(e) 00:07:27

(f) 00:09:29

Figure 3.  Professor Richard Feynman giving a lecture about physics. He is talking about particles moving in all directions at very high speeds (a through e), and “once in a while [they] hit” (f) 5. The time code shown at the bottom right of each freeze is expressed in minutes:seconds:frames. Each frame unit corresponds to approximately 33 milliseconds (there are 29.97 frames per second).



A fresh look at the foundations of mathematics 

The series of gestures shown in the five first pictures correspond to the characterization of random movements of particles at high speeds. The precise finger pointing shown in figure 3f occurs when he says once in a while hit (the stroke of the gesture). The point here is that although Prof. Feynman’s talk was about a very abstract domain (i.e., particle physics), it is still the case that with his finger he is indexing a “particle,” an object with location, extension, and mass, which does exist in the real world. The trajector in this dynamic scene is, an extremely small and fast object, but nonetheless a real entity in the real world. The following gestures – which involve pure mathematical notions – are similar in many respects, but they are even more abstract, in the sense that the entities that are indexed with the various handshapes are purely imaginary entities, like points and numbers in mathematics. Unlike particles in physics, these entities do not exist in the real world. For instance, Figure 4 shows a professor of mathematics teaching how to solve equations by “approximating” a root (a solution to quadratic equations).



(a) 00:08:03

(b) 00:08:13





(c)

00:08:16



(d) 00:08:24

(e) 00:09:07

Figure 4.  A professor of mathematics explaining the concepts underlying the method of finding roots (solutions) of quadratic equations. As he says you want to find sequences which get closer and closer, he iteratively moves his right hand frontward co-timed with the word sequences

 Rafael Núñez

While explaining this idea he says you want to find sequences which get closer and closer. As he says sequences (Figure 4a through 4e, which takes about 1 second) he iteratively moves (as if he is depicting steps) his right hand frontward, while pointing his palm towards his body and forming a modified version (to accommodate the thick pen he is using) of the handshape called baby O in American Sign Language and in gesture studies6. The frontward hand iteration is co-produced with the utterance of the word sequence, which refers to the entity getting “closer and closer.” Notice that what “gets closer” is not an entity relative to him, but an entity “getting closer” to a specific “location,” which is, metaphorically, a numerical value in front of him. Grammatically, this is done via the iterative use of the word “closer” together with the conjunction “and.” Figure 5 shows a professor of mathematics teaching about convergence of sequences of real numbers in a university level class. In this particular situation, he is talking about a case in which the values of an infinite sequence do not get closer and closer to a single value as n increases, but they “oscillate” between two fixed values. His right hand, with the palm towards his left has a baby O handshape. In this gesture the touching tip of the index and the thumb are metaphorically indexing a metonymical value standing for the values in the sequence as n increases (it is almost as if the subject is carefully holding a very tiny object with those two fingers). Holding that fixed handshape, he moves his right arm horizontally back and forth while he says still oscillates.



(a) 00:22:02

(b) 00:22:08

(c) 00:22:14

Figure 5.  A professor of mathematics teaching a university level class on convergence of sequences of real numbers. Here he is referring to a case in which the numbers of a sequence “oscillates”

6. In its prototypical form the index finger and the thumb are touching and are slightly bent while the other three fingers are bent.



A fresh look at the foundations of mathematics 

Hands and arms are essential body parts involved in gesturing. But often it is also the entire body that participates in enacting the inferential structure of an idea. In the following example (Figure 6) a professor of mathematics is teaching a university level course involving fundamental notions in calculus. In this scene he is talking about some particular theorems regarding monotone sequences. As he is talking about an unbounded monotone sequence, he is referring to the important property of being “going in one direction.” As he says this he is producing iterative unfolding circles with his right hand, and at the same time he is walking frontward, accelerating at each step (Figure 6a through 6e). His right hand, with the palm toward his chest, displays a shape called tapered O (thumb relatively extended and touching the upper part of an extended index bent in right angle, like the other fingers), which he keeps in a relatively fixed position while doing the iteration circular movement. Two seconds later he completes the sentence by saying it takes off to infinity at the very moment when his right arm is fully extended and his hand shape has shifted to an extended shape called B spread with a fully (almost over) extension, and the tips of the fingers pointing frontward at eye-level.

(a) 00:04:10

(b) 00:04:20

(c) 00:07:04

(d) 00:08:07

(e) 00:08:19

(f) 00:10:22

Figure 6.  A professor of mathematics in a university level class talking about an unbounded monotone sequence “going in one direction” (a through e) and which “takes off to infinity” (f)

 Rafael Núñez

Sometimes, when the sequence exhibits a peculiar property, handshapes adopt specific forms that match the meaning of those properties. In Figure 7 we see the same professor talking this time about a situation where the sequence is constant. As he begins to say so that would be just like a constant sequence, his dominant hand (the right one, with which he has been writing) raises and curls back and forms a slightly modified tapered O handshape (with the fingers a little bit bent pointing downwards), his elbow is bent in 90 degrees and his wrist is maximally bent with the palm oriented towards down (Fig. 7a). Then while keeping that handshape he extends his elbow (and wrist) producing a small and fast frontward (and slightly downward) motion with his right hand. In the meantime his left hand, with palm towards right, rises slowly, forming a five handshape with fingers extended flat (Fig. 7b). As he says the word constant, he abruptly stops the forward motion with his right hand marking a location situated a couple of inches in front of his open five left hand (Fig. 7c). While keeping his left hand totally fixed holding the same five handshape, he iterates a couple of times the same frontward movement with his right hand always stopping sharply at the same location, just a few inches from the open palm of his left hand. These abruptly stopped movements performed with the curled handshape while referring to a constant sequence sharply contrast with the smooth openended fully extended arm, hand and fingers, of the previous example produced when referring to an unbounded monotone sequence (Fig. 6).



(a) 00:01:03

(b) 00:01:20

(c) 00:01:28

Figure 7.  A professor of mathematics in a university level class talking about a constant sequence



A fresh look at the foundations of mathematics 

It is important to notice that in the last three cases the blackboard is full of mathematical expressions containing formalisms like the ones we saw earlier (e.g., static existential and universal quantifiers): formalisms, which have no indication or reference to motion. The gestures (and the linguistic expressions used), however, tell us a very different conceptual story. In these cases, the mathematicians are referring to fundamental dynamic aspects of the mathematical ideas they are talking about. In the example shown in Figure 5, the oscillating gesture matches, and it is produced synchronically with, the linguistic expressions used. In the following one (Figure 6), the iterative unfolding circular gesture matches the inferential structure of the description of the iteration involved in the unbounded monotone sequence, and the entire body moves forwards as the sequence unfolds. Since the sequence is unbounded, it “takes off to infinity,” idea which is precisely characterized in a synchronous way with the full frontal extension of the arm and the hand. In the last example (Figure 7) the curled shaped hand moves slightly forward but “hits” repeatedly the same location never being able to go further, thus embodying the meaning of a constant sequence always unfolding the same number as outcome. In sum, what we have learned from these gesture examples is the following: – First, gestures provide converging evidence for the psychological and embodied reality of the linguistic expressions analyzed with classic techniques in cognitive linguistics (e.g., metaphor and fictive motion). In these cases gesture analysis show that the mathematical metaphorical expressions we saw earlier are not cases of dead metaphors. The above gestures show that the dynamism involved in these ideas have full psychological and cognitive reality. – Second, these gestures show that the fundamental dynamic contents involving infinite sequences, limits, continuity, and so on, are in fact constitutive of the inferential structure of these ideas. Formal language in mathematics, however, is not as rich as everyday language and cannot capture the full complexity of the inferential structure of those mathematical ideas. It is cognitive science’s job (via, in this case, a detailed study of gesture production) to characterize the full richness of mathematical ideas. 6. Conclusion One of the main points of this chapter was to show that the study of gestures can be an extremely powerful tool for complementing the investigation of the (cognitive, metaphorical) foundations of mathematics. Mathematics, which is perhaps the most abstract conceptual system we can think of, is ultimately grounded in the nature of our bodies, language, and cognition. Conceptual metaphor and fictive

 Rafael Núñez

motion, being a manifestation of extremely fast, highly efficient, and effortless cognitive mechanisms that preserve inferences, play a fundamental role in bringing many mathematical concepts into being. We analyzed several cases involving dynamic language in limits and continuity in calculus, domains in which, according to the corresponding formal definitions and axioms, no motion should exist at all. Via the study of gestures, we were able to see that the conceptual metaphors underlying the linguistic metaphorical expressions in calculus were not simply cases of “dead” expressions. Thanks to the analysis of mathematicians’ gesture-speech coproduction we were able to provide convergent evidence supporting the psychological and cognitive reality of the conceptual metaphors underlying the notions of limits and continuity of functions (and their inferential organization). Building on gestures studies we were able to tell that these mathematics professors, not only were using metaphorical linguistic expressions, but that they were in fact – in real time – thinking dynamically! It is important to understand that these gestures are not mere teaching aids or explicit attempts to make abstract things more concrete (in fact, as we said, those dynamic gestures embody an inferential organization which is different from the ones expressed by the formalisms). Those gestures are in fact vivid manifestations of the fundamental inferential organization that constitute basic ideas in calculus such as limits and continuity. References Bates, E. & F. Dick (2002). Language, gesture, and the developing brain. Developmental Psychobiology, 40, 293–310. Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J-P Koenig (Ed.), Discourse and cognition (pp. 189–204). Stanford CA: CSLI Publications. Cornejo, C., F. Simonetti, A. Ibáñez, N. Aldunate, F. Ceric, V. López, & R. Núñez (submitted). Gestures and metaphor comprehension: Electrophysiological evidence of the influence of cross-modal coordination by audiovisual stimulation. Courant, R. & H. Robbins (1978). What is mathematics? New York: Oxford University Press. Fauconnier, G. & M. Turner (1998). Conceptual integration networks. Cognitive Science, 22, 133–187. Fauconnier, G., & M. Turner (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books. Gibbs, R. W., Jr. (1994). The poetics of mind. Cambridge: Cambridge University Press. Gibbs, R. W., Jr. (2007). Why cognitive linguists should care more about empirical methods. In M. González-Márquez, I. Mittelberg, S. Coulson, & M. Spivey (Eds.), Methods in cognitive linguistics (pp. 2–18). Amsterdam: John Benjamins. Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA: Belknap. Hersh, R. (1997). What is mathematics, really? New York: Oxford University Press. Iverson, J. & S. Goldin-Meadow. (1998). Why people gesture when they speak. Nature 396, Nov. 19, 1998, p. 228.



A fresh look at the foundations of mathematics  Iverson, J. & E. Thelen (1999). Hand, mouth and brain: The dynamic emergence of speech and gesture. In R. Nuñez & W. Freeman (Eds.), Reclaiming cognition: The primacy of action, intention and emotion (pp. 19–40). Thorverton, UK: Imprint Academic. Jousse, M. (1974). L’anthropologie du geste. Paris: Gallimard. Jousse, M. (1975). La manducation de la parole. Paris: Gallimard. Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M. R. Key (Ed.), The relation between verbal and nonverbal communication (pp. 207–227). The Hague: Mouton. Kendon, A. (1982). The study of gesture: Some observations on its history. Recherches Sémiotiques/ Semiotic Inquiry, 2, 45–62. Kendon, A. (2000). Language and gesture: Unity or duality? In D. McNeill (Ed.), Language and gesture (pp. 47–63). Cambridge: Cambridge University Press. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Kita, S. & J. Essegbey (2001). Pointing left in Ghana: How a taboo on the use of the left hand influences gestural practice. Gesture, 1, 73–95. Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (2nd ed.) (pp. 202–251). Cambridge: Cambridge University Press. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & R. Núñez (1997). The metaphorical structure of mathematics: Sketching out cognitive foundations for a mind-based mathematics. In L. English (Ed.), Mathematical reasoning: Analogies, metaphors, and images (pp. 21–89). Mahwah, N.J.: Lawrence Erlbaum Associates. Lakoff, G. & R. Núñez (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York: Basic Books. Mayberry, R. & J. Jaques (2000) Gesture production during stuttered speech: insights into the nature of gesture-speech integration. In D. McNeill (Ed.), Language and gesture (pp. 199–213). Cambridge: Cambridge University Press. McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92, 350–371. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: Chicago University Press. McNeill, D. (Ed.) (2000). Language and gesture. Cambridge: Cambridge University Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. Núñez, R. (2000). Mathematical idea analysis: What embodied cognitive science can say about the human nature of mathematics. Opening plenary address in Proceedings of the 24th International Conference for the Psychology of Mathematics Education, 1, pp. 3–22. Hiroshima, Japan. Núñez, R. (2003). Fictive and metaphorical motion in technically idealized domains. Abstracts of the 8th International Cognitive Linguistics Conference, Logroño, Spain, July 20–25, p. 215. Núñez, R. (2005). Creating mathematical infinities: The beauty of transfinite cardinals. Journal of Pragmatics, 37, 1717–1741. Núñez, R. (2007). Inferential statistics in the context of empirical cognitive linguistics. In M. González-Márquez, I. Mittelberg, S. Coulson, & M. Spivey (Eds.), Methods in cognitive linguistics (pp. 87–118). Amsterdam: John Benjamins. Núñez, R., L. Edwards, & J.F. Matos (1999). Embodied cognition as grounding for situatedness and context in mathematics education. Educational Studies in Mathematics, 39 (1–3), 45–65. Núñez, R. & G. Lakoff (1998). What did Weierstrass really define? The cognitive structure of natural and ε-δ continuity. Mathematical Cognition, 4, 85–101.

 Rafael Núñez Núñez, R. & G. Lakoff (2005). The cognitive foundations of mathematics: The role of conceptual metaphor. In J. Campbell (Ed.), Handbook of mathematical cognition (pp. 109–124). New York: Psychology Press. Núñez, R. & E. Sweetser (2001). Spatial embodiment of temporal metaphors in Aymara: Blending source-domain gesture with speech. Abstracts of the 7th International Cognitive Linguistics Conference, Santa Barbara, USA, July 22–27, pp. 249–250. Núñez, R. & E. Sweetser (2006). With the future behind them: Convergent evidence from language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401–450. Sweetser, E. (1990). From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge: Cambridge University Press. Sweetser, E. (1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. In Actes du 16e Congrès International des Linguistes. Oxford, UK: Elsevier. Talmy, L. (1996). Fictive motion in language and “ception.” In P. Bloom, M. Peterson, L. Nadel, & M. Garrett (Eds.), Language and space. Cambridge, MA: MIT Press. Talmy, L. (1988). Force dynamics in language and cognition. Cognitive Science, 12, 49–100. Talmy, L. (2003). Toward a cognitive semantics. Volume 1: Concept structuring systems. Cambridge, MA: MIT Press. Thurston, W. (2006). On proof and progress in mathematics. In R. Hersh (Ed.), 18 unconventional essays on the nature of mathematics (pp. 37–55). New York: Springer.

Peircean semiotics meets conceptual metaphor Iconic modes in gestural representations of grammar Irene Mittelberg This paper approaches the gestural sign from a cognitive-semiotic perspective combining Peircean semiotics (Peirce, 1955, 1960) and conceptual metaphor theory (Gibbs, 1994; Johnson, 1987; Lakoff, 1993; Lakoff & Johnson, 1980, 1999; Müller, 2004; Sweetser, 1990). It suggests that these two theoretical strands share central assumptions regarding image-schematic structures, metaphorical projection, and habitual patterns of experience and interpretation (Danaher, 1998) and that exploring spontaneous co-speech gesture reveals additional points of cross-fertilization. In the gesture modality, embodied structures are re-externalized and visualized. The corpus for this study consists of videotaped academic discourse and gestures produced by four linguistics professors during introductory courses. Applying Peirce’s taxonomy to gestural representations of grammatical concepts and structures, I demonstrate in what ways his categories of icon, index, symbol, and especially the less widely used sub-categories of iconicity (image, diagram, metaphor) allow to capture both fine distinctions between and transient cases of sign-object relations within a metaphorically accessed domain. Forms of interaction between iconic and indexical modes are also briefly discussed.

1. Introduction Describing, reasoning, and theorizing about grammatical phenomena are cognitive activities that are central to every linguist, language teacher, and, to a certain extent, to any speaker who engages in some sort of meta-linguistic discourse. The question that motivates the work presented here concerns the ways in which speakers spontaneously draw on multimodal resources to convey abstract knowledge. Of central interest is the role bodily semiotics plays in the teaching of grammar and linguistic theory. Given that conceptual metaphor is assumed to be instrumental in accessing and concretizing abstract domains, one of the underlying assumptions is that figurative thought is at the heart of meaning-making processes

 Irene Mittelberg

in both speech and manual modalities. In this paper, I demonstrate some of the ways in which the co-speech gestures produced by linguistics professors exploit space to represent grammatical concepts, relations, and structures. Work on metaphoricity in gesture, particularly done from a cognitive linguistic perspective, has shown that the manual modality can provide additional evidence for conceptual metaphor. That is, gesture has the potential to depict embodied aspects of abstract concepts, such as source domain information of metaphoric mappings not necessarily expressed in concurrent speech (Bouvet, 2001; Calbris, 2003; Cienki, 1998a, 2005; McNeill, 1992; Müller, 1998, in press; Sweetser, 1998, 2007). These observations suggest that investigating manifestations of figurative thought in media other than language can substantiate claims that are primarily made on the basis of traditional linguistic data (e.g., Gibbs, 1994; Gibbs & Steen, 1999; Johnson, 1987; Lakoff & Johnson, 1980, 1999; Lakoff, 1993; Sweetser, 1990, 1992). In the light of the well-established claim that metaphoric mappings are conceptual and thus “non-propositional” (Johnson, 1992, p. 349), they appear to condition externalizations of mental models in multiple modalities, with each semiotic mode having its characteristic affordances and limitations (e.g., Forceville, 2005; Kress et al., 2001; Mittelberg, 2002; Müller, in press; Tilley, 1999; Whittock, 1995). Exploring bodily communication appears to be a natural endeavor to gain insights into situated aspects of cognition and knowledge mediation, particularly regarding the centrality of embodiment in cognitive linguistics (Gibbs, 2003; Lakoff & Johnson, 1999; Lakoff & Núñez, 2000; Núñez, 2004; Nuñez & Sweetser, 2005; Smith, 2003; Williams, 2004). Building on this body of interdisciplinary research, this paper explores the semiotic work gestures perform in visualizing abstract concepts and structures. It reports on one part of a larger study on both metaphor and metonymy in metagrammatical discourse (Mittelberg, 2006). Of particular interest here are the different types of iconic modes that I was able to discern in gestural representations of the metaphorically conceptualized domain of grammar. Central to this work is, nonetheless, the finding that while conceptual metaphor is central to accessing abstract domains, conceptual metonymy, as the second major figure of thought, assumes an equally prominent role not only in terms of sign formation via synecdoche (Bouvet, 2001; Müller, 1998), but also in terms of contiguity relations between gesturing hands and the metaphorically construed, imaginary objects they may hold or otherwise manipulate (Mittelberg, 2005, 2006; Mittelberg & Waugh, forthcoming).1 1. For additional indications that metonymy partakes in the semiotics of gesture and signed languages see Bouvet (1997, 2001), Wilcox & Morford (2007), Müller (1998), Taub (2001), and P. Wilcox (2004).



Peircean semiotics meets conceptual metaphor 

The focus of analysis is on what Müller (1998, pp. 110–113) calls in her functionalist typology of gestures “referential gestures”; “performative gestures” and “discursive gestures” will not be taken into account here. Referential gestures are defined as depicting objects, attributes of objects or people, actions, behaviors, etc. Depending on the specific nature of the referent of the gestural sign, Müller further distinguishes referential gestures that represent concrete entities from those that represent abstract entities (see also Calbris, 2003 and Cienki, 2005). The approach to multimodal metaphor taken here combines a functionalistcognitivist perspective with Charles Sanders Peirce’s pragmaticist semiotics (Peirce, 1955, 1960). The rationale for combining these frameworks integrates the following three aspects: first, some of the categories widely used in gesture research are based on Peirce’s semiotics (McNeill, 1992, 2005). Second, cognitive metaphor theory and Peirce’s semiotic have been shown to share central assumptions about the link between image-schematic conceptual structures, metaphorical projection, and habitual patterns of experience and interpretation (Danaher, 1998; Hiraga, 1994). And third, Peirce’s notions of icon, index, and symbol as well as the different sub-types of icons he posited (image, diagram and metaphor) still need to be applied to gesture more systematically. Although the scope of this paper only permits the discussion of a few examples, the aim is to demonstrate some of the ways in which working with Peirce’s modes of representation, and particularly with his less widely used sub-categories of object-sign relationships, provides a way to capture both fine distinctions and transient cases regarding the different types of cognitive-semiotic relations. Notably, I will argue that the different iconic modes (image iconicity, diagrammatic iconicity, and metaphor iconicity) may interact in gestural representations of grammatical phenomena, motivating not only the link between gestural signs and denoted concepts, but also regarding the relations holding between single gestural signs that depict, for instance, linguistic units within a larger morphological or syntactic structure.2 Iconicity has been identified as one of the central semiotic modes underlying gestural representations of both concrete and abstract objects and actions. In McNeill’s (1992, p. 14) original gesture classification system, gestures depicting concrete entities are called iconics, and gestures representing abstract entities are called metaphorics. It should be stressed right away that these gesture categories are not mutually exclusive and that metaphoric gestures may comprise layers of several semiotic

2. As discussed in detail in Mittelberg (2006) and Mittelberg & Waugh (forthcoming), Peirce’s distinction also correlates with Jakobson’s view of similarity (i.e., icon/metaphor) and contiguity (i.e., index/metonymy) as two fundamental cognitive strategies that have the potential to structure both verbal and non-verbal messages (Jakobson, 1956; Waugh, 1992; Waugh et al., 2004).

 Irene Mittelberg

modes.3 McNeill and other gesture researchers have shown that metaphoric gestures tend to be at their base iconic in depicting aspects of the concrete source domain of the underlying metaphorical mapping (Bouvet, 2001; Cienki, 1998a, 2005; Parrill & Sweetser, 2004; McNeill, 1992, 2005; Müller, 1998, in press; Sweetser, 1998, 2007; Taub, 2001). Nonetheless, as we will see, a metaphorically motivated gesture does not necessarily represent the source domain iconically; a small imaginary object inside of a closed fist, for instance, first needs to be metonymically inferred before it can be interpreted as a metaphor for a small linguistic unit such as a grammatical morpheme (Mittelberg, 2005, 2006; Mittelberg & Waugh, forthcoming).4 Given the frequently observed mixing of semiotic properties in one and the same gesture, McNeill (2005, p. 41) recently proposed disregarding categorical systems for the classification of gestures and to speak instead of “dimensions” such as iconicity, metaphoricity, etc. Indeed, one and the same gesture, for example a gesture consisting of a hand tracing the outline of a sort of “base in the air,” may potentially refer to either a concrete or an abstract entity. Whether such a gesture depicts something concrete, such as the foundation of a building, or something abstract, such as the foundation of a theory, can only be determined in correlation with the concurrent speech content. If the speaker refers linguistically to the foundation of a building, the gesture receives a concrete interpretation. Conversely, a linguistically expressed figurative understanding, referring to the foundation of a theory, will evoke a metaphorical interpretation of the same gesture. From a cognitivist perspective, the conceptual metaphor theories are buildings (Grady, 1997; Lakoff & Johnson, 1980) may be said to motivate such a multimodal rendition of an abstract idea. In referring to an abstract notion, a metaphoric gesture may also rely on metonymic principles of sign formation (e.g. via synecdoche) in that the hands depict the locally essential elements of the object or action in question, e.g., the base line of a building described above (Bouvet, 2001; Mittelberg, 2006; Müller, 1998). It can further fulfill distinct discourse-pragmatic functions such as demarking a chunk of space for subsequent use, directing the addressees’ attention, etc. (Müller, 1998, p. 113). Ascribing considerable weight to the noted multidimensionaliy of gestural signs goes hand in hand with Peirce’s view that the different semiotic modes at work in a given process of signification may overlap to various degrees. Accordingly, 3. I wish to thank Susan Duncan, discussant of the panel “Metaphor and Gesture” at the 2003 ICLC in Logroño, for an insightful discussion regarding the issue that gesture categories are not mutually exclusive and that metaphorics may be parasitic on other modes. 4. Taub’s (2001) work on American Sign Language has shed new light on embodied cognitive structures and on the ways in which iconicity interacts with metaphor (with relevance to cospeech gesture and spoken language). Bouvet (2001) demonstrates ways in which gestural signs may be predominantly metaphorically or metonymically motivated.



Peircean semiotics meets conceptual metaphor 

the larger part of this paper is devoted not only to the characterization of the different sign-object relationships put forward by Peirce, but also to the interaction of semiotic modes within referential gestures of the abstract. In what follows, I first describe the corpus and methods developed for this study (section 2). I then illustrate Peirce’s basic notions of icon, index, and symbol and his sub-divisions of the icon with examples from the data (section 3). Finally, I trace some common ground between Peircean semiotics and cognitive metaphor theory, particularly in terms of image schemas and their metaphorical extensions (section 4), and offer some concluding remarks regarding bi-directional processes of abstraction and concretization via the human body (section 5). 2. Data and methodology There exists to date no unified empirical methodology for gesture coding and analysis; the approach developed for this study thus represents only one way of transcribing, coding, and analyzing spontaneous co-speech gesture.5 Since this paper serves as one part of a larger study of iconic, metaphoric, and metonymic modes in multimodal representations of grammar (Mittelberg, 2006), I will only describe those parts of the analytical apparatus that are directly relevant for the issues of interest here. The emphasis will be on the prominent hand shapes and movement patterns that emerged from the data and build the material basis for the different iconic modes of sign-object relationships to be demonstrated throughout the paper. Aspects of how the detected schematic images of objects and actions may build the basis for metaphorical projection will be discussed in sections 3 and 4. 2.1

Data

The corpus designed for this research comprises 24 hours of naturalistic academic discourse and co-speech gestures produced by four linguists (all native speakers of American English; three females and one male). The subjects were videotaped while lecturing in introductory linguistics courses at two major American universities. The focus of attention is on the communicative behavior of the teacher lecturing; student behavior and teacher-student interaction are not considered here. Compared to genres such as narrative and conversation, these classroom lectures consist of expository discourse, and although the teachers do at times reach out to 5. For a concise overview of the field see Kendon (1997), for more comprehensive accounts see McNeill (1992, 2000), Müller (1998), and the recently published monographs by Beattie (2003), Goldin-Meadow (2003), Kendon (2004), and McNeill (2005).

 Irene Mittelberg

the audience and interact with students to solicit feedback and answer questions, the nature of the speech is monologic rather than dialogic. Base-level courses seemed to be suitable for this kind of research, as teachers could be expected to make pedagogical efforts when introducing new technical terms, concepts, and theories. Also, the courses were selected such that the data would cover general grammatical issues (such as morphology, syntax, and phonology) as well as different theoretical views of grammar and linguistic theory: generative grammar, emergent grammar (in the context of second language acquisition), and relational grammar. In teaching contexts, speakers usually move about the classroom, write on and point to blackboards, whiteboards, overhead transparencies and screens, interact with the audience by turning and/or walking towards students asking questions, and so forth. Obviously, factors such as the physical environment, the locations of the speaker vis-à-vis the student audience, and the use of artifacts determined how the video camera was set up to ensure that all of the physical elements and the speakers’ movements were captured as completely as possible. The video camera was mounted on a tripod and placed in the back of the classroom. Each lesson was videotaped in its entirety. 2.2

Methodology

The corpus was assessed from a conceptual point of view, selecting and capturing episodes in which referential gestures depicting grammatical phenomena occurred. The goal was to determine how speakers linguistically and gesturally represented linguistic units (morphemes, words, phrases, etc.), grammatical categories (verb classes, cases, semantic roles, etc.), syntactic structures (clauses, sentences, etc.), as well as operations (active-passive transformation, subordination, reiteration, etc.). The speech of each segment was transcribed by adapting the discourse transcription convention provided by Du Bois and colleagues (Du Bois et al., 1993) to the level of detail deemed appropriate for this study. The next step was to code the gestures, thereby accounting for the exact mechanics of speech-gesture synchrony, which entailed documenting systematically and in relation to the concurrent speech the form and dynamics of the manual configurations and actions unfolding in space and time.6 The course of each gestural movement (e.g., onset, peak, hold, 6. Gesture researchers have suggested various schemes for how to graphically capture not only the close temporal relationship between speech and co-speech gesture, but also the kinesic features of gestures (cf. Calbris, 1990; Duranti, 1997, pp. 144–154; Kendon, 2004; McNeill, 1992, 2005; Müller, 1998, pp. 175–199, 284ff.; Parrill & Sweetser, 2004; inter alia). This study has particularly been inspired by the methods of transcription, coding, and analysis developed by the McNeill Lab (McNeill, 1992, 2005), Müller (1998, 2004) and Webb (1996). For more detail see Mittelberg (2007).



Peircean semiotics meets conceptual metaphor 

return to rest, etc.) was translated into typographic representations, superimposed on the written speech transcript. Each gesture was traced from the moment the articulators (here hands and arms) begin to depart from a position of rest or relaxation until the moment when they return to rest, thus accounting for what Kendon (2004, p. 111) calls such a “gesture unit.” Only gestures articulated with hands and arms were taken into account, leaving aside facial expressions, gaze, instances of self-grooming, and movements of the head and torso. To capture the kinesic features of gestures, the most widely used coding parameters are hand presence and hand dominance (left hand and/or right hand), hand shape, palm orientation, movement (trajectory and type), and the location in gesture space where a gesture is performed (McNeill, 1992, 2005; Kendon, 2004; Webb, 1996; inter alia). This array of physical gesture features was also used for this study. For each holistic gestural gestalt and in correlation with the speech content, those features that contributed most significantly to its meaning and function were identified. In certain cases, the movement proves to be more salient with respect to the meaning of a gesture than the particular shape of the hand performing the movement (e.g., in certain pointing gestures); sometimes, the hand shape is more salient than the contextual movements (e.g., in the case of a precision grip gesture), and in yet other cases, both dimensions are significant (e.g., a push with both hands flat and palms facing the addressee, thus building a barrier and evoking the idea of ‘stop’ or ‘rejection’). 2.2.1 Hand shape and palm orientation Opting for a data-driven typology of manual signs, the data were searched for recurring hand shapes and movement patterns across contexts and speakers. Then labels were assigned to each of the prominent forms. For example, one of the most frequently used hand shapes in the data is a flat open hand with the palm turned upwards. Here it seemed worthwhile to build on conventions introduced by Müller (2004) in her study of forms and functions of the palm-up open-hand gesture (hereafter referred to as “puoh”). Each variant of the open hand gesture that occurred in the data was given a similar abbreviation, indicating the orientation of the palm, plus a short name evoking the degree of openness of the hand (“tray,” “cup,” “lid,” etc.) as well as an indication of which hand performed the gesture. For instance, “puoh-tray-lh” stands for a flat palm-up open hand, produced with the left hand, evoking the shape of a tray. Or, “pvoh-box-bh” stands for another frequently produced gesture consisting of two hands held apart, with both palms being held vertically and facing each other, as if they were holding an object between them.

 Irene Mittelberg

2.2.2 Movement and location in gesture space Gestures typically involve some sort of movement through space. Describing the manual actions entailed accounting for the range and trajectory of the performed motion (for example, along horizontal, vertical, or diagonal axes) as well as the manner of the movement (straight line, wave, rotation, etc.). Instances in which a movement is discontinued or a configuration is being held (i.e., gesture holds) were also recorded. The movements observed in the present data constitute several kinds: first, hands tracing links between two or more locations in gesture space; second, pointing gestures whose direction and range depend on the location of the object or person being pointed at; and third, genuine motor actions such as the rotation of the wrist or a sweep performed with hand and forearm. In keeping with the notational conventions used for hand shapes, the prominent movement patterns observed in the data were given labels. For example, “verttrace-rh” signifies a line that is traced vertically with the right hand, and “wristrota-lh” refers to a wrist rotation performed with the left hand. Gestures evolve and vanish in space. Gesture space is relative to, and constituted by, the position and posture of the speaker-gesturer who, in each communicative situation, sets up the coordinates of gesture space around her, according to the dimensions and movements of her body, the range of her gestural articulators, her physical environment, and, if applicable, also according to the interpersonal space spanning between herself and her interlocutor(s). Here, one of the main interests was to determine in what ways the speakers’ use of the different dimensions of gesture space can be linked to spatial metaphor and thus to the theoretical model of sentence structure talked about in a given moment. I will now introduce those parts of the interpretative framework that are relevant in the context of this paper. First, I illustrate Peirce’s modes of representation with examples from the data, and second, I draw connections between the different modes of iconicity (image, diagram, metaphor) on the one hand and image schemas and conceptual metaphor on the other. 3. Peircean perspectives on the gestural sign Peirce’s widely-applicable pragmatist theory, with its dynamic understanding of the sign, is particularly valuable in gesture research (cf. Enfield, 2003; Fricke, 2004; Haviland, 2000; McNeill, 1992, 2005). According to Peirce, “we think only in signs” (Peirce, 1960, p. 169, 2.302). Gestures are undoubtedly part of the semiotic landscape, and particularly part of the bodily semiotics, on which we draw to make meaning and also to make sense of what others try to convey. This is particularly true regarding knowledge mediation and social interaction in teaching contexts.



Peircean semiotics meets conceptual metaphor 

As I hope to show in the course of this paper, exploring what Peirce understood by similarity and contiguity relations in co-speech gesture can account for a wide range of meaning-making processes in the data. Focusing here on similarity relations, the considerations below are also meant to prepare the ground for the ensuing discussion (section 4) of how investigating gesture from this angle may reveal instances in which Peirce’s semiotics and cognitive metaphor theory can crossfertilize each other (Danaher, 1998). 3.1

Peirce’s triadic model of the sign process

Peirce’s triadic model of the sign includes a representamen (the material form the sign takes), an object to which the sign refers, as well as the sense that a sign evokes in the mind of its receiver. Peirce calls the latter, that is, the impression the representamen leaves in the receiver’s mind, the interpretant (not to be confused with the person who receives the sign, the interpreter). Signs point to something outside of themselves and they need an interpretive mind to unfold their meaning: “[a] sign not only represents an object but it does so for, and determines, an interpretant (interpretation), the third moment of the sign-object-interpretant process” (Waugh et al., 2004, p. 42). The interpretant, in turn, gives rise to other signs, thus propelling the semiotic process: a limitless circulation and interpretation of signs that derive their meanings only in the active process of semiosis in a given communicative act. In line with Bakhtin (1981), Peirce sees thought as dialogical and social (Chandler, 2002, p. 33). The following quote encapsulates Peirce’s sign model:7 A sign […] is something which stands to somebody for something in some respect or capacity. It addresses somebody, that is, creates in the mind of that person an equivalent sign, or perhaps a more developed sign. That sign which it creates I call the interpretant of the first sign. The sign stands for something, its object. It stands for that object, not in all respects, but in reference to a sort of idea, which I have sometimes called the ground of the representamen. (Peirce, 1960, p. 135, 2.228)

The part of this model that is not easy to seize is the ground of the representamen; it consists of those qualities of a representamen that actually act as a sign in a specific event of interpretation. It represents the signifying properties of the sign vehicle, whereas other properties might be irrelevant in a given context and thus remain insignificant. For example, in an uttered word, a particular combination of phonemes gives rise to its meaning and interpretation. However, the pitch with which the word is produced, or the individual coloring it receives from voice 7. When referring to Peirce’s original writings, I will indicate both the page on which the respective passage appears and the number of the exact paragraph; for example, “Peirce, 1960, p. 135, 2.228” refers to the Collected Papers, p. 135, paragraph 2.228.

 Irene Mittelberg

qualities of the person who pronounces it, might be irrelevant, even though all these features jointly make up the material sign carrier, e.g. the speech sound (Jakobson & Waugh, 1979, 2002). Similarly, not all the components of a given gesture contribute to its meaning. Peirce established three different trichotomies of the sign (Peirce, 1955, p. 7), one for each of the three elements outlined above: the sign itself, the sign in relation to its object, and the sign in relation to its interpretant. The discussion below will focus on the second triad and its implications for possible relations between gestural signs and their objects. In this way, the perspective taken is that of the interpreting subject (i.e., the perspective of the student addressee which is also that of the gesture analyzer).8 3.2

Interpreting sign-object relationships: Icon, index, and symbol

The triad icon, index, symbol reflects the different ways in which the sign vehicle (i.e., the ground of the representamen) represents the signified object: through similarity (icon), through contiguity (index), and/or through conventionality (symbol). As stated above, there are in principle no pure icons, indices, or symbols; a sign can simultaneously incorporate qualities of all three modes. These sign-object relationships are to be understood as semiotic modes that may layer in a particular process of signification, with the overriding factors determining the local function of the sign (Chandler, 2002, p. 43). Peirce’s understanding of what an object can be is extremely wide and ranges from existing to non-existing things, including possibilities, goals, qualities, feelings, relations, modes of being, acts, etc. (Liszka, 1996, p. 21). Anything can be an object, as long as it is represented by a sign (Shapiro, 1983, p. 25). In what follows, I will discuss each of these modes and provide examples from the gesture data. 3.2.1 Icon Generally, in an icon, the relation between the ground of the sign and the object it represents is based on some sort of conceived similarity: the signifier is taken to resemble the signified, whether in terms of sound, shape, feel, taste, organization, or some other quality. In Peirce’s own words, “Icons have qualities which ‘resemble’ those of the objects they represent, and they ‘excite analogous sensations in the mind’ [Peirce, 1960, p. 157, 2.276]” (Chandler, 2002, p. 39). Examples include photographs, painted portraits, and onomatopoeia in language (Jakobson & Waugh, 1979, 2002). The physical form the sign takes is thus to some degree conditioned by the object or action to which it refers. However, even photographs only resemble 8. For a discussion of the remaining two triads, the different types of objects and interpretants, and the role of abduction see Mittelberg (2006).



Peircean semiotics meets conceptual metaphor 

what they represent in some respects; they reduce, for example, three-dimensionality to two-dimensionality. No matter how truthfully such highly iconic signs represent reality, they always involve some kind of cultural convention in their production and interpretation (Chandler, 2002, p. 40; Kress & van Leuuwen, 1996). What seems to be vital is the recognizable analogy between real world objects or events on the one hand and the products of their semiotic transformation on the other. In a similar fashion, the form of an iconic gestural sign is conditioned by the object or action it refers to (cf. iconics in McNeill’s [1992] typology). A literal interpretation of a referential gesture, such as the one representing the foundation of a building mentioned above, is based on a perceived similarity with a building base structure in the real world. Other examples would be a hand movement that traces the outline of a table by evoking its length and height, or a gesture imitating the action of locking a door using an imaginary key. Such schematic depictions obviously resemble the things and actions they represent only in some pertinent respects, giving sufficient information for the interpreter to metonymically infer, in conjunction with the speech content, their meaning and function (Eco, as quoted in Bouvet [1997, p. 17], calls this the “code of recognition;” see also Müller, 1998).9 Let us consider an example from the data. If we looked at the gesture given below (figure 1) with the speech suppressed, how would we interpret it? It does not take much to detect a similarity between this gesture, consisting of two open hands with palms held vertically and facing each other, and the common action of holding a medium-size object between one’s hands. However, we might ask whether the object itself gets referred to or rather the action of holding or carrying something. It might be a trivial observation, but in order to evoke the idea of a physical object it is not necessary to trace each of its sides or its exact dimensions. The shape and constellation of the hands indirectly allude to the two outer sides by which the imaginary object, occupying the space between them, is seized. The mind may fill the empty space between the hands with a three-dimensional solid object. By ignoring the speech content, one can entertain such speculations about the iconic properties of a gesture; however, the concurrent linguistic elements disambiguate such polysemous gestural signs instantly.

9. Here, in fact, iconicity and synecdoche interact, playing into the gestural sign formation by abstracting essential parts of objects which then get to stand in for the entire referent; see Müller’s (1998) modes of gestural representation and Mittelberg’s (2006) taxonomy of metonymic modes, with synecdoche being one of them.

 Irene Mittelberg

Figure 1.  “Grammar” as an object

Evidently, the gesture shown above may iconically depict a physical object such as a box and/or the action of holding such a box; or, it may also, through a metaphorical interpretation, refer to an abstract entity. As the transcript below reveals (example 1), the speaker refers here to the concept of grammar:10 (1) ((grammar is not a thing)) … maybe a too radical departure for you from the normal way of looking at how grammar is acquired 10. The abbreviations used in the transcript follow the conventions introduced in section 2 (inspired by Müller, 2004: “pvoh” stands for “palms vertical open hand,” “bh” stands for “both hands,” and “rota-bh” for both hands rotating around one another). As for gesture-speech synchrony, the speech segments that coincide with a gesture are set off by square brackets, speech segments highlighted in bold face represent the gesture stroke, and underlined speech segments indicate a post-stroke gesture hold.



Peircean semiotics meets conceptual metaphor 

G1 G2 object-box held between bh fists-bh but it does not see [grammar as a thing] [that you get] / G3 rotation-bh [.. and then u=se] Gesture 1 (G1) expresses the idea that grammar should not be understood as a thing, i.e., as a commodity that one can “get” and hold on to (G2: each hand forms a closed fist) for subsequent “use.” The action performed in gesture 3 – two hands rotating around one another – conveys the idea of an ongoing activity, here probably referring to cognitive processing, or the act of practicing language skills. In fact, Figure 1 illustrates a typical instance of a gesture frequently found in the data, with varying distances between the two hands, representing words, constituents, sentences, or entire discourses respectively. What such renditions have in common is that they reify abstract ideas in the form of graspable entities or bounded space. Seen from a cognitive linguistic perspective, these gestures are manifestations of the conceptual image schemas object or container and reflect the metaphorical concepts ideas are objects or categories are containers (Johnson, 1987; Lakoff, 1993; Lakoff & Johnson, 1980, 1999). One should keep in mind that the array of potential meanings (interpretants) of this kind of gesture can cut across different domains of experience, and in order to determine whether a given gestural sign refers to a concrete or to an abstract phenomenon we need the support of the linguistic and often also extra-linguistic context. Articulated in Peircean terms, the gestures discussed above possess a heavily iconic base. As most of the meta-grammatical gestures under investigation here refer to abstract phenomena, they are essentially metaphorical in nature. They evoke, according to Peirce (1960, p. 157, 2.277), a parallelism between the abstract (ideas and structures) and the concrete and the embodied (manual signs), and may exhibit, as I will illustrate in detail in section 3.3, various combinations of iconic modes. 3.2.2 Index In indexical signs, the relation between sign and object is based on contiguity, that is, on a factual (physical or causal) connection between the two. “An Index is a sign which refers to the Object that it denotes by virtue of being really affected by that object” (Peirce, 1960, p. 143, 2.248). Smoke indicates fire, a footprint the passing of an animal, a bodily symptom such as pain the malfunctioning of an organ, etc. In language, the use of deictic expressions, or shifters (Jakobson, 1957) – such as prepositions, pronouns, demonstratives, verbal tenses, and adverbs of time and

 Irene Mittelberg

space – is highly context-sensitive, and depends on the here and now of the speaker or writer (the origo according to Bühler, 1934). Perception appears to play a crucial role in detecting the link between a demonstrative pronoun and its referent (if the latter is physically present): The demonstrative pronouns, “this” and “that,” are indices. For they call upon the hearer to use his powers of observation, and so establish a real connection between his mind and the object; and the demonstrative pronoun does that – without which its meaning is not understood – it goes to establish such a connection; and so is an index. (Peirce, 1955, p. 14)

Indexical gestures fulfill similar functions; the spatial orientation of pointing gestures (deictics, in McNeill’s [1992] terms), for example, depends each time on the location of the object they are directed towards. Through the act of pointing at something in the proximity of the speaker, for instance on the mention of a demonstrative pronoun, the object is established via a visible vector. Deictic gestures bring to light spatial relationships between speakers and objects, locations, or people, whether they are present in the environment, imagined, or previously introduced in the unfolding discourse (cf. Fricke, 2002, 2007; Furuyama, 2001; McNeill et al., 1993). As naturally contiguous the connection between index and referent may be, pointing practices are also culturally determined and thus to some degree conventionalized (cf. Clark, 2003; Haviland, 2000; Kita, 2003). To illustrate and anchor their explanations, teachers frequently point to information presented on blackboards, whiteboards, or overhead screens. An example of this proto-typical kind of pointing gesture is given below (Figure 2). Interestingly, for a moment the speaker is using both hands simultaneously, though each is engaged in a different type of semiotic act: he points with his right hand to words projected onto the overhead screen behind him (G4 on the mention of “there is”), while he produces with the left hand a cupped palm-up open hand gesture (G5 on the mention of “the main verb”). (2)

((there is the main verb)) G4 rh-index pointing at overhead [... but there is (...) _]

G5 puoh-cup-lh held toward audience what’s [called the main verb]. \



Peircean semiotics meets conceptual metaphor 

Figure 2.  Pointing gesture (“there is”) plus cup gesture (“the main verb”)

An example of the category “main verb” (“taught”) is being pointed at on the screen (concrete deixis), and the abstract category as such is to be imagined as being contained in the cupped hand directed toward the student audience. The second gesture (G5) again reflects a metaphorical understanding of an abstract notion as a concrete object; this object is, however, not represented iconically, but needs to be metonymically inferred as sitting inside of the cupped hand. Put differently, the hand here serves as an index, or a “reference point” (Langacker, 1993), thus providing cognitive access to the metaphorically construed object (see Mittelberg, 2005, 2006, and Mittelberg & Waugh, forthcoming). Besides these gestural signs with specific deictic functions, gestures can be said to be inherently indexical. They are always contextualized and indexical of the object they represent. They point to something other than themselves. Being existentially tied to the speaker’s body, the message, and the context, referential gestures may provide grounding for both concrete elements and abstract ideas in the here and now of the speech event. 3.2.3. Symbol The object-sign relationship in symbolic signs is primarily based on conventionality. Yet they can also incorporate iconic dimensions, as is the case with onomatopoeic

 Irene Mittelberg

words and a great number of sign language signs (Taub, 2001; P. Wilcox, 2000; S. Wilcox, 2004). In Peirce’s terms, “[a] Symbol is a sign which refers to the Object that it denotes by virtue of a law, usually an association of ideas, which operates to cause the symbol to be interpreted as referring to that Object” (1960, p. 2.249). Language is the symbolic system par excellence, yet symbolic relationships are to be found in many other sign systems (e.g., religious iconology in paintings, symbolic elements in rituals, etc.), and without the knowledge of the respective code, it is impossible to grasp the symbolic properties of such signs. In addition to sociocultural and scientific conventions of a law-like nature, there may also exist a habitual link between object and sign which can be fostered over time. The relationship between representamen and object may thus become subject to dynamic change (Chandler, 2002, p. 44). Signs may shift in mode over time; possible factors that bring about such shifts include cognitive and social forces, semantic change, metaphorization, theory building, habituation through learning, and so forth.11 Among the different types of gestures, the truly symbolic signs are codified emblematic gestures (emblems according to McNeill, 1992).12 An example would be the thumbs-up gesture indicating “OK” or “great.” As each culture and sub-culture has its own idiosyncratic inventory of emblems, they cannot be deciphered without the specific cultural knowledge. The interpretation of the same emblematic gesture may, in fact, differ across cultures (Kendon, 2004; Morris et al., 1979; Müller & Posner, 2004). Contrary to emblems, most spontaneously produced gestures cannot said to be, in strictly Peircean terms, inherently and predominantly symbolic. Nonetheless, here, too, a gradual habituation due to recurring gesture-meaning pairings in specific environments can result in an increased degree of symbolization and eventually in a new repertoire of conventionalized, i.e., symbolic forms (see Kress et al., 2001; LeBaron & Streeck, 2000; Ochs et al., 1996; Roth, 2003).

11. Peirce privileged “the symbol-using mind” and posited a general tendency for sign systems to evolve from iconic, the original default mode of signification, to symbolic: “a regular progression […] may be remarked in the three orders of signs, Icon, Index, Symbol” (Peirce, 1960, 2.299; Chandler, 2002, p. 44). 12. As Peirce himself points out, a symbol can be embodied in an indexical gesture: “[…] a constituent of a Symbol may be an Index, and a constituent may be an Icon. A man walking with a child points his arm up into the air and says, “There is a balloon.” The pointing arm is an essential part of the symbol without which the latter would convey no information. But if the child asks, “What is a balloon,” and the man relies, “it is something like a great big soap bubble,” he makes the image a part of the symbol. Thus, while the complete object of a symbol that is to say, its meaning, is of the nature of a law, it must denote an individual and must signify a character. A genuine symbol is a symbol that has a general meaning” (Peirce, 1955, p. 17).



Peircean semiotics meets conceptual metaphor 

Figure 3.  Ring gesture referring to a “node” in the tree structure

As the scope of this paper does not allow the discussion of the intricate processes of habituation taking place in the linguistics classroom, I will instead provide a gesture which only occurs once in the corpus and does not function here as a coded emblem. It is known, however, as an emblematic gesture in several cultures (Kendon, 2004; Müller, 1998). The meanings of this kind of ring gesture, achieved by pressing the tips of thumb and index finger together, include the ideas of precision and perfection. Above (Figure 3), it instead represents a spontaneous attempt to depict a technical term used in the framework of generative grammar, namely the concept of the “node” (G6) designating joints in syntactic tree structures. Both “tree” and “node” are metaphorical understandings of abstract structures (the instructor here analyses a verb phrase consisting of the verb ‘invite’ and its complement ‘Terry’): (3) ((node, branching)) ... The rightmost bra=nch, / goes down to= another, _

G6 ring-lh [(..) node, / (..) on the tree, \ N-O-D-E, ]\

 Irene Mittelberg

which itself branches, _ (..) to ‘invite’ and ‘Terry.’ \ After having exemplified the principal modes of sign-object relationships in Peirce’s theory, I will now turn to the three-fold subdivision of the icon. 3.3

Iconic modes within metaphor

Gestural signs and the objects they represent can be similar in various ways. Iconic gestures, portraying individual objects and actions, are to a high degree image icons and belong as such to one of the three iconic modes of representation: image, diagram, metaphor (Peirce, 1955, p. 10). Diagrams are icons of relations, schematically depicting the relationship holding between two or more things. Metaphors draw attention to similarities between two concepts, thus highlighting a parallelism. In Peirce’s own words: Those [icons] which partake of simple qualities […] are images; those which represent the relations, mainly dyadic, or so regarded, of the parts of one thing by analogous relations in their own parts, are diagrams; those which represent the representative character of a representamen by representing a parallelism in something else, are metaphors. (1960, p. 157, 2.277)

The goal in this section of the paper is to demonstrate some of the ways in which gestural representations of grammar may combine these different types of iconicity. 3.3.1 Image iconicity A prototypical example of an image icon in the data is the gesture shown above in Figure 1. This gesture consists of two open hands with palms facing each other, appearing to hold a physical object between them. The way the hands are being held evokes the two shorter sides of an object, or, we can also perceive a similarity in relation to the action of holding a generic object such as a box between two hands. As was the case in the first example (“grammar is not a thing,” section 3.2), this kind of iconic sign receives a metaphorical interpretation in the context of meta-grammatical discourse. The point here is that the way in which the object represents a single abstract idea, such as grammar, is an example of image iconicity within metaphor. To examine these phenomena further, let us look at example 4 below; the “passive flip-flop” entails the image-iconic portrayal of both an abstract action and an abstract object.



Peircean semiotics meets conceptual metaphor 

(4)

((flip-flop passive)) G7 both arms crossing over chest [The passive basically flipflops _



G7 being held the subject and object of the sentence, \ a=nd _

G7 still being held (...) what we find out by forming this particular passive], _ G8 pcoh-box-bh – held, hands move up and down is [that the string ’John’s sister’ forms a constituent, \ G8 still being held hands, move up and down namely the object of the verb, \ G8 still being held, move up and down, hands retract to fist (..) and that’s an object noun phrase] in fact. \ To illustrate the concept “passive,” the speaker crosses over her forearms (G7, after first holding them vertically and shoulder-width apart). This physical action represents, iconically, the switching around of two things. It stands, metaphorically, for the abstract action of ‘flip-flopping’ two elements in a sentence, here referring to the inversion of subject and object entailed in the syntactic operation of activepassive transformation (according to the generative grammar framework). Following the stroke of the first gesture (G7), which coincides with the mention of the verb “flipflops,” the arms-crossed gesture is being held while the speaker continues to explain the syntactic operation involved (the underline indicates the duration of the gesture hold). She turns her torso leftward towards the overhead screen behind her, walks briefly towards the screen, then turns back and finally faces the audience again. Subsequently, her arms open up again and flow right into the next gesture (G8): both open hands being held more than shoulder width apart and facing each other, seemingly holding an (imaginary) object. Figure 4 below shows both the arms-crossing gesture in its fully executed state and the ensuing object gesture.

 Irene Mittelberg

Figure 4.  arms-crossing gesture (passive “flip-flop”) resulting in large object gesture (“constituent”)

The second gesture (G8) is thus another example of the object gesture reflecting a parallelism between an abstract notion, here “constituent,” and a physical object. As already pointed out, it reflects the metaphorical concept ideas are objects (Lakoff & Johnson, 1980), materialized in the manual modality. Upon close investigation, we can, in fact, detect a metaphor incongruity between speech and gesture: Although the sequence “John’s sister” is linguistically described as a string, the concurrent gesture clearly depicts a relatively large box-like object (G8). In the corpus, variants of this image-iconic gesture were found to depict linguistic units of different complexity above the morphemic level (e.g., categories, words, phrases, sentences, discourses, etc.). 3.3.2 Diagrammatic iconicity Diagrammatic iconicity is relational in nature, i.e., the “relations among the elements in the sign are isomorphic to relations among elements in the object” (Liszka, 1996, p. 37). Hence the similarity expressed does not need to be based on a physical correspondence between the object and the sign (as in a painted portrait), but can instead reside in the organizational analogy of the spatial or temporal constellation of at least two parts of an object. Maps, statistical graphs, timelines, musical scores, or family trees may serve as examples of visual schematic representations, all of which also rely on conventions (Chandler, 2002, pp. 39–43; Jakobson, 1966, pp. 409f.). Moreover, diagrammatic iconicity provides structural threads



Peircean semiotics meets conceptual metaphor 

through complex linguistic signs constructed of individual parts, be they words (Bybee, 1985; Waugh, 1992), sentences (Givón, 1985; Haiman, 1985; Jakobson, 1963), or entire discourses (Waugh et al., 2004). Diagrammatic qualities in iconic signs are of particular relevance for the present study, because a large portion of the gestures illustrate linguistic structure and in doing so make abstract, relational iconicity graspable by depicting morphological and syntactic structure in a spatial way (in this case, the respective position of each element in space carries meaning). The architecture underlying a sentence is, following Peirce, in and of itself an icon of relations. Within the abstract domain of grammar, I identified several types of diagrammatic iconicity within metaphor, two of which I will treat here: first, gesture diagrams that may be created on the fly, and second, gesture diagrams imitating conventional diagrams that already exist in another visual modality (such as tree diagrams or diagrams used in the theory of relational grammar, reminiscent of umbrellas or igloos). An example of the first type, an ad hoc diagrammatic representation of sentence structure, is a hand movement indicating the spatial relationship of linguistic units within a sentence by simply tracing the horizontal alignment of words from the left to the right of the speaker. A slight variation is shown in figure 5 below, where the gesture starts out with both hands joined at the center of gesture space, right in front of the upper torso of the speaker. Then the hands move laterally outward until both arms are fully extended, as if they were tracing a horizontal string or chain of words: (5) ((string of words)) So how do we kno=w which complex syntactic units make up English sentences? G9 string-bh, lateral horizontal outward movement So [we think of a sentence as a string of words … ] G10 sketchy pvoh-object-bh, repeated along the string it’s [obvious that we can group the words together in various ways]. After setting up the sentence structure in gesture space (G9), the speaker seems to group elements with both hands (palms facing each other), indicating boundarylike contours of objects in a rather vague way along the imaginary string. The designated spaces between both hands seem to overlap as the hands move from the middle to the right. It is very likely that this kind of diagrammatic representation of word groups within a sentence was motivated by the fact that the speaker had just explained “bracketing,” a classification system according to which sentence elements are grouped into constituents marked off by square brackets; more complex linguistic

 Irene Mittelberg

units encapsulate less complex units. The schematic graph below stands for a fleshedout sentence, and there is, as in prototypical diagrams, a part-part relationship among the word groupings – alluded to by the bracketing gesture – as well as a partwhole relationship between the individual units and the entire sentence.

Figure 5.  A “sentence” represented as a string of words

There is a clear difference, often stressed by the instructors, between viewing a sentence as a horizontally elongated entity, as shown in Figure 5, and depicting hierarchical relations by exploiting the space spanning between both the horizontal and vertical coordinate axes. It seems safe to assume that the culturally ingrained practice of reading and writing from left to right conditions, at least in Western cultures, basic conceptualizations of words and sentences as horizontally arranged combinations of discrete elements. They can be considered as the default model, also available to speakers without theoretical training in linguistics. By contrast, the characteristic feature of syntactic tree diagrams used in generative grammar is a schematic branching structure (comparable to family trees) that spatially represents hierarchical relationships between constituents. Let us now look at the second type of diagrammatic iconicity within metaphoric gestures which entails direct, image-iconic, renditions of standardized diagrams that are integral to the theoretical apparatus of a given framework. One way in which the tree model is repeatedly reflected in the data is in the form of a triangle-like shape, formed with hands joined at the center top and both forearms held diagonally with elbows pointing outwards. As can be seen in Figure 6 below, the



Peircean semiotics meets conceptual metaphor 

evoked pyramid directly imitates a tree chunk, thus mirroring a part of the diagram figuring on the blackboard behind the speaker. Alternatively, the speaker drew the same kind of geometric configuration in the air, with both hands starting out at center top, fingertips touching (at the node) and then tracing two diagonal lines downward, one to each side of her body (also represented in figure 6 below).

Figure 6.  Triangle depicting a “tree chunk” which branches out laterally

When talking about dependent clauses, the same speaker sketches in example 6 (figure 7) a branch descending toward the lower right of her body (G11 below) by repeatedly moving her right hand first up to eye-level and then diagonally downward to her right side, thereby making a wave-like movement by tilting the hand from side to side. This movement can be assumed to roughly imitate the process of going down along the different nodes and branches of a tree structure. (6)

((wavy embedded clauses)) G11 repeated twice rh diagonal wavy line from head downward to the right … [but this is gonna be another one with embedded sentences, _

(G11 repeated) G12 index-rh, extended arm and index point to ground coming in verb phrases] [all the way down] \.

 Irene Mittelberg

Figure 7.  An “embedded clause” as a diagonally descending wavy line

Figure 8.  An embedded clause “going all the way down”



Peircean semiotics meets conceptual metaphor 

Through the consecutive gesture (G12), we learn that there are certain cases in which embedded sentences go “all the way down,” at which point the speaker directs her fully extended right arm towards the floor and points with her index finger straight to the ground (as shown in figure 8 above). Both lines drawn in the air (G11 and G12) evoke a spatialization of the idea of sub-ordination by reaching into comparably low regions of gesture space. Such gestures are schematic images that are informed by a particular theoretical view of relations holding between elements; without the relevant theoretical background, it would probably be difficult to make sense of them. They are visual renditions of relational iconicity which verbal descriptions cannot provide in an equally economic fashion. 3.3.3 Metaphor iconicity Metaphor iconicity entails a sort of cognitive-semiotic detour in establishing similarity: Two objects, or concepts, are brought into relation due to certain characteristics they share (concerning form, function, qualities, structure, etc.), and one concept is represented through another concept (Liszka, 1996, pp. 37f.). According to McNeill (1992, p. 80) metaphorics are gestures that simultaneously depict two things: A) the base (vehicle or source domain) of the metaphor, namely the concrete entity or action represented in the gesture and B) the referent (tenor or target domain), that is, the abstract concept expressed by it. With reference to Lakoff & Johnson (1980), they are said to refer to an abstract entity by depicting certain aspects of the source domain of an underlying metaphorical mapping in the conceptual system. Within the abstract domain of grammar, linguistic theories themselves tend to be built on specific sets of metaphors, which can be interpreted in Peircean terms as instances of metaphor iconicity within metaphor. In the generative grammar framework, tree structure diagrams combined with the corresponding terminology characterize grammatical relations as well as the behavior of syntactic elements, in terms of both social and spatial hierarchies. It was observed in the data that the conceptual metaphor power is up (Lakoff & Johson, 1980) manifests itself through a division of labor between the two modalities. In the manual modality, which naturally lends itself to convey spatial information, metaphorical expressions from the domain of power relations (dominance, control, command, government, subordination) are accompanied by gestures reflecting compatible spatial metaphors such as a tree structure (cf. the triangle forms and diagonal up-down motions discussed earlier): the most powerful, governing elements are situated at the top of branching structures and the dominated, subordinated elements at lower levels. These gestural representations can be seen as reflecting the metaphors having control is up and being subject to control is down (Lakoff & Johnson, 1980, p. 15) and are,

 Irene Mittelberg

according to the Peircean paradigm, instances of metaphor iconicity within the metaphorically-accessed domain of grammar. Another theory of grammar covered in the corpus is Emergent Grammar (Hopper, 1998) which sees linguistic form as being motivated and shaped by discourse-pragmatic forces, that is, by language use. It questions whether there are in fact a priori categories such as nouns and verbs. The following excerpt (example 7) gives an idea of the understanding of grammar as a dynamic system (it is the continuation of example 1, “grammar is not a thing”): (7)

((emergent grammar)) G13 puoh-tray/cup-lh and puoh-claw-rh with relaxed fingers draw circles over lh [it sees the use and grammar together (...)­_]

G14 as G13, but lh is raised twice toward rh [grammar coming out of the use (...) _ you] learn rather then _

G15 rotation-bh [by doing,] \

G16 pvoh-box-bh, first made to the right of the speaker, then to the left [you learn first the rules and then you do them], \ G17 G18 puoh-box-bh pushed together rotation-bh [it blurs the boundary between (...) [learning.. and doing].\ As illustrated in figure 9, the gesture illustrating the idea of emergence (G13) consists of the speaker’s left (non-dominant) palm-up open hand with slightly curled fingers forming a sort of receptacle and the right hand exhibiting an open palm but facing down and being held above the left hand. The fingers of the upper hand are drawing horizontally oriented circles (seemingly stirring something), and on “coming out of ” the lower hand approaches the upper hand, signifying that something is emerging from below.



Peircean semiotics meets conceptual metaphor 

Figure 9.  “Emergent grammar”: patterns emerging out of language use

Actions such as “doing” (G15) and “learning and doing” (G18) are depicted with a rotation gesture produced with both hands revolving around one other. The idea that boundaries between grammar and language use are blurred is represented by two hands apart palms facing each other (like the container gesture) which are then pushed towards each other (G17) to convey the idea of fusion. From a Peircean perspective, these representations of blurred boundaries and fusion are also instances of metaphor iconicity within metaphor. While the previously discussed gestural diagrams of linguistic structure can be said to originate, at least partly, in conventionalized diagrams – proposed in the literature as well as reproduced on blackboards, overhead transparencies, assignment sheets, etc. – the theory of emergent grammar does not have standardized diagrams. This might be a reason why the speaker trying to explain this view needed to be more creative in her visual descriptions of the idea of emerging patterns (Figure 9 above). 3.4

Summary

The sequences of multimodal communication discussed above give an idea of how teachers draw on linguistic resources and gestural illustrations to navigate themselves and their students through abstract knowledge domains whose access and understanding seems to be guided to a high degree by metaphorically conceptualized models and processes. Metaphor is assumed to be anchored in embodied perceptual and kinesthetic experience in the first place; a close look at the data reveals

 Irene Mittelberg

some of the ways in which gesture may provide a window on the oscillation between the conceptual and the grounded, e.g., between thought, bodily experience, and selfexpression. These observations confirm the claim that metaphoric gesture and speech share the task of meaning construal in situated cognition (cf. Cienki, 1998, 2005; McNeill, 1992; Müller, 1998, in press, Nuñez, 2004; Sweetser, 1998, 2007).13 As has become evident in the course of the above exposition, there are indeed no clear-cut boundaries between the different semiotic modes; that is, the image is present in the diagram, and both image and diagram reside in metaphor (cf. Danaher, 1998; Hiraga, 1994). Likewise, a symbol may encompass iconic and indexical qualities. Table 1 illustrates the relative hierarchy of semiotic modes and indicates the cognitive-pragmatic principles they are rooted in: Table 1.  Modes of representation according to Peirce (1955) sign-object relations

sub-types of iconic modes

icon (similarity) index (contiguity) symbol (conventionality, habit)

image (single icon) diagram (icon of relations) metaphor (parallelism)

In section 4, I suggest ways in which these nesting relations of semiotic modes can shed light onto possible points of cross-fertilization between Peirce’s semiotics and cognitive metaphor theory. 4. Peircean semiotics meets conceptual metaphor: Common ground revealed through gesture Central to the present work is the idea that the intricacies of the iconic modes Peirce distinguished can be discerned, as illustrated above, in metaphoric gestures. In his discussion of cross-fertilizing aspects of Peirce’s semiotics and cognitive metaphor theory (Lakoff, 1993; Lakoff & Johnson, 1980, 1999), Danaher (1998) provides a revealing lens on some of the assumptions the two approaches seem to share. Some of these points of intersection can, as I hope to show here, illuminate and also be illuminated by the principles of gestural sign constitution and 13. “[S]ome of the same metaphors appear in gesture as they appear in spoken language when thinking about and discussing abstract domains. The metaphoric gestures, however, can express spatial and force-dynamic information about the speaker’s conceptualization of the given event or entity that are not necessarily expressed in speech. [… D]ifferent metaphors may be expressed in speech and gesture, with the two representing different aspects of a coherent metaphorical model” (Cienki, 1998a, p. 194).



Peircean semiotics meets conceptual metaphor 

communication. Arguing that both approaches are concerned with the relationship between experience and interpretation and that cognition and semiosis can be seen as two sides of the same coin, Danaher emphasizes the role Peirce’s notion of semiotic translation plays in these processes: meaning (interpretants) can be understood as arising from the translation of a given sign belonging to one sign system (e.g., the linguistic) into another sign system (e.g., the conceptual). Gesture clearly participates here, adding a visual sign system to the linguistic and the conceptual. According to Peirce, icons, indices, and symbols are to be understood as reflections of different types and stages of experience, with an increasing level of abstraction from icon to index, and finally to symbol. A fundamental assumption is that all experience is given to us semiotically (Danaher, 1998, pp. 188–189). Below, I will highlight some ways in which gestures may attest to the “semiotic reality of image-schemas” (Danaher, 1998, p. 190) and to the semiotic reality of other kinds of abstract patterns such as basic geometric shapes. 4.1

Image schemas as iconic symbols

Image schemas occupy a central place in cognitive approaches to meaning. According to Johnson (1987, p. xiv), image schemas are defined as “recurring, dynamic patterns of our perceptual interactions and motor programs that give coherence and structure to our experience.” Danaher (1998, p. 189) argues that image schemas can be described in Peircean terms as iconic symbols which have acquired their symbolic status through habits of interpretation, e.g. through regular interactions between the conceptual and the experiential, resulting in the perpetual formation of similar interpretants (see section 3.2.3 above on symbols). An example of a pervasive image schema is the path schema which consists of a beginning (source), an end point (goal), and a vector tracing a path between the two points (Johnson, 1987, p. 28); it manifest itself in actions such as walking from one point to another, throwing a ball, punching someone, etc. Such schemas are not static images, but malleable, dynamic patterns of activities that can take on different dimensions, depending on the medium of expression (cf. Johnson, 1987 on visual art; Cienki, 2005 on gesture, and Hampe, 2005 for an overview of the latest advances in image schema research). Independently of the medium or media in which they manifest themselves, image schemas can be seen as generalized (conceptual) structures abstracted from our habitual bodily experience, not only based on perception, but also on motor programs and force gestalts (Talmy, 1988). It appears that some of the senses and channels we use to build our knowledge about the world are directly reflected in gesture: we still seem to be doing many of the original actions of comprehending the physical world when we gesture (e.g., move, mold, draw, trace, grasp, push, pull, weigh, etc., according to Müller’s [1998]

 Irene Mittelberg

system of modi of gestural representation; see also Calbris, 2003; Cienki, 1998a, 2005; LeBaron & Streeck, 2000; Streeck, 2002; and Sweetser, 1998, 2007.). Given that concepts are assumed to be multidimensional gestalts comprising image-schematic, perceptual, kinesthetic, tactile and other interactional properties (Johnson, 1987, 2005; Lakoff, 1987), it comes as no surprise that gestures tend to portray those dimensions that correspond to both the logic of their own materiality and the logic of space. Gestures are bodily semiotic acts through which embodied image and action schemas may, to some degree at least, be externalized, made visible, and used for meaningful communication. They can bring to light visual and sensorimotor aspects of mental models that would require laborious paraphrases, e.g., in the case of spatial constellations or the direction and manner of movement through space (see Duncan, 2002; McNeill et al., 2001; Müller, 1998; Slobin, 1996). In the case of the present study, a set of image-schematic and geometric patterns emerged from the data. For instance, the PATH schema materialized in those gestures that trace a horizontal line from the left to the right side of the speaker, thus representing a sentence from its beginning to its end. The patterns are not just derived from the forms; rather, the hand shapes and movements feed into the evocation of the full schema or configuration. The identified salient patterns fall into three categories: (a) image-schematic patterns such as support, surface, contact, object, containment, balance, source-path-goal, scale, linkage, left-right, up-down, center-periphery, part-whole (cf. Johnson, 1987; Lakoff, 1987; Lakoff & Johnson, 1999; Mandler, 1996); (b) geometric shapes such as triangles, circles, and squares; and (c) lines traced along horizontal, vertical, and diagonal axes. These patterns cannot be discussed and exemplified in detail here (cf. Mittelberg, forthcoming), but we will see below some of the ways in which they may build the basis for metaphorical projections that materialize in gesture. It should be emphasized that not all of the abstract forms and structures observed here fit the definitions of image schemas found in the literature and that one needs to carefully distinguish between those patterns that represent geometric shapes (e.g., triangle, circle, square) and directions (e.g., horizontal, vertical, diagonal) on the one hand and those that are reminiscent of image schemas reflecting topological forms (e.g., a closed curve) and qualities such as straightness or curviness (cf. Cienki, 1998b, 2005). Overall, the tendencies observed in the present data offer some insights into how image schemas and other basic abstract forms may motivate gestural representations of abstract entities and structures. It remains to be seen whether they will be supported by investigations across different subject matters and genres.



Peircean semiotics meets conceptual metaphor 

4.2

Conceptual metaphors as frameworks for semiotic translation

Mediators between the concrete and the abstract, image schemas are assumed to provide basic conceptual structures for metaphorical projection: they motivate the iconic properties of certain types of metaphors, i.e., the schematic structure of the source domain that gets mapped onto the target domain (Hiraga, 1994; Lakoff, 1990, 1993; Lakoff & Johnson, 1980, 1999; Johnson, 1987; Sweetser, 1990; Taub, 2001). In strictly Peircean terms, a sign receives a metaphorical interpretation when calling to mind an icon based on a parallelism (Köller, 1975). Danaher (1998) conceives of conceptual metaphors (such as LIFE IS A JOURNEY) as cognitive symbols, i.e. translation frameworks for metaphoric linguistic expressions (such as “my life is at the crossroads”). They originate in habits of interpretation (Shapiro, 1983 p. 39), whereby a habit is, in the Peircean sense, a “generalizing tendency;” or, put differently, a framework for the association of ideas (1960, 4.478; see also Jakobson, 1956). The set of image-schematic patterns that emerged from the data (see above) seems to play into the metaphoric and metonymic meaning construction in the ongoing multimodal discourse. As demonstrated above, one of the schemas that is heavily exploited for metaphorical projection is the OBJECT schema (figure 1, examples 1, 2, 4). Using various hand configurations and actions, speakers frequently handle virtual objects, of different sizes, which stand in for linguistic units of different degrees of complexity. Another schema that figures prominently in the data is the SUPPORT schema (Mandler, 1996). This schema was instantiated, for example, in a sequence in which the speaker extends, on the mention of “a noun,” a relaxed palm-up open hand toward the audience. As shown in figure 10 below, the open hand serves as a surface, or support structure, on which the speaker seemingly presents an imaginary object, standing in this case metaphorically for the category “noun” (this is a “puoh” variant, cf. Müller, 2004). Similarly to what we have observed regarding the cupped hand in figure 2, here, too, metaphor and indexicality (metonymy) interact in interesting ways: the imaginary object is not represented iconically, but needs to be metonymically inferred as being part of the basic action of holding or presenting something (presentation for presented; action for object involved in action; cf. Panther & Thornburg, 2004). The open palm points to the geometrically undefined object resting on it (for a more detailed discussion on “reference points” [Langacker, 1993] and perceptual and cognitive salience [Panther & Thornburg, 2004] in co-verbal gesture see Mittelberg, 2005, forthcoming; and Mittelberg & Waugh, forthcoming).

 Irene Mittelberg

Figure 10.  An extended palm-up open-hand gesture providing a surface for a “noun”

The concept of containment proved to be relevant in several ways, since smaller linguistic units were conceived of as being contained in larger units (categories are containers, constituents are contents; Grady, 1998; Johnson, 1987; Reddy, 1979). Phrases and sentences were represented in terms of either a container, a path with a source and goal (figure 5, example 5), a tree diagram (figures 6, 7, 8, example 6), a sequence of adjacent locations in space (example 5), or, put more generally, as spatial structure (conceptual structure is physical structure; Sweetser, 1998; for an experimental study on manifestations of the image schemas container, cycle, force, object, and path in gesture see Cienki, 2005).14 Coming back to Danaher’s view, the question that arises here concerns whether metaphoric gestures, too, may provide frameworks for semiotic translation and aid as such in the mediation of knowledge about grammar and linguistic theory. One indication that this might be the case is that gestures were found to reflect conceptual metaphors even if the concurrent linguistic expression was a non-metaphorical technical term such as “noun,” “main verb,” “object of verb,” etc. In another example from the data, an object gesture is used to refer to the non-figurative 14. Regarding language-based research on metaphorical models of language, thought, and speech communication see Goossens (et al.), 1995; Grady, 1998; Reddy, 1979/1993; Sweetser, 1990, 1992; and see Sweetser, 1998 for findings concerning meta-linguistic co-speech gesture.



Peircean semiotics meets conceptual metaphor 

notion of “infinitive,“ thus suggesting that an infinitive (as abstract and unfamiliar a concept it might be to some of the students) can be seen as a graspable object. This may instill some sort of sensorial reassurance in the students who need to orient themselves in new abstract knowledge domains; it can also be understood as making reference to visually represented words and sentences with which the students are familiar due to their experience with reading and writing. More research is of course needed to fully understand these mechanisms, but if we find that, across speakers and contexts, metaphorical understandings systematically shine through in gesture, even more frequently and more strikingly than in the speech modality, we have reason to maintain that figurative thought based on embodied schemas does seem to structure, at least in part, the gestural representation of ideas, emotions, and conceptual structure more generally. In a dynamic fashion, that is, through visible action (Kendon, 2004), gesture appeals more intuitively to our senses and sense-making capacities than purely linguistic academic discourse could (see also, e.g., Goldin-Meadow, 2003; McNeill, 1992; Nuñez, 2004 and Smith, 2003 for work on mathematics, Kress et al., 2001, Ochs et al., 1996 and Roth, 2003 on physics; Williams, 2004 on time telling; Sweetser, 2007 and Webb, 1996 on philosophy; and Cienki, 1998b, 2005 on moral values). 5. Concluding remarks The results of this study support the claim that gesture can provide additional insights into how humans conceptualize abstract concepts via metaphor. By demonstrating how the manual modality renders abstract spatial information more tangible, the findings are in accord with what McNeill et al.  (2001, p.12) maintain concerning descriptions of concrete spatial configurations (e.g., living space): “Gestures with hands are seen in abundance when people describe spatially organized information.”15 Gestural representations of grammar were found to have the propensity to reveal metaphorical understandings, e.g., to portray a linguistic unit as an imaginary object, container, line, or as a location in space, even if the concurrent speech was not per se metaphoric, but rather featured technical terms such as noun, main 15. McNeill et al. (2001, p. 28; italics in the original): “There is evidence – in our view, critical evidence – that the underlying force behind the integrated triad of kinetics, spoken discourse structure and prosody, is gesture, especially with topics where spatial description plays a crucial role. Whether more abstract topics are also differentiated via gesture remains to be seen. The evidence that gesture leads the way, at least in living space descriptions, takes the form of a greater differentiation of information in gesture that in the other legs of the triad at the same points in the discourse.”

 Irene Mittelberg

verb, sentence, etc. Grammatical categories as well as operations (e.g., subject-verb agreement or passivization) get translated into the concrete manual modality by evoking correspondences (parallelisms, according to Peirce) between the intangible and the embodied, by providing perceptible action and spatial information, even if only in the form of an extended open hand or a sketchy line. Within the metaphorically accessed domain of grammar, Peirce’s iconic modes were identified as follows: in gestural representations of single linguistic elements (image iconicity), in those that depict relationships holding among parts of a linguistic structure or a theoretical framework (diagrammatic iconicity), and in theory-dependent metaphorical understandings of the behavior of grammatical categories such as dominance, emergence, etc. (metaphor iconicity). It was further concluded that iconicity and metaphor alone cannot explain the meaning-making processes observed in the data and that iconic modes interact with indexical modes in making abstract entities present and graspable in the immediate context. It is, in certain cases at least, through metonymy (i.e., indexicality based on contiguity) that the perceivable manual articulators provide cognitive access to the metaphorically construed objects speakers seem to manipulate when talking about abstract categories and structures (see Mittelberg & Waugh, forthcoming). Drawing on both Peirce’s semiotics and cognitive metaphor theory has revealed common ground between the two theories. The observations presented here, based on meta-grammatical discourse, complement previous research on manifestations of iconicity and metaphor in both co-speech gesture and signed languages (see work cited above), which has inspired this study in the first place. Although it remains to be seen whether the tendencies observed so far hold across languages and contexts, this body of work suggests that image schemas and conceptual metaphor may motivate, at least partly, spontaneous gestural representations of abstracta. Since Peirce’s pragmaticist semiotic theory was not developed exclusively based on language but has a wide range of applicability, and since gesture can (as spoken language) only be described pragmatically, his fine-grained paradigm is well suited to be applied to gesture. Inversely, the fact that gestures are communicative acts makes them an ideal semiotic system to put Peirce’s doctrine to the test and demarcate potential limitations. Whereas Peirce proposed several sub-types of iconicity (similarity), all of which where identified in the data, he did not, to the best of my knowledge, devise sub-types of indexicality (contiguity), which calls for further research in this direction (cf. Fricke’s [2007] typology of linguistic and gestural deixis and Mittelberg’s [2006] taxonomy of indexical/metonymic modes in metaphoric gestures). Studying bodily semiotics from the angle proposed here represents only one way of doing so; yet, as I hope to have shown, it can shed light on the externalization, or “ex-bodiment,” (Mittelberg, 2006), of mental imagery and internalized



Peircean semiotics meets conceptual metaphor 

structures and practices, which uses, just as embodiment does, the body as the medium through which these bi-directional processes of abstraction and concretization are channeled (cf. Bourdieu’s [1991] notions of habitus and bodily hexis regarding social norms and practices). There is also contiguity between the mind and the body, with the body being particularly able to personify dynamic and perceptible aspects of figurative thought. The human body functions as the locus where cognitive-semiotic processes take shape, linking knowledge, linguistic expression, and visible action in the formation of utterances (Kendon, 2004), and thus providing anchorage for thought and theory in both physical surroundings and socio-cultural practices: “Nos premiers maîtres de philosophie sont nos pieds, nos mains, nos yeux [Our first teachers of philosophy are our feet, our hands, our eyes]” (Rousseau, 1969, p. 360). Acknowledgements I particularly wish to thank the editors of this volume, Alan Cienki and Cornelia Müller, for their insightful comments on earlier drafts of this paper and Linda Waugh for sharing her expertise on the works of Charles Sanders Peirce and Roman Jakobson. I am also grateful to Susan Duncan, Shimon Edelman, James Gair, Shweta Narayan, Rafael Núñez, Maria Serrano, Wolfgang Settekorn, Nathaniel Smith, Michael Spivey, Eve Sweetser, Hongyin Tao, and Rebecca Webb for their valuable suggestions and comments on earlier stages of this work. All errors are of course mine. I thank the Cornell Cognitive Studies Program for financial support, as well as Richard Feldman and Andrew Page (Cornell Language Resource Center) for technical support. I am also indebted to the subjects of this study for their generous participation and to Allegra Giovine, Joel Ossher, and Daniel Sternberg for their invaluable help with database design and gesture analysis. And I thank Yoriko Dixon for providing the gesture drawings.

References Bakhtin, M. M. (1981). The dialogic imagination: Four essays. M. Holquist (Ed.). Austin, TX: University of Texas Press. Beattie, G. (2003). Visible thought: A new psychology of body language. Hove: Routledge. Bourdieu, P. (1991). Language and symbolic power. Translated by G. Raymond and M. Adamson. J. B. Thompson (Ed.). Cambridge: Polity Press and Blackwell. Bouvet, D. (1997). Le corps et la métaphore dans les langues gestuelles: A la recherche des modes de production des signes. Paris: L’Harmattan. Bouvet, D. (2001). La dimension corporelle de la parole. Les marques posturo-mimo-gestuelles de la parole, leurs aspects métonymiques et métaphoriques, et leur rôle au cours d’un récit. Paris: Peeters. Bühler, K. (1934/1965). Sprachtheorie. Zur Darstellungsfunktion der Sprache. 2nd ed. Stuttgart: Fischer.

 Irene Mittelberg Bybee, J. (1985). Morphology: A study of the relation between meaning and form. Amsterdam/ Philadelphia: John Benjamins. Calbris, G. (1990). The semiotics of French gestures. Bloomington, IN: University of Indiana Press. Calbris, G. (1998). Geste et images. Semiotica, 118 (1/2), 105–120. Calbris, G. (2003). From cutting an object to a clear-cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture 3, 19–46. Chandler, D. (2002). Semiotics: The basics. London: Routledge. Cienki, A. (1998a). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford: CSLI Publications. Cienki, A. (1998b). STRAIGHT: An image schema and its metaphorical extensions. Cognitive Linguistics, 9, 107–149. Cienki, A. (2005). Image schemas and gesture. In B. Hampe (Ed.), From perception to meaning: Image schemas in cognitive linguistics (pp. 421–442). Berlin: Mouton de Gruyter. Clark, H. H. (2003). Pointing and placing. In S. Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 243–268). Mahwah, NJ: Lawrence Erlbaum Associates. Danaher, D. (1998). Peirce’s semiotic and cognitive metaphor theory. Semiotica, 119 (1/2), 171–207. DuBois, J., S. Schuetze-Coburn, S. Cumming, & D. Paolino (1993). Outline of discourse transcription. In J. A. Edwards & M. D. Lampert (Eds.), Talking data: Transcription and coding in discourse research (pp. 45–87). Mahwah, NJ: Lawrence Erlbaum Associates. Duncan, S. D. (2003). Gesture, verb aspect, and the nature of iconic imagery in natural discourse. Gesture, 2, 183–206. Duranti, A. (1997). Linguistic anthropology. Cambridge: Cambridge University Press. Eco, U. (1986). Semiotics and the philosophy of language. Bloomington, IN: Indiana University Press. Enfield, N. J. (2003). Producing and editing diagrams using co-speech gesture: Spatializing nonspatial relations in explanations of kinship in Laos. Journal of Linguistic Anthropology, 13, 7–50. Forceville, C. (2005). Cognitive linguistics and multimodal metaphor. In K. Sachs-Hombach (Ed.), Bildwissenschaft: Zwischen Reflektion und Anwendung (pp. 264–284). Cologne: Von Halem. Fricke, E. (2002). Origo, pointing, and speech: The impact of co-speech gestures on linguistic deixis theory. Gesture, 2, 207–226. Fricke, E. (2007). Origo, Geste und Raum – Lokaldeixis im Deutschen. Berlin/New York: Mouton de Gruyter. Furuyama, N. (2001). De-syntacticizing the theories of reference maintenance from the viewpoint of poetic function of language and gesture: A case of Japanese Discourse. Doctoral Dissertation, University of Chicago. Gibbs, R. W., Jr. (1994). The poetics of mind: Figurative thought, language, and understanding. Cambridge: Cambridge University Press. Gibbs, R. W., Jr. (2003). Embodied experience and linguistic meaning. Brain and Language, 84, 1–15. Gibbs, R. W., Jr. & G. J. Steen (Eds.) (1999). Metaphor in cognitive linguistics. Amsterdam/Philadelphia: John Benjamins.



Peircean semiotics meets conceptual metaphor  Givón, T. (1985). Iconicity, isomorphism, and non-arbitrary coding in syntax. In J. Haiman (Ed.), Iconicity in syntax (pp. 187–219). Amsterdam/Philadelphia: John Benjamins. Goldin-Meadow, S. (2003). Hearing gesture: How our hands help us think. Cambridge, MA/London: The Belknap Press of Harvard University Press. Goossens, L., P. Pauwels, B. Rudzka-Ostyn, A.-S. Simon-Vandenbergen, & J. Vanparys (Eds.) (1995). By word of mouth: Metaphor, metonymy and linguistic action in a cognitive perspective. Amsterdam/Philadelphia: John Benjamins. Grady, J. (1997). THEORIES ARE BUILDINGS revisited. Cognitive Linguistics, 8, 267–290. Grady, J. (1998). The ‘conduit metaphor’ revisited: A reassessment of metaphors for communication. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 205–217). Stanford, CA: CSLI. Haiman, J. (Ed.) (1985). Iconicity in syntax. Amsterdam/Philadelphia: John Benjamins. Hampe, B. (Ed.) (2005). From perception to meaning: Image schemas in cognitive linguistics. Berlin: Mouton de Gruyter. Hiraga, M. (1994). Diagrams and metaphors: Iconic aspects in language. Journal of Pragmatics, 22, 5–21. Haviland, J. (2000). Pointing, gesture spaces, and mental maps. In D. McNeill (Ed.), Language and gesture (pp. 13–46). Cambridge: Cambridge University Press. Hopper, P. (1998). Emergent grammar. In M. Tomasello (Ed.), The new psychology of language: Cognitive and functional approaches to language structure, volume 1 (pp. 155–175). Mahwah, N.J.: Lawrence Erlbaum Associates. Jakobson, R. (1956/1990). Two aspects of language and two types of aphasic disturbances. In L. R. Waugh & M. Monville-Burston (Eds.), Roman Jakobson: On language (pp. 115–133). Cambridge/London: Harvard University Press. Jakobson, R. (1963/1990). Parts and wholes in language. In L. R. Waugh & M. Monville-Burston (Eds.), Roman Jakobson: On language (pp.110–114). Cambridge/London: Harvard University Press. Jakobson, R. (1966/1990). Quests for the essence of language. In L. R. Waugh & M. MonvilleBurston (Eds.), Roman Jakobson: On language (pp. 407–421). Cambridge/London: Harvard University Press. Jakobson, R. (1957/1971). Shifters, verbal categories, and the Russian verb. In Selected writings II: Word and language (pp. 130–147). The Hague: Mouton. Jakobson, R. & L. R. Waugh (1979/2002). The sound shape of language. Berlin/New York: Mouton de Gruyter. 3rd. ed. Johnson, M. (1987). The body in the mind: The bodily basis of meaning, imagination, and reason. Chicago: University of Chicago Press. Johnson, M. (1992). Philosophical implications of cognitive semantics. Cognitive Linguistics, 3, 345–366. Johnson, M. (2005). The philosophical significance of image schemas. In B. Hampe (Ed.), From perception to meaning: Image schemas in cognitive linguistics (pp. 15–33). Berlin/New York: Mouton de Gruyter. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Kita, S. (Ed.) (2003). Pointing: Where language, culture, and cognition meet. Mahwah, NJ: Lawrence Erlbaum Associates. Köller, W. (1975). Semiotik und Metapher. Untersuchungen zur grammatischen Struktur und kommunikativen Funktion von Metaphern. Stuttgart: Metzler.

 Irene Mittelberg Kress, G., C. Jewitt, J. Ogborn, & C. Tsatsarelis (2001). Multimodal teaching and learning: The rhetorics of the science classroom. London/New York: Continuum. Kress, G. & T. van Leuuwen (1996). Reading images: The grammar of visual design. London/New York: Routledge. Lakoff, G. (1987). Women, fire, and dangerous things. What categories reveal about the mind. Chicago: University of Chicago Press. Lakoff, G. (1990). The invariance hypothesis: ‘Is abstract reason based on image-schema?’ Cognitive Linguistics, 1, 39–74. Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (pp. 202–251). Cambridge: Cambridge University Press. 2nd ed. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh: The embodied mind and its challenge to Western thought. New York: Basic Books. Lakoff, G. & R. E. Núñez (2000). Where mathematics comes from: How the embodied mind brings mathematics into being. New York: Basic Books. Langacker, R. (1993). Reference-point constructions. Cognitive Linguistics, 4, 1–38. LeBaron, C. & J. Streeck (2000). Gestures, knowledge, and the world. In D. McNeill (Ed.), Language and gesture (pp. 118–138). Cambridge: Cambridge University Press. Liszka, J. J. (1996). A general introduction to the semeiotic of Charles Sanders Peirce. Bloomington, IN: Indiana University Press. Mandler, J. (1996). Preverbal representation and language. In P. Bloom, M. A. Peterson, L. Nadel, & M. F. Garrett (Eds.), Language and space (pp. 365–384). Cambridge, MA: MIT Press. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. McNeill, D. (Ed.) (2000). Language and gesture. Cambridge: Cambridge University Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. McNeill, D., J. Cassell, & E. T. Levy (1993). Abstract deixis. Semiotica, 95 (1/2), 5–19. McNeill, D., F. Quek, K.-E. McCullough, S. D. Duncan, N. Furuyama, R. Bryll, & R. Ansari (2001). Catchments, prosody and discourse. Gesture, 1, 9–33. Mittelberg, I. (2002). The visual memory of grammar: Iconographical and metaphorical insights. Metaphorik.de, 2/2002, 69–89. Mittelberg, I. (2005). Interaction of metaphor and metonymy in gestural representations of grammar. Manuscript (March 2005), Cornell University, New York. Mittelberg, I. (2006). Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar. Doctoral Dissertation, Cornell University, New York. Mittelberg, I. (2007). Methodology for multimodality: One way of working with speech and gesture data. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson, & M. J. Spivey (Eds.), Methods in cognitive linguistics (pp. 225–248). Amsterdam/Philadelphia: John Benjamins. Mittelberg, I. (Forthcoming). Geometric and image-schematic patterns in gesture space. In V. Evans & P. Chilton (Eds.), Language, cognition, and space: The state of the art and new directions. London: Equinox. Mittelberg, I. & L. R. Waugh (Forthcoming). Multimodal figures of thought: A cognitive-semiotic approach to metaphor and metonymy in co-speech gesture. In C. J. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor. Berlin/New York: Mouton de Gruyter. Müller, C. (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag A. Spitz.



Peircean semiotics meets conceptual metaphor  Müller, C. (2004). Forms and uses of the Palm Up Open Hand: A case of a gesture family? In C. Müller & R. Posner (Eds.), The semantics and pragmatics of everyday gestures. The Berlin conference (pp. 233–256). Berlin: Weidler Verlag. Müller, C. (In press). Metaphors. Dead and alive, sleeping and waking. A dynamic view. Chicago: Chicago University Press. Müller, C. & R. Posner (Eds.) (2004). The semantics and pragmatics of everyday gestures. The Berlin conference. Berlin: Weidler Verlag. Núñez, R. E. (2004). Do real numbers really move? Language, thought, and gesture: The embodied cognitive foundations of mathematics. In F. Iida, R. Pfeifer, L. Steels, & Y. Kuniyoshi (Eds.), Embodied artificial intelligence (pp. 54–73). Berlin: Springer-Verlag. Núñez, R. E. & E. E. Sweetser (2005). Looking ahead to the past: Converging evidence from Aymara language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 1–49. Ochs, E., P. Gonzalez, & S. Jacoby (1996). ’When I come down I’m in the domain state:’ Grammar and graphic representation in the interpretive activity of physics. In E. Ochs, E. Schegloff, & S. Thompson (Eds.), Interaction and grammar (pp. 329–369). Cambridge: Cambridge University Press. Panther, K.-U. & L. L. Thornburg (2004). The role of conceptual metonymy in meaning construction. Metaphorik.de, 06 /2004, 91–113. Parrill, F. & E. E. Sweetser (2004). What we mean by meaning: Conceptual integration in gesture analysis and transcription. Gesture, 4, 197–219. Peirce, C. S. (1955). Logic as semiotic: The theory of signs (1893–1920). In J. Bucher (Ed.), Philosophical writings of Peirce (pp. 98–119). New York: Dover. Peirce, C. S. (1960). Collected papers of Charles Sanders Peirce (1931–1958), Vol. I.: Principles of philosophy, Vol. II: Elements of logic. C. Hartshorne & P. Weiss (Eds.). Cambridge: The Belknap Press of Harvard University Press. Reddy, M. (1979/1993). The conduit metaphor – a case of frame conflict in our language about language. In A. Ortony (Ed.), Metaphor and thought (pp. 284–324). Cambridge: Cambridge University Press. 2nd ed. Roth, W.-A. (2003). From epistemic (ergotic) actions to scientific discourse: The bridging function of gestures. Pragmatics & Cognition, 11, 141–170. Rousseau, J.-J. (1969). Émile ou de l’éducation. Œuvres complètes, t. IV (Bibliothèque de la Pléiade). Paris: NRF-Gallimard. Shapiro, M. (1983). The sense of grammar: Language as semeiotic. Indiana: Indiana University Press. Slobin, D. (1996). From ‘thought and language’ to ‘thinking for speaking’. In J. J. Gumperz & S. C. Levinson (Eds.), Rethinking linguistic relativity (pp. 97–114). Cambridge: Cambridge University Press. Smith, N. (2003). Gesture and beyond. BA Honors Thesis, Program in Cognitive Science, University of California at Berkeley. Streeck, J. (2002). A body and its gestures. Gesture, 2, 19–44. Sweetser, E. E. (1990). From etymology to pragmatics: Metaphorical and cultural aspects of semantic structure. Cambridge: Cambridge University Press. Sweetser, E. E. (1992). English metaphors for language: Motivations, conventions, and creativity. Poetics Today, 13, 705–724. Sweetser, E. E. (1998). Regular metaphoricity in gesture: Bodily-based models of speech interaction. Actes du 16e Congrès International des Linguistes (CD-ROM), Elsevier.

 Irene Mittelberg Sweetser, E. E. (2007). Looking at space to study mental spaces: Co-speech gesture as a crucial data source in cognitive linguistics. In M. Gonzalez-Marquez, I. Mittelberg, S. Coulson, & M. J. Spivey (Eds.), Methods in cognitive linguistics (pp. 201–224). Amsterdam/New York: John Benjamins. Talmy, L. (1988). Force dynamics in language and cognition. Cognitive Science, 12, 49–100. Taub, S. (2001). Language from the body: Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press. Tilley, C. (1999). Metaphor and material culture. Oxford: Balckwell. Waugh, L. R. (1992). Presidential address: Let’s take the con out of iconicity: Constraints on iconicity in the lexicon. American Journal of Semiotics, 9, 7–48. Waugh, L. R., S. N. Smith, E. Specker, S. Steinhart, & J. Wang (2004). Peircean theory, diagrammatic iconicity, and academic texts: Global structure, abstracts, and the role of narrative. Logos and Language. Journal of General Linguistics and Language Theory, V, 1 (special issue on Aspects of Iconicity in Contemporary Linguistics), 39–62. Webb, R. (1996). Linguistic features of metaphoric gestures. Doctoral Dissertation, University of Rochester, New York. Whittock, T. (1995). Metaphor and film. Cambridge: Cambridge University Press. Wilcox, P. P. (2000). Metaphor in American Sign Language. Washington, D.C.: Gallaudet University Press. Wilcox, P. P. (2004). A cognitive key: Metonymic and metaphorical mappings in ASL. Cognitive Linguistics, 15, 197–222. Wilcox, S. (2004). Cognitive iconicity: Conceptual spaces, meaning, and gesture in signed languages. Cognitive Linguistics, 15, 119–147. Wilcox, S. & J. Morford (2007). Empirical methods in signed language research. In M. GonzalezMarquez, I. Mittelberg, S. Coulson, & M. J. Spivey (Eds.), Methods in cognitive linguistics (pp. 171–200). Amsterdam/New York: John Benjamins. Williams, R. (2004). Making meaning from a clock: Material artifacts and conceptual blending in time-telling instruction. Doctoral Dissertation, University of California at San Diego.

Unexpected metaphors* David McNeill Metaphoric gestures are ‘expected’ in the sense that the metaphoric image is predictable from cultural standards, such as those found in many verbal examples (cf. the ‘cup of meaning’ image in both “the film had no content” and a gesture formed with a palm upright open hand). But many metaphoric gestures are ‘unexpected’ – iconic gestures that are revealed to be metaphoric only when the function of the gesture is examined in the surrounding discourse. I illustrate three unexpected metaphors: a gesture depicting a bowling ball being thrust down, a metaphor for the idea of an antagonistic force; a series of gestures of one character trying to reach another, a metaphor for inaccessibility; and a spatial contrast presenting the idea of a contrast itself. None of these metaphors draws on canonical or cultural metaphoric schemas. Unexpected metaphors show a proclivity for making virtually any concrete image a vehicle for something else, even when the image is not among the canonical metaphoric images of a culture.

I shall explain what I mean by ‘unexpected metaphors’ in due course but I begin with a more familiar type, the ‘expected metaphor’. So doing will set the stage and clarify what is actually unexpected about unexpected metaphors. 1. Expected metaphors The hallmark of an expected metaphor is that it follows an established cultural pattern. Such a metaphor can appear verbally as well as gesturally, although not necessarily in the same utterance. It is ‘expected’ in the sense that, given a repertoire of metaphors embodied in a culture, form and con­tent are more or less predictable. The accompanying drawings1 present ex­amples of such gestures. Figure 1 illustrates, in a gesture, what Reddy (1979) called the ‘conduit’ metaphor in spoken * I am grateful to Nobuhiro Furuyama and Shuichi Nobe for very useful discussions of points in this essay. Grants from NSF and ARDA supported the preparation of the article. 1.

Fey Parrill is the artist.

 David McNeill

and written language. The gesture displays the image of a container or sub­stance, an image that embodies the idea of the “next scene” in this case. The speaker’s open, upright palm ‘is’ or ‘contains’ this scene. Conduit metaphors are widespread but do not ap­pear in all cultures (cf. McNeill, 1992).

Figure 1.  Conduit metaphoric gesture accompanying “and the next scene is” in speech

Although in principle speech could have conveyed this metaphor it did not (presumably to avoid cumbersomeness). Also, as is again often the case, the gesture carried information related to the meta-level of discourse – the reference was to the structure, qua structure, of the story being recounted. The second illustration (Figure 2) includes two metaphors bundled into one gesture – time is a moving object, in this case the temporal dynamics of a computer program, displayed as the speaker’s own transverse motion, joined by a metaphor in which a process is a rotation (example due to Eve Sweetser). The combination metaphorizes what was also in speech, “the dynamics (metaphorically, a rotation of hands) of how you get through (metaphorically, a transverse motion of whole body)”. Again, the metaphors are in gesture only or at least the rotation metaphor is gestural only. Expected metaphoric gestures have been widely studied (for example, McNeill, 1992; Webb, 1996; Cienki, 1998; Parrill & Sweetser, 2004; Müller, 2004, this volume). Cultural standards and the potential for counterparts in speech set expected metaphors apart from the ‘unexpected’ metaphors, described next.

Figure 2.  Process as motion metaphoric gesture accompanying the speaker’s utterance of “the dynamics of how you get through the* # right?”



Unexpected metaphors 

2. Unexpected metaphors Metaphoric thinking is not limited to cultural metaphors like the conduit or time is a moving object. There are other gestures that seem at first purely iconic, not metaphoric at all, but on examination turn out to incorporate metaphoric thinking. This metaphoric content is fleeting, one-off, and created afresh and instantaneously, as one speaks. Such gestures suggest a mode of thinking that is fundamentally metaphoric but also spontaneous and improvisational.  The ‘obviously iconic’ gesture of a bowling ball being thrust down (Figure 3), in an oft-examined example (e.g., McNeill & Duncan, 2000), presents both an image of thrusting down and an abstract idea in the form of this image (an idea, I will explain shortly, of an antagonistic force). This essay examines this and other examples of such unexpected metaphors; identifies their role in speech, thought, and discourse; and aims to uncover some of the functions that explain how and why they occur. Unexpected metaphoric gestures are not random. I propose they have both a discourse and an utterance creation function. They form a bridge between the core idea unit or growth point of the utterance at the moment of speaking, and the larger discourse framework. In this way, these gestures, because they are metaphors, help to maintain the coherence of the idea units in question with respect to the larger discourse, and organize speech output itself within this discourse. Unexpected metaphors also complete growth points by providing the imagery they require in combination with linguistic content. Expected metaphoric gestures have the same potential, and the two kinds – expected and unexpected – can be seen as two manifestations of an imagery-language dialectic as it engages abstract content, in the one case the culture providing the imagery ready-made and in the other the individual speaker generating it iconically and investing it with metaphoric significance on her own. The logic is that unexpected metaphors arise from the need to create images when the culture does not have them readily at hand. These images join linguistic content as growth points and differentiate what Vygotsky (1987) called psychological predicates, or points of contrast in the immediate ongoing context of speaking. Unexpected metaphors, precisely because they are outside the conventions of language and culture, can capture abstractions in novel ways and provide the fluidity of thought and language that is the essence of ongoing discourse. I would not want to say that such metaphors are ‘derived from’ or ‘based upon’ iconicity even though iconicity is present; instead, like Lakoff & Johnson, I imagine that we think in terms of these metaphors directly. The iconicity aspect is important for a different reason. When the idea is abstract, when, that is, strictly speaking it is nonimageable, the aforementioned dialectic, to take place at all,

 David McNeill

draws on metaphoricity. This is how metaphoricity comes in – it fulfills an imagery-language dialectic. So, to quote Nobuhiro Furuyama (from an email communication), iconicity is a crucial part of metaphoric semiosis. But, if anything, metaphor sets the stage for iconicity, rather than the other way around. When the culture provides an image ready-made, we have an expected metaphor. When it does not and the speaker, fulfilling the imperative of the dialectic, sees the image as something else, we have an unexpected metaphor (cf. Müller’s tripartite theory of metaphor: it is this seeing the image as something else that conveys that a metaphor is present). 3. The bowling ball metaphor I have presented this example in the past as a case study of the growth point and how the growth point incorporates context (e.g., McNeill, 2000). Now I shall bring out what has been implicit in this example, that the gesture is also an unexpected metaphor. The gesture (Fig. 3) is iconic for the bowling ball being thrust down, but it also is a metaphor of the idea of an antagonistic force. The speaker has just watched an animated color cartoon (one of the Sylvester and Tweety sagas, Canary Row), and is recounting it to a listener, who has not seen the cartoon but will retell it from the first speaker’s rendition. There was no mention of gesture in the instructions to the participants – the task was described as storytelling. The spoken part of the example is shown in line (2) below (line (1) will be referred to later). The first thing to notice is that the timing of the gesture stroke (boldface) is somewhat off, if we think gestures should line up with lexical affiliates. The stroke excluded the verb, “drops”; it coincided with “it down,” and in this way combined two constituents, the Figure and Satellite (using Talmy’s [2000] categories), but excluded another, the Activating Process, which in the sentence structure actually comprises a unit more tightly coupled with the Figure (numbers here and below refer to the sequential order of the utterance in the narration):2 2. Notation: [is the onset of the gesture phrase, when the hands move from rest or a previous gesture into position to perform the stroke; ] is the end of the gesture phrase; boldface is the gesture stroke itself, the meaning-bearing phase of the gesture, performed with effort, and the only phase that is obligatory; underlining is a pre- or poststroke hold, a brief cessation of motion that tends to ensure the synchrony of stroke and targeted speech. Gesture phrases can occur inside other gesture phrases and this is marked by a double ‘[[‘ and ‘]]’. The preparation phase is the interval between the onset of motion ‘[‘and the beginning of the stroke or prestroke hold; the retraction phase is that between the end of the stroke or poststroke hold and the end of motion ‘]’. In the speech transcription, a ‘/’ is a silent pause and a ‘*’ is a self-interruption. The onset of preparation is the first indication the idea unit in the stroke has come to life – in this



Unexpected metaphors 



(1) he tries going [up] [the insid][e of the drainpipe] and Tweety Bird runs



(2) and gets a bowling b[all and drops it down the drainpipe]

Figure 3.  Iconic gesture with “and drops it down the drainpipe” – the object being thrust down, a bowling ball, is also a metaphor for the concept of an antagonistic force

(brackets showing the onset and completion of the gesture phrase; boldface the gesture stroke; underlining pre- and poststroke holds). This timing is not a mystery, however, if we regard the gesture as a metaphor for an antagonistic force. From this vantage point the synchronized gesture and linguistic segments agree precisely. The “it” indexes the bowling ball, which metaphorizes the antagonistic force, and the “down” refers to the direction the force is taking. The verb, “drops,” in contrast, refers to Tweety, the character who was the agent of the dropping or thrusting but not the force in question.3 In short, “drops” was outside the metaphor and was accordingly excluded from the gesture that carried the metaphoric meaning. The exclusion of “drops” was no mere slippage; timing of gesture and speech shows that it was motivated. First, the preparation phase has two features that skip the verb. Preparation began at the first mention of the bowling ball in the preceding clause. This shows that the bowling ball was already the focus. And preparation continued right through the verb, suggesting that the verb was irrelevant to this focus. Further, a brief prestroke hold seems to have preceded “it down” (although coding varies), which, if present, targeted the stroke on the “it down.” Finally, an unmistakable poststroke hold lasted exactly as long as it took to complete the articulation of “down,” preserving the semantic synchrony of the gesture stroke example, with the word “ball” in the preceding clause. A prestroke hold suggests the linguistic material co-occurring with the stroke was targeted. A poststroke hold suggests the stroke and its speech are not merely co-occurring but are a single production. Finally, the end of retraction can be seen as the switching off of the idea unit. 3. More precisely, Tweety was the original antagonistic force but the causative construction, “and (Tweety) drops it down,” with “drops” the verb, transferred the antagonistic force to the bowling ball – the entity the speaker realized had to be the carrier of the force (see McNeill, 2005 for a full discussion).

 David McNeill

with the articulation of the downward path in speech. So the stroke fully and exactly timed with just two words, “it down,” and excluded the lexically affiliated third, “drops.” The question is: why? 3.1

Catchments

How can we tell, in general, what unexpected metaphoric meaning a gesture may have, or that it has such a meaning at all? For this, we must look to the context prevailing at the moment of the gesture. The curved two-handed gesture was part of a family of similar gestures, a ‘catchment’. Surveying the full catchment reveals the thematic idea of an antagonistic force whenever a curved two-handed gesture occurred in this stretch of the narration (the direction of motion, trajectory, and details other than two-handedness and curvature varied according to the individual events; that is, the gestures combined object and meta-level information). A catchment in general is recognized from recurrences of gesture form features over a stretch of discourse. It is a kind of thread of consistent visuospatial imagery running through the discourse. The logic is that discourse themes are linked to images. These images produce gestures with recurring features. Thus, working backwards, the catchment offers clues to the discourse themes in the text with which it co-occurs. In this example, several catchments can be identified from hand use: C1 – 1 handed = Sylvester as a solo force. One-handed gestures occur in items (1) and (6), and tie together references to Sylvester as a solo force. The single hand has a pointing ‘G’ shape (as the classic extended first finger, the rest of the fingers and thumb curled into the palm is known in American Sign Language), a feature also tied to the Sylvester theme (but with a different significance – the shrinkage he had to undergo to get his plump body inside the pipe, as well as directedness).

(1) he tries going [up] [the insid][e of the drainpipe]



(6) [and he comes out the bottom of the drai][npipe

C2 – 2 similar hands = the bowling ball as an antagonistic force. Two-handed symmetrical gestures in items (2), (7), (8) and (9). They group descriptions where the bowling ball is the antagonist, the dominant force. The 2-handed symmetric shape highlights the bowling ball throughout this catchment.

(2) Tweety Bird runs and gets a bowling b[all and drops it down the drainpipe] (7) and he’s got this big bowling ball inside h][[im (8) and he rolls on down] [into a bowling all]][ey (9) and then you hear a sstri]ke



Unexpected metaphors 

C3 – 2 different hands = the relative spatial positions of the bowling ball and Sylvester inside the pipe. Two-handed asymmetrical gestures are in items (3), (4) and (5), and group items in which the bowling ball and Sylvester are equals, differing only in their positions in space and direction of motion. Both hands have a pointing ‘G’ shape, again possibly signifying shrinkage, except at (5), when the lower Sylvester hand opened slightly for the swallowing.

(3) an[nd / / as he’s coming] [up



(4) and the bowling ball’s coming do][wn



(5) he sswallows it / ]

While it’s true that the ‘it down’ gesture stroke was the first occurrence of the catchment, the C2 catchment with its antagonistic force theme apparently was already active, arising with Viv.’s (the speaker’s) first mention of the bowling ball at the end of the preceding clause, in (2). It was then the preparation phase for the stroke started. What seems decisive is that at the start of this preparation Viv.’s hands were shaped the way they were for the downward stroke itself, the shape of hands over the bowling ball. The hands were also facing down in the agent’s shape, the orientation they have for Tweety’s launch. So all the ingredients of the antagonistic force metaphor seem to have been present at that moment – the bowling ball as a force, Tweety as the avatar of the force, and the transference of the force from Tweety to bowling ball. 3.2

The explanation

None of this was copied from the animated stimulus itself – there, Tweety supported the bowling ball from the bottom rather than curling hands over the top, and released it rather than shoved it into the pipe; in other words, the gesture in Figure 2 is the speaker’s construction of the event to have the properties we discover, together with metaphoricity, the whole enabling an imagery-language dialectic and a discourse focus on the antagonistic force theme, while keeping the narrative line and assorted dramatis personae true to the facts of the cartoon. The gesture with “it down” was in the symmetrical C2. It was thus part of the various guises in which the bowling ball appeared in its role as an antagonist. It is not a ‘symbol’ of this force; it symbolizes the bowling ball and its motion downward, as such, but here the bowling ball and this movement presented the idea of a force contrary to Sylvester. The speaker at this point had construed the cartoon as a paradigm of antagonistic forces – bowling ball (surrogate for Tweety) against Sylvester, and the forces differed in direction (down for the Tweety-bowling ball force, up for Sylvester). Becoming a metaphor for the antagonistic force, the bowling ball

 David McNeill

was a bridge from the thematic content of the catchment to the individual growth point. The metaphoricity of the gesture was in this way an essential feature of speech and thought, and the linkage of both to context. 4. The upper story window metaphor The next example presents the idea of inaccessibility as a separation in space. The metaphor uses an iconic image depicting two characters from the cartoon stimulus, one in the upper story window of a hotel and the other at street level, below, to present this abstract idea. The common thread, the catchment, is this image. It links the metaphor to the successive iconic depictions of swinging, climbing, etc. Again, iconic imagery fits the abstract meaning, and again the image ties a growth point to discourse. The speaker begins with an explicit spoken reference to inaccessibility and has her two hands in a configuration of spatial separation (Figure 4.1 – left panel shows the hands just before the stroke, right panel the configuration at the end of the stroke; the stroke occurred during the boldface portion of speech, “ty Bird’s ings* inaccessible” – the * signaling morphological discontinuity). She then repeats the configuration with more descriptions, 7 in all – Sylvester has to climb up somehow (Figure 4.2 – left panel just before the stroke, right panel just after it; stroke during “climbs”). He devises various strategies, including swinging on a rope from another building (Figure 4.3 – left panel at end of the prestroke hold phase shown in the first underlining, right panel at end of stroke and during the poststroke hold phase in the second underlining; stroke itself during “across”) and walking on nearby trolley wires (not illustrated). In the inevitable logic of the cartoon none succeeds, hence the continuing relevance of the idea of inaccessibility. The metaphor appears in the final configuration of each gesture. In Figure 4.2, the climbing gesture iconically conveys climbing; then ends with the hands in the metaphoric ‘inaccessibility’ configuration, and a similar sequence of iconic depiction followed by the metaphor of inaccessibility occurred in each of the gestures. The discourse effect is to link the iconic depictions (climbs, swings, etc.) to the inaccessibility theme, not unlike how an expected metaphor conveyed a discourse structuring function in the “next scene” example.



Unexpected metaphors 

4.1.1–1.2 part of the problem is that [Tweety Bird’s ings* inaccessible] because he can’t really just go in the front door of this hotel

4.2.1–2.2 [so he has to climb]

4.3.1–3.3 / tryi[ng to swing across by a rope #] Figure 4.1–4.3 with subnumbers  Gestures where spatial separation is the metaphor for the idea of inaccessibility.

5. The two spaces metaphor My third example utilizes space to convey the idea of an opposition, which was a narrative theme for this speaker. The catchment was a spatial map embodying this idea in the form of the left space versus the central space. The spaces did not have other values and in fact the references of each shift, but opposition is a constant feature. Figure 5 shows the speaker’s depiction of what he took to be a moral

 David McNeill

ambiguity in a Hitchcock film (Blackmail).4 In his opinion there was a conflict between canonical ascribed morality and actual effective morality. He uses space to metaphorize this opposition, as a concept in itself. In panel (a) he is saying “everyone’s morals are very ambiguous ’cause [they’re sup]posed to be the good guys,” and indicates the space to the left (right hand pointing). In (b) he continues with “[but she] really did kill him”, and points to his front space. The opposition is continued in (c), with the central space again indicated but this time for ascribed morality; he says “and [he’s a] bad guy”, and then concludes with the left space for real morality in (d), “[but he really] didn’t kill him” (McNeill, 1992, p. 155). Morality, whether fake or real, is laid out like a map. Which side (center, left) has which value is indeed unexpected (e.g., the ‘sinister’ side is the ‘real’ pole in (d)) and variable; instead space maps the idea of contrast itself, and does so in a running commentary on the discourse: the next opposite value = the opposite of current space. This is an image not given culturally (cultural uses would be, e.g., Future is In Front – or Back, cf. the Aymara system of temporal metaphors, Núñez & Sweetser, 2006; or Good is Up, Bad is Down, Lakoff & Johnson, 1980).

Figure 5.  Space as the metaphor for moral ambiguity in the Hitchcock classic, Blackmail

6. What unexpected metaphors do, and how they do it We have examined three unexpected metaphoric gestures from different speakers (all speaking English; however speakers of other languages presumably also create them). In each an iconic gesture presented another, more abstract conception. The metaphors are bridges from thought to context, from ideas to discourse. This section offers hypotheses of how and why unexpected metaphors come about. For the bowling ball gesture to be a component in a growth point, the content it carries had to be differentiatable from its C2 catchment, and this required it to be 4. Laura Pedelty is the artist. This figure originally appeared as Fig. 6.7 in McNeill (1992).



Unexpected metaphors 

a metaphor: the bowling ball conceived of as a kind of downward moving antagonistic force. This catchment-metaphor relationship is an important clue to how unexpected metaphors work. The metaphors are essential features of thinking in context. They function in conjunction with fields of oppositions (the array of potential contrasts that constitute the context at each moment) and engineer the differentiation of growth points or psychological predicates from these fields. In the bowling ball example, we infer from the catchment that the field of oppositions was ‘ways of countering Sylvester with a bowling ball’. Then, within this field, a growth point consisting of “it down” plus the image of thrusting down was differentiated: one way to counter him, said the embodied idea, was to thrust the bowling ball down the pipe – this motion was not purely descriptive of the bowling ball but also conveyed the direction of the antagonistic force, and it was as this metaphor of a downward force that it could be differentiated from the context of ways of thwarting Sylvester. In the growth point model a field of oppositions and the differentiation of a growth point within it are both necessary as the chief source of meaningfulness; they function jointly, this combination of the point of differentiation and the field from which it is differentiated. The speaker creates a field of oppositions, like ways to counter Sylvester, in order to make the differentiation possible with the intended significance. Both the field and the growth point are parts of the meaning creation. Metaphoricity is an integral part of this process in cases with abstract content. Such is one reason why metaphor would be an essential engine of language and thought. Putting these conclusions together, we can say that a growth point that includes an iconic component can also have a metaphoric component that utilizes the same image, and this double layering is essential to the growth point when the intended content is an abstract idea in context. Metaphoric imagery using iconicity is a vital component of speech in this model. Unexpected metaphors do not spring from a vacuum of course. The upper story window metaphor linked to the idea that something overhead can be inaccessible; so the image of Tweety above and Sylvester below was stabilized and connected to the idea of inaccessibility. Similarly, the two spaces metaphor depended on the obvious fact that two spaces are not the same space, hence could be linked the idea of a contrast. Shared features of significance are the links whereby the metaphor draws forth an image, an operation I imagine to be similar to the blending described by Parrill & Sweetser (2004) in terms of mental space theory (Fauconnier & Turner, 2002). Also, some unexpected metaphors share imagery with expected metaphors. Being outside convention, however, they express it in novel ways. The unexpected metaphor of a force against Sylvester, the bowling ball, links to an expected metaphor of causation as a forced movement (cf. Lakoff & Johnson, 1999). Although Lakoff and Johnson do not speak of a category of cultural

 David McNeill

metaphors, they provide linguistic examples that seem to fit the concept (gesture versions of which presumably would mostly include directed motions), for example: “FDR’s leadership brought the country out of the depression. The news propelled the stock market to record heights. The trial thrust O.J.’s attorneys into the limelight.” (p. 184, italics in original). Looking at just the linguistic forms, all are verbs. The bowling ball metaphor, however, actively excluded its verb – “drops” – a verb that parallels “thrust” in the last example. The unexpected metaphor focused not on ‘thrusting’ or ‘dropping’ but on the bowling ball itself and its direction of motion – this was the antagonistic force. In common with the verbal metaphor the bowling ball gesture was a directed motion, but the growth point, the combination of linguistic categorial content and image, inhabited causation in a novel way. Such is the freedom with unexpected metaphors. Furthermore, each unexpected metaphor implies a catchment although not every catchment implies a metaphor, because some catchments are threads of solely object-level content.5 Catchment incidence is huge (cf. Furuyama, 2001), thus opening the way for unexpected metaphors, but the adventitious intersection of genre, content, and culture sets the actual frequency. The readiness of the mind to generate metaphors without cultural guidance strongly reinforces the conclusion by Lakoff & Johnson and many others, that metaphoricity is an integral part of cognition. Unexpected metaphors exhibit the two-way interactions that I.A. Richards argued must occur in all metaphors (Richards, 1936). The vehicle, he said, informs the referent, and also the referent the vehicle; metaphor alters both. The antagonistic force was changed by the bowling ball into a particular instantiation of force with qualities (such as the ability to roll) that became constitutive of the force in the episode logic. Simultaneously, the antagonistic force changed the bowling ball into an abstract idea. The upper story and the two spaces in their metaphors changed into inaccessibility and contrast, respectively, and these meanings in turn altered the upper story and spatial images by giving them abstract content. In other words, the interactions that characterize cultural metaphors also emerged with unexpected metaphors. 7. Dynamic and static To finish this essay I will place the discussion into a wider context of two dimensions of language. On the static dimension language appears as an object of great internal complexity, but not a process. Expected conceptual metaphors have the 5. The four living space catchments described in Ch. 5 of McNeill (2005), for example, are iconic and/or deictic but seemingly not metaphoric (contrary to what I once believed).



Unexpected metaphors 

requisite qualities to be fully visible on this dimension – they are listable, fixed, and identified through the synchronic method; the verbs cited previously from Lakoff & Johnson are a convenient illustration – verb in English is the form in which the metaphor emerges statically. Classically, language on this dimension has been situated in a paradigm established by Saussure which continues in many grammatical models to the present day, including cognitive grammars (cf. Langacker, 2000). To see unexpected metaphors, however, requires looking at language on a dynamic dimension, a view more familiar from the Vygotskian tradition. On the dynamic dimension language appears as a process, not as an object. And a metaphoricity comes into view on this dimension with far wider scope. The dependence of unexpected metaphors on context is one sign of the dynamic dimension and their role in differentiating psychological predicates is another. The bowling ball going downward as a metaphor of an antagonistic force depended crucially on context – it was a psychological predicate, was differentiated within a catchment, and this catchment realized the thematic content of ways-of-thwarting. Without these dynamic perspectives the metaphor would be invisible and not conceptualized. As a minimal unit of an imagery-language dialectic the growth point combines the static and dynamic dimensions at given moments, hence requires both dimensions. A growth point can be formed from imagery sourced with either expected or unexpected metaphors, but has different shadings depending on which – more conventional with the expected (such as moving the body forward for time), more improvised with the unexpected (such as the bowling ball as a force). Given contexts may call out different balances of these poles. The lecturer in Figure 2 had an abundance of ready-mades for the workings of a computer.6 Cartoon narrations however find little ready-made imagery for themes like inaccessibility or antagonistic forces. To differentiate these themes narrators created fresh metaphors out of instances of iconicity. Speakers display a capacity for making virtually any concrete image into a vehicle for something else. Other dynamic cases exist where cultural metaphors are ‘awakened’ with some inventive modification that goes beyond convention (cf. the “gefunkt” or ‘sparked’ metaphor in Müller, 2004, an expected metaphor performed in an unexpected way for the ‘sparking’ of love at first sight). The distinction between the static and dynamic dimensions of language, and ways to combine them are discussed at length in the book cited previously (McNeill, 2005).

6. Computation seems to be both a source of metaphors and itself is often described by metaphors like motion through space and rotation.

 David McNeill

7.1

And gesture morphemes

Although not strictly germane to this essay, the search for gesture morphemes exerts a strong appeal to researchers seeking to understand the systematicity of gesture, and some discussion of it may not be out of place, especially since often the puzzle in defining gesture morphemes is to avoid mixing them up with metaphors of the expected kind. A morpheme in the linguistic sense is a recurring form-meaning pair maintained by a linguistic culture; hallmarks are that it is listable and (above all) held to socially maintained standards of form, hence is part of the kind of system that Saussure called langue. A strong case for morpheme status in this sense can be made for some gestures; many of the Neapolitan examples described by Kendon (2004) are performed, as he emphasizes, with ‘crispness’ – a word implying attention to form, as standards would require (I have been personally corrected by Kendon in the performance of one of these gestures during a lecture demonstration; while not concocted in advance, the stunt was a striking exhibition of standards). While unexpected metaphors lack these essential properties, expected metaphoric gestures do recur with overlapping forms and conceivably could be incipient gestural morphemes. Many of the examples cited as potential morphemes seem to include variations on the conduit gesture (cf. Webb, 1996; Müller, this volume) – the palm up, open hand paired with the meaning of a discursive presentation; a claw hand next to the head for ‘an idea in the mind’, and many others. Indeed, a morpheme consists of such a pairing of form and meaning. However, it is important to acknowledge the implication of the fact that each of these gestures does embody a cultural metaphor: the metaphor itself is also a recurring pair of form and meaning. No further standard of gesture form, qua a good form, is implied or required. The conduit metaphor informs the palm up, open hand gesture and gives it the significance of a container or presentation; likewise the claw next to the head (the head is another metaphor or, more precisely, metonym for the presumed inhabitant of the head; cf. Ishino, 2007). In a logical sense, a morpheme interpretation thus lacks necessity and Occam’s dreary razor shaves it off. Tellingly, expected metaphoric gestures lack the crispness that Kendon notes in the Neapolitan examples. Any maneuver within broad limits is good enough so long as it contacts the metaphor base (indeed, the claw hand was presumably palm down in Webb’s, 1996 example). If expected metaphors are just the cultural ready-mades that unexpected metaphors otherwise provide, we would not find a fundamental break between the two in semiosis. In all cases, expected and unexpected, the mechanism is to use gesture for structuring and focusing the discourse. It is likely that some expected metaphoric gestures become involved in a grammaticization process that, in time, turns them into gesture morphemes; indeed, some of the



Unexpected metaphors 

Neapolitan examples Kendon describes include elements that seem to have begun as metaphoric gestures, then stabilized as cultural forms. To summarize, the following layout shows a continuum of gestures differing in how gesture form is established: Autochthonous

Cultural images with meaning

Cultural forms with meaning

Unexpected metaphors

Expected metaphors

Gesture morphemes

In this scheme, unexpected metaphors like the bowling ball as an antagonistic force are autochthonous creations by an individual speaker; expected metaphors like the conduit cup of meaning are culturally specified imagery; and gestural morphemes like the “OK” sign and Neapolitan gestures have standards of form. The continuum also aligns with the dimension of conventionalization, its degree, and the locus within the gesture where it is felt. Unexpected metaphors are conventionalized only in the sense that human actions in general are (for example, an American and non-American may move in culturally recognizable ways) but not as metaphors or as forms. Expected metaphors are conventionalized as metaphors but not as gesture forms. Morphemic gestures have conventionalized standards of form (hence the ‘crisp’ performances of Neapolitan gestures and to some degree emblems like “OK”). References Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koening (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford: CSLI Publications. Fauconnier, G. & M. Turner (2002). The way we think: Conceptual blending and the mind’s hidden complexities. New York: Basic Books. Furuyama, N. (2001). De-syntacticizing the theories of reference maintenance from the viewpoint of poetic function of language and gesture: A case of Japanese discourse. Ph.D. Dissertation, University of Chicago. Ishino, M. (2007). Metaphors and metonymies in gesture and discourse. Ph.D. Dissertation, University of Chicago. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh: The embodied mind and its challenge to western thought. New York: Basic Books. Langacker, R. (2000). Grammar and conceptualization. Berlin: Mouton. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press.

 David McNeill McNeill, D. (2000). Catchments and contexts: Non-modular factors in speech and gesture production. In D. McNeill (Ed.), Language and gesture (pp. 312–328). Cambridge: Cambridge University Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. McNeill, D. & S. Duncan (2000). Growth points in thinking for speaking. In D. McNeill (Ed.), Language and gesture (pp. 141–161). Cambridge: Cambridge University Press. Müller, C. (2004). Metaphors dead and alive, sleeping and waking: A cognitive approach to metaphors in language use. Habilitationsschrift, Free University Berlin. Müller, C. (This volume). What gestures reveal about the nature of metaphor. Núñez, R. & E. Sweetser (2006). In Aymara, next week is behind you: Convergent evidence from language and gesture in the crosslinguistic comparison of spatial construals of time. Cognitive Science, 30, 401–450. Parrill, F. & E. Sweetser (2004). What we mean by meaning: Conceptual Integration in gesture analysis and transcription. Gesture, 4, 197–219. Reddy, M. (1979). The conduit metaphor: a case of frame conflict in our language about language. In A. Ortony (Ed.), Metaphor and thought (pp. 284–297). Cambridge: Cambridge University Press. Richards, I. A. (1936). The philosophy of rhetoric. New York: Oxford University Press. Talmy, L. (2000). Toward a cognitive semantics. Cambridge, MA: The MIT Press. Vygotsky, L. S. (1987). Thought and language. Edited and translated by E. Hanfmann and G. Vakar (revised and edited by A. Kozulin). Cambridge, MA: MIT Press. Webb, R. (1996). Linguistic features of metaphoric gestures. In L. Messing (Ed.), Proceedings of WIGLS: The Workshop on the Integration of Gesture in Language and Speech. Applied Science and Engineering Laboratories, Newark and Wilmington, Delaware (Oct. 7–8).

Catchment, growth point, and spatial metaphor Analyzing Derrida’s oral discourse on deconstruction Jacques Montredon, Abderrahim Amrani, Marie-Paule BenoitBarnet, Emmanuelle Chan You, Régine Llorca, & Nancy Peuteuil Based on a televised interview of the philosopher Jacques Derrida regarding his concept of deconstruction, this research explores the respective roles of image, speech, and gesture in the production of meaning. After having studied the linguistic characteristics of the interview in its entirety, the authors conduct a verbal and non-verbal analysis of a short excerpt, in which they examine: 1) the relevance of two recent conceptual tools developed by David McNeill et al. (1992 to present) – catchment and growth point – and 2) the use of Tuite’s schema (1993) to formalize speech production. At the end of the study, the authors propose a revised version of Tuite’s model, which integrates McNeill’s concepts of catchment and growth point. Their analysis clearly supports the hypothesis that image is influential in activating the whole process of speech production. However, the role of gesture remains unclear. Might gesture not also, and above all, be the scaffolding on which verbal production is based? The multimodal approach to Derrida’s speech presented here appears significant and revealing with regard to this question.

In December 2002, the Image, geste et parole seminar at the Centre de Linguistique Appliquée, Besançon, France decided to undertake an analysis of the verbal and bodily aspects of Jacques Derrida’s discourse in a televised interview. The interview took place on November 4, 2000 and was hosted by the journalist Edwy Plenel in his programme Le Monde des idées on channel LCI. Plenel regularly interviews well-known writers and philosophers on the occasion of their latest publication. The genre of this programme will be examined further on in our paper. Our aim was threefold: first, to use two recent conceptual tools developed by David McNeill and his colleagues (1992 to present) for our analysis: catchment and growth points;

 Jacques Montredon et al

second, to confirm their relevance, and third to enhance Tuite’s (1993) schema of gesture production, by integrating McNeill’s concepts into this model. Our approach was as follows: based on the work of McNeill et al. and correspondence with the author himself, we attempted to clarify the meaning of the terms catchment and growth point by situating them in a broader theoretical framework. We then looked at the notion of deconstruction, once again redefined by Derrida – this time for a wide public – and within this framework, examined the genre of the programme hosted by Edwy Plenel. After having studied the linguistic characteristics of the text in its entirety, we proceeded with a multimodal analysis of a short excerpt from the exchange between the philosopher and the journalist. Finally, we re-examined the validity of the analytical tools proposed by the approach mentioned before and the respective roles of image, speech, and gesture in the production of meaning in Derrida’s discourse. 1. Catchment and growth point First, we shall look at the two concepts borrowed from McNeill. In their article, McNeill et al. (2000, pp. 2–3) defined catchment as follows: A catchment is recognized from recurrences of gesture form features over a stretch of discourse. Catchments are recognized from 2 or more gestures (not necessarily consecutive) with partially or fully recurring features of shape, movement, space, orientation, dynamics, etc. A catchment is a kind of thread of consistent dynamic visuospatial imagery running through the discourse segment that provides a gesture-based window into discourse cohesion. The logic of the catchment is that the discourse themes produce gestures with recurring features; these recurrences give rise to the catchment. Thus, working backwards, the catchment offers clues to the cohesive linkages in the text with which it co-occurs.

We had considerable difficulty in translating the term catchment in French. The Robert and Collins Senior Dictionary gives captage as the French equivalent of catchment. They also define the term in combination with area or basin; a catchment can designate a hospital or school district or water basin. The French newspaper Le Monde (October 23, 2004, p. 2) defines a catchment as a bassin de retenue, a type of reservoir. In his correspondence with our group, McNeill refers to the hydrological sense of the term: “A catchment in the hydrological sense is the ground area that the water drains from, into a pond or lake” (January 2003). In the end, we decided to keep the English term, which clearly evokes the idea of capture. It is true that in McNeill’s catchment, there is the idea that recurring gestures for a



Catchment, growth point, and spatial metaphor 

given unit are mobilized and that these gestures are mobilized as if to testify to a physical or metaphoric context. We were recently given a French translation for catchment in scientific terms that appears to be more relevant: “bassin versant”. The second concept introduced by McNeill et al., growth point, deals with the production of speech. In the same correspondence (January 2003), McNeill clarified the meaning of this term and in so doing, helped us to understand the concept of growth point: A growth point is the theoretical starting point, in a microgenetic sense, of a speech-gesture combination – ‘growth’ in the sense that it is the seed out of which speech and gesture grows. A growth point is a combination of imagery and linguistic categorial content. It is a single idea unit. It is a unit, moreover, that holds not only content but also implies the immediate context of speaking from which it has been differentiated.

Although growth point translates easily into point de croissance, we opted to use the original term. 2. Tuite’s model We chose this model presented by Tuite (1993, pp. 83–105) for the importance accorded to the activity of image in speech production. Moreover, as Tuite himself said, “David McNeill with other researchers contributed significantly to the preparation of this paper with helpful comments” (see the notes section of that paper). Tuite’s model takes its place in the continuity of an approach which views gesture as “an illustrator,” to use Ekman and Friesen’s terminology (1969). Freedman, in 1977, put forth a particular concept of the role of gesture. To quote Kendon on Freedman, (1983, p. 27) He believes that representational gesturing makes manifest in concrete, motoric form, the imagery that is to be given representation in speech. He argues in addition, however, that representational gesticulation plays a facilitating role in the process whereby content is transformed into spoken form. He has proposed that representational gestures assist in this process because they help to keep available, in the form of concrete representations, the material that is to be encoded in verbal form. He suggests that the representation gesture “acts to cement [the image’s] connection to the symbol, and it is this connecting process which seems to be the central psychological function of this activity. Through confirmation of the image, and through the work of connecting the image to the world, object focused activity [i.e., gesticulation] ensures the continuity of representing.”

 Jacques Montredon et al

In the same vein, Kendon himself, in Gesture and speech: Two aspects of the process of utterance (1980), puts forth and illustrates the hypothesis that gesture and speech are co-produced and spring from a single underlying representation which combines both linguistic and imagistic forms.

pre-constructed*

inner speech (verbal imagistic)

image

internal representation

construction*

product*

grammar

speech

structuring*

gesticulation

rhythmic component (viscero-motor)

*annotations: Jacques Montredon

In 1985 McNeill identified this representation as “an image and an inner speech symbol, which are generated at the same time. Inner speech symbols have syntactic implications. These implications and the imagistic properties that the gesture exhibits are generated at the same time, and can develop internally together” (1985, p. 367). Tuite’s model is based on McNeill’s conception in which language production is a process of unpacking global-synthetic representations – associations of images and inner-speech symbols – into external speech and gesture.



Catchment, growth point, and spatial metaphor 

Following the research of Kendon (1972) and Bolinger (1982), Tuite (1993, p. 99) attributes the kinesic organisation of speech production to a rhythmic pulse which governs both gesture and intonation: If expressed gesturally, the pulse peak corresponds to the stroke portion of the gesture. If expressed orally, the pulse peak is represented by an intonational peak, which – presumably because of the more elaborate processing requires by speech production – may be somewhat delayed relative to the gestural stroke.

The model proposed by Tuite, a model for verbal and non-verbal production, leads us to see these two manifestations as twins. In addition the non-verbal twin would appear before the verbal (see Calbris, 1995). This model is original in that the role of iconicity is taken into account in the uttering process, as a partner, so to speak, of language. 3. Deconstruction As for the term deconstruction and its meaning, which we use Derrida’s own words to analyze it. The Robert, Dictionnaire Historique de la Langue Française gives the following information: “deconstruct: transitive verb (1798) and the derivative deconstruction n.f. (1845) take on special meaning when used by the philosopher Jacques Derrida to critically analyze a system by taking its elements apart.” As we shall see further on, the definition given in the dictionary limits the scope of the concept invented by Derrida, who explains the circumstances which led him to borrow this term in a Letter to a Japanese Friend (Derrida, 1985, pp. 1–2): When I chose the word, or when it imposed itself on me – I think it was in Of Grammatology – I little thought it would be credited with such a central role in the discourse that interested me at the time. Among other things I wished to translate and adapt to my own ends the Heideggerian word Destruktion or Abbau. Each signified in this context an operation bearing on the structure or traditional architecture of the fundamental concepts of ontology or of Western metaphysics. But in French “destruction” too obviously implied an annihilation or a negative reduction much closer perhaps to Nietzschean “demolition” than to the Heideggerian interpretation or to the type of reading that I proposed. So I ruled that out. I remember having looked to see if the word “deconstruction” (which came to me it seemed quite spontaneously) was good French. I found it in the Littré. The grammatical, linguistic, or rhetorical senses [portées] were found bound up with a “mechanical” sense [portée “machinique”]. This association appeared very fortunate, and fortunately adapted to what I wanted at least to suggest. Perhaps I could cite some of the entries from the Littré. “Deconstruction: action of deconstructing. Grammatical term. Disarranging the construction of words in a sentence. ‘Of

 Jacques Montredon et al

deconstruction, common way of saying construction’, Lemare, De la maniére d’apprendre les langues, ch.17, in Cours de langue Latine. Deconstruire: 1. To disassemble the parts of a whole. To deconstruct a machine to transport it elsewhere. 2. Grammatical term... To deconstruct verse, rendering it, by the suppression of meter; to prose. Absolutely. (‘In the system of prenotional sentences, one also starts with translation and one of its advantages is never needing to deconstruct,’ Lemare, ibid.) 3. Se deconstruire [to deconstruct itself]... to lose its construction. ‘Modern scholarship has shown us that in a region of the timeless East, a language reaching its own state of perfection is deconstructed [s’est deconstruite] and altered from within itself according to the single law of change, natural to the human mind,’ Villemain, Préface du Dictionnaire de l’Académie.”

4. The genre of the television program Many times, notably in radio interviews, Derrida was asked to reformulate or comment on the value of this word in his philosophy. To our knowledge, the 2000 televised interview with Edwy Plenel is a first of its kind. With regard to the interview, the genre of this program we recorded is difficult to define. To use Derrida’s method of defining, we will not attempt to qualify the program; to do so would be to invent. Le Monde des idées is broadcast on a cable channel and therefore, seen by a limited number of viewers. Consequently, and contrary to other programs, it has not established itself as a genre. The setting – television cameras, broadcasting – is not one of everyday interaction, such as a dialogue or a conversation, nor is it a cultural debate, or a talk show with deliberately staged pros and cons or a program with an educational vocation, the objective of Le Monde des idées. It is none of these, but at the same time adopts some of the characteristics of each of these genres to form its own genre, half way between that of the classic interview of a celebrated intellectual and a cultural debate. Whatever the case may be, it is important to stress both the televisual character of the corpus and, on another level, its relation to the world of philosophy and the fact that it is accepted by academia. This relation allows us to think that the force of habit is at play here, especially in the way Derrida presents things, which is well thought out and far from being accidental. It can therefore be said that Derrida is addressing not a wide public, but rather a limited one, doubtlessly aware of, but not necessarily engrossed in, his philosophy – a public composed of viewers whose cognitive perception of Le Monde des idées varies considerably. A minority of abstract sequential1 individuals (Gregorc’s 1. “Abstract sequential,” one of the four thinking processes identified by Gregorc, comprises the following attributes: intellectual, logical, analytical, correlative.



Catchment, growth point, and spatial metaphor 

terminology, 1982) can comprehend such speech with little difficulty, whereas concrete individuals, be they sequential or not, have much more difficulty. Derrida was certainly aware of this difficulty and because of his desire to have people share his philosophical standpoint, he gradually moved from an abstract definition of deconstruction to a more concrete one, as we will show further on in an excerpt in the last part of our analysis. 5. General linguistic characteristics of the text 5.1

Reformulation

What is striking about the text is that Derrida does not repeat a concise, refined, fossilized definition or content himself with pre-constructed statements. Quite the contrary, he undertakes to re-define for the viewer the notion of deconstruction, a notion he himself created and which, therefore, did not emerge from a common heritage. He does, however, use common language to explain the notion of deconstruction, but the words he uses are not, in fact, transparent and do not convey meaning in an obvious way. He therefore is led to restructure the notional value of his words within the framework of his discourse and this leads to a cascade of further redefining and notional restructuring. In some cases, Derrida does this by restricting the notional field. When he redefines the notion of deconstruction at the beginning of the excerpt, he begins by emphasizing that which does not belong to the notional field. He proceeds by identification and elimination of the interpretations to be dismissed. He begins by saying (lines 16–19): “First of all, it’s to indicate that deconstruction is not a philosophy, it is first of all the deconstruction of philosophy it is not a philosophy nor a doctrine nor a method …” and goes on to say (lines 23–24) that “… deconstruction is not only in the university, in books …”. 2 Derrida also restructures by extending the notional field or by including an element in the field (lines 23–27): “… deconstruction is not only in the university, in books, it is [implying also] in the political, economic, military area. A war is also, or what happens on the stock market contributes to a deconstruction …”. More examples, further on in the text, illustrate the same mode of restructuring: Lines 35–36: … escaping from the hegemony of one culture, uh without disowning it of course …

2. Line numbers refer to excerpts from the French transcript, in the Appendix.

 Jacques Montredon et al

Lines 54–57: … it’s a yes, I often say it, to the affirmation, an affirmation which nevertheless at the same time questioning the history of values of criticism, the values of questioning, attempts to … Lines 69–71: … and the notion of analysis itself like that of the question of criticism is subject to a deconstructive process. 5.2

Rhetorical categories

The first rhetorical category might be considered the category of evocation. By evocation, we mean describing, exposing or referring to something – to an event or an issue – without taking a stand. This category would be characterized by low intonation (levels A and B in our range, see Melodic organization, section 6.1) and an absence of body movements. This category would include different levels of speech acts, such as refuting (interactional), hesitating (the utterance) and parenthesis (organization of speech and strategy). An example of refuting can be found in the utterance: “contrary to what some want to do with it.” By contrast, when the utterances are situated between levels C and D (see Melodic organization, section 6.1), and are accompanied by numerous body movements, even specific constant gesture forms (opening the hand) which also present certain criteria (speed, amplitude, average or high tension), they come under the rhetorical category we would call positioning with respect to an object. Positioning or self-positioning implies the notion of defence or taking position without going so far as to attack. This category includes description, exposition, explanation, and sometimes refutation. Most of the utterances in the text we studied fall under this category. 6. Analysis of Derrida’s conclusion The bold letters below indicate stressed syllables. et pour c’la naturellement il faut pouvoir euh cesser de considérer – euh cet héritage – euh culturel linguistique – euh comme une chose naturelle – c’est une chose qui est historique – et par conséquent construite – par conséquent sédimentée – et la déconstruction c’est avant tout (h) un effort d’analyse – pour désédimenter – pour retrouver – la construction des couches qui qui euh – fondent une culture – et la notion d’analyse elle-même comme celle – de question de critique est soumise à une histoire déconstructive. Our translation: and that, of course is why we have to be able to uh cease considering uh this cultural uh linguistic heritage uh as something natural it’s historic and consequently



Catchment, growth point, and spatial metaphor 

constructed, consequently sedimented and deconstruction is above all an attempt to analyze in order to de-sediment so as to rediscover the construction of the layers uh which which are the basis of a culture and the very notion of analysis itself like that of the question of criticism is subjected to a deconstructive process. We selected this part of Derrida’s discourse because the speaker is clearly at ease both on the verbal and non-verbal level. Up to this point, Derrida seemed to be entangling himself in an abstract definition, which was inaccessible to the neophyte, especially in an oral discourse situation and despite his manifestly good intentions. Suddenly, his words become clear with a concrete image materializing from his fingers. This change in approach leads to a change in bodily actions. His gestures now give way to pictographs or kinetographs, to use the terminology of Ekman and Friesen (1969, pp. 49–98), which seems best to fit the phenomenon we observed. Unfortunately, nothing allows us to decide here between a gesture which outlines a referent in space (the construction of layers in the excerpt we analyzed) and a gesture which depicts a corporal action (descending to the lowest level, removing a layer, de-sedimenting, also in the excerpt analyzed). Both hands are now mobilized, whereas until now, Derrida kept his elbow – cautiously, one might think – on the table. Thus, it is by means of a developed metaphor that the philosopher manages to restructure his definition for a wider public than the one he is used to addressing in his seminars and conferences. 6.1

Melodic organization

In this excerpt, analyzed on a 5500 Kay sonograph, Derrida’s intonation varies around 4 pitch values. These values can be considered not only as criteria for analysis, but as the references that guide the production of his discourse. They are repeated regularly, as if they were musical references for the speaker’s voice and ear, whatever the melodic configurations that separate them. In terms of acoustics, these values correspond to narrow ranges of frequency. These ranges are not used as transitory passages of the voice but as levels to which melodic plateaux of varying lengths adhere. The ranges we recorded are located at the following values, with variations assimilated to the most commonly used frequency values: – – – –

level D: above 200 hz level C: between 150 and 180 hz level B: between 120 and 130 hz level A: between 90 and 100 hz

 Jacques Montredon et al

Generally speaking, Derrida’s prosodic phrase begins at level B and ends at level A (the lowest level). Levels C and D correspond to accentuated syllables. The accentuation continues thus, with 2 degrees of amplitude: average pitch and higher pitch. 6.2

Correspondence between voice and movements

Similarly, we detected 4 degrees of amplitude in Derrida’s body movements, be they forward or lateral. They are as follows: – degree 1: hands moving near the chest – degree 2: hands moving slightly away from the chest – degree 3: medium distance between the chest and hand movements; elbow slightly unbent – degree 4: greater distance between the chest and hand movements; arm nearly straight. As the energy used for the muscular contraction serves both vocal emission and gestural production, the accents of the voice, perceived on prominent syllables, coincide with gestural accents, indicated by impulse. Our gestural and linguistic (including melodic) analysis of the Derrida’s conclusion, as one could see in the following pages, shows such parallelisms. 6.3

Growth points and catchments

Using the notions of catchment and growth point to analyze Derrida’s discourse in his conclusion, we were able to discern three catchments derived from three different growth points: (1) the comparison of cultural heritage to an object outlined by Derrida, (2) desedimentation as a metaphor for deconstruction, and (3) the image of going back to illustrate a deconstructive attitude. We inferred these three growth points from the semantic content of the gestures and their synchrony with speech. For all three, we cite McNeill and Duncan (2000, p.142): “The confluence of speech and gesture suggests that the speaker was thinking in terms of a combination of imagery and linguistic categorical content.”



Catchment, growth point, and spatial metaphor 

6.3.1 Catchment 1 et pour c’la naturellement il faut pouvoir euh cesser de considérer and that, of course is why we have to be able to uh cease considering

Figure 1

Figure 2



Figure 3



Figure 4

euh cet heritage uh this heritage



euh uh



culturel cultural

 Jacques Montredon et al

Figure 5

Figure 6

Figure 7

linguistique linguistic

comme as

une chose naturelle something natural

Figure 8

Figure 9

Figure 10

c’est une chose qui est it’s something which is

historique historic

et par conséquent construite and therefore constructed



Catchment, growth point, and spatial metaphor 

Figure 11

Figure 12

Figure 13

par conséquent consequently

sédisedi-

mentée mented

Two hands – figures 1 to 13: The recurring gestures are the opening and closing of both hands parallel to each other and with palms facing in. The fingers are bent as if to hold something securely between the two palms facing each other. This catchment deals with cultural heritage, something – a concept – that Derrida opens and redefines, and has limits. Here we could say that there is a gestural metaphor in that Derrida is reifying “cultural heritage” by showing its boundaries as if it were an object. The alternating movement of the palms closer together and then apart punctuates the speech and breaks the utterance into segments: – euh cet héritage euh culturel linguistique (Fig. 3) uh this cultural uh linguistic heritage – comme une chose naturelle (Fig. 7) uh as something natural – c’est une chose qui est historique et par conséquent construite (Fig 10) it’s historic and consequently constructed The space between the hands shows the abstract object ‘held’ in the hands. It coincides with the verbal definitions of this object (Calbris, 2003, pp. 93–95): – (cet heritage) euh culturel, linguistique (figure 3) uh this cultural uh linguistic heritage – (c’est une chose qui est) historique et par consequent construite (figure 10) it’s historic and consequentely constructed. The catchment ends with Derrida lowering his hands – palms down – to represent the first layer of sedimentation. Note that Figure I represents a hold: in this frontal view, both hands are closed with the back of each hand facing outward, thumbs touching and index fingers

 Jacques Montredon et al

resting on the thumbs. The hands form a sort of barrier and express the idea of putting an end to a traditional conception of cultural heritage. They are reunited, on hold, as if they were on the starting line, concentrated before the long development to come. 6.3.2 Catchment 2

Figure 14

Figure 15

et la deconstruction c’est avant tout and deconstruction is above all

un effort d’analyse an attempt to analyze

Figure 16

Figure 17

Figure 18

pour désedimenter in order to desediment

pour so as to

retrouver rediscover



Catchment, growth point, and spatial metaphor 

Figure 19

la construction the construction

Figure 20

Figure 21

des couches qui fondent of the layers which are the basis of

une culture a culture

Two hands – figures 14 to 21. The gestural metaphor for desedimentation unfurls in two phases, (1) going back in time (2) reconstructing the sedimentary and historical progression: 1. the first phase, illustrated by both hands moving backwards, has to do with desedimenting: “et la déconstruction c’est avant tout un effort d’analyse pour désédimenter, pour retrouver” (figures 14 to 18) and deconstruction is above all an attempt to analyze in order to desediment so as to rediscover 2. in the second phase, both hands, still in a frontal position represent “(la) construction des couches qui fondent une culture” the construction of the layers which are the basis of a culture, by three successive frontward movements (figures 19 to 21). The concept of desconstruction, more precisely the concept of desedimentation, is gesturally presented as a decomposition of the stages of the construction. In fact the gestural metaphor does not represent a physical sedimentation (the superposition of horizontal layers) but the idea of a construction, a work of time, therefore an historical sedimentation. Derrida’s gestures represent layers piled up over the course of time-space, which in our western culture, is perceived as advancing from the rear to the front (a succession of vertical layers). In fact, sedimentation is the result of time. The gestural metaphor represents both concepts by combining the two images, one illustrating piled-up layers and the other, the progressive stages of construction. Deconstruction implies going back to the first phase of the sedimentation. The growth point of the metaphor for sedimentation unfurled (figures 14 to 21) appears in figures 12–13, at the end of the first catchment, when Derrida divides the new word sedimented, saying sedi-mented. The word and the gesture, both divided, mark a turning point between two ideas: the historical construction

 Jacques Montredon et al

of sedimentation and the analytical process of desedimentation. The hands, held vertically apart and parallel (figure 11) up to this point, are reoriented horizontally during the two first syllables « sedi » (figure 12), before turning upwards and outwards, with the two last syllables « mented » (fig.13). 6.3.3 Catchment 3

Figure 22

Figure 23

Figure 24

et and

la notion the notion

d’analyse of analysis

Figure 25

Figure 26

Figure 27

elle-même itself

comme like

celle de question that of the question



Catchment, growth point, and spatial metaphor 

Figure 28

Figure 29

Figure 30

de critique of criticism

est soumise is subjected

à une histoire to a process

Figure 31

Figure 32

déconstrucdeconstruct-

tive tive

This catchment – figures 22 to 32 – begins when Derrida lowers his left shoulder and starts to move his right arm forward. He holds his thumb and index finger slightly apart and bends his other fingers. His hand and forearm address a remark to the listener, then trace three circles backwards as if to come back to an important point. His thumb and index finger touch when he refers to the notions of analysis itself and of the question of criticism. Characterized mainly by the use of one hand in recurrent movements in a circle or in an arc, this catchment illustrates the concrete etymology of reflect as well as to its figurative meanings. According to the Robert, Dictionnaire Historique de la Langue Française, réfléchir  (reflect) comes from the Latin word  reflectere to bend back or bend over, figuratively to bring back, return or divert, to place one’s attention on; re- meaning back and flectere

 Jacques Montredon et al

meaning bend or yield. It is as if the speaker were acting out the very transition from the concrete to the abstract that, in the generation of language, gives rise to the linguistic manifestation of a concept. This, in turn, reactivates the motivation for the use of this particular sign to the speaker. At the same time, Derrida’s gestures support him in his reflection and facilitate the reconstruction of meaning by the audience. As Derrida says, analysis and criticism themselves can also be subjected to a reflective attitude, that is, be the object of deconstruction. The last arc, a forward movement, indicates in turn, meaning “in turn, can also be subjected to.” Reflecting on the object led Derrida to reflect on the tool that deals with the object. Hence, the concepts of growth point and catchment enabled us to explain, to a certain extent, Derrida’s production in this particular oral situation while the multimodal approach we used led us to examine what had triggered it. Apart from the speaker’s intention here, to communicate a difficult notion to a wide public, the metaphors drawn from the images we produce in our brains do indeed seem to set our gestures, voice, and speech in motion. However, the alchemy is certainly more subtle than we may think because in this particular situation, Derrida may or may not achieve his goal. The speaker becomes aware of the meaning he is conveying as he is developing it and his gestures show the way, as it were, for his discourse – the way his voice will take. This excerpt helped us to understand this phenomenon. The two hands, which had been forming a barrier, opened quite naturally. First of all, this enabled the speaker to execute an accordion-like movement to redefine and limit a concept. From there, he lowered his hands, turning his palms downward and went on to build the metaphor for sedimentation. The continuity of the movements and the natural transitions, with the end of one gesture becoming the graceful beginning of the next, pose the question of a certain subconscious premeditation. The image is, without a doubt, influential in activating the whole process and the gesture, in materializing it. Might the gesture not also, and above all, be the scaffolding (another metaphor) for the verbal production on the whole? It seems to us that the multimodal presentation we give for Derrida’s conclusion is significant and revealing with regard to this question. With respect to this hypothesis, we would like to take another look at the statement made by McNeill et al. (2000, p. 21): “There is evidence – in our view, critical evidence – that the underlying force behind the integrated triad of kinesics, spoken discourse structure and, prosody, is gesture, especially with topics where spatial description plays a crucial role.”

In our correspondence with McNeill (April 2003), we suggested modifying the end of the sentence slightly to include metaphor: “where spatial description or



Catchment, growth point, and spatial metaphor 

spatial metaphor play a crucial role.” McNeill’s response (April 2003) was positive. “I completely agree with your addition to our conclusion – yes, metaphoric uses of space are more important than the references to objective space that we were looking at in our paper. It would have been good for us to have mentioned this point.” Finally, we thought it worthwhile to go back to Tuite’s 1993 model and integrate the concepts of catchment and growth point. Despite the unknowns in this enhanced model, especially with respect to the physical and cultural constraints on gesture, it does have the virtue of representing the multimodal aspect of the speech act. Thus, in conclusion, we propose the following revision to the model.

 Jacques Montredon et al

pre-constructed*

inner speech (verbal imagistic) diagrammatic competence and linguistic heritage

construction and* globo-synthetic crystallization

filters*

product*

iconic activity (creative or reproductive): image, diagram or metaphor

internal representation / growth point

grammar

speech

structuring*

gesticulation or catchment

rhythmic component (viscero-motor)

* annotations: Jacques Montredon

Translations by Nancy Peuteuil, member of the seminar Drawings by Thomas Henriot



Catchment, growth point, and spatial metaphor 

References Bolinger, D. (1982). Intonation in nondeclaratives. In Chicago Linguistics Society parasession on nondeclaratives (pp. 1–15). Chicago: Chicago Linguistics Society. Calbris, G. (2003). L’expression gestuelle de la pensée d’un homme politique. Paris: CNRS Editions. Calbris, G. (1995). Anticipation du geste sur la parole. In Verbal/Non Verbal, frères jumeaux de la parole. Anèfle, Centre de Linguistique Appliquée de Besançon. Derrida, J. (1985). Letter to a Japanese Friend. In D. Wood & R. Bernasconi (Eds.), Derrida and différance (pp. 1–5). 1–2. XLII. Paris: Navarin. Warwick, UK: Parousia Press. Ekman, P. & W.V. Friesen (1969). The repertoire of nonverbal behaviour: Origins, usage and coding. Semiotica, 1, 49–98. Freedman, L. (1977). Hands, words and mind: On the structuralization of body movements during discourse and the capacity for verbal representation. In N. Freedman & S. Grand (Eds.), Communicative structures and psychic structures: A psychoanalytic approach (pp. 109–132). New York and London: Plenum Press. Gregorc, A.F. (1982). Style Delineator: A self assessment instrument for adults. Columbia, CT.: Gregorc Associates, Inc. Kendon, A. (1972). Some relationships between body motion and speech. In A.Siegman & B. Pope (Eds.), Studies in dyadic communication (pp. 177–210). New York: Pergamon. Kendon, A. (1980). Gesticulation and speech: Two aspects of the process of utterance. In M.R. Key (Ed.), The relationship of verbal and nonverbal communication (pp. 207–227). The Hague: Mouton. Kendon, A. (1983). Gesture and speech: How they interact. In J.M. Wiemann & R. P. Harrison (Eds.), Nonverbal interaction (pp. 13–43). Beverly Hills/London/New Delhi: Sage Publications. Montredon, J. (2002). De la gestualité co-verbale, dimensions cognitives et symboliques. In G. Barrier et N. Pignier (Eds.), Sémiotiques non verbales et modèles de spatialité. Limoges: Presses Universitaires de Limoges. McNeill, D. (1985). So you think gestures are non verbal? Psychological Review, 92, 350–371. McNeill, D. (1992). Hand and mind. Chicago: University of Chicago Press. McNeill, D. (Ed.) (2000). Language and gesture. Cambridge: Cambridge University Press. McNeill, D., F. Querk, K.-E. McCullough, S. Duncan, N. Furuyama, R. Bryll, X-F. Ma, & R. Ansari (2001). Catchments, prosody and discourse. Gesture, 1, 9–33. Tuite, K. (1993). The production of gesture. Semiotica, 93, 83–105. Le Robert Dictionnaire Historique de la Langue Française (1998). Sous la direction d’Alain Rey. Paris: Dictionnaires Le Robert. Robert Collins Senior Dictionary français-anglais anglais-français (2002). Paris: Le Robert VUEF.

Appendix Jacques Derrida: Deconstruction Televised interview by Edwy Plenel, recorded on November 4, 2000 Le Monde des Idées Channel LCI

 Jacques Montredon et al

Transcription key for the French version (1) -: indicates a pause (varied lengths)i (2) bold letters: indicate accentuated syllables (3) (h): subject takes a breath E.P. –  1. …sur la la – l’œuvre réputée difficile (h) il y a une question qui 2. est incontournable – comme on dit mal – (h) qui est euh – ce 3. mot donc de déconstruction ce – concept – dans lequel -on 4. vous enferme aussi ou peut-être qu’on galvaude hein – (h)j’ai 5. lu parfois chez des la plume de confrères c’est un peu Derrida 6. nous invite à déconstruire euh tout ce qui nous aurait 7. encombré dans (h) dans la pensée et dans – les (h) siècles 8. passés euh – (h) qu’est-ce que cette idée de déconstruction – je 9. je rappelle les deux définitions que vous en avez donné – la 10. déconstruction c’est ce qui arrive – et puis une autre – plus 11. d’une langue – la déconstruction – c’est ce qui arrive – et l’autre 12. - plus d’une langue J.D. 13. alors -pour un bref commentaire – sur ces deux essais de 14. définition… E.P. 15. essais de définition … J.D. – 16. d’abord c’est pour marquer que la déconstruction n’est pas 17. une philosophie – c’est d’abord la déconstruction de la 18. philosophie -ce n’est pas une philosophie ni une doctrine – ni 19. une méthode – euh (h) c’est – ce qui se passe chaque fois qu’il 20. y a un événement – imprévu – par définition imprévisible (h) et 21. qui déplace en quelque sorte – l’état des choses – et qui 22. surprend – (h) eh bien cela produit cela a un effet 23. déconstructeur – euh et de la déconstruction il n’y en a pas 24. seulement dans les universités dans les livres – y en a dans 25. dans l’espace politique économique militaire – euh (h) une 26. guerre – est aussi ou euh ce qui se passe en bourse – contribue 27. à une déconstruction – euh comme ce qui arrive – comme 28. l’arrivant – euh ou l’arrivé – ou l’arrivant – euh imprévisible – 29. qui produit l’événement – (h) euh en plus d’une langue – dans 30. la mesure où – c’est plus d’une langue – dans la mesure où – la



Catchment, growth point, and spatial metaphor 

31. 32. 33. 34. 35. 36. 37.

déconstruction – euh (h) est – (h) un effort – alors que je définirai très simplement avant tout de mémoire – de généalogie – (h) savoir d’où nous venons -d’où vient la culture – d’ou vient notre langage – et et pour cela – euh se se comment dire – se soustraire – à l ‘hégémonie – d’une culture par exemple de la culture – euh sans la renier n’est-ce pas – je suis aussi pour l’héritage – pour la fidélité dans l’héritage

E.P. 38. d’une forme d’écriture aussi d’une forme de langue… J.D. 39. oui c’est-à-dire euh c’est une attention – à aux 40. différences – à la pluralité – euh (h) à la traduction – mais 41. pas une traduction – en vue d’une espèce de d’espéranto – de 42. d’une langue universelle transparente et sans odeur et sans 43. sans couleur (h) mais une une traduction qui fasse œuvre en 44. quelque sorte – et qui qui contresigne -euh la langue de 45. l’autre– (h) de ce point de vue là – le le motif de la 46. déconstruction – ne doit pas – contrairement à ce que certains 47. veulent en faire -ne doit pas – euh (h)ni – faire peur – comme 48. s’il s’agissait d’une une entreprise de démolition – je j’insiste 49. souvent surtout sur le fait que c’est d’abord – avant tout – une 50. affirmation… E.P. 51. c’est un cheminement plus qu’une théorie… J.D. 52. c’est une expérience – c’est une expérience – c’est l’expérience 53. de ce qui arrive justement – euh (h) et une expérience comme 54. comme gage donné – comme affirmation comme c’est un oui 55. je le lie souvent à la l’affirmation du oui et (h) une affirmation 56. - qui néanmoins – tout en interrogeant – l’histoire des valeurs – 57. de critique – des valeurs – de questions euh – tente de se porter 58. là où (h) il n’y a pas encore – où – il n’y a déjà plus la critique 59. et la question c’est l’affirmation (h) où il y a l’affirmation – 60. donc il s’agit en fait d’un acte de fidélité et de mémoire hein – 61. (h) mais qui traverse – l’histoire euh – euh ou la généalogie – de 62. ce dont nous héritons – 63. et pour c’la naturellement il faut 64. pouvoir euh cesser de considérer – euh cet héritage – euh

 Jacques Montredon et al

65. 66. 67. 68. 69. 70. 71. 72.

culturel linguistique – euh comme une chose naturelle – c’est une chose qui est historique – et par conséquent construite – par conséquent sédimentée – et la déconstruction c’est avant tout (h) un effort d’analyse – pour désédimenter – pour retrouver – la construction des couches qui qui euh – fondent une culture – et la notion d’analyse elle-même comme celle – de question de critique est soumise à une histoire déconstructive.

Form, meaning, and convention A comparison of a metaphoric gesture with an emblem Fey Parrill Speakers tend to produce very similar-looking gestures in very similar discourse contexts. This chapter probes the sources of such uniformity with reference to a particular gesture form. When producing this gesture form, a speaker’s hand moves towards her interlocutor with the palm facing up, as though an object were being presented. Some researchers have claimed that this gesture form is pervasive because people metaphorically conceptualize ideas as though they were objects – the gesture reflects this conceptualization. An alternative hypothesis, however, is that the gesture form is constrained by a cultural convention, in which case the gesture form may no longer be motivated by metaphoric imagery. The present chapter presents an experiment which explores these two hypotheses. Participants were asked to provide a rating of naturalness for non-canonical versions of the gesture form described above and non-canonical versions of a gesture form which is assumed to be wholly conventional. If a gesture form is conventional, participants should be resistant to seeing its normal production form violated. Instead, however, there was a high degree of variability among participants in what was considered acceptable for both gesture forms. These results produce a more complex picture of how convention shapes gesture, which will have implications for theories of language production.

1. Introduction This paper takes up the case of a particular metaphoric gesture and assesses the extent to which its production form is constrained by a culturally established standard. The assessment is made via an experimental comparison between this gesture and an emblem.1 Emblems can be distinguished from other kinds of gestures 1. This project was initially designed to explore questions raised in David McNeill’s discussion of morphological properties of gesture (McNeill, 2005, chap. 2). The focus of the project has shifted somewhat, but interested readers are referred to McNeill’s discussion.

 Fey Parrill

by their possession of a verbal counterpart and a socially constituted standard of form (McNeill, 1992). Whether or not this metaphoric gesture resembles the emblem in having a standard of form is of interest because a standard of form is an index of the gesture’s degree of conventionality. Conventionality, in turn, is of interest because it has implications for theories of speech and gesture production. I will set the stage for my presentation of the experiment by discussing the contrasts between representational gestures and emblems. These categories are defined more fully below, but for the moment I will note that the category of representational gestures includes metaphoric gestures. I will discuss the semiotic properties which can be shared between members of these categories and the motivation for nevertheless classifying representational gestures and emblems as two different kinds of cognitive object. I then provide an illustration of the gesture form in question, which I will refer to as the presenting gesture. Using the framework of conceptual metaphor theory (Lakoff & Johnson, 1980, 1999), I explain why this form has been considered to be metaphoric. Next I describe the experiment itself. The experiment explores the extent to which the presenting gesture is comparable to the emblem often described as the okay gesture.2 In the experiment, observers saw variants of these gestures that differed from the forms with which they are canonically produced. They were asked to rate the gestures on a scale of naturalness. Reactions to variants of the presenting and okay gestures were compared to determine whether participants were more sensitive to the violation of form in one case or the other. If a gesture is more conventional, participants should be more resistant to seeing its production form violated. Participants’ judgments and the time required to make a decision (reaction times) are discussed. Participants were also asked to respond to two probe questions. The first probe explored whether participants who recognized a gesture as being unnatural could explain their reasoning. Because a standard of form is part of cultural knowledge, members of the culture should have access to this information. That is, they should say that the gesture is unnatural because the form is wrong. The second probe explored whether participants would assign meaning to the atypical features of the gesture. 2. Representational gestures and emblems Gestures have been categorized according to numerous schemes; Kendon (2004) provides an excellent comparative summary of representative examples. At least some taxonomies, however, distinguish between gestures that represent something in the accompanying speech and gestures that have independent word-like meaning 2. For ease of reading, the two gesture forms compared have been put into small capitals.



Form, meaning, and convention 

(Ekman & Friesen, 1969; Kendon, 1988, 2004; McNeill, 1992, 2005). While various labels have been assigned to these categories, I will refer to the first as representational gestures, and the second as emblems. These two kinds of gesture differ in several respects. In an effort to systematize these differences, David McNeill has set up a series of continua along which various types of gesture can be arranged (McNeill, 2000, 2005).3 The gestures considered are gesticulation (including representational gestures), pantomime, emblems and sign. Their relative locations on the continuum differ depending on which dimension is under consideration. McNeill explores the following four dimensions: relationship to speech, linguistic properties, convention, and character of semiosis. While I present McNeill’s four continua below (Figures 1–4), I am primarily concerned with representational gestures and emblems, and thus will not discuss pantomime or sign in any detail. In the next section, I define representational gestures and emblems with reference to these four dimensions. This presentation in necessarily quite brief, but readers are referred to McNeill 2000 and 2005 for fuller explication. 2.1

Representational gestures

Representational gestures are, as noted above, gestures which represent something in the accompanying speech. The category is a subset of McNeill’s gesticulation (McNeill, 1992), and consists of iconic and metaphoric gestures. With iconic gestures, there is a correspondence between aspects of the gesture form (the hand’s shape, motion, etc.) and a referent – the gesture iconically depicts something. Metaphoric gestures, on the other hand, iconically represent abstract content. In addition to iconic and metaphoric gestures, McNeill’s category of gesticulation includes deictic gestures, which point at something (abstract or concrete), and beats, or rhythmic baton-like gestures. It is important to bear in mind that, while two of these categories are labeled as semiotic properties (iconic and deictic), the properties themselves may be applied to gestures within all four categories. A gesture that describes a character’s motion by tracing a path is iconic, but is also deictic, in that the gesture is made with respect to an origo. Even beats are often made in a particular location which the speaker has given further structure (e.g. by setting up an entity there and repeatedly referring to it in that spatial location). Metaphoric gestures are de facto iconic gestures, given that gestural metaphor entails iconicity.

3. These continua are an elaboration of McNeill’s previous analysis of gestures as occupying locations on a single continuum, which he named ‘Kendon’s Continuum’ (McNeill, 1992). Kendon’s Continuum is based on McNeill’s summary of Adam Kendon’s work on the various relations between speech and gesture (1988).

 Fey Parrill

The multi-dimensional nature of gesture semiosis will be discussed again with reference to emblems. The first of McNeill’s four continua has to do with the relationship between gesture and speech. Representational gestures generally occur with speech. Indeed, one of their defining features is that they represent something being talked about. Without the accompanying speech, such a gesture’s meaning is usually highly underdetermined.4 McNeill’s second continuum has to do with the extent to which a gesture possesses the properties of a linguistic element. The particular properties McNeill lists are: the existence of morphology (conventionalized formmeaning pairings), phonological form constraints, and the potential for syntactic combination with other gestures (McNeill, 2000, 2005). According to McNeill, representational gestures do not have these properties, in part because they are not conventionalized, as discussed further below. The third continuum shows the relationship of the gesture to conventions: representational gestures are not conventionalized. A convention comes into being when a community of users reaches an agreement about something, whether it is the proper side of the road to drive on or that a certain acoustic signal should be associated with a meaning, as in the case of the words of a spoken language. With a representational gesture, on the other hand, it is not a collective agreement that motivates the use of a certain form, but visuo-spatial thinking. The fourth continuum, what McNeill calls the character of the semiosis, also has to do with how the gesture comes to have the meaning it does. Two semiotic dimensions are contrasted: global-segmented and syntheticanalytic. Representational gestures are global and synthetic. They are global because each part of the gesture does not have an independent meaning which is combined compositionally to generate the whole, as in the assembly of a sentence from its individual words (McNeill, 2000). While the parts which make up a gesture (the shape of the hand and its motion, for example) are potentially separable by an analyst, their meaning derives from the context in which they occur. Representational gestures are synthetic because a single gesture can represent a character’s motion, position and shape holistically. Linguistic items, conversely, represent information more selectively (e.g. motion alone might be expressed with a verb). In summary, representational gestures occur with speech, are non-linguistic, not conventionalized, and are global and synthetic. Emblems differ according to all four dimensions.

4. This is not to say that such gestures must represent a particular word or phrase in the speech, à la Morrel-Samuels & Krauss’ lexical affiliate (1992).



Form, meaning, and convention 

2.2

Emblems

Emblems are gestures which can be glossed, usually in one or two words (Ekman & Friesen, 1969). Some examples are insults, but others include the thumbs-up gesture to indicate a positive evaluation, and the okay gesture described below. While the form-meaning relation of an emblem is often less transparent than it is with a representational gesture, emblems, like representational gestures, may be iconic and metaphoric. For example, many offensive emblems iconically depict some aspect of a taboo act. However, the fact that there may be cross-cultural differences in how these gestures are to be interpreted is one indication of the existence of a convention. Emblems may be used without speech, though they occur with speech as well. In addition, they seem to have some linguistic properties, and are, according to McNeill, partly conventionalized. They can be used without speech because there is a convention about their form and meaning, but they are not like the signs of signed languages in that they do not function as part of a system of paradigmatic oppositions. For example, the okay gesture has a fixed, glossable meaning (positive evaluation), and a form which is consciously accessible to its users (thumb and index in contact, other fingers spread), in which respects it is word-like. However, McNeill notes that these properties are not sufficient to make the okay gesture fully linguistic. There are differences between well-formed and not-well-formed ways of making the gesture. Placing the middle finger on the thumb results in a gesture with some kind of precision meaning, but is not recognizable as the OK sign. The OK gesture, like a word, is constrained to assume a certain phonological shape. Yet, these constraints are limited and they don’t by any means amount to a full language. There is no way to reliably reverse the OK sign, for example. Forming it and waving it back and forth laterally (another emblem that, on its own, conveys negation) might convey “not OK” but it also might be seen as meaning the opposite of negation – waving the hand could call attention to the OK sign, or to suggest that many different things are OK – a flexibility that is basically not linguistic in character. (McNeill, 2005, pp. 8–9)

Emblems, unlike representational gestures, are analytic – they do not represent information holistically, but express a relatively narrow meaning irrespective of context. To sum up, emblems and representational gestures contrast with respect to each of the dimensions identified by McNeill: relation to speech, linguistic properties, convention, and character of semiosis. McNeill’s four continua are shown in their full form as Figures 1–4. (Figures are based on McNeill, 2002, pp. 2–5.)

 Fey Parrill

Figure 1.  McNeill’s continuum 1: Relationship to speech

Figure 2.  McNeill’s continuum 2: Relationship to linguistic properties

Figure 3.  McNeill’s continuum 3: Relationship to conventions

Figure 4.  McNeill’s continuum 4: Character of the semiosis

2.3

Summary

I have argued that representational gestures and emblems are two different kinds of cognitive object, one far more language-like than the other. This claim is not terribly controversial.  As noted above, many taxonomies distinguish between



Form, meaning, and convention 

these two types of gesture (Ekman & Friesen, 1969; Kendon, 1988, 2004; McNeill, 1992, 2005). The claim that the presenting gesture belongs to the category of representational gestures, however, has not yet been justified. I will now discuss the motivation for describing this gesture form as representational, and specifically as metaphoric. 3. The presenting gesture In this section, I provide an illustration of the presenting gesture and describe why it has sometimes been classified as a metaphoric gesture. When producing the presenting gesture, the gesturer’s palm faces up and the fingers are loosely spread. The hand moves away from the body and slightly down. The gesture may be performed with one hand or two. This gesture has also been discussed extensively by Müller as the Palm Up Open Hand (Müller, 2004a) and by Kendon as the Open Hand Supine (Kendon, 2004). It should be noted, however, that these authors do not describe it as a metaphoric gesture, a point to which I will return below. An example of the presenting gesture is provided in Figure 5. As the speaker produces the gesture he says, Here’s what I think we should do. The gesture and speechgesture timing are shown in the figure.

[ here’s gesture produced

what I think we should do ] gesture held

Figure 5.  Example of a presenting gesture

The speaker’s hand has the form used when holding and displaying a physical object. He could have easily produced the gesture with speech describing something

 Fey Parrill

concrete. It is the content of his speech which tells us that there is no physical object involved. This kind of gesture has been described by David McNeill and others as metaphoric, because it iconically depicts something abstract (McNeill, 1992; Sweetser, 1998). In this case, the accompanying speech is metaphoric as well.5 In the next section I will elaborate on this point, but I must first provide a brief summary of the approach to metaphor which underlies McNeill’s analysis, Lakoff and Johnson’s conceptual metaphor theory. 3.1

Conceptual metaphors in gesture

The basic premise behind conceptual metaphor theory is that concrete domains of experience can serve as a framework for reasoning about abstract topics (Lakoff & Johnson, 1980, 1999). For example, according to Lakoff and Johnson, we reason about time in terms of movement in space, giving rise to expressions like, the meeting is coming up fast, that vacation really flew by, etc. A conceptual metaphor is more than a verbal expression, however – it is a pattern of reasoning. This particular pattern of reasoning is described by Lakoff and Johnson as the conceptual metaphor time is space, where the abstract target domain (time) is conceptualized in terms of the concrete source domain (space).6 Similarly, we reason about ideas in terms of physical objects which can be held, exchanged, and so on. This pattern of metaphorical thinking is described as the conceptual metaphor ideas are objects.7 The metaphor gives rise to expressions like she gave me an idea, we tossed some ideas around, etc. In the example above (here’s what I think we should do), the same metaphorical mappings from the concrete source domain of physical objects to the abstract domain of thought are evident, though less overt. The speaker uses spatial deixis (here) to refer to an idea (what he thinks should be done).8 The use of a spatial term to refer to a discourse segment is metaphoric: there is no entity occupying space which can be indexed by the deictic term. Metaphoric gestures are gestures which depict some aspect of the source (or concrete) domain of a conceptual metaphor (Cienki, 1998; McNeill, 1992, 2005; Müller, 2004b; Parrill & Sweetser, 2004; Sweetser, 1998). In the example in 5.

Metaphoric speech need not always accompany a metaphoric gesture (Cienki, 1998).

6. The names of conceptual metaphors are conventionally put in small capitals. Note that I also use this notation for the two gestures compared here – these two different uses should be borne in mind to avoid confusion. 7. A more extended discussion of the conceptualization of communication as exchange of objects can be found in Reddy (1979). 8. This is also a case of discourse deixis, where a referring expression indexes a segment of discourse. Many linguistic examples of this metaphor are discourse-deictic.



Form, meaning, and convention 

Figure 5, I argue that the conceptual representation underlying both speech and gesture is organized by the ideas are objects metaphor. The gesture has the form it does because the speaker is thinking of his idea in terms of a physical object which can be presented to his interlocutor. Just as an iconic path gesture can be motivated by a speaker’s visuo-spatial representation of a moving entity’s path, a gesture depicting the manipulation of an object can be motivated by imagery involving objects. 3.2

The pragmatic function of gesture

While some researchers consider the presenting gesture to be a classic metaphoric gesture (McNeill, 1992; Sweetser, 1998), others have discussed it without describing it as metaphoric (Kendon, 2004; Müller, 2004a). These diverging perspectives seem to arise from a difference in which aspects of gesture are of greatest interest to the researchers and in how their interests shape the classification schemes they use. The present study is not designed to determine which of these perspectives is correct. To begin with, Kendon does not make use of the category of metaphoric gestures at all, thus my research question – is the presenting gesture as conventional as the okay gesture – does not have immediate consequences for his taxonomy. While Müller has written extensively about metaphor and metaphoric gesture (2004b), she does not classify the presenting gesture as either a metaphoric gesture or an emblem, thus my attempt to discover which it most closely resembles is orthogonal to her discussion. Because both Kendon and Müller have worked extensively on this gesture form, however, I will briefly discuss their views, and how my own perspective differs. Müller and Kendon maintain a distinction between the referential and pragmatic functions of gesture (Kendon, 2004; Müller, 2004b, p. 121). The referential function allows a gesture to refer to an object as an object, either by representing it or pointing at it (Kendon, 2004). The pragmatic function, in contrast, describes the use of gesture to contribute non-referential or non-propositional meaning to the accompanying utterance (Kendon, 2004, p. 158). If a speaker shrugs her shoulders and turns both palms upwards, for example, her gesture may indicate that an utterance is intended as a question rather than an assertion. The gesture thus provides information about the speaker’s epistemic stance or about the illocutionary force of her utterance. We have seen that the presenting gesture tends to occur with non-referential (meta-narrative) speech. Thus, the presenting gesture qualifies as a pragmatic gesture, and has been described as such by (Kendon, 2004) and Müller (2004a). The division between referential and pragmatic functions of gesture, however, is not exploited here. In part this is because it is a functional and not a psychological division, thus its consequences for production or cognition are not

 Fey Parrill

spelled out. In addition, I prefer not to separate the presenting gesture from other metaphoric gestures which would not qualify as pragmatic. This is because the difference between the two can be seen as a difference of target domain rather than function. In other words, the difference between a metaphoric gesture which depicts time as a sweep of space traced by the hand (thus exemplifying the time is space metaphor) and a gesture which depicts a discourse theme as an object (thus exemplifying the ideas are objects metaphor), is that in the latter case the target domain is discourse. This is not to say that the gesture does not have a pragmatic function – its function is simply not viewed as a reason for excluding the gesture from the category of metaphoric gestures. 3.3

Summary

The presenting gesture occurs with high frequency in very similar discourse contexts. But does the sameness with which different speakers produce the gesture come from shared underlying imagery or from the existence of a cultural convention? To put it another way, where should the presenting gesture be located on the four continua discussed above? The analysis presented above assumes the gesture is so pervasive because the conceptual metaphor in which ideas are conceptualized in terms of objects is so pervasive. On the other hand, the gesture’s production form may be governed by a culturally established standard. The presenting gesture may accompany a common discourse move (meta-narrative comment) via convention, in the way that the okay gesture can accompany a positive evaluation via convention. In the next section I present an experiment designed to explore the extent to which the presenting gesture is conventionalized. 4. Experimental assessment of the presenting and okay gestures If we wish to determine where a frequently occurring metaphoric gesture falls on a continuum of conventionality, we can compare it to a category of gesture which is expected to represent one extreme, the emblem. In the study described here, I examine the extent to which the presenting gesture is like the okay gesture insofar as it has a conventional form.9 I do this by asking whether observers are equally sensitive to violations of the canonical production forms of these two gestures. 9. The conventional form of okay gesture is widely assumed, thought it has not been experimentally demonstrated so far as I know. This project does not explicitly test the okay gesture’s conventional form, but rather takes that assumption as a starting point. My results, however, may shed some light on the question of whether or not the assumption is a good one.



Form, meaning, and convention 

In addition to assessing observer’s sensitivity to violations of good form, I also explored participants’ intuitions about the gestures. Participants were asked to respond to two probe questions. The first probe asked participants who recognized a gesture as being unnatural to explain why they felt that way. Because a standard of form is part of cultural knowledge, members of the culture should have access to information about the form. The second probe asked participants to extend the meaning associated with the gesture in a way that related to the change in form. Such an extension would provide more information about participants’ consciousness of the form associated with a gesture. In summary, the project addressed the following questions. 1. Are participants as sensitive to non-canonical variants of the presenting gesture as they are to non-canonical variants of the okay gesture? 2. Do they know why they object to non-canonical variants? 3. Can they reason about the non-canonical forms of these gestures? 4.1

Methods

The questions posed above were addressed with a two-part experiment. In the first part of the experiment, participants were presented with thirty-two stimuli (described in detail below) consisting of a video clip of a gesture produced with a sentence. Participants were asked to decide as quickly as possible where the stimulus fell on a one-to-five scale from highly natural to highly unnatural. Reaction time data were collected as well as judgments of naturalness. Each participant saw sixteen variants of the presenting gesture and sixteen variants of the okay gesture in random order. Thirty University of Chicago students participated in the experiment for payment. After rating the stimuli, participants were asked to look at eight of the stimuli just rated, and to explain their judgments. The stimuli chosen for these explanations were those which the participant had rated as least natural. Participants were asked to respond to the following two probe questions. As noted above, the function of these probes was to explore speakers’ intuitions about gesture form. 1. You gave this a rating of “highly unnatural”. Can you tell me why? 2. Is there a situation where you might give this a more natural rating? 4.2

Stimuli

This study used constructed gesture stimuli. Sentences and gestures were constructed separately, then edited together, as explained below.

 Fey Parrill

4.2.1 Presenting sentences To create the sentences to be used for the presenting stimuli, thirty-two sentences which were accompanied by a presenting gesture were collected from videotaped lecture data. Data from three different speakers were used. Gesturespeech combinations were selected from different portions of each speaker’s lecture. Selection was constrained by the fact that the number of syllables needed to be constant for all stimulus sentences, as a means of roughly equating sentence length. These sentences were then recorded by the experimenter. Following recording, the sentences were normed independently of gesture in order to reduce the chance that the sentence itself would influence judgments. A separate group of forty participants was presented with the sentences and asked to decide how natural each was, on a scale from one (highly natural) to five (unnatural). Of the sentences which were rated as highly natural, the first sixteen were used in the experiment. These sentences appear in Appendix A. 4.2.2 Presenting gestures Gesture video clips were then constructed. Each gesture variant was performed with a single sentence normed and rated as highly natural. To create a matrix of gesture variants ranging from natural to unnatural, two dimensions of variation were selected. The first was the degree to which the thumb overlapped the palm. The second was the degree to which the fingers were curled. Both dimensions ranged from none to extreme. Crossing these two dimensions yielded a total of sixteen gestures, which ranged from fairly normal to quite bizarre, shown in Table 1. Because crossing each gesture variant with each of the sixteen sentences would have created 256 stimuli, each gesture was instead combined with a random subset of the normed sentences. Five sentences were randomly selected for each gesture variant. The sentence and gesture variant were edited together to create a total of eighty gesture-speech presenting video clips. To avoid a mismatch between the speech a participant heard and the movement of the stimulus speaker’s lips, the speaker’s head was not shown in the video (see Figure 6). After completing the experiment, participants were asked if they had been aware that gesture and speech were combined after production. None reported having had any suspicion.



Form, meaning, and convention 

Table 1.  Matrix of variants for the presenting gesture No Overlap

Slight Overlap

Medium Overlap Extreme Overlap

No Curl

Slight Curl

Medium Curl

Extreme Curl

Figure 6.  Example stimulus

4.2.3 Okay sentences The okay gesture occurs far less frequently in natural discourse. For this reason, sentences were constructed by asking naïve speakers to imagine the sorts of sentences which they might produce with the gesture. Thirty two examples were collected. Because the length of the sentence needed to be constant across the whole experiment, these sentences were then edited slightly. The sentences were again normed by a separate set of forty participants, following the procedure outlined above. A set of sixteen sentences rated as highly natural was compiled. 4.2.4 Okay gestures Two dimensions of variation were also selected in order to create a range of natural and unnatural okay stimuli. The first dimension was the degree to which the fingers were curled. The second dimension was the angle formed by the index finger

 Fey Parrill

and thumb contact. This angle will be referred to as pinch. Combining these two dimensions again yielded a total of sixteen gestures, shown in Table 2. Table 2.  Matrix of variants for the okay gesture No Pinch

Slight Pinch

Medium Pinch

Extreme Pinch

No Curl

Slight Curl

Medium Curl

Extreme Curl

These gestures were again combined with a random subset of the normed sentences, to produce a total of eighty gesture-speech okay video clips. Taking both conditions together, there was a total of thirty-two individual gesture types (sixteen presenting, sixteen okay) and 160 possible video clips (thirty-two gesture types crossed with five possible sentences). Each participant saw sixteen randomly selected presenting stimuli and sixteen randomly selected okay stimuli. Participants sat in front of a computer monitor and were presented with the stimulus. They were then prompted for a judgment and pressed a key on the computer keyboard to enter it. After completing this portion of the experiment, participants were asked the two probe questions described above. This portion of the experiment was videotaped to capture any gestures produced as part of an explanation. 4.3

Predictions

If the presenting gesture is less conventional than the okay gesture, participants should be more tolerant of violations of its form, accepting more variants of it. The okay gesture, on the other hand, should be rejected more frequently because its form is culturally specified. Participants may also take longer to make decisions about the presenting gesture. Since the okay gesture has a well-known form, participants should quickly reject instances which do not fit that canonical form.



Form, meaning, and convention 

With respect to the probes, participants should cite the non-canonical form of the gesture as their reason for judging it to be unnatural for both kinds of gesture. Other reasons may be offered for the presenting gesture, however. Participants may also be more likely to respond to the second probe when asked about the presenting gesture. The conceptual metaphor which is argued to underlie the imagery of the presenting gesture can have variable mappings, whereas the okay gesture is assumed to have a set form. 4.4

Results

4.4.1 Results of judgment task In order to determine whether there was a significant effect of gesture type (presenting or okay), analysis of variance (ANOVA) was carried out separately for both reaction time and for judgment, with participant, gesture type and sentence as factors. A significant effect of participant occurred for both tests (F(2, 28) = 3.64, p speech pausesd gestural metaphor (8) repeated & held =>metaphoric gesture fills in acc.obj. & finishes clause

Figure 2.  “setting the tracks”: Continuous activation of metaphoricity in speech and gesture over six verbo-gestural and two gesture-only metaphors

What we see in this longer piece of utterance are six closely linked verbo-gestural metaphoric expressions and one gestural metaphor which is produced and repeated in a speech pause. Note that this gesture-only metaphor is used to finish the clause grammatically, i.e., it fills in the information of the omitted accusative object. All gestural metaphors are variations of one hand-shape (a pointing or directional gesture, palm[s] open and laterally oriented) and all verbal metaphors are specifications

 Cornelia Müller

of one initially introduced verbo-gestural metaphor “Weiche” (“switch”) with a twohanded directional gesture reformulated later as “Weichenstellung” (“switching point”) again with a two-handed directional gesture. Speech as well as gesture expresses the idea of a career as a path that is determined through job choices one makes when entering the job market. The sequence begins with what we term the ‘local establishment’ of a verbogestural metaphor (1).17 What happens is that the speaker uses a verbal metaphoric expression “setting the tracks” and with it he produces a gestural representation of aspects of what it means to literally set tracks: he sets up a direction for a path to follow by pointing with his both hands (loosely) to the left. This verbo-gestural metaphor is the beginning of his counterargument and with it he more or less generally states that the first job choice after leaving university does indeed “set tracks” (i.e. determines career paths) for the future. What follows are different specifications that go beyond this general statement. In (2) he repeats what he has said in (1) but at the same time he adds a gestural specification, namely a clear forward orientation (both palms laterally oriented, touching each other, directed forward, clearly articulated hand-shape). In (3) he repeats the gesture (hand-shape, orientation, and position maintained, beat added) and specifies verbally how already one particular choice of academic field presupposes a future career path (“When you say: I will study medicine”). The subsequent three verbo-gestural metaphors provide a three-item list of examples with a different verbo-gestural metaphor each time (Selting, 2004): (4) “or German Studies & bh pointing left,” (5) “or Agriculture & bh pointing right,” and (6) “or if you become a tennis coach & bh pointing upwards and clap.” In (7) he reformulates and restates his (metaphorical) point: choices of academic topic do set tracks “this DOES set the tracks & both hands maintain position and hand-shape of (6) but do not repeat the clapping.” In (8) he comes to his conclusion “but after finishing university you really have to think about which (speech pauses) & rh pointing forward, palm lateral.” This gesture is performed (in contrast to all the others in the sequence, which were both handed) with his right hand only in an extremely well articulated forward pointing gesture (again with a palm lateral hand-shape but the hands are now clearly straight and tense in contrast to the others that were performed with a very loose hand-shape); in addition, the left hand is used to support his right forward pointing arm; moreover this pointing gesture is being moved into the focus of the interactive attentional space of the coparticipants and repeated with an up and down movement (a beat) precisely when 17. The term ‘local’ refers to the fact that meaning is constructed in the here and now of coparticipants or in a specific moment in time as conversation and interaction move along. It is a technical term that is used in conversation analysis and approaches to distributed cognition (Goodwin, 2000) and it highlights the dependency of meaning constitution on the precise sequential position of a gesture or a verbo-gestural metaphor.



What gestures reveal about the nature of metaphor 

the (metaphoric) accusative object would have to be uttered verbally. But instead of using a metaphoric noun, in (8 repeated) the gesture is repeated, fills in the speech pause, and finishes the clause. This is a truly rich and fascinating instance of metaphor and gesture interplay and enchainment, but what does it reveal about the nature of metaphor? In the first section we have argued that metaphors appear in all kinds of different modalities and that this indicates that they might be more appropriately characterized as products of a general cognitive process. Characterizing metaphors as products, however, conceptualizes them as single well-delineated items or objects, which is precisely how metaphor has mostly been discussed and conceived of hitherto. But as the analysis of a sequence of successive metaphorical expressions (verbal, gestural, and verbo-gestural) in this section has documented, metaphors can be products of the metaphoric conceptualization of an entire argument, they can be outcomes of a process which may in fact structure extended stretches of discourse. The reported observations once more underline that metaphors appear to be grounded in ad hoc creations and/or activations of metaphoric expressions in speech, gesture, or speech and gesture conjointly, and the verbal, gestural, and verbo-gestural metaphors we have encountered are outcomes of an underlying metaphorical conceptualization or idea. In McNeill and Duncan’s terms these might be considered a metaphorical “Growth Point,” in which there is a constant maneuvering back and forth between verbal and gestural modes of thinking (cf. McNeill, 1992; McNeill & Duncan, 2000).18 The single metaphorical products (the succession of verbo-gestural metaphors and a gestural metaphor) are outcomes of an ongoing cognitive process in which a metaphorical idea or a Growth Point is active over the time span of an argumentative move. This property of metaphors questions the established concept of metaphor as being one clearly delineated thing at a time, be it conceptual, verbal, or gestural. 5. Metaphors are dynamic The dynamic character of metaphors is another core facet of the nature of metaphor that a close study of gestures in talk and discourse reveals. Again, this perspective directs us to the sequential linear structuring and the interrelatedness of metaphors in speech and gesture as they unfold over time. As the example (2) has shown, this interrelatedness only becomes visible when both modalities are 18. In McNeill’s cognitive theory of speech and gesture a “growth point” is considered a minimal idea unit in which visuo-spatial and propositional forms of thought interact, containing, in a nutshell, all the elements of a verbo-gestural utterance.

 Cornelia Müller

carefully considered as they evolve in the course of discourse. In this section an example will be presented which indicates that not only may the activation of metaphoricity be sustained over quite extended utterances, but that, in fact, metaphoricity may be subject to processes of foregrounding by a speaker. In this third and last example the speaker summarizes her first love relationship which lasted for several years but was not easy and contained many highs and lows that eventually lead to its break-off. The speaker characterizes this course of her relation as a kind of up and down, a path that went up and down many times but had an overall tendency downwards. (Ex. 3)  The course of a love relation as “a kind of up and down” (1)

(2)

JA es war eben ein relatives auf und [ab |

mit der= |mit der ständigen tendenz bergAB\] (.hh)

Yes it was | basically a kind of [up and down [gaze to hand – – – – -->

with the=| with the permanent tendency downhill(.hh) gaze to hand]

Verbo-gestural metaphor established:

Verbo-gestural metaphor reformulated & specified: – love relation has “permanent tendency downhill” – head moves down, larger and slower movement – right hand with glass moves downward too & gaze to hand => increasing activation of metaphoricity

– love relation as “a kind of up and down” – head moves down, slightly & quickly

& gaze to hand => onset of activated metaphoricity



What gestures reveal about the nature of metaphor 

(4 start)

(4 end)

[aber es ging= | ne es startete SO (---)

und flachte dann so:: (.) weiter ab]

but it went= well it began like THIS (---) [– – gaze to hand – – -

and then flattened like this (.) on out

Breaks off verbal metaphor -

Verbal metaphor reformulated & specified “flattened...out” Gesture (4) continued

Gesture reformulates metaphor “up and down” – changes gesturing hand – left hand outlines huge amplitude & gaze follows gesture => gesture performed in focal attentional space => high activation of metaphoricity

- – – gaze follows hand ---------]

& gaze continues to follow gesture => gesture still in focal attentional space => high activation of metaphoricity

Figure 3.  The course of a love relation as “a kind of up and down”: successive and increasing foregrounding of metaphoricity

In this example the speaker uses gesture, speech, gaze, and spatial position as interactive resources in the process of successive foregrounding of metaphoricity. Foregrounding is literally made visible in the gestures and the ways in which this is achieved indicate an increasing activation of metaphoricity in the speaker. In segment (1) the speaker begins her description of the course of this love relationship with a verbal metaphoric expression (“Yes it was basically a kind of up and down”), but when uttering the final part of it (“down”) she adds a small downward movement of the head or more technically – a head gesture. Since the head

 Cornelia Müller

gesture shares the source domain of the verbal metaphoric (“down”) we may argue that metaphoricity is activated for the speaker at the beginning of her narration. Then (in segment 2) she specifies the metaphoric image ‘up and down’ and says that not only were there regular ups and downs, but in addition to that, the relationship had a “permanent tendency downhill.” While verbally specifying the metaphor, she moves her head once again downwards, but this time she adds a manual gesture which also moves downwards (her right hand moves down with the glass) but this time the movement is larger and takes longer, and – she directs her gaze to it. Directing the gaze at something has strong interactive implications: it indicates speaker’s focal attention which might be characterized as principles of “What I am looking at is what I am talking about” and “Therefore this is what you should also look at.” Put differently, gaze direction functions here as a pointing gesture, i.e. gaze points to the gestural metaphor expressed with a manual gesture and turns the gesture into an interactively significant “object” (cf. Streeck, 1993). And in fact (at least as far as it is detectable from the video) the listener also turns her gaze to the gesture. The source domain of the verbal metaphoric expression inspires a hand and a head gesture both depicting ‘downwardness’, so that ‘downwardness’ is simultaneously expressed three-times (once in speech and twice in gesture), creating a kind of multi-modal cluster of metaphoric expressions. Moreover the size of the manual metaphor is larger and takes longer, which ensures greater visibility for the coparticipant. Since the speaker puts more expressive effort into the metaphoric expression than in segment (1) she makes sure that metaphoricity is in shared focal attention and therefore is not to be dismissed by the listener. In this way, she responds to the lack of ratification of her first formulation of the metaphor by the listener. Therefore, we may conclude that the metaphoricity of the verbal metaphoric expression ‘up and down’ is more active in segment (2) than it was just before in segment (1). In segment (3) the speaker offers a second reformulation of her description (the first one not having been verbally ratified by the listener, this creates an interactive ‘must’ for providing further details): while saying: “well it began like this (---)” she produces a very elaborated amplitude gesture. It is not random that she switches hands to perform this gesture. She now uses her left ‘glass free’ hand, starts high up, over her head, and draws a graph with decreasing amplitudes and says: “well it began like this.” The deictic particle “so” (“like this”) refers to the gesture and integrates the gesture syntactically into the verbal utterance (cf. Streeck, 1993). It indicates the specific syntactic location in the verbal utterance when the listener is supposed to be looking at the gesture, to pick up the information provided gesturally, and to relate it to the verbal part of the utterance. For the co-



What gestures reveal about the nature of metaphor 

participant this is not to be missed, because the gesture carries semantic information that is not provided verbally and – it completes this part of the utterance. It is with the gesture that the speaker specifies the course of the relationship as decreasing in amplitude: high at the onset and steadily becoming lower and smaller towards the end. As in segment (2), the speaker’s gaze follows the gesture; but now this is performed in very marked manner (all through segments 3 and 4), the speaker’s gaze meeting the gesture in the center of the attentional space, where it is unavoidable for the co-participant to notice it, and takes it with her all along the gestured path down to the edge of the gesture space. Towards the end of the amplitude gesture – in segment (4) – speech and gesture become semantically co-expressive again, but the speaker continues to gaze at it until almost its very end. While saying “and then flattened like this (.) on out” she outlines the ending part of the decreasing amplitude. Metaphoricity continues to be highly activated in segment (4), since it is expressed in two modalities at a time (speech and gesture) and because the gestural metaphor continues to receive the speaker’s gaze and is once again syntactically integrated into the verbal part of the utterance via the deictic particle “so” (“like this”). In segments (3) and (4) metaphors are at the core of focal attention. Metaphoricity is hence even more activated than in segment (2) and there is no way that the listener could dismiss it. This activation becomes visible through a powerful combination of five different foregrounding devices which turn activation of metaphoricity into an interactively relevant phenomenon: gaze direction (3) & (4), replacement of verbal metaphor (3), syntactic integration of gestural metaphor (3) & (4), visibility (size and duration) (3) & (4), and in (4) iconic reinforcement of metaphor (multi-modal clustering). Moreover, the speaker’s gaze is directed towards the gesture and hence documents that indeed the interactive focus of attention is on the metaphoric gestural expression. Let us reconsider how metaphoricity is dynamically activated over this entire piece of discourse: in the first segment of this example we may infer that metaphoricity is activated because the verbal metaphor is accompanied by a gesture drawing upon the same source domain. In the second segment metaphoricity is expressed in three modalities at a time and interactively foregrounded through gaze (but the gestures are brief and small). Therefore metaphoricity appears to be more activated than in segment one. In the third segment in turn metaphoricity is highly activated because now it is integrated with a deictic particle into the sentence and replaces the verbal metaphoric, in addition, it is larger and longer and receives the speaker’s and recipient’s gaze (i.e. it is in the focus of attention of both participants). Finally, in segment four, metaphoricity continues to be highly activated because the metaphor is now again expressed in speech and gesture (iconic

 Cornelia Müller

reinforcement), is also syntactically integrated, and continues to receive the gaze and to be highly visible. The microanalysis of this piece of discourse documents a dynamic activation of metaphoricity that is organized in different modalities simultaneously and successively. We have encountered metaphors in hand as well as in head gestures, we found them in speech as elaborations and specifications, and we also saw how they may combine with speech on the one hand and function independently of each other on the other hand. The sequential and procedural microanalysis offers further support for the proposal that metaphoricity is a cognitive activity which is modality independent and that this activity may structure entire pieces of discourse. However, this third example also reveals a facet of the dynamics of metaphoricity, not yet introduced: namely its potential gradability. Over a stretch of discourse, metaphoricity may not only be present continuously but it may gradually increase. Increasing metaphoricity is an achievement, probably often an interactive achievement, namely when the speaker reformulates verbo-gestural metaphors because of the absence of ratifications from a listener. Yet psychologically it is always a consequence of the dynamic orientation of focal attention of the speaker. If focal attention is on the metaphoric expressions, then metaphoricity is activated. Focal attention in turn, is empirically graspable in foregrounding techniques. What is perceivably foregrounded is where the focal attention of the speaker is, and in most cases this focal attention will be a shared one. 6. Summary The analyses revealed some fundamental facets of metaphor which hitherto were only scarcely recognized. They could only be discovered because a successive and sequential microanalysis was performed and describes precisely which expressive forms of metaphors were produced in linear succession at what precise moments in the flow of utterance and in the process of interaction. Other aspects which were carefully described were: how the different metaphors (gestural, verbal) relate to and combine with each other simultaneously (do they substitute, supplement, or co-articulate), how these metaphors are placed and integrated syntactically and semantically, and if and how they were turned into interactively significant objects (foregrounding of gestural metaphor through gaze and its syntactic and semantic integration in the utterance, foregrounding of verbal metaphor by simultaneously co-articulated gestural metaphors). The applied methodology turned out to be a valuable ‘discovery’ procedure. It revealed that metaphoricity is in principle independent of any modality, at least as far as words and head and hand gestures are concerned. This observation offers



What gestures reveal about the nature of metaphor 

further support for the assumption of metaphors as products of a cognitive activity. Moreover it revealed, that this online activity may persist over longer periods of time, structure longer pieces of discourse, and trigger verbal, gestural and verbo-gestural elaborations.In addition, the cognitive process of activating metaphoricity appears to be shaped via other general cognitive processes such as attention and foregrounding, i.e. via these processes metaphoricity may be subject to grading, so that in one context of use the activation may be low, in other ones it may be high. Different degrees of activiation are visible in foregrounding resources or techniques that co-participants make use of. While clearly directing the recipient’s attention to a specific focus – they first and foremost display where the attentional focus of the speaker is at a given moment of time in the flow of discourse: The more communicative effort is used to represent a certain concept (speech only, speech & gesture, speech & hand & head gesture) and the more explicit something is being marked as focal object (verbal deictic points to gesture, gaze points to gesture, gesture replaces word) the more focal, the more foregrounded, the more salient it is. 7. Conclusion The study of gestures as they are used during discourse offers insights into traits of metaphor production and hence may shed light on the ways speakers conceive of metaphors as they are using them. Looking at how gestures are coordinated, integrated, and foregrounded over longer pieces of discourse shows that the metaphoricity of words, gestures, or gesture-word combinations critically depends on the organization of the attentional focus of the speaker. In turn, this is a response to the communicative intent of the speaker (“What is it you want to tell?”) and interactive needs of the participants (“Requesting specification,” “Ratifying information”). The varying degrees of metaphoricity that we have encountered, demonstrate that there is a constant flow of attention (as part of what Chafe, 1994, 1996 terms flow of consciousness) which may successively build up peaks or focal points that themselves may trigger subsequent peaks in this dynamic movement of attention. Analyzing step-by-step how metaphoric meaning is organized and distributed over multiple modalities uncovers this very property of the attentional structure within the flow of consciousness. What is in the focus of attention of the speaker is foregrounded in a verbo-gestural utterance and what is foregrounded by the speaker are the salient aspects of the communicative “message” for the listener. Attention regarded in this way becomes empirically verifiable or at least observable through foregrounding techniques and their resulting salience effects. This is because the organization of the speaker’s attention is interactively relevant; it is

 Cornelia Müller

presumably what makes a speaker tell a story the way she or he wants a listener to hear it (and this is what we as analysts may exploit systematically). This organization of information within a given discourse may be best characterized as a twofold activity, one that brings together intrapersonal and interpersonal processes. Attention is the trigger behind the activation of metaphoricity. Once metaphoricity comes into the focus of attention then it may be as much activated as the speaker’s focal attention holds on to it. When metaphors are maintained over discourse and successively foregrounded, their form becomes clearer and more articulated, and one is tempted to infer that this clarity in form goes along with clarity and precision in thoughts that are being expressed. This could be a trace of what Heinrich von Kleist termed “Über die allmähliche Verfertigung der Gedanken beim Reden” (“The successive clarification of thoughts in speaking”). Furthermore these observations indicate that Slobin’s (1987, 1991) “thinking for speaking” is actually indeed an ongoing process of conceptual reorganization as we speak and interact. Gestures therefore reveal something about how meaning is organized that transcends metaphors and their activation. Metaphoric expressions are products of one general process – establishing or creating metaphoricity – but this process only gets started if they move into the attentional focus within the speaker’s flow of consciousness. Put differently, this process needs another process to get started. Moreover, as the examples above have shown, activation is not an on-off process; it holds on over time and it comes in degrees and so does attention. Metaphoricity is subject to a dynamic management of information flow, which means that the activation of metaphoricity is to be characterized as a dynamic process that unfolds over time, and that is part of an online and ad-hoc process of constructing and structuring meaning between co-participants. This is why Müller (in press) introduces one dynamic category of vital metaphor. A vital metaphor carries a potential of metaphoricity that may be activated as it is used (cf. Cameron, 1999b, p. 108). Potential metaphoricity may be activated in actual contexts of use, or, to put it differently, a particular metaphoric linguistic expression may be waking in one context and sleeping in another one. Metaphors in language (but also in gesture) are to be considered as inherently dynamic, and the degree of activated metaphoricity is a consequence of cognitive processes such as foregrounding and focusing of attention. This dynamic view is consistent with other recent approaches to cognition and language use. Most recently David McNeill (2005) has advocated a dynamic perspective on language and gesture and Cameron’s (1999a) proposal for an applied linguistics framework advocates a theoretical stance which takes metaphor use as point of departure for a theory of metaphor and integrates careful distinctions of products and processes of metaphors, using Gibbs’ terms (Gibbs, 1999), and accounts for different kinds of metaphors on the levels of system and use (Steen,



What gestures reveal about the nature of metaphor 

2007; Steen & Gibbs, 1999). The observations presented in this chapter further support these approaches in that they show how empirical observations may inform theoretical conceptualizations (cf. also Cienki, 2005a, 2005b). The nature of metaphor reveals itself in the multiple and creative ways in which people make use of it. Studying gesture and metaphor in this way uncovers that metaphors are the product of a cognitive activity (a process) of activating metaphoricity and that metaphoricity is therefore a dynamic feature of metaphors in language use. References Aldrich, V. C. (1996). Visuelle Metapher. In Anselm Haverkamp (Ed.), Theorie der Metapher (pp. 142–159). Darmstadt: Wissenschaftliche Buchgesellschaft. (First published 1968). Becker, K. (2004). Zur Morphologie redebegleitender Gesten. MA thesis, Freie Universität Berlin. Black, M. (1993). More about metaphor. In A. Ortony (Ed), Metaphor and thought (2nd ed.) (pp. 19–41). Cambridge: Cambridge University Press. Bouvet, D. (2001). La dimension corporelle de la parole: Les marques posturo-mimo-gestuelles de la parole, leurs aspects métonymiques et métaphoriques, et leur rôle au cours d’un récit. Paris: Peeters. Calbris, G. (1985). Espace-Temps: Expression gestuelle du temps. Semiotica, 55 (1/2), 43–73. Calbris, G. (1998). Geste et images. Semiotica, 118 (1/2), 105–120. Calbris, G. (2003a). L’expression gestuelle de la pensée d’un homme politique. Paris: CNRS Éditions, collection CNRS Communication. Calbris, G. (2003b). The semantic structure of gestures (Plenary lecture abstract). Proceedings of First Congress of the International Society for Gesture Studies: Gesture–The Living Medium. University of Texas: Austin. http://www.gesturestudies.com/ Calbris, G. (2003c). From cutting an object to a clear cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture, 3, 11–46. Cameron, L. (1999a). Operationalising ‘metaphor’ for applied linguistic research. In L. Cameron & G. Low (Eds.), Researching and applying metaphor (pp. 3–28). Cambridge. Cambridge University Press. Cameron, L. (1999b). Identifying and describing metaphor in spoken discourse. In L. Cameron & G. Low (Eds.), Researching and applying metaphor (pp. 105–132). Cambridge. Cambridge University Press. Cameron, L. (2003). Metaphor in educational discourse. London: Continuum. Cameron, L. & G. Low (1999). Researching and applying metaphor. Cambridge: Cambridge University Press. Chafe, W. (1994). Discourse, consciousness, and time: The flow and displacement of conscious experience in speaking and writing. Chicago/London: University of Chicago Press. Chafe, W. (1996). How consciousness shapes language. Pragmatics & Cognition, 4, 55–64. Cienki, A. (1997). Motion in the metaphorical spaces of morality and reasoning as expressed in language and gesture. International Journal of Communication, 7, 85–98. Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford, CA: Center for the Study of Language and Information.

 Cornelia Müller Cienki, A. (2000). The production of metaphoric gestures: Evidence for on-line cognitive processing of metaphoric mappings. Unpublished manuscript. Cienki, A. (2005a). Metaphor in the “Strict Father” and “Nurturant Parent” cognitive models: Theoretical issues raised in an empirical study. Cognitive Linguistics, 16, 279–312. Cienki, A. (2005b). Gesture and the question of literal versus non-literal reference. In S. Coulson & B. Lewandowska-Tomaszczyk, (Eds.), The literal and nonliteral in language and thought (pp. 281–298). Frankfurt am Main: Peter Lang. Cienki, A. (This volume). Why study metaphor and gesture? Cienki, A. & C. Müller (In press). Metaphor, gesture and thought. In R. W. Gibbs, Jr. (Ed.), Cambridge handbook of metaphor and thought. Cambridge: Cambridge University Press. Corradi Fiumara, G. (1995). The metaphoric process: Connections between language and life. London/New York: Routledge. Croft, W. & D. A. Cruse (2004). Cognitive linguistics. Cambridge: Cambridge University Press. Deppermann, A. (1999). Gespräche analysieren. Eine Einführung in konversations-analytische Methoden. Opladen: Leske und Budrich. Evans, V. & M. Green (2006). Cognitive linguistics: An introduction. Edinburgh: Edinburgh University Press. Forceville, C. (1996). Pictorial metaphor in advertising. London/New York: Routledge. Gibbs, R. W., Jr. (1994). The poetics of mind: Figurative thought, language, and understanding. Cambridge: Cambridge University Press. Gibbs, R. W., Jr. (1998). The fight over metaphor in thought and language. In A. N. Katz, C. Cacciari, R. W. Gibbs, Jr., & M. Turner (Eds.), Figurative language and thought (pp. 119–157). New York/Oxford: Oxford University Press. Gibbs, R. W., Jr. (1999). Researching metaphor. In L. Cameron & G. Low (Eds.), Researching and applying metaphor (pp. 29–47). Cambridge: Cambridge University Press. Gibbs, R. W. (2006a). Embodiment and cognitive science. Cambridge: Cambridge University Press. Gibbs, R. W. (2006b). Embodied simulation in metaphor interpretation. Mind & Language, 21, 434–458. Gibbs, R. W., Jr. & G. J. Steen (Eds.) (1999). Metaphor in cognitive linguistics. Amsterdam/Philadelphia: John Benjamins. Gilot, F. & C. Lake (1964). Life with Picasso. New York: McGraw-Hill. Goodwin, C. (2000). Action and embodiment within situated human interaction. Journal of Pragmatics, 32, 1489–1522. Hougaard A. R. & Hougaard, G. R. (This volume). Implications of cognitive metaphor and gesture studies for ethnomethodology and conversation analysis and vice versa. Hutchby, I. & R. Woofitt (1998). Conversation analysis: Principles, practices and applications. Oxford: Blackwell. Kendon, A. (1988). Sign languages of Aboriginal Australia: Cultural, semiotic and communicative perspectives. Cambridge: Cambridge University Press. Kendon, A. (1990). Conduction interaction: Patterns of behavior in focused encounters. Cambridge: Cambridge University Press. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. Kleist, H. von (2002). Über die allmähliche Verfertigung der Gedanken beim Reden. Kleist-Archiv Sembdner, Internet Editionen. www.kleist.org Lakoff, G. (1990). The Invariance Hypothesis: Is abstract reason based on image-schemas? Cognitive Linguistics, 1, 39–74.



What gestures reveal about the nature of metaphor  Lakoff, G. (1993). The contemporary theory of metaphor. In A. Ortony (Ed.), Metaphor and thought (pp. 202–251). Cambridge: Cambridge University Press. Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago/London: Chicago University Press. Lakoff, G. & M. Johnson (1999). Philosophy in the flesh. New York: Basic Books. Lakoff, G. & M. Turner (1989). More than cool reason: A field guide to poetic metaphors. Chicago: Chicago University Press. Langacker, R. W. (1987). Foundations of cognitive grammar. Vol. 1. Stanford: Stanford University Press. Langacker, R. W. (1988). An overview of cognitive grammar. In B. Rudzka-Ostyn (Ed.), Topics in cognitive linguistics (pp. 3–48). Amsterdam/Philadelphia: John Benjamins. Langacker, R. W. (1991). Concept, image and symbol: The cognitive basis of grammar. Berlin: Mouton de Gruyter. Langacker, R. W. (1999a). Grammar and conceptualization. Berlin/New York: Mouton de Gruyter. Langacker, R. W. (1999b). Assessing the cognitive linguistic enterprise. In T. Janssen & G. Redeker (Eds.), Cognitive Linguistics: Foundations, scope, and methodology (pp. 13–59). Berlin/New York: Mouton de Gruyter. Liddicoat, A. J. (2006). Introduction to conversation analysis. London/New York: Continuum. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. McNeill, D. (This volume). Unexpected metaphors. McNeill, D. & S. Duncan (2000). Growth-points in thinking-for-speaking. In D. McNeill (Ed.), Language and gesture (pp. 141–161). Cambridge: Cambridge University Press. Mittelberg, I. (2005). Metonymic modes in co-speech gesture. Paper presented at the 2nd Conference of the International Society for Gesture Studies, Lyon, June 2005. Mittelberg, I. (2006). Metaphor and metonymy in language and gesture: Discourse evidence for multimodal models of grammar. Unpublished PhD dissertation, Cornell University. Mittelberg, I. (2007). Internal and external metonymy in co-verbal gesture: Jakobsonian and cognitivist views on contiguity relations within and across modalities. Paper presented at the 10th ICLC, Kraków, Poland, July 2007. Mittelberg, I. & L. Waugh (In press). Multimodal figures of thought: A cognitive-semiotic approach to metaphor and metonymy in co-speech gesture. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor. Berlin/New York: Mouton de Gruyter. Mittelberg, I. & C. Müller (eds.) (In preparation). Metonymy in gesture and signed languages. Special Issue. CogniTextes. http://aflico.asso.univ-lille3.fr/cognitextes/ Müller, C. (1998). Redebegleitende Gesten. Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag A. Spitz. Müller, C. (2000a). Chicago lectures. Unpublished manuscript. Müller, C. (2000b). Zeit als Raum. Eine kognitiv-semantische Mikroanalyse des sprachlichen und gestischen Ausdrucks von Aktionsarten. In E. W. B. Hess-Lüttich & H. W. Schmitz (Eds.), Botschaften verstehen. Kommunikations-Theorie und Zeichenpraxis. Festschrift für Helmut Richter (pp. 211–228). Frankfurt/Main: Peter Lang. Müller, C. (2003). Gestik als Lebenszeichen ‘toter Metaphern’. Tote, schlafende und wache Metaphern. Zeitschrift für Semiotik, 1–2, 61–72.

 Cornelia Müller Müller, C. (2004a). The Palm-Up-Open-Hand. A case of a gesture family? In C. Müller & R. Posner (Eds.), The semantics and pragmatics of everyday gestures. The Berlin conference (pp. 233–256). Berlin: Weidler Buchverlag. Müller, C. (2004b) Gesture and speech cross-culturally. Dimensions of cultural variation. Unpublished manuscript. Müller, C. (2007) A dynamic view of metaphor, gesture and thought. In S. Duncan, J. Cassell, & E. Levy (Eds.), Gesture and the dynamic dimension of language (pp. 109–116). Amsterdam: John Benjamins. Müller, C. (In press). Metaphors. Dead and alive, sleeping and waking. A dynamic view. Chicago: University of Chicago Press. Müller, C. (In preparation) Iconic devices in gestures and sign-language. Müller, C. & A. Cienki (In press). Metaphors, gestures and beyond: Forms of multimodal metaphor in the use of spoken language. In C. Forceville & E. Urios-Aparisi (Eds.), Multimodal metaphor. Berlin/New York: Mouton de Gruyter. Müller, C. & H. Haferland (1997). Gefesselte Hände. Zur Semiose performativer Gesten. Mitteilungen des Germanistenverbandes, 3, 29–53. Müller, C. & G. Speckmann (2002). Gestos con una valoración negativa en la conversación cubana. DeSignis, 3, 91–103. Morris, D. (1977). Manwatching. London: Jonathan Cape. Ochs, E., E. Schegloff, & S. Thompson (1996). Interaction and grammar. Cambridge: Cambridge University Press. Parrill, F. (This volume). Form, meaning, and convention: A comparison of a metaphoric gesture with an emblem. Ricœur, P. (1986). Die lebendige Metapher. München: Wilhelm Fink. Selting, M. (2004). Listen: Sequenzielle und prosodische Struktur einer kommunikativen Praktik – eine Untersuchung im Rahmen der Interaktionalen Linguistik. Zeitschrift für Sprachwissenschaft, 23, 1–46. Slobin, D. (1987). Thinking for speaking. Proceedings of the thirteenth annual meeting of the Berkeley Linguistics Society (pp. 435–445). Berkeley, CA: Berkeley Linguistics Society. Slobin, D. I. (1991). Learning to think for speaking: Native language, cognition, and rhetorical style. Pragmatics, 1, 1–25. Stählin, W. (1914). Zur Psychologie und Statistik der Metaphern. Eine methodologische Untersuchung. Leipzig/Berlin: Verlag Wilhelm Engelmann. Steen, G. J. (2007). Finding metaphor in grammar and usage. Amsterdam: John Benjamins. Steen, G. J. &. R. W. Gibbs, Jr. (1999). Introduction. In R. W. Gibbs, Jr. & G. J. Steen (Eds.), Metaphor in cognitive linguistics (pp. 1–8). Amsterdam/Philadelphia: John Benjamins. Streeck, J. (1993). Gesture as communication 1: Its coordination with gaze and speech. Communication Monographs, 60, 275–299. Streeck, J. (This volume) Metaphor and gesture: A view from the microanalysis of interaction. Teßendorf, S. (In preparation a). From everyday action to gestural performance: Metonymic motivations of a pragmatic gesture. In I. Mittelberg & C. Müller (Eds.), Metaphor and metonymy in gesture and sign. Special Issue. CogniTextes. http://aflico.asso.univ-lille3.fr/cognitextes/ Teßendorf, S. (In preparation b). Pragmatic and recurrent: The case of the brushing aside gesture. In J. Bressem & S. Ladewig (Eds.), Recurrence in coverbal gestures: Investigating form and function. Amsterdam: John Benjamins.



What gestures reveal about the nature of metaphor  Taub, S. F. (2001). Language from the body: Iconicity and metaphor in American Sign Language. Cambridge: Cambridge University Press. Zbikowski, L. (2002). Conceptualizing music: Cognitive structure, theory, and analysis. Oxford: Oxford University Press.

Commentaries on the value of studying metaphor and gesture from the perspectives of different disciplines

Metaphoric gesture and cognitive linguistics Ronald W. Langacker

Metaphoric gesture is relevant for cognitive linguistics in two broad respects. On the one hand, it provides an independent source of evidence for cognitive linguistic notions. On the other hand, it poses the issue of whether such gesture should itself be considered linguistic in nature, i.e. an inherent aspect of language structure. The ways in which gesture is relevant as a source of evidence are for the most part quite apparent. Obviously, the prevalence of metaphoric gesture attests to the view of cognitive linguists that metaphor is a fundamental aspect of conceptual organization. Gestures which represent information not otherwise expressed corroborate the position that conventional metaphors are independent of any particular linguistic expression. The form and application of metaphoric gestures give palpable indication of a basic claim of cognitive semantics, namely the embodied nature of meaning and the grounding of abstract conceptions in perceptual and motor experience. In addition to these general claims, co-occurring gestures can offer independent evidence supporting the semantic analysis of particular elements and expressions. They can further be used to indicate which facets of an overall conception are active at a given moment, thus providing clues to the shifting of attention in online processing. This may well prove vital for determining the time course of linguistic conceptualization (Langacker, 2001a) and developing an integrated account of grammar, processing, and discourse. Since the need for such an account is generally acknowledged by cognitive and functional linguists, the question arises as to whether co-occurring gestures – rather than being extrinsic to language structure – are best considered part of it, hence one aspect of the full characterization of expressions. There is no single or simple answer to this question. Language is not a discretely bounded entity such that particular factors either belong to it exclusively or are wholly excluded (Langacker, 1987, § 2.1). It recruits and adapts a wide array of physical structures, neural circuitry, knowledge, and cognitive abilities that exist independently and serve other functions. Moreover, the processing activity we reify and recognize as linguistic expressions unfolds at various levels, with respect to both meaning and form. Meaning specifications pertain to the situation being described, information

 Ronald W. Langacker

structure, as well as interaction and speech management. Aspects of form include segmental content, prosody, and gesture. These levels are neither sharply distinct nor strongly autonomous: there are conventional patterns for coordinating structures at different levels, and some elements function simultaneously at more than one (Langacker, 2001b). For instance, the pointing gesture that accompanies demonstratives is an element of meaning as well as form (crucial to the situation described is the very fact that the speaker is pointing to a certain entity). These considerations provide a possible basis for regarding metaphoric gesture as an integral part of linguistic expressions. Both descriptive content and the gestural components of form are complex and multifaceted. Besides “literal” description, the former includes all manner of mental constructions – metaphor, conceptual blending, mental space configurations, fictive entities – characterized at any level of specificity/schematicity. Besides manual gestures, the latter include facial expression, actions performed more globally (e.g. a shrug), and even factors like posture and body language. These may all be closely bound up with linguistic expressions, in which case they can hardly be excluded from “language” on an a priori basis. Manual and facial gesture are of course the primary means of expression in signed languages. And not only do many gestures convey definite meanings, but they can also be associated with particular vocalizations. When a baseball umpire yells Safe! and simultaneously gives the standard gestural signal to this effect (raising both arms together to shoulder level and then sweeping the hands outward, palms down), why should only the former be analyzed as part of the linguistic symbol? Why should a pointing gesture not be considered an optional component of a demonstrative’s linguistic form? The fact that metaphor is a ubiquitous aspect of linguistic meaning, and gesture a common feature of linguistic form, would seem to offer ample basis for according linguistic status to the metaphoric gestures that accompany speech. It does not however follow that every such gesture has this status. A language does not include every gesture used while speaking, any more than it includes every conceptualization entertained by speakers or every vocalization produced. While it has no fixed boundaries and cannot be precisely delineated, a language does have certain limits. Partly these are given by degree of approximation to indisputably linguistic structures – “core” elements of the form and meaning of expressions. But even if we limit ourselves to core structures, comprising conceptual and segmental content drawn from units of a language, it is not the case that just any expression we might construct will be part of the language or even judged well-formed. A structure per se qualifies as an element of a language just to the extent that it is entrenched in the minds of speakers and conventional in the speech community. A novel structure reflects the language, being accepted as well-formed with respect to it, by virtue of instantiating conventional linguistic units.



Metaphoric gesture and cognitive linguistics 

It is these factors which determine whether gestures ought to be considered linguistic. If a gesture is both familiar and conventional in a speech community, bearing a systematic relation to the expressions it occurs in, its exclusion from “the language” would be arbitrary. If a particular gesture is novel, it may nonetheless reflect the linguistic system by instantiating a conventional gesture pattern characterized at a certain level of schematicity. This is not to say, of course, that all occurring gestures qualify as linguistic. But to the extent that they do, the full description of a language has to account for them. Granted their linguistic nature, it is important to investigate how gestures relate to other facets of language structure. One dimension of this problem is to ascertain how gestural content is integrated with co-occurring elements, both conceptually and formally. Another dimension is to investigate how the functions served by gestures relate to those served by elements of other kinds. It is worth exploring, for example, whether the locative prefixes that regularly occur on verbs in Cora (Langacker, 1990, ch. 2) do the same work as gestures in other languages. The burning of a candle, for instance, is described by the stem u-ká-taa (insidedown-burn), where the prefixes indicate the location and path of the verbal process. In fairly systematic fashion, prefix combinations invoke a schematic spatial configuration representing the clausal process either literally or metaphorically. To what extent are these morphological elements analogous to manual gestures? Obviously, the various issues raised here can only be resolved by careful and extensive descriptive work. But whatever might be the ultimate judgments, it seems quite clear that metaphoric gesture is intimately associated with language structure and revelatory of its nature. References Langacker, R. W. (1987). Foundations of cognitive grammar, vol. 1: Theoretical prerequisites. Stanford, CA: Stanford University Press. Langacker, R. W. (1990). Concept, image, and symbol: The cognitive basis of grammar. Berlin/ New York: Mouton de Gruyter. Langacker, R. W. (2001a). Dynamicity in grammar. Axiomathes, 12, 7–33. Langacker, R. W. (2001b). Discourse in cognitive grammar. Cognitive Linguistics, 12, 143–188.

Metaphoric gestures and cultural analysis Naomi Quinn

For cultural anthropologists, language is an important, if not primary, window into that part of culture that has been internalized and hence exists largely or only in people’s heads. Cognitive and other psychological anthropologists, in particular, rely heavily on the methods that linguists provide (see Quinn, 2005). So psychological anthropologists will be interested in the methodological potential of this new set of methods for studying metaphoric gesture. In my own work, specifically, I have used metaphors to identify components of one cultural schema, Americans’ model of marriage. Because speakers select metaphors to emphasize or clarify or expand upon (whether for themselves or for their listeners) the spoken points they are making, a systematic analysis of these metaphors, across speakers, can expose the shared concepts that underlie metaphor use. This methodological stance is consonant with David McNeill’s distinction in this volume, between what he calls “static” and “dynamic” metaphors – or what I have elsewhere (Quinn, 1999) characterized as “naturalized” versus “intentionally selected” ones. It is intentionally selected, dynamic metaphors that have provided me with insight into Americans’ schema for marriage. My analysis of this discourse on marriage leads me to depart sharply from McNeill’s (section 6 in his chapter) assertion, however, that “unexpected” metaphors are “without cultural guidance” – by which he means that individual speakers generated them on their own “when the culture does not have them readily at hand.” I found, to the contrary, that metaphors for marriage were chosen according to a set of underlying concepts that they were used to express – concepts that were shared by my American interviewees. Indeed, in several hundred hours of talk about marriage by 22 interviewees, all the metaphors used for marriage fell in to just eight classes. In my analysis, these classes – mutual benefit, lastingness, sharedness, compatibility, difficulty, effort, risk, and success/failure – were related in a shared sequence of marital events. I was able to reconstruct this shared event sequence through a further analysis that relied on the reasoning about marriage that these speakers did. In their discourse, speakers revealed their expectations that marriage be mutually beneficial in the sense of psychologically fulfilling to

 Naomi Quinn

both spouses, that it be shared, and that it be lasting. They reasoned that a marriage that was not mutually beneficial would not last, a dilemma in terms of the expectations they held. But, for the marriage to be beneficial, spouses had to be compatible, which was bound to raise marital difficulties, since Americans ordinarily married precipitously, without any full sense of how compatible they might be. Interviewees considered that the way to deal with these inevitable marital difficulties, as any difficult task, was to work to overcome them, so that the marriage would succeed and last. The risk was that a marriage might fail and end in divorce despite the best efforts of both spouses. By comparison with research presented in this volume, my analysis of metaphors for marriage, conducted in the 1990s, now seems downright primitive, relying as it did on audio tape recording alone, and treating the metaphors I found in speech, to use Müller’s term from section 4 of her chapter, “as products” – “as single well-delineated items or objects” rather than more extended processes. Attention to metaphoric gesture of the sort illustrated in this volume holds promise as an accessory to reliance on spoken metaphor in the reconstruction of culture. I will only suggest this promise, very briefly, with examples from my own research on American marriage. My suggestions are necessarily sketchy, absent actual data on the gestures that accompanied the interview discourse I collected. The reader will just have to imagine with me what I might have been able to learn from video recording that captured the metaphor in gesture. I will focus on just two features of metaphoric gestures. The first such feature, perhaps best illustrated in Müller’s chapter, is but one implication of viewing gestures dynamically – the realization that, as she puts it in section 4 of her paper, gestural metaphors “can be outcomes of a process which may in fact structure extended stretches of discourse.” Consider just two brief examples from my material, picked literally at random. In the first, a husband expresses confidence that the fulfillment of his wife’s and his own needs will lead to a lasting marriage – its lastingness emphasized in a succession of the three metaphors “we’re here to stay,” “we got it,” and “it’s concrete.” In the second excerpt, a woman is talking about the marriage of a friend, who has to decide whether the balance of needs that are and are not being met in her marriage is sufficiently negatively skewed to justify ending it. 2H-8: It’s like I know everything that she needs. That – you know, and the things that I don’t know are things that will come up in the future. I feel like she will tell me, you know, and if it’s something that I can’t handle then, you know, maybe perhaps our relationship will allow her to get it, you know, and vice versa. So I feel like, you know, we’re here to stay, you know. I think we got it. You know, I feel like – well I really feel it’s, you know, concrete.



Metaphoric gestures and cultural analysis 

5W-9: Because from all I could see, from all she could talk to me about, from all I could understand, there is very little chance that he is ever going to change in a way that will encourage her growth and her development and her meeting her own needs. I don’t believe he will. So she needs to either sort out, “Am I willing to give up this much of myself in order to have this security and some of the good stuff we have et cetera. Or am I going to indeed make the split and be on my own and live with the sadness and the loss of not having any of that stuff with him.” And I think that’s what she’s been doing. Trying to work that out.

Just try to imagine the way gestures accompanying these interview segments might have carried thought forward. How, for example, were the “things” and “stuff ” of the marriage conceptualized in gesture, and were they captured in the same basic gesture across these stretches of discourse (or even across these and other speakers)? Or, consider the causal relations between need fulfillment and lastingness, which are expressed in the first excerpt by the so in “So, I feel like we’re here to stay, etc.” and in the second by the and in “make the split and” not have “any of that stuff with him.” Did the accompanying gestures extend across each chain of causality, supporting and elaborating the bare language of causal reasoning? Might there have been a gestural bridge, in the second of the two passages, between “sort out” and the later “work out” that sustained the thought embodied in both? Is there gestural continuity, in the first passage, from the initial introduction of “everything that she need,” through “things that will come up in the future” and “something that I can’t handle,” to the final “will allow her to get it” – an extended gesture that would delineate, beyond the simple repetition in these four expressions, a more complex conceptual development of thought across them? For a second illustration, consider the possibility raised in section 2.2 of Cienki’s paper of a range of gestural spaces as yet unidentified, beyond just the space captured in the conduit metaphor. Cienki observes that past over-attention to that metaphor for gestural space is largely an artifact of what has up until now been the dominant method in metaphor research, that of showing subjects cartoon films and having them retell the film narratives. Talking about marriage, speakers may create other spaces. There could be, for example, the metaphoric gestural space of a container in which the “things” and “stuff ” of marriage exist. Another likely gestural space, equally abstract, is a metaphoric representation of the relation between two spouses – whether the couple themselves, another couple they know, or a hypothetical married couple. Here are three brief examples in which such a metaphoric space representing relation seems to figure, and might be expected to play out in gesture. 9W-2: I can say now I really feel married. I really feel married. Well, I mean that. Not one but a unit that our plans – that we make plans together. The future of one of us is the future of the other. That our future gets planned and operates together.

 Naomi Quinn

And the reason that I said that we were a unit and not one is that we’re still two very distinct people. 3H-2: Between two people, you can only go so far. You know, without – you’re not – you may be married, you know. You don’t – you’re not cemented together physically, you know. You can only get so close, and it’s not good to get that – it’s good to know your partner as well as you can know another human being possibly. 9W-2: I mean, I know lots of couples that where it meant maximizing the career they would want to do it separate and see each other on weekends or periodically. I have a – know a woman at the University of Iowa and her husband is in St. Louis and she talks about their long-distance phone calls and everything. For Rich and me that would not work because we just would grow apart, like I say we would become very independent once apart. And we can’t maximize the career without worrying about the couple. And if we do, I think we both agree that the marriage will be ending. That it might not end at that moment, you know, but in essence we’re – we’d be ending it.

Attention to the metaphoric gestures accompanying these passages of discourse would have surely alerted me to the centrality of the potential conflict, inherent in the American schema for marriage, between autonomy and dependency – or, put in less psychoanalytic terms, between relatedness and self-definition. These three speakers all see the resolution of this conflict in preserving a balance between relatedness and self-definition, or what one wife refers to as being “a unit, and not one” – that is, being “close” as a couple while still recognizing that they are “still two very distinct people.” The first two speakers are quick to defend autonomy in the face of the extreme intimacy that the marriage relationship entails; while the third speaker, considering a commuter marriage, worries about preserving relatedness against encroaching self-definition or “independence.” Gestures within a relation space – likely to occur in company with phrases like “a unit but not one,” “still two very distinct people,” “you’re not cemented together physically,” “you can only get so close,” “we just would grow apart,” “we would become very independent once apart,” or “worrying about the couple” – are intriguing not only because they bring to light the relatedness/self-definition tension in these speakers’ cultural understandings of marriage. It would be interesting to see, further, if the same basic gestures, within the same gestural space, were standard among American speakers. And it would be useful to identify other gestural spaces that Americans use to talk about marriage, in addition to marriage conceived of metaphorically as a container or as a relation. Two other likely candidates are the metaphoric conception of marriage as a trajectory (in terms of which the famous JOURNEY metaphor for interpersonal relationships, among other possibilities, can be instantiated) and marriage as an object (with strong foundations, or



Metaphoric gestures and cultural analysis 

capable of being blown apart, for example). Of course, in this and other domains, still other gestural spaces, metaphoric and non-metaphoric, are bound to be established. (Some recent videotape of an amateur astronomer sharing his passion for that subject, recorded by me as part of a learning exercise, revealed use of three different gestural spaces.) What might be added to a cultural analysis such as mine by the analysis of metaphoric gesture can only be conjectured at this time, but the possibilities are exciting. References Cienki, A. (This volume). Why study metaphor and gesture? McNeill, D. (This volume). Unexpected metaphors. Müller, C. (This volume). What gestures reveal about the nature of metaphor. Quinn, N. (1999). Metaphor and culture. In R. Wilson & F. Keil (Eds.), The MIT encyclopedia of the cognitive sciences (pp. 537–538). Cambridge, MA: MIT Press. Quinn, N. (2005). Finding culture in talk: A collection of methods. New York: Palgrave Macmillan.

Metaphor and gesture A view from the microanalysis of interaction Jürgen Streeck

The perspective from which I want to reflect on the promise and challenge of analyzing gestures as metaphors is that of the microanalysis of interaction, instantiated, for example, by conversation analysis. The first question that conversation analysts ask about an object – a bit of talk, a gesture – is why that now?: what is accomplished by it in a sequence of turns and actions? The focus, thus, is on the doings of conversational objects, including gestures. Among the many things that people do through gesture are the mapping out of aspects of the structure of their utterance, the display of its illocutionary role or the preferred uptake it is to receive, and the managing of the interactional process of taking and allocating turns. I call these practices pragmatic gesture. The idea of metaphoricity is particularly applicable to pragmatic gesture. For example, pragmatic gesture abounds with instantiations of the conduit metaphor (McNeill, 1992), that is, gestures that articulate aspects of the process of speaking in interaction (and interacting through speaking) as if it were a process of transaction of physical objects. In my own research (Streeck, to appear), I have identified – among other such gestures or “gesture families” (Kendon, 2004) – open-handed turn completors, i.e. gestures which, occurring during the completion of a turn-attalk, figure acts of “offering,” “giving,” or “handing over,” and appear to metaphorize acts such as the offering of an opinion or assertion, as well as the turn itself, as a tangible, physical act (see also Kendon, 2004; Müller, 2003); schematic acts of moving something aside, which occur with discourse markers such as anyway or metacommunicative statements such as that’s another story; gestures of the open hand, palm facing the interlocutor, which accompany verbal acts of negation and rejection; and shrugs, during which the hand is symbolically retracted from its engagement with the world. I would also include in this category beats (McNeill, 1992), that is, gestures that have been said to only mark the rhythm of speech – but which may nevertheless, given that their forms vary, provide additional structure that aids recipients in its parsing.

 Jürgen Streeck

The “doing” of pragmatic gestures often seems to be plainly transparent, and their metaphorical nature obvious. The open-handed “palm up” gesture varieties, for example, that are cross-culturally so common at turn-endings, can often be identified without much doubt as schematic versions of acts of offering or handing over that fit the relevances of the moment well: it makes easy sense that turntransfer be figured as a handing-over, or the voicing of an opinion or the making of a statement as an offering. Just as often, however, pragmatic gestures are opaque: we can neither be sure what aspect of the communicative process is elucidated by them, nor can we easily identify the “source domain” of the schemata that they instantiate (as we can identify the domain “object transfer” in the case of conduit gestures). And yet participants somehow seem to understand these gestures and be able to see the talk and interaction of the moment in their light. We seem to recognize semblances of inherently meaningful instrumental acts in them, but we are hard-pressed to say what they are. Let me illustrate this problem of opacity with two examples. One is a gesture, made by a young man in the U.S., who is trying to get the turn for a telling or informing of some sort during a family dinner conversation; the gesture is simply an uncurling of the fingers that were previously closed to make a loose fist, used by the owner of the hand as a headrest, holding his chin. The speaker opens his hand briefly, palm to the side, so that the fingers vaguely point in the direction of the audience. The gesture is made twice, in both cases as company to the turn-preface to be honest, by which the imminent telling is framed as a confession.1 1 __|____ | | 1 Albert: To be hon [ 2 Mother: I’d like mine in the shape of an E for Esther please= [ 3 Niece: Yes 4 Albert: =To be honest hhh [ 5 Niece: Mine in a heart (–––) 6 Mother: ((laughs))

1. When the speaker succeeds in getting his confession heard, the response he gets from his mother is “I don’t wanna hear it”; lines 12, 14: the family has been talking about farting.



Metaphor and gesture 

2 ___|___ | | 7 Albert: To be honest I’d like I > li:ke < [ 8 Daughter: Help, we can’t, we can’t we can’t 9 Albert: I’ll do that quite a bit in u:hh ( – – – – ) 10 tsk on the b(h)us r(h)ide back ho:me (0.8) 11 Daughter: ((bursts out with sound)) 12 Mother: I jus’ don’t wanna hear it 13 Albert (( )) [ 14 Mother: ↑Don’t wanna hear it 15 Albert: Ye-, yes yuh d:oo 16 Mother: No I don- Hey who’d you sit with on the bus? 17 Albert: Jeremy:::, Ke:vin This is a single case, and it is difficult to make any claim about the relationship between form and sense. We can not know whether this gesture is the speaker’s idiosyncracy or available as a “prefab,” “rented” (Bakhtin, 1986) from the community. All we know, really, is that the speaker makes the gesture twice out of the three times that he utters “to be honest,” i.e., that for him, on this occasion, the gesture is coupled with a pre-confession. Somehow, the gestalt of the gesture “fits” or “goes with” the context in which it occurs: a speech act-preface by which an utterance is begun (opened) and a confession (an opening up) is being announced. For my other example, consistent form-context relationships have been established (Streeck & Hartge, 1992). This is a “palm up” gesture that shows up repeatedly in a corpus of conversations of speakers of Ilokano in the North of the island Luzon in the Philippines, where it clearly is a convention, at least for older speakers (although the gesture can also be observed in U.S. conversations). The gesture consists of an open hand, palm facing up, coupled with a moment of intent gaze to it by the gesturer. However, no movement of the hand in the direction of the recipient is involved. Often, this gesture is then, in a next step, elaborated by embodied acts of enumeration: either the index finger of the other hand repeatedly points to the palm, or thumb and index-finger grasp one finger after another. It is not difficult at all to establish the function of the gesture: an exhaustive search of the corpus reveals that it always and only occurs along with or immediately prior to a telling or an installment of a telling. With some simplification, we can call it a “story-telling gesture.” The acts of counting that elaborate it are coupled with

 Jürgen Streeck

various kinds of listings – of protagonists, circumstances, or events – as they are common in recountings. Here, too, we perceive a good fit between the gesture and the context or position in an action sequence where it is made: the intent, sometimes extended, look at the palm seems to display, but also to perform, the speaker’s collecting of her thoughts. Playing on a line from a Wallace Stevens poem (1990), we might say that the intending speaker looks at the palm at the end of her mind: the narrative universe, the memory from which she must now construct a story. We could even speculate that the gesture expresses the cultural notion that telling a story from memory is like reading from a book; after all, the speakers look like they are reading from their hands! But of course, these are all conjectures. And yet, the opacity of these gestures seems to be an artefact of our method; co-participants have no difficulty making sense of and adequately reacting to them (as demonstrated in Streeck & Hartge, 1992)2. What we need is an account that can explain the fit between context and form. A first step would be to reconceive these gestures as abstractive actions, that is, as actions that take configurations or schemata that are readily at hand (available to the hand) to give form to abstract senses of the situation or speech act at hand, in a highly constrained sequential context. What I mean to suggest is that we take seriously what the hand qua hand contributes to the figuration: a prehensile posture (Napier, 1960), an action schema (holding, taking, setting, and so on), or the shape and role that the hand is in just prior to its deployment in a gesture. The Latin term abs-trahere means to “pull off of ” or “peel away from”: we can pull off/peel away and take home with us a hand configuration – a prehensile posture – from an act of handling an object and thus end up with a manual category and form, derived via abstraction from a haptic act, and full of symbolic potential when transferred to other contexts so as to articulate them. Gestures classify in the first place by virtue of the acts that they are, not by what they look like or what they resemble. In other words, it is the specific physical act itself – its particular, if underdetermined3, grip, hold, push, etc. – that organizes the target in terms of a source domain. The hands, in other words, bring to gesture heterogeneous habits and schemata that collectively offer a distinct way of knowing the world, which is why gesture is a distinct way of showing and making sense of it. (The medium, as it were, has a message.) These schemata include action 2. In the case of conventional gestures this could simply be explained as learning; but many pragmatic gestures do not seem to be conventionalized. 3. Gestures are inevitably underspecified, generic, schematic acts, because in making them the hands lack contact with matter; it is contact with specific matter that otherwise gives the practical actions of our hands (and thus their shapes and motions) specificity.



Metaphor and gesture 

patterns such as opening and closing or curling and spreading; parts with their specific features and affordances (fingers lending themselves to individuation, a big, flat palm; opposable thumb, mirror symmetry); habitual motor (action) patterns; and prehensile postures, among other habits and forms. To clarify how these, in their specific instantiations, articulate with the communicative context at hand must be our first order of business. It is recommended, then, that the hands not be treated as malleable for expressive purposes (although they most emphatically are this too, as documented by their uses in ritual, dance, or precise gestural depiction), but wise: familiar with experienced reality and capable of coping with it. The communicating body is not one which in its original state is a quiet, unmoving one, which is then set in motion by the engine of the mind to exteriorize mental content. Rather, the natural state of the body is that of active and haptic engagement (German handeln) and grasping (er-fassen). This, I think, is the essence also of Johnson’s view of “the body in the mind”: Our perceptual interactions and bodily movements within our environment generate... schematic structures that make it possible for us to experience, understand, and reason about our world (Johnson, 1987, p. 19).

I offer this view as an alternative to the prevailing intellectualist view of metaphor in gesture, which asserts that gesture expresses conceptual metaphors that exist independently of them4. I do not deny that such gestures are made – quite commonly made, in fact, especially in ritual.  But I do not believe that they are the prevailing variety of gesture in the domain that is at issue here, pragmatic gesture. I have coined the term ceiving to refer to the heterogeneous, yet distinct ways in which the hands, in gesture, make sense – how we con-ceive through them (Streeck, 2005, to appear). The habituated schemata that result from acts of ceiving are called cepts. I use these terms because they highlight the original metaphor (or insight) included in the Latin root, -cap- (take, grasp, take hold of): that grasping an object, taking it in one’s hands, is a fundamental way of knowing; that grasping is, in the first place, just that: grasping. While grasping is making sense in the way of the hand, ceiving means grasping an intangible domain (one not accessible to the hands) by drawing upon the wisdom of the hands: by finding, for example, a prehensile posture in terms of which it can be comprehended.5 4. Among the contributions to this book, the one that comes closest to my view is Calbris’; see also Calbris (2003). 5. Talmy (2000) also uses the term ception, in order to group together perception and conception. While this usage is not incompatible with mine, I was at the time unaware of Talmy’s use. In any case, I mean to highlight the root meaning of -cap- and what its occurrence in terms such as conception tells us about the manual foundations of understanding.

 Jürgen Streeck

There is one final quality of the ceiving of communicative action in pragmatic gestures that I need to mention, and that is the fact that they are also proprio-cepts, given to their makers by way of kinesthetic perception (proprioception). Thus, when we make a pragmatic gesture, our hands tell us what we are up to right now, with our present or imminent utterance: whether we are in the process of moving something aside or putting it out in front of everybody or letting go of it or rubbing it between our fingertips. Our gestures let us experience our “abstract” communicative acts as concrete physical acts, acts which we intuitively understand. They keep us appraised of the things that we are doing with words. References Bakhtin, M. M. (1986). Speech genres and other late essays (trans. V. W. McGee). C. Emerson & M. Holquist (Eds.). Austin, TX: University of Texas Press. Calbris, G. (2003). From cutting an object to a clear cut analysis: Gesture as the representation of a preconceptual schema linking concrete actions to abstract notions. Gesture, 3, 19–46. Johnson, M. (1987). The body in the mind. Chicago: University of Chicago Press. Kendon, A. (2004). Gesture: Visible action as utterance. Cambridge: Cambridge University Press. McNeill, D. (1992). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Müller, C. (2004). Forms and uses of the Palm Up Open Hand. In C. Müller & R. Posner (Eds.), The semantics and pragmatics of everyday gestures. The Berlin conference (pp. 234–256). Berlin: Weidler. Napier, J. (1980). Hands. New York: Pantheon. Stevens, W. (1990). The palm at the end of the mind. Selected poems and a play. H. Stevens (Ed.). New York: Vintage Books. Streeck, J. (2005). Pragmatic aspects of gesture. In J. Mey (Ed.), Encyclopedia of language and linguistics, vol. 5: Pragmatics (pp. 71–76). Oxford: Elsevier. Streeck, J. (to appear). Gesturecraft. The manu-facture of meaning. Amsterdam: John Benjamins. Streeck, J. & U. Hartge (1992). Previews: Gestures at the transition place. In P. Auer & A. di Luzio (Eds.), The contextualization of language (pp. 138–158). Amsterdam: John Benjamins. Talmy, L. (2000). Toward a cognitive semantics. Cambridge, MA: MIT Press.

Implications of cognitive metaphor and gesture studies for Ethnomethodology and Conversation Analysis and vice versa Anders R. Hougaard & Gitte R. Hougaard

1. Introduction Despite continued discussion concerning methodology and the omnipresence of the phenomenon, two and a half decades of cognitively informed studies of metaphor seem to have convinced most researchers within the fields of human sensemaking that metaphor exists as a real “cognitive” phenomenon1. This volume continues, expands, and strengthens the “tradition” of cognitively informed studies of metaphor by demonstrating how the study of gesture adds further substance to and further means of gaining insight into the phenomenon. In this paper we will briefly discuss what consequences the type of research presented in this volume and its results may have for Ethnomethodology (EM) and Conversation Analysis (CA) studies as well as the consequences that EM/CA studies of metaphors constructed, and gestures employed, in interaction may have for cognitive studies hereof. In addition and in relation hereto the paper very briefly introduces an approach that aims at uniting empirical research in interaction and the “cognitive” and “psychological” aspects of sense-making. 2. Gesture and language When two (or more) people communicate, an extremely complicated process of sense-making is taking place. In order to describe (aspects of) this process dominant trends in linguistics have extracted systems of language structure from the 1. The quotation marks indicate that though metaphor is a “real,” “cognitive” phenomenon to us as it is to cognitive linguists in general, we understand these categories differently from cognitive linguistics in general.

 Anders R. Hougaard & Gitte R. Hougaard

communicative situations and studied these on their own terms. Accordingly, even though the field of cognitive linguistics (CL) has in many respects revolutionized the study of language by for instance emphasizing the general cognitive underpinnings of language, this field has, too, from the beginning tended to focus on the language system by itself, mostly disregarding other situationally integrated resources of sense-making. However, the present volume testifies to a broadening of CL’s scope by including gesture into its studies of the sense-making events involving spoken language. This seems to correspond to the research interests within the field of EM/CA. Nonetheless CL finds a seeming anti-thesis in EM/CA. The former views language as a “window to the mind,” that is as a product and representation of cognitive processes. EM/CA positions language with respect to its primary goal of studying and describing 1) the local, situated construction of social order in and through which human beings make sense of their social life and thus 2) how they build society as well as how society builds them. In this picture language is not a focal object by itself, but of interest in so far as it serves as a resource in the construction of social actions. “Language” as such, one can infer from most EM/CA studies, is understood in terms of sentences which have syntactic, semantic and pragmatic structure. The notion of “language” within EM/CA is thus comparable to the one within CL and most other linguistics “schools” even though the former considers it to be a resource for (intersubjective) sense-making and the latter a product and an expression of (typically individual) meaning. This being said, it must again be emphasized that “language” as such is not by itself a research object of EM/CA and the study of it is not considered to be the key to describing how sense is achieved by participants in interaction2. As a matter of fact EM/CA studies document participants’ use of resources in addition to language in the procedures through which sense is achieved, such as the sequential positioning of actions, pitch, pauses (even down to 0.2 seconds), in-breaths, physical artifacts in the environment, talk-accompanying activities, and of course gesture. CA studies hereof (Goodwin, 1984, 1995, 1996, 2003; Schegloff, 1984; and Laursen, 2002) suggest that gesture is inevitable in the study of sense-making in face-to-face-interaction. However, despite the achievement of such insights, gesture has only been the object of a disappointingly small amount of research as compared to the overwhelming literature on talk in interaction.

2. Of course linguistic studies are being done using CA methodology (cf. Steensig, 2001; Ochs, Schegloff, & Thompson, 1996) in order to describe the grammar of interaction.



Implications of cognitive metaphor and gesture studies for Ethnomethodology 

3. Metaphor in discourse The present volume testifies to the relevance of gesture as a way of constructing, in the sense of expressing, metaphors or metaphorical thought in discourse. In this way, the volume broadens the notion of “language” in the sense that gesture may be considered to be a part of the total systematic, communicative and “cognitive” work that is being done. The studies in this volume also support the claim made in the conceptual metaphor literature that metaphor is a general cognitive phenomenon. Finally, by studying metaphor in discourse the authors take their studies out into concrete social reality. From our perspective, this is where metaphor studies belong. Following the thrust of Wittgenstein (1968), Ryle (1949), and Coulter (1970), we consider descriptions of metaphor not as descriptions of subjective mental experience but as descriptions of shared actions. They gain their intelligibility not from some individual mental processes but from the particulars of concerted, social procedures. As Brouwer, Hougaard, and Hougaard (2006) shows, there are no criteria in the founding literature for identification of metaphors that are useful in the context of face-to-face social interaction. As opposed to identifying metaphor by linguistic criteria (“life is empty for him,” Lakoff & Johnson, 1980) and/or by conventionality (“A dog’s tail” versus “a loyal dog” where the latter is conceived of as conventionally a metaphor, Lakoff & Turner, 1989), Brouwer, Hougaard, and Hougaard (2006) proposes a procedural approach to the identification and description of metaphor in interaction. That study shows that neither isolated sentences nor considerations of smaller or larger lumps of context that surround these suffice to capture metaphor in interaction. Only by studying how sequences of interaction give a temporally contingent meaning to utterances is it possible to capture metaphoricity in interaction. Metaphoricity is not a static, permanent feature of particular utterances in interaction. Interlocutors constantly have to work out whether something, in the making or having been said, is metaphorical or not. Hence if we wish to hold on to context as a specifier of meaning, we must revise the notion of context in terms of constantly developing sequences of interaction. Thus while an EM/CA approach to metaphor supports the CL approach by offering empirical, interactional substance to the claim of the reality of metaphor, we also endorse a critical evaluation of the ways in which metaphor has been identified and conceived of as a feature of single utterances supported by static frames of context. Interactional studies allow for the identification of a dynamic development of metaphoricity in interaction and for interactional descriptions of procedures for metaphor as well as of procedures for NOT being metaphorical and for being ambiguously metaphorical/non-metaphorical (see Brouwer, Hougaard, & Hougaard, 2006).

 Anders R. Hougaard & Gitte R. Hougaard

Though the preceding discussion has indeed focused on utterances (the Brouwer, Hougaard, and Hougaard study was conducted on data from a Danish call-in show), we wish to emphasize that by the term “procedural” we not only imply the sequential development of talk but the temporarily contingent, systematic production of senseful actions of any sort. As we will discuss in section 5, human beings produce and make sense of actions in and through sounds, gestures, gazes, and manipulations of materials in the surroundings, or typically in and through an integration of these resources. First, however, we will discuss the cognitive status of the socio-interactional findings within EM/CA. 4. Interaction and cognition According to traditional EM/CA, the phenomenon of “metaphor” is best understood in terms participants’ social behavior rather than in terms of individuals’ mental processes. In general in EM/CA, intra-individual phenomenological issues are translated into phenomenological issues of social, collaborative activity (Maynard, 2003). Thus, most scholars in this field have abandoned cognitive analysis from their research program. There may be many different reasons for this, some of which are methodological, i.e. the view that autonomous studies of text and talk should be carried out as far as it goes before resorting to speculative cognitive explanations of the phenomenon at hand. The reasons may, however, also be theoretical (cognitive insights are gained through discursive studies, not the other way around) or empirical (we have no direct access to private or personal minds) in nature. When invited to address the issue of “cognition” as in Potter & Molder (2005), most CA analysts construct a dualistic view which typically considers cognition an “inner” phenomenon that is separated from the “outer” social world. This, in fact, corresponds to the notion of cognition as it is applied in much present day cognitive science. However, this dualism is rather surprising if one takes into consideration Wittgenstein’s influence on EM/CA (see Coulter, 1970, and Williams, 1999). Furthermore, it does not accord with the notion of “cognition” entertained by, for instance, psychologists entering the field of CA, such as Billig (2006), Edwards & Potter (2005), and Hepburn (2004), and by ourselves (cf. Hougaard & Hougaard, 2006). According to these authors’ work there are no separate “inner” and “outer” spheres. The dualistic CA take on cognition and CA’s sole focus on the non-mental, social-behavioral aspects of sense-making may have the consequence that CA seems to study “mindless social actors” (van Dijk, 2006) and that “hardcore” CA is a behaviorist discipline. In fact some sociologists, like Coulter (2005), advocate describing language – which we would take as including gesture – in behaviorist



Implications of cognitive metaphor and gesture studies for Ethnomethodology 

terms rather than in terms of cognitivist speculation: “Our relationships to each other in and through the use of language are not primarily, nor in most subsequent instances, relationships best expressed in terms of “knowing” but, rather, of behaving. Not of “thinking”, but of acting” (p. 84). Coulter distances himself from Skinnerian behaviorist theses about “verbal behavior” as conditioned linguistic “responses” to “stimuli” but is almost agnostic about talking of “mind” as this, according to Coulter, can only be done if “one buys into the whole Cartesian bag of tricks from the outset” (p. 92). CA’s (and EM’s) research interests in social human beings’ common sense “knowledge” and thus the paradigms’ interests in “knowledgeable” social actors may be perceived in terms of Coulter’s version of behaviorism3. This position, however, has difficulties, from our point of view, with coming to terms with certain aspects of EM/CA research interests, namely the social construction of social order: How can a description that is done solely in terms of concrete actions in interaction – accomplished in and through gesture and talk – account for the participants’ understanding and recognition of that order as being normative? In other words, from our perspective, interlocutors do show – or rather do do – a reflective or thoughtful behavior: they are not social “dopes,” as EM would have it. In sum then, whereas Coulter warns against the threats of including cognition and mind into the socio-interactional vocabulary, we recommend doing so, but without buying into the whole “bag of individualistic cognitive science tricks.” According to our perspective (as well as to Coulter’s, we assume) the social and the cognitive aspects of the construction of metaphors in interaction are inseparable. It makes little sense to describe procedures of sense-making in terms of a (inside/ outside) dualistic view. In our understanding doing or accomplishing social actions through gesture and talk is being cognitive participants in interaction, and sense-making procedures should be conceptualized and described as such. 5. Fused bodies From our perspective then, understanding and knowledge are “done,” i.e. accomplished, in and through gesture and talk, here-and-now, and are not characteristically preceded by or in other ways indirectly linked to any internal cognitive processing. The cognition is in the social action, in the very doing of it. This though does not mean that participants enter any new interaction totally “blank,” waiting for cognition to happen. They are knowledge-able of things that they reconstruct, re-achieve 3. Other researchers such as Cicourel (1974) and Schegloff (1991, 2006) consider the EM/CA interests to be transparently relevant to cognitive science.

 Anders R. Hougaard & Gitte R. Hougaard

and reestablish; they are characterized by using memory as a means for recognition (or re-cognition) of ongoing action. By this – and in accordance with the discussion in the previous section – we deliberately object to typical cognitive science jargon that conversationalists (or people in general) “possess” or “store” knowledge which can then be activated. Social knowledge is rightfully thought of as reestablished or as coming into being as actions are being constructed. This reconstructed knowledge/knowledge-coming-into-being can be described as mind (as an analysts’ category in contrast to ordinary members’ categories; see for instance Coulter, 1979): In order to make sense, participants have to assume (Garfinkel, 1963) that knowledge is social (or “shared”), that is they have to assume that as they are themselves knowledgeable of what “people-like-us” know, so are their co-conversationalists. Further, they have to assume that their co-conversationalists, as they do themselves, act in accordance with and orient towards that knowledge. The assumed knowledge is as we said earlier oriented-to-knowledge, i.e. is done in interaction. When making sense, bodies in interaction “share” a “we-centric space” (Gallese, 2006). In Gallese’s terms the we-centric space is a primitive space that, for instance, the infant shares with others (mothers). According to his hypothesis, the shared we-centric space enables the social bootstrapping of cognitive and affective development because it provides a powerful tool to detect and incorporate coherence, regularity and predictability in the course of interaction. In our understanding that “space” does not cease to exist once people have become socialized individuals. The we-space remains a fundamental condition of human life. Without it people will be treated as and thus become deviants or non-members of any society.4 Sense-making then is fundamentally connected to acting or doing. A participant’s senses are neurologically combined when making sense. So when for instance hearing things (such as the sound social), seeing things (like lips moving) has an influence on what we hear (cf. the McGurk effect). Our brain integrates all types of impulses and so does our observable interaction. Analysts’ a priori hierarchical division in terms of the relevance of these various senses, and thus of the hierarchical division of resources (first language/talk and then gesture and other conduct) that participants employ, is to be avoided. Instead all resources and actions that are accomplished in interaction and that are constantly co-systematized in the hereand-now of the interaction such that knowledge in the end is co-(re)constructed, co-(re)established and co-(re)accomplished; all of these should, from our perspective, be included in systematic descriptions of how human beings make sense in systematic ways. This entire systematics of resources for interaction we (in opposition to typical linguistics/language studies and EM/CA in general) call “language.” 4. Ultimately, if an individual is not categorized as a member of any society, social, or cultural group, that individual is not even granted humanity (cf. certain wild children).



Implications of cognitive metaphor and gesture studies for Ethnomethodology 

Lastly, but of fundamental importance to the approach we endorse, we wish to point out that to make sense, participants in interaction rely upon the full inventory of neurons, hormones, modulators, neurotransmitters and neural circuits, muscles, and molecules. When participants in interaction make sense, bodies, that is entire social and biological organisms (as opposed to Cartesian minds, brains and computers) are at work. Bodies grasp the world, including the interactional world.5 Bodies orient towards each other in the world. Sense-making then, as we perceive it, is the sum of processes and products of socio-interactional sensing by knowledge-able, social bodies. Imagine for instance the perfect union of a couple of skillful dancers who accomplish a minutely concerted dance. They are as if one. When embodied everyday interaction is at work, it carries within it such a dancing couple. And we should not let considerations of deceit, hidden motives, conflicting intentions, and the like fool us to believe that it does not. References Billig, M. (2006). A psychoanalytic discursive psychology: From consciousness to unconsciousness. Discourse Studies, 8, 17–24. Brouwer, C., A. R. Hougaard, & G. R. Hougaard (2006). Metaphors in interaction. Paper at the Language, Culture, and Mind Conference, Paris (July 17–20). Cicourel, A. (1974). Cognitive sociology. Harmondsworth: Penguin. Coulter, J. (1979). The social construction of mind. London: Macmillan Press. Coulter, J. (2005). Language without mind. In H. te Molder & J. Potter (Eds.), Conversation and cognition (pp. 79–92). Cambridge: Cambridge University Press. Dijk, T. van (2006). Introduction: Discourse, interaction and cognition. Discourse Studies, 8, 5–7. Edwards, D. & J. Potter (2005). Discursive psychology, mental states and descriptions. In H. te Molder & J. Potter (Eds.), Conversation and cognition (pp. 241–259). Cambridge: Cambridge University Press. Gallese, V. (2006). Intentional attunement: A neurophysiological perspective on social cognition and its disruption in autism. Brain Research, 1079, 15–24. Garfinkel, H. (1963). A conception of, and experiments with, “trust” as a condition of stable concerted actions. In O. J. Harvey (Ed.), Motivation and social interaction: Cognitive approaches (pp. 187–238). New York: Ronald Press.

5. A bodies-social-world is the resource of knowledge. Bodies in the jungle (like animals) know pain as well (innate knowledge), i.e., to act upon it, to avoid it, etc.; but it takes a social world to know, conceptualize, and reflect upon pain, i.e., to understand pain, which by the way also entails knowledge about how to handle pain: i.e., when crying is acceptable and when not. In other words it takes a social world to create, develop, and elaborate understandings, impressions, and beliefs. Sense-making beings, then, are not just a matter of neurology and physiology; they crucially develop minds too. These do not precede social life; they are a product of it.

 Anders R. Hougaard & Gitte R. Hougaard Goodwin, C. (1984). Notes on story structure and the organization of participation. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action (pp. 225–246). Cambridge: Cambridge University Press. Goodwin, C. (1995). Co-constructing meaning in conversations with an aphasic man. Research on Language and Social Interaction, 28, 233–260. Goodwin, C. (1996). Transparent vision. In E. Ochs, E. Schegloff, & S. Thompson (Eds.), Interaction and grammar (pp. 370–404). Cambridge: Cambridge University Press. Goodwin, C. (2003). Pointing as situated practice. In S. Kita (Ed.), Pointing: Where language, culture, and cognition meet (pp. 217–242). Mahwah, NJ: Lawrence Erlbaum. Hepburn, A. (2004). Crying: Notes on description, transcription, and interaction. Research on Language & Social Interaction, 37, 251–291. Hougaard, A. R. & G. R. Hougaard. (2006). Fused bodies: Sense making as a phenomenon of interacting, knowledgeable, social bodies. Paper at the German Cognitive Linguistics Conference, Munich (October 5–7). Lakoff, G. & M. Johnson (1980). Metaphors we live by. Chicago: University of Chicago Press. Lakoff, G. & M. Turner (1989). More than cool reason: A field guide to poetic metaphor. Chicago: University of Chicago Press. Laursen, L. (2002). Kodeskift, Gestik og Sproglig Identitet på Internationale Flersprogede Ar­ bejdspladser. Doctoral dissertation, Det Humanistiske Fakultet, Syddansk Universitet. Maynard, D. (2003). Bad news, good news: Conversational order in everyday talk and clinical settings. Chicago: University of Chicago Press. Ochs, E., E. Schegloff, & S. Thompson (Eds.) (1996). Interaction and grammar. Cambridge: Cambridge University Press. Potter J. & H. te Molder (Eds.) (2005). Conversation and cognition. Cambridge: Cambridge University Press. Ryle, G. (1949). The concept of mind. Chicago: University of Chicago Press. Schegloff, E. (1984). On some gestures’ relation to talk. In J. M. Atkinson & J. Heritage (Eds.), Structures of social action (pp. 266–296). Cambridge: Cambridge University Press. Schegloff, E. (1991). Conversation analysis and socially shared cognition. In L. Resnick, J. Levine, & S. Teasley (Eds.), Perspectives on socially shared cognition (pp. 150–171). Washington, D.C.: American Psychological Association. Schegloff, E. (2006). On possibles. Discourse Studies, 8, 141–157. Steensig, J. (2001). Sprog i virkeligheden: bidrag til en interaktionel lingvistik. Aarhus: Aarhus University Press. Williams, M. (1999). Wittgenstein, mind and meaning: Toward a social conception of mind. New York: Routledge. Wittgenstein, L. (1968). Philosophical investigations (trans. G. E. M. Anscombe). Oxford: Basil Blackwell.

Sign and gesture Towards a new paradigm Sherman Wilcox

For too many years, decades in fact, gesture has been a taboo word among signed language researchers. While the scholarly world was discovering the significance of gesture and demonstrating the undeniable fact that gesture is linked socially, psychologically, and neurologically with language, signed language linguists were stuck in a backwater eddy, denying the obvious: there is a natural link between signs and gestures. It is not too difficult to see why sign linguists were so gun-shy about gesture. For centuries, the common understanding was that signed languages were not languages. They were simply gestures used by people who, because they could not hear, could not acquire or use the only true human means of expressing language – speech. In fact, for most of history language and speech were so confounded that it was impossible to tell the difference. Speech was synonymous with language. Gesture was not language. Fortunately, the theoretical currents have changed. Sign researchers have begun to explore the gesture-language interface and to discover what it reveals about the nature and development of language. Linguists, at least some, are no longer quite so certain that there is a categorical distinction between gesture and language. The latter conceptual shift can be attributed directly to the pioneering work of David McNeill, whose 1985 article in Psychological Review boldly and directly challenged the received wisdom: “So you think gestures are nonverbal?” Once linguists and others were able to include gestures and speech within a unified conceptual framework as aspects of a single underlying process (McNeill, 1990), it demanded that signed language linguists re-examine the relation between sign and gesture: in short, to embrace a paradigm shift in how signed languages are viewed. One contribution to this paradigm shift was the publication of Gesture and the nature of language (Armstrong, Stokoe, & Wilcox, 1995), which made the case that scholars could legitimately examine gesture and signed language as related phenomena. Of course, GNL, as it has come to be called, went beyond the claim that gesture

 Sherman Wilcox

and language might be related. It put forward the hypothesis that all language is gestural, and that the origins of human language can be traced to visible gestures. GNL relied on the cognitive linguistic model, and in fact was instrumental in bringing together a nexus of gesture-language-sign research informed by cognitive linguistics. Cognitive linguistics dramatically changes the perspective of how we view language, and it does so in a way that allows linguists to learn from gesture researchers, an option that simply was not possible under the formalist framework. Soon sign researchers were examining in new and exciting ways phenomena that had been noted and then discarded in the early days of sign research, such as metaphor, iconicity, and metonymy. A perfect example of this is the work reported in Lynn Friedman’s On the other hand: New perspectives on American Sign Language (1977). I can remember when this book first came out. I was a new student of signed language at the time, but not yet a linguist. With an interest in signed languages, yet having no ideological stake in linguistic theories, I found this book tremendously stimulating. Several years later, however, in the mid 1980s, I recall chatting with a colleague who had recently graduated from Gallaudet University. When I happened to mention Friedman’s book, this person immediately chastised me: “That work has been totally refuted.” The implication was that any attempt to describe the type of phenomena that On the other hand focussed on, especially iconicity, was to deny the linguistic status of American Sign Language. It was only when I later discovered McNeill’s Psychological Review article that I could begin to return to the work of Friedman, Mandel, DeMatteo and the others who had the misfortune of being ahead of their time. The paradigm shift we are witnessing goes beyond a new view of signed languages; it also is changing the way we regard language in general. The driving force behind the shift is the convergence of these two new lines of research – gesture studies and cognitive linguistics, as exemplified by a number of the contributions in the present volume. The synergism is strong and enticing. We see the emergence of research produced by scholars who are charting a new course in the integration of signed language, gesture, and cognitive linguistics. These include linguists such as Paul Dudis, Terry Janzen, Maria Josep Jarque, Barbara Shaffer, Sarah Taub, and Phyllis Wilcox, to name just a few. My own work on cognitive iconicity (Wilcox, 2004a), routes from gesture to grammar (Wilcox, 2004b), and the expression of modality in American Sign Language (Wilcox & Shaffer, 2006) also grows organically out of this new paradigm. The chapters in Metaphor and gesture vividly demonstrate how gesture studies and cognitive approaches to language can open new vistas for sign research. Alan Cienki’s chapter “Why study metaphor and gesture?” not only answers this question for gesture researchers, it points to the importance of investigating the gestural



Sign and gesture 

and metaphorical heritage of signs. The issues of form, meaning, and convention described by Fey Parrill tie directly to understanding how gesture enters the sign linguistic system. Geneviève Calbris’ work has been an inspiration to spoken and signed language linguists. Her chapter in this volume once again provides exceptional insights into a still poorly understood area of signed language linguistics – the symbolic use of space. The research of Rafael Núñez provides deep insights into the neuroscience of motor action and language, which in turn points to an evolutionary link between gesture and language (Armstrong & Wilcox, 2007). Some may see the surge of interest among sign linguists in gesture studies, metaphor, and other cognitively-motivated phenomena as merely an attempt to capitalize on a passing fad. I couldn’t disagree more. What we are seeing is the exhilaration of a theoretical sea change, a new way of thinking about language and how it is that humans make meaning. Metaphor and gesture is an indispensable contribution to this endeavor. References Armstrong, D. F. & S. E. Wilcox (2007). The gestural origin of language. Oxford: Oxford University Press. Armstrong, D. F., W. C. Stokoe, & S. E. Wilcox (1995). Gesture and the nature of language. Cambridge: Cambridge University Press. Calbris, G. (This volume). From left to right… Coverbal gestures and their symbolic use of space. Cienki, A. (This volume). Why study metaphor and gesture? Friedman, L. A. (Ed.) (1977). On the other hand: New perspectives on American Sign Language. New York, NY: Academic Press. McNeill, D. (1985). So you think gestures are nonverbal? Psychological Review, 92, 350–371. McNeill, D. (1990). Hand and mind: What gestures reveal about thought. Chicago: University of Chicago Press. Núñez, R. (This volume). A fresh look at the foundations of mathematics: Gesture and the psychological reality of conceptual metaphor. Parrill, F. (This volume). Form, meaning, and convention: A comparison of a metaphoric gesture with an emblem. Wilcox, S. E. (2004a). Cognitive iconicity: Conceptual spaces, meaning, and gesture in signed languages. Cognitive Linguistics, 15, 119–147. Wilcox, S. E. (2004b). Gesture and language: Cross-linguistic and historical data from signed languages. Gesture, 4, 43–73. Wilcox, S. E. & B. Shaffer (2006). Modality in American Sign Language. In W. Frawley (Ed.), The expression of modality (pp. 207–237). New York: Mouton de Gruyter.

The study of metaphor and gesture A critique from the perspective of semiotics Paul Bouissac

1. Introduction The problem of understanding which gestures make sense, what kind of sense they make, and how they achieve this result, obviously belongs to the range of issues that traditionally have been addressed by semiotics. It is usually accepted that gestures, as opposed to gesticulation, are signs by definition in as much as they signify, gesticulation being all sorts of movements with no specific intentional meaning. But the situation is murkier than it may appear at first. If “signifying” is taken to mean “being interpreted” rather than “deliberately communicating” some information, then gesticulation can be construed by an observer as symptom (e.g., motor pathology, insecurity), signal (e.g., distress, frustration), or sign in the fullest sense of the term (e.g., calling for attention, pretending any of the above). In all these cases the distinction between gesticulation and gesture dissolves. On the other hand, gesturing is most of the time so spontaneous, and is usually so intimately woven into speech that this behavior can hardly be considered to be consciously planned. This, however, does not disqualify gestures as objects relevant to semiotics in as much as the information that is thus conveyed influences the behavior of the interacting agents. The first semiotic analyses of gesture (e.g., Efron, [1941] 1972; Birdwhistell, [1954] 1960, 1970; Ekman & Friesen, 1969) have classified them according to broad Peircean sign categories (index, icons, symbols, and their derivatives). Iconicity has often been foregrounded in these endeavors because of the obvious capacity of the hands to approximately imitate or model objects and movements observed in the environment. Therefore, focusing on the metaphorical use of iconic gestures comes naturally under the purview of semiotics. But the notion of iconicity itself is very problematic, and it probably should not be taken for granted as readily as is usually the case in gesture research. These few comments will heuristically question some of the assumptions made in some of the chapters in this volume.

 Paul Bouissac

2. Can a metaphor explain another metaphor? “Growth point” and “catchment” are two metaphors that were coined by David McNeill in order to figuratively capture two distinct dynamic patterns of gestures associated with speech. These terms have caught the imagination of researchers who make extensive use of them when they try to sort out functional elements in the flow of hand and arm movements that accompany verbal interactions. The danger is to forget that these terms are metaphors designed to broadly characterize some gestures artificially isolated for the purpose of phenomenological descriptions. They have a merely heuristic value in the sense that they can indeed focus the attention of the researcher on some recurring patterns of movements. They are definitely suggestive but their functions and properties are not well established. The correspondence with McNeill that is quoted in the article by Montredon et al. shows that the definitions of these terms are rather fuzzy in the very words of their promoter. They seem to simply mean respectively “point of departure” and “point of arrival” since a catchment is, technically, a collecting basin. Ultimately, most, if not all gestures start with lifting a limb against the force of gravitation, then end by returning to a resting downward equilibrium. Gestures are the product of neuro-muscular energy and the law of gravitation. They differ from one another by the paths limbs follow, their configuration, their speed and intensity, the energy they spend, and their rhythms. Certainly, “growth point” and “catchment” are useful notions for segmenting the dynamic flow of limb movements accompanying speech. However, these fancy metaphors may stand in the way of the analysis as they may be promising more than they can afford in terms of explanation. The paths from point of departure to point of arrival comprise a limited number of possible trajectories that are constrained by the interplay of bones, muscles, nerves, and articulations. Thus, there is a restricted repertoire of movements such as extending, retracting, and rotating within angles determined by the anatomy and physiology of the upper limbs. Any of these possible movements can be interpreted as “resembling” the motion or morphology of natural or artificial objects, and therefore can be interpreted as metaphors that select such objects as the vehicles of their meaning. But such imitative movements are highly polysemic. Their relation to any particular “object” is ambiguous, and requires an interpretation. In many experimental examples presented as evidence of gestural metaphors, the cues are provided by the lexical context. One may thus wonder whether the gesture is specifically generated by the root metaphor that provides the theme of the particular speech segment being elicited and observed, or whether the gesture belongs to a more general repertoire of movements that merely specify some semantic features which are difficult to express lexically or syntactically. This means that these gestures



The study of metaphor and gesture 

would be cases of semantic supplementation rather than the redundant expression of a metaphor in two different modalities. It would thus ensue that what would be an ambiguous gesticulation if it were produced out of context is being construed as a gestural metaphor thanks to the cues offered by the speech utterance it complements. Only extensive comparative experiments, designed to provide the means of falsifying rather than confirming the general hypothesis, would make it possible to settle the issue. 3. Illustrators vs. metaphors Even if it could be demonstrated that, in a given instance, both speech and gesture are constrained by the same metaphor, characterizing the gesture as metaphorical would remain questionable. The assumption that both segments (linguistic and gestural) are generated by a kind of “trans-modal” metaphor could only be proven on a deep neuro-cognitive level. Even infra-second measurements could not yield a decisive proof because the comparative timings could be indicative of the length and complexity of the paths rather than absolute priority of one origin over another one. If, as it was suggested in the preceding section, the production of a gesture is constrained by, and subservient to, the cognitive metaphor that is first realized through speech, the iconic gesture would be at best an illustrator (in Ekman and Friesen’s sense, i.e., imitating some property of the referent of a word or a sentence) but not a metaphor in the rhetorical sense of the term. An illustrator is redundant with respect to a speech metaphor but not a metaphor in its own right. This does not mean that there cannot be true gestural metaphors, that is, gestures that refer to a stereotyped action different from, but metaphorically pertinent to, the action to which a speech segment refers (itself being possibly, but not necessarily, a metaphor). If I say that someone sweet-talks someone else, and, at the same time, make the gesture of playing the violin, this could undoubtedly be considered a gestural metaphor, not a mere iconic illustrator of “sweet-talk”. Or, still more so, if my gesture is a comment on someone else reporting that someone has obtained the support of a former adversary. Other examples could include the gestures of vomiting, yawning, pointing a gun, to gesturally metaphorize respectively a linguistic expression of dislike or disgust, boredom, and enmity. From a cognitive point of view, it is important to distinguish at least three typical cases: (i) plain speech (that is, not metaphorical) with gestural metaphor; (ii) metaphorical speech with congruent illustrator; (iii) metaphorical speech with gestural metaphor. The study of recorded interactions would probably indicate that true gestural metaphors mostly belong to the category of “emblems,” that is, stereotyped gestures that

 Paul Bouissac

have a lexical identity either through convention or because they have a metonymic relation to crafts or social functions. But this is only a hypothesis. 4. The naturalistic fallacy When analyzing gestures that occur with speech, many researchers appear to be the victims of a naturalistic fallacy. First, gesticulation is construed as being a-cultural and not proper to convey any specified meaning. But even a superficial survey of available documentaries shows that gesticulation differs from culture to culture, at the very least with respect to the degree to which it is allowable, but also in other features which must be acquired during development. The same patterns are repeated with some possible idiosyncratic features. It appears that gesticulating people have a relatively limited repertoire with identical dynamic patterns recurring with a great variety of topics, or in conjunction with some syntactic constructions independently of their meaning, or even perhaps whenever the speaker encounters some inhibiting blockage, such as Derrida, in the Montredon et al. article, having difficulty providing a conceptually clear definition of “deconstruction” because being straightforward would disclose its direct derivation from Heidegger, and (horribile dictu) Nietzsche, and would thus jeopardize Derrida’s claim to absolute originality. Interestingly, the derivation is both acknowledged and denied. Furthermore, the speaker also finds himself in a double bind since cultural deconstruction is also inconsistent with the politically correct duty of preserving ethnic heritage. It seems from Montredon’s description that the sedimentation gestures obfuscate rather than clarify the meaning in spite of their purely apparent iconic simplicity. Having created a self contained language that has become the lingua franca of a restricted group, the philosopher obviously cannot cope with the task of translating it into the plain language of a television audience. Technically, “deconstruction” means the demonstration that the very basis of some fundamental philosophical or cultural assumptions is logically inconsistent. The gestural metaphor of “sedimentation,” with its implication of depositing layers upon layers, is more a construction than a deconstruction figure. Note that the speech accompanying these gestures is hardly articulate, full of hesitations and incomplete sentences. Secondly, a large subset of meaningful gestures is considered to be iconic, that is, to imitate or reproduce properties of natural objects. This latter category provides the stock of examples for metaphoric gestures. But there are many ways of imitating a process or an object with the hands. These ways are culture-dependent, not only because the artifacts to which they refer are different (driving a car vs. a horse cart), but also because the environments greatly vary in their properties and their perceptions. Equally varied are the cultural gestures that can function as



The study of metaphor and gesture 

metaphors (e.g., stereotyped polite, apotropaic, religious, artistic gestures.). In any case the context (both cultural and linguistic) is needed, and pragmatics is absolutely required if one is to understand their metaphorical function. Another consequence of the naturalistic fallacy is that, by analogy with natural languages, gestures are credited with more creativity than they can actually afford because they are not produced by a dually patterned system. With the possible exception of the fingers, which each can conceivably be “on” or “off ” (hence the origin of “digital” code), gestures cannot provide a rich combinatorial resource. Therefore, analyzing gesture on the model of linguistic metaphors can only be a strained exercise because it consists of comparing the limited potential of a single articulation system with the unlimited potential of a double articulation system, or, in other words, the messages produced by a digital code with those produced by an analogical code. This might be the source of the difficulties that have been identified in metaphorical gesture research. Fauconnier’s “mental space,” which is often called upon as a crucial theoretical notion in such research, is cognitively constrained and conventionally “furnished” like a dwelling. It does not seem to function as a matrix upon which a calculus can operate. 5. Conclusion At a time when neuroimaging drastically renews the methods of investigation in the behavioral and cognitive sciences, it can be anticipated that the problems raised by the notion of gestural metaphors will find elegant and novel solutions. Methods that stick to the notion that the brain is a black box and that researchers are limited to the observation of the outputs (in this case language utterances accompanied by gestures) and their description are missing an opportunity of discovering actual brain processes that would shed some light on the functional relations between language and multimodal perception. Explanations based on mere inferences regarding inaccessible cognitive processes are at best hypotheses that should be tested through the numerous neuroimaging techniques that are now available to researchers. References Birdwhistell, R. L. (1960). Kinesics and communication. In E. Carpenter & M. McLuhan (Eds.), Explorations in communication (pp. 54–64). New York: Beacon. Birdwhistell, R. L. (1970). Kinesics and context: Essays on body motion communication. Philadelphia: University of Pennsylvania Press.

 Paul Bouissac Efron, D. ([1941] 1972). Gesture, race, and culture. The Hague: Mouton. Ekman, P. & W. Friesen (1969). The repertoire of nonverbal behavior: Categories, origins, usage and coding. Semiotica, 1, 49–98. Montredon, J. et al. (This volume). Catchment, growth point, and spatial metaphor: Analyzing Derrida’s oral discourse on deconstruction.

The neuroscience of metaphoric gestures Why they exist George Lakoff

It is the fall of 1980. Metaphors we live by was published only a few months earlier. David Rumelhart was running a cognitive science conference at the University of California, San Diego, and David McNeill and I were invited as presenters. McNeill’s paper was stunning. In it he presented his first results on types of spontaneous gestures: beats, deictics, iconic – and metaphoric gestures, correctly explaining them in terms of the theory of conceptual metaphor. McNeill’s first example came from a videotape, in which a woman was saying, “I couldn’t decide whether to stay at home or go to the movies.” As she said “decide,” she put her hands out in front of her, slowly moving one hand up and the other down, and then the first down and the second up, and repeating this. McNeill observed that the metaphor was Choosing Is Weighing (the alternatives), in which the hands become the two sides of a scale with one side going up as the other goes down. The word “weighing” doesn’t occur in the sentence, but the idea is in the source domain of a common conceptual metaphor for making a decision. The metaphor was in the mind and the gesture but not the language. McNeill’s second example was the case of someone saying, “Let me tell you …” and, while saying “tell,” tracing the path from the speaker to the hearer. McNeill correctly pointed out that this was in instance of the Conduit metaphor for communication (Reddy, 1979), with the gesture tracing the conduit. It has now been nearly three decades since those first examples. Researchers have found examples by the hundreds – across languages – and I have noticed examples by the thousands in everyday life. One of the most interesting is Alan Cienki’s (1998) example in which college students, who are talking, gesture with the past at the left and the future at the right. The metaphors Past is Left and Future is Right only work for gestures, not spoken language. The obvious theoretical conclusion is that metaphoric gestures provide evidence – independent of words – for the theory of conceptual metaphors.

 George Lakoff

They also provide evidence that gesture is an inherent part of language – gestures work as signs communicating thoughts. Critics of this view have sometimes claimed – incorrectly – that iconic gestures are just pantomimes. But metaphoric gestures dispel that view: the iconic gestures are in the source domain but what they express is in the target domain – usually abstract subject matter. 1. The neuroscience of metaphoric gesture In thinking about metaphoric gesture it is useful to bring some neuroscience into the picture. What I am going to do is put together the best hypotheses I know about how the brain works for phenomena other than metaphoric gesture to show how they combine to make metaphoric gesture happen. – The cognitive unconscious. Just as most thought is below the level of consciousness, and just as grammar and phonology are mostly used in normal speech below the level of conscious control, so spontaneous gesture, for the most part, is also unconscious and automatic, rather than consciously controlled. – Mirror neurons. There are neural fibers connecting the pre-motor cortex (which “choreographs” complex actions) and the parietal cortex (which integrates perceptions). As a result, neurons in certain pre-motor areas fire whenever (i) we perform an action, or (ii) we perceive the same action being performed by someone else. These neurons link actions and perception. Each gesture is “choreographed” by pre-motor neurons and the same gesture by someone else can be recognized by the mirror neurons. – Imagination. When we imagine that we are moving or perceiving, part of the same neural structure is activated as when we actually move or perceive. The same is true for dreaming, remembering, planning, etc. – Simulation. The general term for this is “simulation” – the brain simulates moving and perceiving in imagining, dreaming, remembering, planning, and “mind reading” – generating expectations of what someone will do next on the basis of what that person has just done. Simulations can be either conscious or unconscious. – Simulation semantics. Jerome Feldman has suggested that linguistic meaning is simulation. The basic idea is this: Take the sentence, “Harry picked up a book.” If you can’t imagine what it would be to pick up a book, then you don’t understand the sentence. Let us pause for a moment. Suppose someone is saying, “Harry offered Bill a sandwich,” while moving the right hand, turned upward, toward an interlocutor, and stopping in between himself and the interlocutor. The gesture is understood via



The neuroscience of metaphoric gestures 

simulation. The prototypical offer is of a physical object directly transferred. But the offer has to be accepted. The sandwich is held out, and if the offer is accepted, the sandwich is taken. The gesture is iconic, tracing out the motion, with the hand up as if holding the thing offered. But the gesture is understood via simulation (usually unconscious) in which there is an imagined sandwich held out to someone in a position to take it. – Neural binding. Suppose you are looking at a red Volkswagen bug. The shape is perceived in one part of the brain and the color in another. Yet you see a single colored object, not a shape separated from its color. The brain mechanism that accomplished this is called neural binding. It requires connecting “binding circuitry.” Our best current theory is that neural binding is accomplished via synchronized firing, accomplished by the binding circuitry. Whenever two things are understood as the same thing, neural binding is happening. Thus, the offering gesture – choreographed and perceived via the mirror neurons in the pre-motor cortex – is neurally bound to the much richer imagined simulation, which involves the activation of some of the mirror neurons in the same area. The moral: Iconic gestures are always understood via neural binding to an imagined simulation (which may be unconscious). – Convergence zones. There are physical structures in the brain where separate neural pathways converge. They are called “convergence zones.” They have two functions: They integrate incoming signals from different brain regions, via neural binding. They also pass signals in the opposite direction, activating neurons in other brain regions and neurally binding them together. Take the problem of face recognition. I show you a picture of Marilyn Monroe and you say “Marilyn Monroe.” What is involved? The picture activates groups of neurons in many regions: one for a face template, one for eye shapes, one for hair-style shape, one for color, and so on. A Face-zone has to integrate all the incoming signals, binding them into a coherent whole. Then there is a Person-zone that integrates information about faces and names. The Face-zone activation for Marilyn’s face activates a Person-zone, which activates the name in the zone for phonology. If I give you the name and ask you to imagine the face, the opposite activation-flow occurs. Mirror neurons form convergence zones – linking perceptual information in the parietal cortex with pure motor choreography in the pre-motor cortex. The motor choreography forms a convergence zone linking the various regions in the primary motor cortex, which actually controls bodily movement. When you are making the offering gesture, the simulation of the offering as a whole activates some of the arm-movement neurons in the mirror neurons (characterizing the arm movement in the imagined simulation), which in turn provide

 George Lakoff

activation to the pre-motor neurons, which send activation to the neurons in the motor cortex, which mover the arm in just the right way. When we perceive the gesture, the reverse process occurs, which activation from perception going through all the linked convergence zones to ultimately activate a simulation, in which the gesture makes sense. – X-schemas. Srini Narayanan (1997) discovered the structure of the general neural circuitry that carries out motor programs: it has a single structure with variations. He then showed that the same neural structure can compute the semantics of linguistic aspect in all the world’s languages – the structure of all events and actions as we understand them. These are called “executing schemas” – or X-schemas for short. They can be combined in various ways – one embedded in another via neural binding of a whole X-schema to a role in another X-schema. The general X-schema has initial and final states, initial and final actions, central actions, a check to see if a purpose has been met, an iteration (a loop back to the central state), and a post-final “consequence state” which characterizes significant consequences of actions. X-schemas apply to mirror neurons and so can structure both actions and perceptions of those actions. Each gesture has an initial and final state before and after the gesture, an initial and final action – the approach to and retreat from the gesture, and a central action: the “stroke” that constitutes the gesture. In the theory of neural computation, the stroke is an action characterized independently as an X-schema that is bound to the central action of the overall gesture X-schema. – Frames. These are familiar from Fillmore’s (1982 and elsewhere) work on frame semantics. Each frame has a collection of semantic roles with properties of the roles and relations between them. In addition, frames may have scenarios, that is, X-schemas that can impose a linear structure on the frame. The roles, properties, relations, and scenarios constitute the frame elements. – Functional clusters, nodes, and control nodes. A functional cluster is a group of between dozens and hundreds of neurons that are jointly activated when an action is carried out (or a perception is perceived). The neurons in the function are connected by circuitry that allows for the X-schemas and bindings needed for the action to be carried out. The same neurons may play a role in various different functional clusters that control different functions. Such a functional cluster is called a node in the theory of neural computation. The frame elements in a frame are brought together by a control node that integrates and controls the activity of the frame elements. When the control



The neuroscience of metaphoric gestures 

node is active, all the frame elements are active. If enough of the frame elements are active, the control node will become active, which will activate the entire frame. Each frame contains the minimal information needed to activate a class of simulations, which differ depending on context, but which each fit the informational structure of the frame. Thus, there is a frame for taking a drink, which fits different kinds of drinks in different kinds of containers in different situations. The frame elements include functional clusters over the mirror neurons, which control actual drinking in different situations, and the perception of drinking in such situations. – Neural recruitment. Regularly recurrent activation of a collection of functional nodes, e.g., frame elements, results in spreading activation from those nodes, so that another cluster of neurons is activated when those elements are all activated. When such activation recurs, the synapses on those neurons change and the cluster is “recruited” as a control node over the elements. That is how frames are “conventionalized.” – The neural theory of metaphor. Complex metaphors, as we know, decompose into simpler primary metaphors. As Joseph Grady (1997) observed in his dissertation, the primaries arise when there is a regular correlation in experience. Thus, Affection is Warmth arises from experiencing affection while being held by one’s parents and simultaneously experiencing their body warmth. In the brain, emotional areas and areas that register temperature are in different places. When they are simultaneously activated over and over, activation spreads outwards from these areas, further and further each time until the activated circuitries “meet” and form a single circuit linking the two areas. That linking circuitry constitutes the metaphorical mapping, and a control node for the mapping is recruited. When the source and target are activated the control node is activated and activation flows between source and target. When the control node and the source are activated, then the target is activated and activation flows. When the target is activated some activation flows to the various control nodes for the metaphors with that target, making it easier to activate one or more of the metaphors. Narayanan has argued that the asymmetry of the metaphors arises from the fact that one area or the other is used more, and hence the neurons there will have more synaptic receptors and so will fire more strongly, inducing an asymmetry of flow. Thus, in More is Up, verticality is computed constantly, while issues of quantity arise less often. In Affection is Warmth, temperature is computed constantly, while affection isn’t. – Complex metaphors. Complex metaphors are formed by neural bindings of a number of primary metaphors and frames.

 George Lakoff

Now we can better understand the initial example of a metaphoric gesture that McNeill discovered – the Choosing is Weighing example. The subject was discussing a decision, remembering a decision she had made. She was simulating the decision process via the activation of the Decision Frame. That activation spread to metaphors for decision, including Choosing is Weighing, unconsciously activating that metaphor. Its source domain is weighing with a scale with two pans, alternately going up and down. That image is simulated as well, with a metaphorical link to the decision-making frame. That simulated scale image is simulated over the mirror neurons linking perception with action. The mirror neurons bind the pans of the scale in the simulated image to the hands, which carry out the same movement as the pans of the scale in the image! The activated circuitry of the Choosing is Weighing metaphor links the gesture bound to the image to the decision-making. And that is why the gesture is meaningful. – Pre-motor cortex and Broca’s area. Broca’s area contains mirror neurons linking speech and hearing. That area is adjacent to the mirror neuron portion of the pre-motor cortex. The lexicon is a convergence zone with control nodes linking conceptual structures (frames, metaphors, and so on) with phonemic sequences in Broca’s area. That is a pre-motor area choreographing activity in the primary motor cortex, where actual mouth movements are controlled. Broca’s is also a mirror neuron area linking pre-motor activity for speech with the auditory cortex. The main mirror neuron area in the pre-motor cortex for general action is right next to Broca’s area. Neural connections from the conceptual system (via the lexical convergence zone) that come to Broca’s area spread to the adjacent mirror neuron area, allowing gestures to have meaning, just as words do. This is my best attempt to put together the known pieces from neuroscience to give an account of what metaphoric gesture looks like from the brain’s perspective. Each piece is independently motivated. Those pieces from neuroscience and neural computation have been postulated with no thought given to metaphoric gesture. Together, however, they explain why metaphoric gestures exist, and make good neural sense of this utterly natural phenomenon.



The neuroscience of metaphoric gestures 

References Cienki, A. (1998). Metaphoric gestures and some of their relations to verbal metaphoric expressions. In J.-P. Koenig (Ed.), Discourse and cognition: Bridging the gap (pp. 189–204). Stanford, CA: Center for the Study of Language and Information. Fillmore, C. J. (1982). Frame semantics. In Linguistic Society of Korea (Ed.), Linguistics in the morning calm (pp. 111–138). Seoul: Hanshin. Grady, J. (1997). Foundations of meaning: Primary metaphors and primary scenes. Ph. D. dissertation, University of California at Berkeley. Narayanan, S. (1997). KARMA: Knowledge-based action representations for metaphor and aspect. Ph. D. dissertation, University of California at Berkeley. Reddy, M. (1979). The conduit metaphor: A case of frame conflict in our language about langauge. In A. Ortony (Ed.), Metaphor and thought (pp. 284–297). Cambridge: Cambridge University Press.

Metaphor and gesture Some implications for psychology Raymond W. Gibbs, Jr. 1. Metaphor and gesture: Some implications for psychology The new work on metaphor and gesture, as seen in the chapters in this volume, represents a major advance in the study of metaphor, and the psychology of human thought and communication. It comes as no surprise that people produce metaphoric gestures given their frequent use of metaphor in speech and writing. Nevertheless, the emerging literature on metaphor and gesture adds a significant source of evidence in support of the idea that abstract concepts are understood and communicated metaphorically. As several authors note in this volume, some cognitive scientists have raised skeptical questions about the possibility of metaphoric concepts because of the heavy reliance on linguistic evidence in making arguments for the existence of conceptual metaphor (e.g., Glucksberg, 2001; Murphy, 1996). These scholars argue that speaking about something metaphorically should not be seen as reliable data as to whether people actually think metaphorically, and urge that nonlinguistic evidence be gathered to support any claims about the metaphoric nature of ordinary thought. Indeed, the various kinds of research showing the prominence of metaphor in gesture in this volume not only provides nonlinguistic data skeptical scientists demand to support the possibility of metaphoric thought, but also shows how fundamental aspects of abstract thought are rooted in, and expressed by, recurring patterns of bodily experience and action. Most generally, the various explorations of metaphor in gesture are consistent with the “embodiment premise” as a research strategy in cognitive science (Gibbs, 2006a, p. 9): People’s subjective, felt experiences of their bodies in action provide part of the fundamental grounding for language and thought. Cognition is what occurs when the body engages the physical, cultural world and must be studied in terms of the dynamical interactions between people and the environment. Human language and thought emerge from recurring patterns of embodied activity that constrain ongoing intelligent behavior. We must not assume cognition to be purely internal,

 Raymond W. Gibbs, Jr

symbolic, computational, and disembodied, but seek out the gross and detailed ways that language and thought are inextricably shaped by embodied action.

Thus, the empirical study of metaphoric gesture, across languages and cultural contexts, aims to explore the “gross and detailed ways that language and thought are inextricably shaped by embodied action” (ibid.). Moreover, the empirical demonstrations of metaphoric gesture certainly reflect the embodied quality of abstract thought, but also illustrate how “cognition is what occurs when the body engages the physical, cultural world and must be studied in terms of the dynamical interaction between people and the environment” (ibid.). Under this view, metaphoric gestures are not simply the outward manifestation of an inner, symbolic, disembodied idea, but exhibit the dynamic creation, and recreation, of metaphoric thought in the bodily act of online communication. The following suggests a few directions for future research on metaphoric gesture in the broad discipline of Psychology that takes off from these themes. 2. Some needed empirical facts Psychologists like to think in quantitative ways and often, for better or worse, like to see numbers of some sort before they evaluate any behavioral phenomenon or theory. The growing literature on metaphor and gesture is primarily centered on linguistic research, much of which is concerned with the analysis of different case studies. But to more firmly establish the psychological reality of metaphoric gesture, there is a need for some basic information. What are the different kinds of metaphoric gestures? What are the specific, reliable criteria that distinguish a metaphoric gesture from one that is metonymic, or more generally, iconic? How often do people use metaphoric gestures compared to various other gestures? How often do metaphoric gestures accompany metaphoric and non-metaphoric speech? In what contexts are metaphoric gestures found most frequently? Do people gesture metaphorically when speaking with no observers, such as when talking on the telephone? Do children produce metaphoric gestures and what is the developmental progression in the use of these enactments? What are the cross-linguistic and cross-cultural similarities in the kind and amount of metaphoric gestures enacted? How often do people create novel metaphoric gestures? Are there regularities in the source domains recruited in conventional and novel metaphoric gestures? Do people mix their metaphoric gestures like they sometimes do with their verbal metaphors? This list of questions reflects some of the likely concerns that psychologists would have for empirical research on metaphoric gestures. Psychologists have



Metaphor and gesture 

frequently studied gesture in experimental situations, yet claims about the relationship between metaphoric gesture and thought demand more explicit criteria for defining, counting, and categorizing metaphoric gestures and the circumstances under which they are enacted, as well as how often they are produced. Although linguistic research offers some tentative answers to some of the above questions, studies in which a scholar selectively identifies certain metaphoric gestures for analysis and discussion must be complemented by larger empirical studies that provide comparative, quantitative data on metaphoric gesture production. 3. Relationship between speech and gesture The growing interest in gesture over the past twenty year has provided the basis for serious debates among psychologists on the relationship between speech and gesture. At present, there are several ways of characterizing the speech-gesture relationship (Gibbs, 2006a; Iverson & Thelen, 1999). First, speech and gesture are conceived as separate communicative systems, which occasionally become linked due to the cognitive demands associated with speech production (Butterworth & Hadar, 1989). For example, gesture serves to support speech production activities by compensating when speech is temporarily disrupted (e.g., by coughing) or when speakers are unable to put their thoughts into words. But gestures do not influence underlying speech production processes. A second view holds that there are deep cognitive linkages between speech and gesture, presumably located at the physiological encoding stage (i.e., the stage at which words’ forms must be accessed from lexical memory) (Krauss, 1998). Thus, gestures are especially useful when speakers experience difficulty retrieving words, because the production of gesture activates relevant spatio-dynamic features of the concept (i.e., thought) that a speaker had in mind. The link between speech and gesture is limited, then, to a particular stage of speech production. Third, speech and gesture are sometimes assumed to be communicative activities that are grounded in common thought processes (Iverson & Thelen, 1999; McNeill, 2005). Under this view, speech and gesture have a strong reciprocal relationship through the entire process of speech production, ranging from phonological encoding up through producing syntax, semantics, and discourse. Even though speech and gesture may communicate different aspects of people’s thoughts, the tight coupling of these activities suggests that any disruption in one (e.g., gesture) will have negative effects on the other (e.g., speech). Gesture appears to play a critical role in the conceptualizing and planning of messages. Research on metaphoric gestures may provide important evidence relevant to these theoretical possibilities. Of course, metaphoric gesture may be helpful to

 Raymond W. Gibbs, Jr

speakers when they are unable to convey their thoughts in words, and therefore may assist lexical retrieval, yet also reflect common metaphoric thought processes in the same way that many aspects of metaphoric speech does. If some metaphoric gestures reflect common metaphoric mappings, similar to those motivating different words and phrases, then producing these gestures should facilitate metaphoric speech production given these gestures’ fundamental role in conceptualization. Thus, gesturing a grasping motion with one hand may both reflect some natural conceptualization of the idea of a concept, but may also help a speaker verbally articulate the idea of “grasping a concept” as in “I just couldn’t grasp that concept.” The physical gesture of grasping, or attempting to grasp, which is related to the conceptual metaphors of ideas/concepts are physical objects and understanding is grasping, sets up the easy formulation of the conventional statement “I just couldn’t grasp that concept” as metaphorically meaning, “I just couldn’t understand that concept.” In this view, gestures that may be related to parts of conventional metaphoric concepts help people understand metaphoric messages that have embodied source domains. There are no experimental studies that explicitly show whether metaphoric gestures facilitate the formulation of metaphoric utterances. But there is psycholinguistic evidence demonstrating that when people make a hand gesture or body movement, such as a hand grasping motion, appropriate to the meaning of a metaphoric phrase, such as “grasp the concept,” they are subsequently quicker to comprehend the phrase than if they had previously made some other gesture (e.g., a pushing away gesture) (Gibbs, 2006b). Furthermore, people’s speeded understanding of metaphoric phrases is facilitated when they first simply imagined making the appropriate gesture before seeing the phrase. These experimental results suggest that engaging in body movements associated certain metaphoric ideas (i.e., an actual grasping motion related to the possibility that concepts can be metaphorically conceptualized as physical entities which can be grasped, fumbled, dropped, etc.) enhances the simulations that people create to form a metaphoric understanding of abstract notions related to these gestures. This same set of studies also shows the grasping gesture, to take one of many examples in the experiments, does not simply activate a literal word “grasp” as people associated a variety of words with this action. But the gesture is metaphorically meaningful precisely because people think, once more, of concepts as things that can be grasped. People therefore appear to create embodied simulations of speakers’ messages that involve moment-by-moment “what must it be like” processes that make use of ongoing tactile-kinesthetic experiences such as grasping gestures. Seeing metaphoric gestures may enhance the creation of simulations when people encounter language that is abstract, or refer to actions that are physically impossible to perform (i.e, “grasp the concept,” “throw out an idea”).



Metaphor and gesture 

Does observing a metaphoric gesture make a speaker’s accompanying metaphoric utterance easier to understand and remember? There is good reason to suspect that this may be the case. First, psychological studies indicate that the accuracy of hearing spoken sentences is twice as high when the sentences are presented combined with gestures than when they are not (cf. Gibbs, 1999). Second, research in neuropsychology demonstrates that observing another person’s simple actions activates relevant motor areas of the brain (i.e., “mirror neurons”), as if the observer were doing the very action being seen (Decety & Grezes, 1999). In an extended way, observing a metaphoric gesture may activate appropriate motor regions of the brain that could be linked to the embodied source domains of many metaphoric concepts (Gallese & Lakoff, 2005), which may also facilitate the embodied simulation launched to comprehend a speaker’s communicative message when saying something like “I just couldn’t grasp that concept.” Empirical research is needed to study this possibility in more detail. Of course, the temporal coordination between the production of a gesture and the production of a specific lexical item or phrase will also need to be systematically explored to see under what conditions observing metaphoric gestures facilitates speech understanding, particularly compared to cases where literal gestures accompanying literal speech (e.g., gesture before words, gesture simultaneous to words, gesture immediately after words). Furthermore, the differences between metaphoric and metonymic gestures must also be studied. One possibility here is that metonymic gestures facilitate both production and understanding of individual lexical items, while metaphoric gestures best enhance people’s understanding of abstract concepts that are not as conventionally associated with individual lexical items. 4. Analyzability of metaphoric gestures Gesture scholars differ as to whether metaphoric gestures are compositional or not (Calbris, 1990; Kendon & Versante, 2003; McNeill, 1992; Müller, 1998). However, psycholinguistic studies of verbal metaphors suggest that even highly conventional kinds of metaphors, such as idioms motivated by metaphor, are analyzable to varying extents (Gibbs, 1994). Listeners comprehend metaphoric idioms (e.g., “John popped the question to Mary”), for example, through analysis of how each word makes independent contributions to the phrase’s overall figurative meaning, although this analysis links parts of metaphoric expressions with metaphoric, and not literal, meanings. This work implies that even the most clichéd metaphoric phrases are not understood through simple retrieval of their meanings stored in a phrasal, mental lexicon.

 Raymond W. Gibbs, Jr

Are metaphoric gestures understood as wholes, through retrieval of their meanings from some gestural lexicon or might these be interpreted compositionally with parts of each gesture referring meaningfully to parts of their overall metaphoric messages? One way of answering this question is to present observers with metaphoric gestures and their typical metaphoric meanings and ask to what extent different parts of the temporal articulation of each gesture are associated with parts of its metaphoric meaning. More complex studies could present gestures with appropriate metaphoric speech and ask observers/listeners to judge, more once, the extent to which aspects of the gesture correspond with parts of the metaphoric speech (both in terms of the words spoken and the overall figurative meaning conveyed). Similar to conventional metaphoric speech, people may alter parts of their metaphoric gestures to convey metaphoric meanings in novel ways. Novel gestures need not be completely new, but may be slightly creative instantiations of conventional articulations, which possibly communicate additional cognitive effects, compared to typical metaphoric gestures. The extent to which people gesture in these slightly novel ways, and whether people actually infer different cognitive effects, or meanings, when observing these novel instantiations, are also empirical questions worthy of empirical study. This last issue raises a more general challenge for future psychological studies on metaphor and gesture. Both linguists and psychologists studying metaphoric language understanding typically describe what is understood in rather crude terms, often simply noting that the metaphoric meaning of some statement can be roughly paraphrased in a manner that is different from the meaning of a literal expression. Yet a closer look at what people really infer when hearing metaphoric language reveals that even highly conventional expressions, like metaphoric idioms, are understood as communicating complex figurative meanings that are not easily paraphrased (Gibbs, 1994). Novel metaphoric utterances often express even greater degrees of aesthetic and interpersonal meaning. One challenge for metaphor scholars of both speech and gesture is to empirically examine the kinds of meanings, or cognitive effects, people draw from seeing metaphoric gestures, both with and without accompanying speech. As mentioned earlier, metaphoric gestures with speech are likely not just communicating redundant information, but should, in many cases, express something different. Determining what those meanings are, and the conditions under which they are best inferred, should be a significant topic of study in both linguistics and psychology.



Metaphor and gesture 

5. Coordination between participants in talk/gesture exchanges It is important to note that observing a metaphoric gesture is not necessarily just an added piece of information to ease understanding of a speaker’s linguistic, metaphoric message. Instead, a person’s metaphoric gesturing is part of the ensemble of meaning that is communicated by the speaker’s overall bodily actions, which can include voice, hands, and other parts of the body in action. More accurately, a metaphoric gesture is part of the dynamic interaction between speakers that enables interpersonal coordination and intersubjectivity. One concern is whether metaphoric gestures are indeed understood as intentional communications. People appear to have two different capacities for displaying information (Gibbs, 1999). Information “given” is information intentionally emitted by a person and recognized by another in the manner intended by the actor. On the other hand, information “given-off ” is information interpreted for meaning by another person even if it had not been intended to convey that meaning. Thus, when Jack spies on Sally, he can draw various meaningful inferences about what Sally might be doing even though Sally’s gestures and actions are only providing information that is given-off, not intentionally given to Jack. Studies show that people can differentiate, even without speech, between gestures that are intended to convey meaning and gestures that only seem to emphasize what a speaker is saying (Feyereisen, van de Wiele, & Dubois, 1988). This conclusion may be extended to suggest that metaphoric gestures, facial expressions, and so on, are also specifically produced to be understood as part of a person’s overall communicative intentions and must be recognized as such for successful interpersonal interactions to occur (see Müller, this volume). Although there is no experimental evidence to support the above conjecture, as of yet, there is a growing literature showing that people appear to specifically design their gestures for intentional purposes. Studies show, for example, that during causal conversation between first and second language speakers of Japanese, first language speakers will often complete statements using gestures when they believe that their addressees, the second language users, may not be able to understand the words spoken (Mori & Hayaski, 2006). Second language users observed these “embodied completions” and showed their understanding of what was intended by continuing the talk and focusing on the meaning of what was gestured, and not just the words spoken. It appears, then, that speakers’ awareness of their co-participants’ linguistic background fosters the use of gestures to enhance intersubjectivity during talk exchanges. One possibility is that speakers will employ metaphoric gestures particularly in cases where there are concerns over whether the addressee will be able to understand a specific metaphoric statement, with these metaphoric gestures sometimes serving as embodied completions of what speakers were attempting to verbally articulate. Thus, metaphoric gestures surely reflect

 Raymond W. Gibbs, Jr

“growth points” in which a speaker dynamically reconfigures what is said and gesturally enacted to meet the demands of the social context (McNeill, 2005). More generally, metaphoric gestures are not just part of the meaning expressed and understood by conversationalists, but a major part of how speakers and listeners ground their joint understandings. For instance, a person may gesture with palms up, as if offering, to suggest that some message being communicated by another is obvious. Many other instances of face-to-face interaction are structured by the conduit metaphor such as when a listener uses subtle hand and finger movements to suggest that the speaker should continue speaking as if the message is a physical entity being transferred from one person to another. When discussing difficult topics, speakers will sometimes rapidly wave their hands to indicate the complexity of what they are trying to communicate, or wave their hands in front of their faces as if to ward off the multiple messages being rapidly hurled at them. Speakers may place their arms out beside themselves and act as if they are pushing alternately against different forces to indicate a struggle to maintain balance in instances where they are experiencing difficulty with the burden of what is being communicated. People will even display metaphoric facial expressions to communicate their metaphoric reaction to a speaker’s message, such as when a listener displays a disgust face to suggest displeasure with what a speaker is saying as if the speaker were presenting rotten food in front of the listener’s nose. These observations on the prominence of metaphoric gestures in face-to-face communications are indirectly supported by research on human and computerized embodied agent interactions, where there are major attempts to get computerized figures to gesture appropriately during talk with humans (Cassell et al., 2001; Yan, 2000). There is, as yet, not much explicit attention paid to the appropriate production of metaphoric gestures by embodied computerized agents, although some of the protocols available of human-computer verbal interactions suggest that some agents may be doing so already to some degree (see Cassell et al., 2001). There may be important positive benefits of incorporating appropriate metaphoric gestures in human-computer interaction, a possibility that is also ripe for future empirical study in psychology and cognitive science. Finally, the frequently repeated wisdom in social psychology that people’s body actions are often more truthful than their words spoken raises one interesting question about metaphoric gestures. Might people use metaphoric gestures more often when they are trying to conceal something, or avoid responsibility for the impact of what they are communicating? A recent study suggests this may be the case (Letiza et al., 2006). Hand gestures were experimentally studied during truth telling and deception, and in situations with either weak or strong suspicion. Participants in this study were interviewed twice about the possession of an object. In one interview they were asked to lie and in the other asked to tell the truth



Metaphor and gesture 

(veracity factor). In both conditions, participants were then accused of lying by the interviewer and asked to repeat their account a second time (suspicion factor). Interestingly, lying was associated with a decrease in deictic gestures, but with a significant increase in metaphoric gestures. These findings suggests that the increased use of metaphoric gestures, at least in some situations, compared to some previously established baseline, may indicate a speaker’s attempt to deceive others. Whether a conversationalist can detect variations in the amount and kind (i.e., metaphoric) of gesturing in real-world contexts is not yet established, but this topic is clearly worthy of additional psychological research with implications for theories of interpersonal communication. 6. Moving beyond the hands Most of the work on metaphoric gesture has focused on how the hands work to communicate metaphoric ideas. But, as noted above, people move other body parts, and even their whole bodies, to express metaphoric messages. Even further, people sometimes perform whole body actions that clearly represent metaphoric ideas. During the World Cup football/soccer matches in 1994, players who had just scored a goal were seen mimicking a person pulling a gun out of a holster, shooting the gun forward, and replacing it back in a holster, to convey the message that scoring a goal was metaphorically like slaying an opponent in a gun battle or duel. Another European football player once, after scoring a goal, ran the white end-line, kneeled down and pretended to snort the white substance there, as if he were ingesting a drug like cocaine. This action is perhaps doubly metaphoric in that not only is scoring a football goal metaphorically like experiencing the rush of ingesting a powerful drug, but the idea of “scoring” in football is linked with “scoring” some drug. Other simple, widely known metaphoric actions include cutting one’s throat and hanging oneself by a rope, both of which are especially salient gestures because they refer, like the previous examples, to taboo topics (i.e., killing someone, doing illegal drugs, killing oneself), which is perhaps why these gestures can be enacted without one verbally saying what one really means. Metaphoric gestures like these are parodies of well known body routines and convey a rich set of meanings that would be impossible to communicate via words, especially in cases where the audience is unlikely to hear any words spoken, as in a large, noisy football stadium.

 Raymond W. Gibbs, Jr

7. Conclusion Making and understanding metaphoric gestures are psychological events, within cultural contexts, and must be studied in a systematic way that acknowledges several major factors that likely shape production and understanding of these actions. These include (1) the participants (e.g., their background abilities, interests, beliefs, motivations), (2) the participants’ immediate discourse goals (e.g., fill in for missing words, communicate something different from what was spoken, add redundancy to the spoken message, be provocative), (3) the participants’ criteria for production and understanding (e.g., to express an exact vs. approximate meaning, to generally understand, to remember how something was expressed), and (4) the stimuli (e.g., conventional or novel metaphoric gesture, whether it accompanies speech and the temporal relation with that speech). Metaphoric gesture production and understanding will depend on the complex interaction of these different factors, which should cause psychologists, and others, to be cautious in the generalizations they make from the data collected and explained. But the ground-breaking body of work seen in this volume testifies to some of the many empirical possibilities on human thought and communication processes that cry out for new empirical study. At the very least, the research described in this volume suggests the increasing possibility for fruitful interdisciplinary collaboration between linguistics and psychology on the topic of metaphor. References Butterworth, B. & U. Hadar (1989). Gesture, speech and computational stages: A reply to McNeill. Psychological Review, 96, 168–174. Calbris, G. (1990). The semiotics of French gestures. Bloomington, IN: Indiana University Press. Cassell, J., T. Bickmore, L. Campbell, H. Vihjammsseon, & H. Yan (2001). More than just a pretty face. Conversational protocols and the affordances of embodiment. Knowledge-Based Systems, 14, 55–64. Decety, J. & J. Grezes (1999). Neural mechanisms subserving the perception of human action. Trends in Cognitive Sciences, 3, 172–178. Feyereisen, P., M. van de Wiele, & F. Dubois (1988). The meaning of gestures: What can be understood without speech. Cahiers de Psychologie Cognitives, 8, 3–25. Gallese, V. & G. Lakoff (2005). The brain’s concepts. The role of sensory-motor system in conceptual knowledge. Cognitive Neuropsychology, 22, 455–479. Gibbs, R. W. (1994). The poetics of mind: Figurative thought, language, and understanding. New York: Cambridge University Press. Gibbs, R. W. (1999). Intentions in the experience of meaning. New York: Cambridge University Press.



Metaphor and gesture  Gibbs, R. W. (2006a). Embodiment and cognitive science. New York: Cambridge University Press. Gibbs, R. W. (2006b). Embodied simulation in metaphor interpretation. Mind & Language, 21, 434–458. Glucksberg, S. (2001). Understanding figurative language: From metaphor to idioms. New York: Oxford University Press. Iverson, J. & E. Thelen (1999). Hand, mouth, and brain: The dynamic emergence of speech and gesture. Journal of Consciousness Studies, 11–12, 19–40. Kendon, A. & L. Versante (2003). Pointing by hand in Neapolitan. In S. Kita (Ed.), Pointing: Where language culture and cognition meet (pp. 109–137). Hillsdale, N.J.: Lawrence Erlbaum. Krauss, R. (1998). Why do we gesture when we speak? Current Directions in Psychological Science, 7, 54–60. Letizia C., F. Maricchiolo, M. Bonaiuto, A. Vrij, & S. Mann (2006). The impact of deception and suspicion on different hand movements. Journal of Nonverbal Behavior, 30, 1–9. McNeill, D. (1992). Hand and mind. Chicago: University of Chicago Press. McNeill, D. (2005). Gesture and thought. Chicago: University of Chicago Press. Mori, J. & M. Hayashi (2006). The achievement of intersubjectivity through embodied completions: a study of interaction between first and second language speakers. Applied Linguistics, 27, 195–219. Murphy, G. (1996). On metaphoric representations. Cognition, 60, 173–204. Müller, C. (1998). Redebegleitende Gesten: Kulturgeschichte – Theorie – Sprachvergleich. Berlin: Berlin Verlag Arno Spitz. Yan, H. (2000). Paired speech and gesture generation in embodied conversational agents. MA thesis, MIT Media Laboratory, Cambridge, MA.

Index A abstract deixis  17 abstract referential gesture  8, 9 abstraction  96–98, 119, 143, 149, 262 action schema  144, 262 activation of metaphoricity  10, 219, 221, 228, 231, 234, 235, 237, 238, 240 American Sign Language (ASL)  25, 108, 154, 160, 224, 245, 274, 275 analogical connection  29, 30 analogical sign  28, 29 anchored blend  59, 61, 67, 85, 87 asymmetry  44, 46, 50, 287 attention  1, 5, 7, 10, 22, 40, 63, 68, 69, 73, 118, 132, 168, 187, 193, 199, 222, 232, 238–240, 249, 255, 277, 278, 298 attentional focus  239, 240 axis back-to-front  37, 48 dissymmetrical  52 of reading and writing  28 of symmetry  30, 36, 38, 50, 52 of walking  38 sagittal  44, 48, 51 spatio-temporal  27, 38, 52 symbolic  27, 49–51 transverse  27–29, 37, 42, 44, 46–51 upward  49 vertical  47, 48, 53 Aymara  15, 48, 53, 114, 153, 164, 170 B balance schema  46 beats  7, 8, 14, 197, 259, 283 blend  11, 56–62, 64–75, 77–83, 86 See also anchored blend, blended space, conceptual blending, conceptual integration, metaphoric blend,

online integration blended space  57–61, 64, 65, 83 C calculus  93, 96, 97, 102, 109, 112, 281 Cartesian  104, 269, 271 catchment  2, 160–167, 171–173, 180, 183, 185, 187–189, 278 causal reasoning  255 Chomsky, N.  18, 94 cognitive ethnography  56, 62, 63, 88 cognitive linguistic(s) (CL)  16, 22–25, 53, 56, 87, 88, 93–95, 97, 101, 105, 111–113, 116, 127, 150–152, 154, 220, 242–244, 249, 266, 267, 274 cognitive science  94 cognitive semantic(s)  114, 151, 170, 224, 249, 264 conceptual blend(ing)  56–65, 74, 88, 97, 154, 250 conceptual integration  57–59, 64, 66, 81, 83 conceptual mapping  5, 9, 55–63, 66, 68, 75–77, 83–87 conceptual metaphor  1–3, 5, 6, 9–11, 13–16, 18–20, 55–60, 74, 76, 93, 95–98, 102–106, 112, 114–116, 118, 122, 142, 145, 146, 148, 168, 196, 202, 204, 209, 214, 220, 221, 263, 267, 275, 283, 291, 294 conceptual metaphor theory  1, 2, 6, 16, 18, 98, 115, 196, 202 conceptual model  55, 60–62, 64, 65, 68, 74, 76, 77, 81, 84–86 conceptual structure  56, 117, 144, 146, 147, 220, 288 concrete referential gesture  8, 9, 21 conduit metaphor(ic model)  7, 25, 151, 153, 168, 170, 215, 255, 259, 289, 298

conduits  22 conventional linguistic unit  250 conventionality  6, 9, 10, 124, 129, 196, 204, 212–214, 267 convergence zone  285, 286, 288 Conversation Analysis (CA)  23–25, 112, 151, 224, 241, 244, 251, 265–270, 289 Cora  251 co-speech gesture  115, 116, 119, 123, 148, 150, 152, 243. See also co-verbal gesture Courant, R., 98, 99, 112 co-verbal gesture  10, 145, 243 See also co-speech gesture cross-domain mapping  1, 5, 15, 55, 56, 59 See also metaphoric mapping cultural analysis  253, 257 D Danish  268 dead metaphor  98, 105, 111 deconstruction  2, 171, 172, 175–177, 179, 185, 188, 280, 282 deictic  35, 127, 129, 197, 202, 236, 237, 239, 299 deictic gesture  128, 197, 299 deictics  7, 128, 283 deixis  24, 38, 53, 129, 148, 150, 152, 202 demonstrative  128, 250 discourse gesture  8 dynamic  2, 20, 97, 99, 100, 102–104, 107, 111–113, 122, 130, 140, 143, 147, 149, 153, 167, 172, 219, 222, 223, 233, 238–241, 244, 253, 267, 278, 280, 292, 293, 297, 301 dimension  168, 244 E emblem  6, 8, 10, 22, 25, 130, 131, 169, 195–197, 199, 200, 203, 204, 213, 214, 244, 275, 279

 Metaphor and Gesture embodied cognition  16, 102 embodiment  114, 116, 149, 242, 291, 300 emergent grammar  120, 140, 141 property  60 structure  57, 58, 64, 65 English  7, 10, 11, 13, 19, 105, 113, 119, 135, 153, 164, 167, 172 Ethnomethodology (EM)  265–270 Euler, L., 101 expected metaphor  19, 155, 156, 158, 162, 165, 167–169. See also unexpected metaphor eye-gesture  73 F fictive motion  97, 103–106, 111 focal attention  219, 236–238, 240 focus of attention  119, 237, 239, 240 foregrounding  219, 222, 234, 235, 237–240 French  1, 23, 27, 29, 31, 33, 34, 41, 45, 46, 150, 172, 173, 175, 192, 300 Fused bodies  269, 272 G gaze  10, 18, 20, 44, 61, 71–73, 76, 121, 225, 235–239, 244, 261 genre  3, 7, 21, 119, 144, 166, 171, 172, 176, 264 German  1, 9, 223, 226, 228, 232, 263, 272 gestalt  121, 143, 144, 261 gesticulation  7, 173, 197, 277, 279, 280 gestural accents  180 continuity  255 metaphor  5, 183, 185, 197, 228, 231–233, 236–238, 254, 278–281 See also metaphoric gesture metaphoric  5 spaces  255–257 stroke  175 gesture family  153, 215, 244, 259 forms see asymmetry, eye-gesture, hand shape, left hand,

okay, palm up, pantomime, pointing gesture, precision grip, right hand, standard(s) of form, symmetry, thumbs up functions see abstract referential gesture, beats, concrete referential gesture, deictic gesture, deictics, discourse gesture, iconic gesture, iconics, metaphoric gesture, metaphorics, performative gesture, pragmatic gesture, presenting gesture, referential gesture, representational gesture phases see poststroke hold, prestroke hold, gesture stroke, stroke semiosis  199 space  10, 20, 121, 122, 135, 139, 151, 152, 225, 237 stroke  6, 63, 68, 158, 159, 161 Greek  105 growth point  2, 157, 158, 162, 164–167, 171–173, 180, 185, 188, 189, 278, 282, 298 H hand shape  10, 15, 19, 107–110, 119, 121, 122, 144 Heidegger, M., 175, 280 hermeneutic  3 I iconic gesture  21, 85, 134, 155, 164, 197, 277, 279, 284 See also iconics mode  2, 116, 117, 119, 127, 132, 142, 148 signs  8, 125, 135 iconicity  115, 117, 118, 122, 132, 134–136, 139–141, 148, 154, 157, 158, 165, 167, 175, 197, 274, 275, 277 iconics  7, 117, 125 See also iconic gesture idioms  228, 295, 296, 301 illustrator  173, 279 Ilokano  261 image schema  23, 27, 52, 119, 122,

127, 143–146, 148, 150, 151 indexical  68, 85, 115, 127, 129, 142, 148 modes  115, 148 signs  127 inferential organization  93, 95–98, 101–103, 105, 111 structure  57, 58, 59, 75, 101, 105, 109, 111 infinity  96–98, 101–103, 109, 111 interpersonal  122, 240, 296, 297, 299 intersubjective  60, 266 intersubjectivity  297, 301 intonation unit  12 J Jakobson, R., 124, 127, 134, 135, 145, 149, 151, 243 Japanese  150, 169, 175, 191, 297 Jospin, L. (LJ)  27, 29, 31, 32, 34, 35, 37, 38, 40, 41, 45, 50 Jousse, M.  53, 113 K Kendon’s continuum  197 Kepler, J., 101 L Laban, R., 6, 24 Latin  10, 46, 105, 187, 262, 263 Le Monde des idées  171, 176 left hand  12, 13, 28–37, 40–42, 45, 50–52, 110, 113, 121, 122, 128, 140, 232 left to right  15, 27, 28, 37–42, 48, 51, 52, 65, 67, 136, 275 left/right  15, 17, 37, 53 left-hand  33, 35, 36, 39, 45, 50 left-hander  33, 45, 50 Leibniz, G.  102 lexical affiliate  158 limits  93, 96, 98, 100, 102, 103, 105, 111, 112, 168, 175, 183, 250 linguistic system  251, 275 locative prefix  251 M mapping  9, 17, 55–59, 62–71, 73, 74, 76–78, 80, 81, 84–87, 99, 100, 102, 116, 209, 212, 259, 287 material anchor  61, 65, 68–71, 73, 80 mathematics  2, 62, 74, 88, 93,



Index  95–100, 102, 104, 105, 107–114, 147, 152, 153, 275 mental constructions  250 mental space  16, 17, 25, 57, 58, 60, 63, 65, 75, 154, 165, 250, 281 metaphor in interaction  267 metaphoric blend  2, 58, 59, 74, 80, 83, 84, 87 metaphoric gesture  2, 3, 5, 7, 8, 14, 17, 20–22, 25, 27, 40, 86, 117, 118, 136, 142, 146, 148, 154–157, 168–170, 195–197, 201, 203, 204, 226, 242, 244, 249–251, 253, 254, 256, 257, 275, 280, 283, 284, 288, 291–300 see also gestural metaphor, metaphorics metaphoric(al) mapping  10, 11, 13, 19, 56, 74, 77, 81, 85, 95, 116, 118, 139, 154, 202, 242, 287, 294 See also cross-domain mapping. Individual mappings: a process is motion along a path from source to goal  76 anger is heat  220 abstract is concrete  16, 17 being subject to control is down  139 categories are containers  127, 146 competition is physical struggle  58 concepts are physical objects  294 conceptual structure is physical structure  146 constituents are contents  146 degree is distance  12 future is right  15 good is up  3, 11, 19 having control is up  139 ideas are objects  127, 134, 202–204 important is big  57 knowing/understanding is seeing  22, 84 life is a journey  10 love is a journey  220 more serious is higher

up  12 numbers are locations in space  105 past is left  15 power is up  139 purposes are destinations  60 theories are buildings  118, 220 time is space  80, 202, 204 understanding is grasping  294 metaphorical projection  46, 115, 117, 119, 144, 145 metaphoricity  5, 8–11, 14, 17, 20, 23, 25, 53, 114, 116, 118, 153, 158, 161, 162, 166, 167, 215, 219, 221, 222, 234–241, 259, 267 metaphorics  7, 117, 139 See also metaphoric gesture metaphors for communication  19, 151, 283 conceiving  103 decision  288 deconstruction  180 inaccessibility  155 interpersonal relations  256 language  153 marriage  253, 254 speech and thought  7 metonymic modes  119, 148 metonymy  52, 97, 102, 116, 145, 148, 151–153, 243, 244, 274 See also synecdoche mirror neuron  284–288, 295 multimodal  22, 24, 25, 55, 62, 87, 115, 117–119, 141, 145, 150, 152, 171, 172, 188, 189, 219, 224, 225, 236, 237, 243, 244, 281 N narrative  2, 7, 17, 21, 22, 119, 154, 161, 163, 203, 204, 223, 226, 255, 262 naturalness judgment of  205, 211 rating of  195 scale of  196 see also unnaturalness neural binding  285­287 neuroimaging  281 neurons  271, 284– 288 neuroscience  93, 95, 275, 283, 284, 288

Newton, I., 102 Nietzsche, F., 175, 280 O okay (OK)  6, 130, 169, 196, 199, 203–205, 207–214 online  56, 59, 219, 222, 225, 239, 240, 249, 292 (cognitive) processes  225, 242, 249 integration  56 meaning construction  59 ontological metaphor  14, 16, 17 P palm up  7, 8, 10, 12, 14, 19, 155, 168, 203, 260, 261, 298 open hand (PUOH)  12, 14 pantomime  197, 284 Peirce, C.S.  2, 115, 117–119, 122, 123, 124, 127, 128, 130, 132, 135, 139–143, 148–150, 152–154, 277 performative gesture  8, 117 Plenel, E.  171, 172, 176, 191 pointing  17, 22, 28, 31, 39, 47, 48, 51, 65, 71–73, 75–78, 83–86, 107–110, 121, 122, 128, 136, 150, 160, 161, 164, 203, 231, 232, 236, 250, 279 gesture  121, 122, 128, 232, 236, 250 poststroke hold  159, 162 Pragglejaz group  18 pragmatic function  118, 203, 204 pragmatic gesture  203, 244, 259, 260, 263, 264 precision grip  121, 226 presenting gesture  196, 201, 203–214 prestroke hold  159, 162 primary metaphor  287 proprioception  264 prosody  10, 152, 188, 191, 250 psychological anthropologists  253 psychological reality  16, 93, 95, 98, 105, 106, 275, 292 R reference point  129, 145 referential function  203 referential gesture  8, 9, 21, 117, 119, 120, 125, 129 representational gesture  173, 196–201, 214

 Metaphor and Gesture right hand  12, 22, 28, 29, 32–42, 44–46, 50–52, 107–110, 121, 122, 128, 137, 140, 164, 232, 236, 284 right to left  28 right-handed  29, 32, 34, 35, 45, 46 right-hander  29, 34, 35, 46, 50 Robbins, H., 98, 99, 112 Russian  46, 53, 151 S schematicity  250, 251 semiosis  123, 143, 158, 168, 197–200 semiotic  1, 2, 3, 5, 115–119, 122–125, 128, 139, 142–146, 148–150, 152, 153, 196–198, 224, 242, 243, 277 theory  2, 148 semiotics  23, 115, 117, 119, 122, 123, 142, 148, 150, 277, 300 sensory-motor  29, 38, 300 sign(ed) language  15, 16, 130, 148, 154, 199, 243, 250, 273–275 simulation  242, 284–287, 294, 295, 301

standard(s) of form  168, 169, 196, 205, 213 static dimension  166 stress accent  12 stroke  6, 12, 69, 107, 133, 158–162, 175, 286 phase  6, 12 symbolic gesture  1 symmetry  28, 29, 31, 35–37, 39, 42–46, 49–51, 263 synecdoche  116, 118. See also metonymy T thinking for speaking  17, 21, 25, 153, 170, 240, 243 three-item enumeration  37, 45 thumbs up  6, 8, 10, 11, 130, 199 topology  61, 96, 144 trace(d)  6, 67–69, 71, 73, 85, 119, 121, 122, 125, 143, 144, 187, 204, 210, 240, 274 tracing  68, 69, 71, 79, 85, 118, 122, 135, 137, 143, 197, 283, 285 turns  66, 93, 133, 168, 203, 224, 236, 259 at talk  17, 224

U unexpected metaphor  155, 157, 158, 160, 164–169. See also expected metaphor unnaturalness  210, 212 V verbal metaphor  13, 16, 20, 23, 27, 28, 112, 150, 166, 169, 214, 226, 228, 231, 232, 235–238, 241, 289, 292, 295 verbal metaphoric expression  13, 16, 23, 112, 150, 169, 214, 232, 235, 236, 241, 289 verbo-gestural metaphor  227, 228, 231–233, 238 Vygotsky, L., 157, 167, 170 W Wundt, W., 1, 4

In the series Gesture Studies the following titles have been published thus far or are scheduled for publication: 3 2 1

Cienki, Alan and Cornelia Müller (eds.): Metaphor and Gesture. 2008. ix, 306 pp. Streeck, Jürgen: Gesturecraft. Manufacturing understanding. Forthcoming Duncan, Susan D., Justine Cassell and Elena T. Levy (eds.): Gesture and the Dynamic Dimension of Language. Essays in honor of David McNeill. 2007. vi, 328 pp.