Handbook of Visual Analysis

HANDBOOKOF VISUAL ANALYSIS

EDITEDBY THEOVAN LEEUWENAND CAREYJEWITT

OSAGE

Los Angeles. London. New Delhi. Singapore

@Theo van Leeuwenand CareyJewitt2001 Compilationand Introduction @ PhilipBell2001Chapter2 @ MalcolmCollier2001 Chapter3 @ MartinListerand Liz Wells 2001 Chapter4 @Theo van Leeuwen2001 Chapter5 @ GertraudDiem-Wille2001 Chapter6 @ CareyJewittand RumikoOyama2001 Chapter7 @ CharlesGoodwin2001 Chapter8 @ Rick ledema2001 Chapter9 Firstpublished2001 Reprinted2002, 2003, 2004, 2006, 2007, 2008 Apart from any fair dealingfor the purposesof researchor privatestudy, or criticismor review, as permittedunder the Copyright,Designs and Patents Act, 1988, this publication may be reproduced, stored or transmittedin any form, or by any means,only with the priorpermissionin writing of the publishers,or in the case of reprographicreproduction,in accordancewith the terms of licencesissued by the CopyrightLicensing Agency. Inquiriesconcerningreproductionoutsidethose terms shouldbe sentto the publishers.

SAGE PublicationsLtd 1 Oliver'sYard 55 City Road L o n d o nE C l Y 1 S P Inc SAGE Publications 2455 TellerRoad ThousandOaks California91320 SAGE PublicationsIndiaP\i/t.Ltd B1/11MohanCooperativeIndustrialArea MathuraRoad.New Delhi 110 044 lndia Asia-PacificPte Ltd SAGE Publications 33 Pekin Street#02-01 Far EastSquare Singapore048763 British Library Cataloguing in Publication data A cataloguerecordfor this book is availablefrom the BritishLibrary lsBN 978-0-7619-6476-6 (pbk) ISBN978-0-7619-6477-3 Typesetby Keystroke,JacarandaLodge,Wolverhampton Printedand boundin Great Britainby CromwellPressLimited,Trowbridge,Wiltshire

Contents List offigures List of tables Noteson contributors Acknowledgements

vii ix xi vl11

Introduction Theovan Leeuwenand Careylewitt Content analysisof visual images Philip Bell

l0

Approachesto analysisin visual anthropology Malcolm Collier

35

Seeingbeyond belief:Cultural Studiesasan approachto analysingthe visual Martin Listerand LizWells

61

Semioticsand iconography Theoyan Leeuwen

92

A therapeuticperspective:the useof drawingsin child psychoanalysis and socialscience GertraudDiem-Wille

119

Visual meaning:a socialsemioticapproach Careylewitt andRumiko Oyama

134

Practicesof seeingvisual analysis:an ethnomethodologicalapproach CharlesGoodwin

r57

Analysingfilm and television:a socialsemioticaccountof Hospital:an UnhealthyBusiness Rickledema

r83

Index

207

Figures 2.I

The first twenty coversof a quarter of a century of Cleomagazine

tl

(re72_4). 2.2 3.1

The last twenty coversof a quarter of a century of Cleomagazine (1ee6-7).

T2

Antonio Romo in his wheat field, Rio Chiquito/Talpa,New 37 Mexico (USA).Photographby Iohn Collier,Jr.,1939. 3.2(a) Excerptfrom largerpanoramaof TaosValley,New Mexico 4l (USA), made in 1964.Photographby Malcolm Collier. 3.2(b) Excerptfrom larger panoramaof TaosValley,NewMexico 42 (USA), made in 1996.Photographby Malcolm Collier. 3.3 Unloading firewood in Romo year,Talpa,New Mexico (USA). 47 Photographby |ohn Collier,lr., c.1939. 3.4 Conversationin Santistevan'sauto repair shop, Ranchosde 50 Taos,NewMexico (USA). PhotographbyMalcolm Collier, 1984. 3.5 Alberto Tafoyahome, Rio Chiquito/Talpa,New Mexico (USA). Photographby |ohn Collier,|r., 1930s. 53 3.6 Excerptfrom photo map of StocktonStreet,SanFrancisco 55 Chinatown madeby classin Asian American Studiesat San FranciscoStateUniversity. Photographsby FrancesLee, Hernan Cortez,SuzanneCanierosand Mark Johnson,1985. 3.7 A photo map spreadout for analysis,Asian American Studies class,SanFranciscoStateUniversity. Photographby Malcolm Collier, 1986. 4 . 1 Marlboro cigaretteadvert,c. 1995. 66 4.2 RobertDoisneau.'An obliqueglance',194819. 67 4.3 De Fluminibus.Woodcut, 1483. 7I 4.4 RobertMapplethorpe.'Portraitof Clifton', 1981. 74 4 .5 David Hampshire.Photographof schoolboys,c. 1986. 77 4.6 NewYear's Day, Korem camp,Wollo province,Ethiopia, 1985. 78 4 . 7 Mike Goldwater.Drought migrant, Za\azeIetransit centre,Tigray, 1983. 78 4 . 8 Chris Steele-Perkins. USAid,Sudan,1985. 82 4 . 9 Mike Wells. A food convoy passingthrough Korem Refugee 82 Camp on its wayto Makele Camp in Tigray, February1985. 4 . r 0 Carl Lewisphotograph for a Pirelli advertisement. 87 4 . 1 1 SeanBonnell. Chalk Down, from the seriesGroundings, 1996. 88 5 . 1 'The Third World in Our Street'(from Bolset al., 1986). 93 5.2 GeneralEmilio Aguinaldo representedasa black dancinggirl. 104 Cartoon by Victor Gillam in Harper'sWeekly,1899.

vill

FIGURES

5.3 Advertisementfor a Dutch brand of crisps(The Netherlands,1987). 5.4 Carlo Grivelli:'Madonnaand Child enthronedwith donor'. 5.5 fan van Eyck 'St Jeromein his study'. 'In 5.6 the good old summertime', Americanpostcard,1907. 5.7 American advertisementfor grapefruit, 1930s. 5.8 Advertisementfor a Dutch brand of bananasfeaturingthe runner Nellie Cooman, L987. 'Allegory 5.9 Titian: of prudence'(1569). 6.1 Rupert'sdrawing. 6.2 Daphne'sfirst drawing. 6.3 Daphne'sseconddrawing. 6.4 Daphne'sthird drawing. 6.5 Hermann'sdrawing. 6.6 Erika's drawing. 'Explore 7.1 the possibilities'(City and EastLondon Health promotion). 7.2 From'4 Boys'(FamilyPlanningAssociation). 'If 7.3 Sketchof he won't usea condom' (Health EducationAuthority poster). 7.4 Onion celltext by Ramendeep, aged1l years. 7.5 Onion celltext byAmy, agedl l years. 'exit 7.6 British and Japanese this way' signs. 8.I Gazebetweenspeakersand hearers:transcript of Pam and Ann. 8.2 Transcript of archaeologists coding and recording the colour of the dirt they are excavatingthrough useof a Munsell chart. 8.3 Transcript of Ann (the senior archaeologistat the site) drawing a map and Sue(her student) making measurements. 8.4 Transcript of useof video in the trial of RodneyKing. 9.I Frameone: frame from a sceneabout 7 minutes into the documentary. 9.2 Opening sceneof Hospital:An UnhealthyBusiness. 9.3 An operationscene.

105 110 111 ll2 ll2 113 tt4 t24 r26 t26 t27 130 t3l r37 t39 r42 148 t49 153 158 168 172 t76 183 193 198

Tables 2.1

2.2 2.3 2.4 2.5 2.6 5.1 5.2 5.3 9.I 9.2 9.3

Genderdepictions inmagazineadvertisements:hypothetical valueson four variables. Duration of content emphasiscategoriesin rank order, pre-election period, per seandaspercentageof total Channelg broadcasttime. M odality and gender in magazineadvertisements. Hypothetical data from a pilot trial of 100classificationswith four coders. In our hypotheticalstudythree variablescanbe defined,with severalvalueson each. Classificationof distance,modality and behaviourof femalemodels on the front coversof Cleo,1972-4 andl996-7. Denotativesignifierand signifiedin Figure5.1. Connotatorand connotationin Figure5.1. 'photogenia'connotators in a chapteron'Netherlanders'. Useof Six levelsof tele-film analysis. The sectionon'stenting'. Summarizingthe analysis.

17 19 2l 22 32 33 99 99 100 189 196 t97

Contributors

Philip Bell is the Foundation Chair in Media and Communications, University of New South Wales,Australia. He has conductedcontent-analysisbasedresearchfor variousAustraliangovernmentbodiesand publishedwidely on the representationof socialand political issuesin the media,including: the media representationof alcohol and drug misuse,and road safety.His books include TheMedia Interview(with Theo van Leeuwen)and Implicated:the [JnitedStatesof Australia (with RogerBell). Malcolm Collier holds an MA in Anthropology from SanFranciscoStateUniversity (1977),however,his primary training in visualanthropologyhasbeenin field settings in the United States,Latin America and the Arctic. He has worked for many years in the Collegeof Ethnic Studiesat San FranciscoStateUniversity and has worked extensivelyin the fieldsof visual anthropology,anthropologyand education,applied anthropology, and ethnic studies. He is co-author of the publication, Visual Method,andis currentlythe presidentof the Anthropology:PhotographyasaResearch Societyfor Visual Anthropology in the American AnthropologicalAssociation. Gertraud Diem-Wille is AssociateProfessorat the Institute for Interdisciplinary Researchand Education,Universitiesof Vienna and Klagenfurt,Austria. Her fields organisationdevelopment,and civic of researchinclude applied psycho-analysis, education. Her recent publications are, Diem-Wille, G (1996) Femininity and (1997)'Observedfamiliesrevisited- two yearson: a follow up study', Professionalism in Reid,S. Infant Observation:the Tavistockmodel. Charles Goodwin is Professorof Applied Linguistics at UCLA. His books include andHearers,andRethinking Speakers Conversational Organization:Interactionbetween Context:Languageas an InteractivePhenomenon(edited,with AlessandroDuranti). In recentyearshe haspublisheda rangeof articlesrelevantto visualcommunication, including 'ProfessionalVision' (AmericanAnthropologist),'seeingin Depth' (Social 'Transparent Vision' (in Ochs,Schlegoffand Thompson, eds, Studiesof Science), 'Practices of Color Classification'(Mind, Culture and Interaction and Grammar), 'Action Activity) and and Embodiment within SituatedHuman Interaction' (Journal ofPragmatics).His interestand teachingincludevideo analysisof talk-in-interaction, gesture,gazeandembodimentasinteractivelyorgantzedsocialpractices,languagein the professions,and the ethnographyof science.

xtl

CONTRIBUTORS

Rick Iedemacurrentlyworks asa SeniorResearchAssociatewith the Schoolof Health ServicesManagement,Centre for Hospital Managementand Information Systems Research,University of New SouthWales(NSW), Australia.He hasbeena researcher in the areaof mental health policy planning (this alsoprovided the basisfor his PhD on organisationalsemiotics)and haswritten two major researchreportsfor the NSW Departmentof Education,one focusingon the print and broadcastingmedia and one on the discoursesof administration and bureaucracy.In addition, he haspublished articleson a rangeof linguistic and semiotictopics. Carey Jewitt is an experiencedsocial researcherwith a particular interest in visual analysis.Sheis a SeniorResearchOfficer at the Institute of Education,and is currently working on an ESRCfunded project, The Rhetorics of the ScienceClassroom: a MultimodalApproach. Her researchinterestsinclude,multimodal communication, and visual socialsemiotics.Her recentpublicationsinclude a socialsemioticanalysis of how discoursesof masculinitiesare visually realised,'Imagesof men' (1997) Sociological Research Online,Vol 2, Issue2. Martin Lister is Headof the Schoolof Cultural Studies,UniversityofWest ofEngland. He haslecturedin a rangeof subjects,including art history, photo-media and critical studies,and computersand culture.He hasundertakenresearch,and publishedwidely in the area of cultural studies, including, Youth, Culture and Photography(with A. Dewdney),MacMillan (1988),and edited ThePhotographiclmageinDigitalCulture (1995)London and New York: Routledge.In addition he hascuratedand produced numerous photo-text touring exhibitions dealing with aspectsof contemporary culture. Rumiko Oyamataught in the literature and linguisticsdepartmentof Kobe College, and the departmentof Intercultural Communication, Universityof Kobe,Japan.She is currently researchingthe area of cross-cultural semiotics, and is particularly interestedin the waysin which cultural value systemsare realisedvisually. Theo van Leeuwenis Professorof Communication Theory at the London Collegeof Printing. He worked asa film and televisionproducer beforestudyinglinguisticsand becomingan academic.He haspublishedwidely in the areasof socialsemioticsand critical discourseanalysisand, with Gunther Kress,is the author of Readinglmages TheGrammarof VisualDesign(1996),London: Routledge.His currentwork is on the semioticsof speech,musicand sound. ElizabethWells is a seniorlecturerin film, video and photographicstudiesand Course Leader,MA Independentfilm and Video, at the School of Media, The London Institute: London College of Printing. She is editor of Photography:a Critical Introduction,Routledge(1997),and curatedVier,vfindings- women photographers: 'landscape' and environment,which toured Britain in 1994195. Shehas contributed numerousessaysand reviewarticlesto books andmagazineswithin the field ofvisual and cultural studies.

Acknowledgements

The authorsand publisherswish to thank the following for permissionto usecopyright material. Network PhotographersLtd., for figure 4.2 tJnRegardOblique,RobertDoisneauand figure 4.8 FoodConvoy,Mike Wells Art and CommerceAnthology Ltd. for figure 4.4 Clifton, 1981Copyright @1981The Estateof Robert Mapplethorpe Magnum PhotosLtd. for figure 4.7 Daily Life in Chad,Chris Steele-Perkins The AdvertisingArchivesfor figure 4.I0 CarI Lewisfor PireIIi Tyres SianBonnellfor figure4.ll ChalkDown National GalleryCompany Limited for figure 5.4 Virgin and Child with SaintsFrancis and Sebastian,Carlo Crivelli and figure 5.9 An Allegory of Prudence,Titian EastLondon and the City Health Authority for figure 7.I ExploreThePossibilities Family Planning Association and Comic Company for figure 7.2 for imagesfrom 48oys Every effort has been made to trace all the copyright holders,but if any have been overlooked, or if any additional information can be given, the publishers will be pleasedto make the necessaryamendmentsat the first opportunity.

Hru WK Kffi

Introduction THEO VAN LEEUWEN AND CAREY JEWITT

INTRODUCTION This book evolved from our different relationships to the visual and from our experienceswith visual analysis.Careyhad recentlyevaluateda young men's sexual health clinic which had included analysisof the images in sexual health leaflets and posters availablefor young men (Jewitt, 1997, 1999).The imagesreinforced stereotypedforms of masculinity which, had they beenput into words, would have beenunacceptableto most sexualhealth workers.Interestingly,theseresourceshad been rejectedby many of the young men who used the clinic; the imageswere key in this rejection, but most people found it difficult to articulatewhy they disliked 'articulatewhy', to understand'what might otherwiseremain them. Finding a way to at the level of vaguesuspicionand intuitive response',had been difficult (Iedema, usefulinformation aboutvisual this volume:201).One of the difficultieswasaccessing analysis.Theo had often been involved in a searchfor this kind of information by postgraduatestudentsand colleaguesin linguistic discourseanalysisand pragmatics, and had rarely beenableto fully satisfr them. At our initial meetingto discusssome of the difficulties involved in visual analysiswe cameup with the idea for this book. We wanted above all to produce a book which would be a useful resource for researchersinvestigatingthe visual representationof significant social issues,and which provided exemplification of a range of methods and perspectivesof visual analysis in sufficient detail to make it possible for readers to actually use the approachesexplainedin the book. By way of introduction we will first give a brief description of the chapters and then discusssomeissuesthat arisefrom readingthem. One of our aims in doing the latter is to show how elementsfrom different approachesmight be combined accordingto the requirementsof specificresearchprojects.

D I F F E R E N T P E R S P E C T I V E SA N D M E T H O D S O F V I S U A L ANALYSIS Content analysis(Chapter 2) has long been associatedwith investigationsof the way socialissuesarerepresentedin the massmedia,and it hasthe distinct advantage of being understoodand acceptedby most people,including journalists.Philip Bell

ISUAL ANATYSIS

showshow it can handle large quantitiesof data,typically in relation to confirming comparative hypotheses('expectations'),such as that women are more often representedas engagedin domesticactivitiesthan men, or that professionalsin the movies or televisionseriesof a given period are more often playedby white rather than black actors.But, asBell points out in the chapter,it can equallywell be applied to formal issues(for example, whether yellow is more often used on magazine coversthan green), or, indeed, to any issuethat allows the formulation of clearly definablecategoriesand comparativehypotheses.The chapterprovidesenough detail to allow readersto construct their own content analyses,including the necessary statistics. Visual anthropology (Chapter 3) is concernedwith the useof visual records for the description of the present and past ways of life of specific communities. In the caseof past ways of life, visuals are often (r,ery successfully)used to elicit memoriesfrom informants. In this chapter,Malcolm Collier drawson a wide range of examples,including studiesof Anglo immigration in New Mexico agricultural communities, student behaviour in Cantonesebilingual classesin San Francisco, and community schools in Alaska and the Navajo Nation in Arizona. It is very much oriented towards the practice of visual research,discussingwhat kinds of photographic and video records are most useful for the purposesof anthropological research,and what kinds of contextualinformation to keep.The chapter endswith a challengingcall for the developmentof a visual languagefor intellectual discourse. Cultural Studies(Chapter 4) has recently developeda specificsub-field of visual cultural studies,which, as Martin Lister andLiz Wells describe,is premised on the unprecedentedimportance of imaging and visual technologiesin contemporary society,and concernedwith all kinds of visual information, its meanings, pleasuresand consumption, including the study of all visual technologies,from oil painting to the Internet. The chapter seesCultural Studiesas an interdisciplinary field, and describesit, not as a specificmethodology,but as an agendaof questions and issuesfor addressingspecificimages.Thesequestionsand issuesarethen matched to the conceptualframeworksand methodologiesof a rangeof different disciplines. This approach makes the chapter particularly useful as a model for integrating the various approachesdiscussedin this volume as a whole. The chapter draws primarily on examplesof mass media and art photographic images,from global cigaretteadvertisingcampaignsto documentaryphotographsof faminesin Africa the latter, as Lister and Wells point out, having played a key role in constructing a 'economically Eurocentric view of Africa and its peoples as and technologically weak,dependentvictims of natural disaster'(p. 78). The chapteron semioticsand iconography(Chapter 5) written by Theo van Leeuwendiscussesthe visual semioticsof Roland Barthesand the iconographical method of visual analysis developed by art historians such as Edgar Wind, Erwin Panofskyand Meyer Schapiro.Both methods are premisedon the idea of layered meaning, of imagesconsisting first of all of alayer of representationalor denotativemeaning (the layer of who and what are depictedhere) on which is then superimposeda layer of connotativeor symbolic meaning (the layer of what doesit

INTRODUCTION

all mean). Both methodsprovide specificpointers for distinguishingand analysing these layers, and specific criteria for arguing whether or not a layer of symbolic or second-ordermeaning is present.The main differencebetweenthe two is that iconography usesboth textual and contextual criteria for arguing symbolic meaning, while Parisschoolsemioticsrestrictsitselfmostlyto textualcriteria,to pointerswithin the image itself. Although iconography has mainly been applied to art works from the past,the chapterattemptsto demonstratethat it can alsobe appliedto contemporary images,using Jan NederveenPieterse's( 1992)history of the European and North American depiction of Africans and Afro-Americans as a main source of examples. The chapter on psychoanalyticalimage analysis (Chapter 6), written by Gertraud Diem-Wille, a Viennesepsychoanalystspecializingin the treatment of children, arguesthat children, within the specialcontext ofthe psychoanalyticsession, produce drawings that are basedon the sameprimary processesof representation just asdreams.Their meaningscanthereforebe brought out through psychoanalysis, like those of dreams.Diem-Wille then goeson to apply this method to what could be calleda socio-psychoanalytical study of what driveshighly successfulcareermen and women. Here the projective drawing technique is seento lower interviewees' defences,enabling them to visually expresswhat they are inhibited from verbally expressing.Although Diem-Wille is reluctant to generalize,the casesshe discusses revealpatternswhich havewider validity in understandingthe influenceof parental relationshipson careerchoices,and paths,and the role of work in the formation of identity. Socialsemiotic visual analysis(Chapter 7) provides a detailed and explicit method for analysingthe meaningsestablishedbythe syntacticrelationsbetweenthe people,placesand thingsdepictedin images.Thesemeaningsaredescribedasnot only representational,but also interactional (images do things to or for the viewer), concernedwith the modality or perceivedtruth value of images,and compositional (for example,positioning imagesand written text in certain ways). In this chapter CarcyJewittand Rumiko Oyamacharacterizesocialsemioticsasconcernedwith the study of imagesin their socialcontext,and asa critical form of visualdiscourseanalysis which does not necessarilystop at description but may also seekto influence the semiotic practices it describes.The chapter shows the method at work in three differentresearchprojects:the studyof sexualhealthmaterialsalreadymentioned,an ethnographic/semioticstudy of the primary schoolscienceclassroomand a study of someof the cross-culturaldifferencesin visual syntaxwhich distinguishBritish and |apaneseexit signsand magazineadvertisements. In the caseof conversationanalysisand ethnomethodology (Chapter B), visual analysisis not so much a matter of analysing images (with or without considerationof their context) as of analysingthe dynamic unfolding of specific socialpracticesin which non-verbal communication (pointing, gazework, and so on) and images (including signs, maps and diagrams) play a role. The chapter beginsby showing how, without taking account of visual communication, conversation analysismight not only miss out on information but lead to inaccurate conclusions.Charles Goodwin discussesthe role of a range of different kinds of

HANDBOOK OF VISUAL ANALYSIS

image in the work of scientistsand other experts.Both the production of images (for example,map-makingby archaeologists) and their interpretation (for example, the interpretation of the RodneyKing videotapeby police expertsduring the infamous trials) take placein situatedinteractionsthat usea variety of communication modes - speech,non-verbal communication (for example,pointing and gazework) and images(for example,the archaeologists'map or the RodneyKing video). The same images,moreover,ffi?y be useddifferently by different participants (for example,a scheduleof plane arrivalsand departuresis useddifferentlybybaggagehandlersand gate agents).The chapter provides specificpointers for analysingexisting images (and for producing useful video recordings for researchpurposes), and argues powerfully for studying visual communication in its socially specific and multimodal contexts. The chapteron film and televisionanalysis(Chapter 9) discusses a television documentary which depicts the conflicts between the clinical and administrative sectionsof a large hospital, and demonstrateshow this film is systematicallyconstructedto favour one point of view over others,and how a study of this kind of bias might be conducted. Combining elementsfrom film theory and social semiotic genreanalysis,Rick Iedemadescribesthe different levelsat which film and television texts can be studied (the frame, the shot, the scene,the sequence,the stageand the genre),and then looks at key variablesand methodsrelevantto eachof theselevels. At the time of writing the chapter,the author wasworking as a researchconsultant for a large hospital, and hencethe chapteris an exampleof applied visual analysis, of a form of critical analyticalpractice taking place inside an institution, with the aim to influenceand changethe practicesof that institution.

THE IMAGE AS RECORDAND AS CONSTRUCT Someofthe chaptersin thisvolume describethe analysisof imagesspeciallyproduced for researchpurposes(Chapters3 and, in part, 8). Such imagesare producedto serveas records of reality, as documentary evidenceof the people, places,things, actionsand eventsthey depict. Their analysisis a matter of extractingjust that kind of information from them. The same applies to the therapeutic use of drawings describedin Chapter 6. The analysisof thesedrawingsmust bring out, for instance, what the family relationsin a given subject'schildhood were like and how they were experiencedby the subject.Art historians,too (Chapter 5), often analyseimagesas sourcesof factual information, even though, in this case,the imagesmay not have been speciallyproduced for this purpose.Researchers who use imagesin this way are of courseawareof the limitations of images(including photographicand video images)assourcesof factualinformation. Good researchimages,asCollier describes them (Chapter 3), should not be overly constructed,or complex, and therefore harderto readthan imagesproducedfor the media or asart images;alsothey should not be isolatedfrom the seriesof imagesto which they belong (for ashe points out, singleimagesand imageswithout extensivecontextualannotation are problematic for researchpurposes).However,despitetheir limitations,imagesare,in this context,

INTRODUCTION

regardedas a reliable source of factual evidence,and Collier in fact prefers them 'deceptiveworld of words' (p. 59). over the In other casesimagesareanalysed,not asevidenceofthe who, whereandwhat of reality,but asevidenceof how their makeror makershave(re-)constructedreality, asevidenceofbias, ideologicallycolouredinterpretation,and so on. This is common (for example,Chapters4,5,7 and 9), and in Cultural Studiesand semioticanalyses in ethnomethodologicalresearchwhen the processof (re-)constructingrealityitself is documented,suchasin studieson the way scientistschangethe apparentlyunruly and messyworld of photographsinto the more orderly world of diagramsby 'filtering', 'uniforming', 'upgrading' and'defining' photographs(Lynch, quoted in the imageis more unreliableand slipperyas Chapter8: 163).From theseperspectives a sourceof factual information. In this volume Lister and Wells, like Sekulawhom 'evidence'ofphotojournalism and documentaryphotography: they quote, mistrust the '. when photographsare uncritically presentedas historical documents, they are .. transformed into aestheticobjects.Accordingly, the pretenceto historical understanding remains although that understanding has been replaced by aesthetic experience'(Sekulain Chapter4: 89). The point for us is not to constructtheoreticalargumentsin favour of reality or construction,or to arbitratein the debateson this issuefrom our editorialposition. Ratherwe think the point is to urgeyou to keepthe distinction in mind when reading the chaptersin this volume, and to considerthat the choiceof an appropriatemethod of analysisis dependenton the nature of the project in which it is to be used,on the visual material that is being investigated,and on the goalsof the researchproject. One of us is currentlyengaged Indeed,sometimesseveralmethodsmaybe necessary. in a researchproject dealingwith children's toys. The project investigatesboth the meaningsoffered to the child by the toy industry and the massmedia (through the texts and pictureson toy packaging,in toy catalogues,in toy advertisements,and so on) and the way in which these(and other) meaningsare taken up in parent-child interactions and in the child's actualplaying with the toys. Clearlythe former requires a mode of analysisin which the variouspackagingtexts,cataloguesand advertisements are treatedasconstructs,and the latter the analysisof videotapesspeciallyproduced for the researchasethnographicevidenceof parent-child interaction and children's playwith toys. 'record' versus'construct' existsbecausemany imageshave an The issueof elementof both and so require a mode of analysiswhich is sensitiveto both. Clearly advertisingimagesare in the first place constructsand their analysismust reveal the nature of theseconstructs.Equally clearlyconstruction has to be minimal (or, where imagesare speciallyproduced for researchpurposes,minimized) if images are to be used as recordsof people,places,things, actionsor events.Again, it is no accidentthat studiesaiming at changingpracticesof representationoften choosea detailedand explicit method of analysingconstruction and its effects,so asto avoid the idea that it is all in the eyeof the beholder,which in this casewould be counterproductive (cf. Chapterc2,7 and 9); alternativelythey could of coursedocument of that construction,ashappensin ethnomethodologicalresearch the very processes (Chapter8).

6

HANDBOOK

OF VtSUAt

ANALYSIS

THE UNITS OF ANALYSIS: SINGLE IMAGES VERSUS COLLECTIONSOF IMAGES Someof the approachesto visual analysisdescribedin this volume are basedon the analysisof collectionsof images,others on the analysisof singleimages.Chapter2 showsthat content analysisrequiresat leasttwo different setsof data (for example, imagesfrom two different periods or publications) for the purposeof comparison, and that each of thesesetsneedsto contain a suffrcientlylarge number of similar images(for example,all advertisementscontaining imagesof women from a given period of time) in order to be both representativeand statisticallysignificant.Visual anthropology (Chapter3) alsousescollectionsof images,but for different purposes. Collier describestwo kinds of collections:first, collectionsof many different images of the same subject (for example,wide views, details and a range of anglesof the samestreet)which are put togetherto allow patternsto becomevisible;and second, collectionsof imagesmade to help identifr what is depictedin a given image (for example,a 60-year-oldimage of an elderly man engagedin wheat harvestingwas comparedto other photos of the sameman in different settings,other photos of the areaand other photos of wheat harvesting,in order to establishexactlywho the man was,where he was photographedand what he was doing). As discussedin Chapter 5, art historians also use this method when they want to establishthe exact who, where and what of art works of the past. As noted by Collier and Goodwin in Chapters3 and 8, films and videosare alwayscollectionsof images. In other casessingle imagesare discussed.But, as indicated in Chapters2, 5 and 7, any method of visual analysiswhich provides a wide enough range of clearlydefinedspecificimagefeaturesand connectsthem convincinglyenoughwith particular meaningsand/or communicative effectscan be usedeither for the analysis of singleimagesor quantitatively.

TEXT, CONTEXT, SOCTALPRACTICE Thereis anotherway in which the variousapproachesdescribedin this volume do not use the sameunits of analysis.Visual analysismay be basedonly on what is visible within the imageor collectionof images(in the text), asis the case,by and large,with contentanalysis(Chapter2) andalsowithvarioustypesofsemiotic analysis(Chapters 5,7 andg). It may draw on contextualinformation, whethergleanedfrom interviews, as in the caseof visual anthropology (Chapter 3) and the therapeuticinterview (Chapter 6), or from archival researchand background reading, as in the case of iconography(Chapter5). Or, as in the caseof Cultural Studies(Chapter4),it may use a range of different kinds of information. The approachof Chapter 8 is different again.Here the unit of analysisis not the text, or the text together with external,contextual information, but the enactedsocialpracticesin which images areused. The sameappliesto the questionofword and image.Imagesmaybe analysed without any recourseto the verbal or written information which may accompany

INTRODUCTION

them (for example,the catalogueof an exhibition or the introduction to a book of artistic photographs).The imagesmay have been designedto be self-sufficient,or capableof being insertedinto many different contexts,as certainly is the casewith much contemporaryfine art and'classic'photojournalism (suchasCartier-Bresson in Chapter4). At the veryleastthereis asmuch of a caseherefor analysingthe broader, supra-contextualmeaning potential of images,asfor analysingtheir meaningsin a particular context, for examplea particular exhibition. But visual analysismay also include the accompanyingtext, or evenseeword and imageasone indivisibleunit of analysis,asin the socialsemioticanalysisof layout (Chapter7).In suchan approach the catalogues,text panels, and so on of an exhibition would be included in the analysis,asa secondarysourceof information, serving,for instance,to anchor who or what is depictedor what is symbolized.But the analysiswould still not include the way this meaning potential is actuallytaken up in the interactionsof specificsocial actors visiting the exhibition, as would be the casein an ethnomethodological approach.Again,the choiceofmethod dependson the natureof the materialanalysed and on the goalsof the analysis.There are good argumentsfor analysingimagesin relativeindependenceof their context (for re-contextualizingthem, in other words), and for analysingthem togetherwith the physicalcontext and or socialinteraction in which they are embedded(as,for instance,in the analysisof readingadvertisements on hoardingsor in magazinesin Chapter4).

TEXT, PRODUCER,VTEWER Thesedifferent perspectiveson text, context and socialpracticeshave implications for the way in which the producers and viewers of images may be included or implied in the analysis.A mode of analysiswhich restrictsitself to the evidenceof 'by itself demonstratehow viewersunderstandand the text may not, asBell puts it, value what they seeor hear' (Chapter 2: 26), or what producers, deliberatelyor otherwise,intend to communicate.On the other hand, text analysiscan show what representationsinclude and exclude,what they prioritize and make salient, and what differencesthey constructbetweendifferent people,placesand things. The degreeto which producersand viewersshould be included and how this is to be achievedagaindependson the kinds of imagesanalysed,and on the aims of the research.Researchaimed at discoveringhow a generalaudienceunderstands advertisingimagesshould perhapsnot include information about the production of advertisementswhich such an audiencewould not be familiar with. On the other hand, if a researcherwanted to changethe audience'sperceptions,for instance 'look through media education,a behind the scenes'might becomerelevant.Again, researchaiming at critiquing, say, racism or sexism in certain representational practicesclearlyhasan interestin linking thesepracticesto specificsocialinstitutions, but in the analysisof art images(Chapters4 and 5) producersmay be depicted, not in terms of the formal or informal institutions within which their work is (or was) situated,but as individuals working on the basisof traditions, influencesand inspirations- and herethe audiencewill be lessoften taken into account.Important

HANDBOOK

OF VISUAL ANALYSIS

and innovative researchoften thrives in the 'blind spots' of specific research approachesand methods,applying the methods of one approachto the kind of materialstudiedin another- for instance,consideringthe individual in an areawhere most researchhasconcentratedon institutions, or institutions in an areawheremost researchhasconcentratedon individuals. When socialpracticesare taken asthe units of analysis(asin Chapter 8), the differencebetweenproducersand consumersis much diminished. Both the archaeologistshowingan apprenticehowto constructan archaeological map and the police expert showing a jury how to interpret the Rodney King video usethe resourcesof their expertknowledgeto constructmeaning.This method could alsobe usedto show howthe script of a televisionseriesis produced,or how universitystudentslearn new interpretationsof a televisionseriesin Media Studiescourses- all aspectsof visual communicationwhich could neverberevealed,for instance,through contentanalysis. At the sametime, this approachwould not bring out what aspectsof sociallife televisionseriesmore generallyinclude and exclude,what kind of interpretationsare generallyfavouredin the massmedia, and so on - questionswhich content analysis and semioticsarewell placedto answer.

CONCLUSION Clearlysome methodsof analysisare more methodicalthan others.Somelay down very precisecriteria for analysis,so that the impressionmay arisethat visual analysis can be done 'by rote', and describedas a kind of recipe,a procedureto be followed stepby step,without the needfor any form of initiative,let aloneinspiration.Content analysis,with its more or lessmechanicalstatisticalprocessingof data, and social semiotic analysis,with its proliferation of featuresand precisecriteria for analysing them, tend most clearlyin this direction.Anyonewho hasactuallytriedthesemethods knows that thereis a greatdealmore room for initiative and, indeed,inspirationthan is sometimesacknowledgedin the way thesemethodsare described.Thesemethods remain an art of interpretation,but one that follows certain rules of accountability. Other forms of analysisprovide lesspreciserules for conductingthe analysis. Cultural Studiesand ethnomethodology,for instance,certainlydepart from precise theoreticalpositions,researchquestionsand principles of research,but they do not provide a large number of analyticalcategoriesand nor do they explicitly construct researchwork in terms of a 'step-by-stepprocedure'. The approach describedby Collier in Chapter 3 provides an intermediary position. Collier seesvisual analysis asa complex processwhich alternatesbetweenstagesthat require an intuitive grasp of the whole and stagesthat require the hard work of structured analysis,of careful and methodical checkingand double-checking.For Collier, it is both necessaryto 'obserye the dataaswhole',to look at,listento 'its overtonesand subtleties',to'trust your feelingsand impressions'andto'go through the evidencewith specificquestions - measuredistance,count, compare.Producedetaileddescriptions'(Chapter3:39). He seesvisual analysisasboth art and science:'It is both necessaryand legitimateto allow ourselvesto respond artistically or intuitively to visual images. . . However,

INTRODUCTION

while creativeprocessesare essentialto discovery,artistic processesmay produce only fictitious statementsif not combined with systematicand detailed analysis' (p. 59). In our opinion, this can be usefully appliedto all visual analysis.

REFERENCES Research Online,2 (2). Jewitt,C. (1997)'Imagesof men', Sociological 'A Jewitt,C. (1999) socialsemiotic analysisof male heterosexualityin sexualhealth Methodologt, resources:the caseof images', International Journal of SocialResearch 1 (a): 263-79.

I

ll n

H

ContentAnalysisof Visuallmages PHILIP BELL

INTRODUCTION Cleo is a magazineaimed at female readers from eighteen to thirty-five years of age.First published in Australia in 1972,it celebratedits twenty-fifth birthday byreproducing all threehundred ofits front coversin its November 7997spectacular with Kylie Minogue asits model on the cover.The first and last twenty coversfrom the quarter of a century of CIeoarereproducedasFigures2.1 and2.2 respectively. What differencescan be seenif one comparesthe two groups of covers published 23-5 yearsapart?How has Cleo's'image' (and the imageswhich make up its'image') changed?To answerthis, either in conversationat the hairdresser'sor as an academicthesis,requiresidentification of observabledimensionsofthe imagesin question(for example,arethe modelswho aredepictedolder or differentlydressed?), as well as a judgement about how frequently various visual featuresappearin the periodsthat one choosesto compare.In short, the answerrequiresa (visual)content analysis. This chapteroutlines the assumptions,practices,limitations and advantages ofexplicit, euantifiableanalysisofvisual contentasa researchmethod.It then returns to the Cleocoversto exempli4rthe process.

CONTENTANALYSIS AND WHEN IT CAN BE USED Generalizationsabout what is shown on television,in the pressor in advertisements requireobservable,more or less'objective'evidence.Considerthe ubiquitous claims and counter-claimsabout, say,the depiction of women in magazineadvertisements or the prevalenceand effectsof televisualor cinematicviolence.When suchissuesare debated,the frequencyand the meaning of identifiableclassesof discretevisual content are at stake(asopposedto more complex forms like narrativestructuresor the complexitiesof different genres). Whether it is explicitlylabelled content analysisor not, making generahzations about the relativefrequenciesofvisual representationsof particularclasses of people, actions, roles, situations or eventsinvolves implicit or explicit classificationand quantification of media-circulatedcontent.

CONTENT ANALYSIS

,g1trsr Flft$

OF VISUAL IMAGES

F$kuS l$?*

$irturt*9)$

Figure 2.1

The first twenty covers of a quarter of a century of Cleo magazine (1972-741.

12

HANDBOOK

OF VISUAL ANALYSIS

N* E*fs

*tf rss,

ldr 1*?7

Figure 2.2

The last twenty covers of a quarter of a century of CIeo magazine (1996-7).

CONTENT ANALYSIS

OF VISUAL IMAGES

Not surprisingly, therefore, content analysishas provided one of the most widely cited kinds of evidencein Media Studiesfor many decades.First, in relation to newspapersand radio (for you can analyseverbal content as well as visual) and,later, directedat televisionand occasionallyat the cinema.Perhapsthe method 'commonsense'way to researchwhat hasbeenwidely usedbecauseit seemslike the the media show,or becauseit appearsto requirelittle theoreticalanalysis.In short, it is the most basicway of finding out somethingabout the media'smeaningand allows for apparentlygeneralstatementsto be made about aspectsof representationwhich non-specialists,journalists and expertsalike can understand.However,aswill be clear in the following discussion,content analysisis quite a technicalprocedure.It is also of limited valuein many researchcontexts,and might bestbe thought of asa necessary but not sufficientmethodologyfor answeringquestionsabout what the mediadepicts or represents.Content analysisaloneis seldomableto support statementsabout the significance,effectsor interpreted meaning of a domain of representation.For example,using content analysisto show that prime-time televisiondepicts a high level of physicalinter-personalaggressiondoesnot, by itself, show that viewersare affectedin any particularway (eitherby imitating what they seeor by inhibiting their own aggressive behaviours).Claimsabout the effectsofwhat is shownraisequestions which need to be addressedby further, different kinds of researchwhich are not consideredin this chapter(seeChapters4 and 9). Let us now become more preciseand technical in defining and describing the range of proceduresfor visual analysisthat content analysisrefers to. First, a generaldefinition: content analysisis an empirical (observational)and objective 'audio-visual' (including verbal) representation procedurefor quantifnrg recorded using reliable,explicitly definedcategories('values'on independent'variables'). To illustrate each component of the definition, considerthe exampleof aboutwomen and men in advertisementsin making generalizations/representations 'men's' 'women's' magazines.To beginto observehow women and men comparedwith are depictedrequiresan explicit hypothesis(or hypotheses)without which the comanalysed. plex field is too diverse,ill-defined and thereforeunableto be systematically (expectation) or So content analysisbegins with some precisehypothesis questionabout well-definedvariables.In our example,thesevariablescould include types of magazine,size of published advertisement,(defined) pose of represented models (say,standing,seated,walking, running) and depictedcontext (e.g.office, home, outdoors). One explicit hypothesismight be that women will be depictedin fewer outdoor situationsthan men, in both kinds of magazines.Only when one or more hypothesisis formulated will the relevantvariablesbecomeapparent. Note that the kinds of hypotheseswhich content analysisusuallyevaluatesare comparative.That is, the researcheris usuallyinterestedin whetherl sol, women are depictedmore or lessfrequentlythan men in relation to somevariableor quality, not in the absolute frequency of certain depictions taken in isolation. Obviously, 'outdoor' settings, if 20 per cent of women in televisioncommercialsare shown in thesedata tell the researchervery little unlessthe comparablefrequencyfor men (in otherwisesimilar contexts)is known. Content analysisis usedto testexplicitly comparativehypothesesby meansof quantification of categoriesof manifestcontent.

13

14

H A N D B O O KO F V I S U A T A N A L Y S I S

Let us continue discussingour (admittedly quite simplified) examplesby drawing attention to other aspectsof content analysis.First, the choice of the scope,number or scaleof the visualcontentto be studiedmust be explicitlydescribed before detailed observationsbegin. This involves deciding what 'corpus' or what 'sample size'or what'field' or 'domain' of representationis to be studied:how many magazines,how many advertisements,how many editions of one or more newspapers, and so on (seeChapter3). For example,becauseit is comparative,a hypothesismight specif. two or more periods (say,five yearsbefore World War II and five yearsafter) during which to comparesomeaspectsof genderrepresentation.BecauseWorld War II brought women increasinglyinto the paid workforce this hypothesiswould be of sociological interest. In all such cases,the question arisesof how many data are enough?Can we generalizefrom our samplesofimages to the population from which it has been selected?This question ultimately requires statistical decisions and inferences,but it is clearlyimportant to ensurethat the corpus (the sample(s))is representativeof the variablesspecifiedin the relevanthypotheses.Obviously, if a researcheronly samplesfrom one women's magazinesheor he could not generalize the findings to all women's magazines,nor evendraw conclusionsabout the particular magazinebeyond the period sampled.(Women's magazineshavechangedradically in the last severalyears.)We will return to this issueafter outlining the technical proceduresto follow when conducting a well-designedcontent analysis. To summarize this introductory discussion:visual content analysisis a systematic,observationalmethod usedfor testinghypothesesabout the waysin which the media representpeople,events,situations,and so on. It allowsquantification of samplesof observablecontent classifiedinto distinct categories.It doesnot analyse individual imagesor individual'visual texts' (comparedwith psychoanalyticalanalysis, discussedin Chapter6, and semioticmethods,discussedin Chapterc4,7 and 9). Instead, it allows description of fields of visual representationby describing the constituentsof one or more definedareasof representation, periodsor typesof images. Typical researchquestionswhich may be addressedusing content analysis include: 1 Questions of priority/salience of mediacontent:howvisible(howfrequently, how large,in whatorderin a programme)differentkindsof images,stories, 'AgendaSetting'studiesof newsbroadcasts eventsarerepresented? wouldbe an exampleof thiskind of question. 2 Questions of 'bias':comparative questions priabouttheduration,frequency, ority or salience of representations poliof, say,politicalpersonalities, issues, cies,or of 'positive'versus'negative' features of representation. 3 Historicalchanges in modesof representation ol for example, gender,occupational,class,or ethicallycodifiedimagesin particulartypesof publications or television genres.

WHAT TO ANALYSE: 'ITEMS'AND 'TEXTS' 'content' What is the of content analysis?As the examplesmentioned already indicate, the material analysedby meansof content analysismay be visual, verbal,

CONTENT ANALYSIS

OF VISUAL IMAGES

graphic, oral - indeed, any kind of meaningful visual/verbalinformation. To analyse is to break a thing down into its constituent elements.So the visual/verbalunits of meaningwhich are the objectsof content analysisare those definedby the medium in which they are produced as isolable,self-containedor separate,like paragraphs, 'texts'whether or not framed images,pagesor news photographs.I will call these (on television) they are verbal.A displayadvertisementis a visual text. A newsitem is also a text, becauseit has a clear frame or boundary within which the various elementsof sound and image'cohere','makesense'orarecohesive.Texts,then, are definedwithin the context of a particular researchquestion and within the theoretical categoriesof the medium (gallerypainting, television)and genres(suchasportraits, newsor'soaps') on which the researchfocuses. The textsthat are to be analysedare the meaningful and cohesiveunits framed within the medium and genre(s)at issue.Visual content analysisusually isolates framed images(in publications) or sequencesof representation(scenesor shotsin television or film). But, unlike semiotic analysis,content analysisclassifiesall the 'variables' in this chapter) to texts on specifieddimensions (what will be called 'reading' or interpreting each describethe field or totality. It is not concernedwith text individually. By contrast, semiotic analysisis qualitative and usually focuseson eachtext or text-genrein much the way that a critic focuseson a particular gallery painting or on the aestheticsor cultural connotationsof a particular film or classof films (seeChapters4, 7 and 9; in addition, Chapter5 demonstratesan iconographic approachto individual textsand Chapter6 describesa psychoanalyticalapproachto individual texts).

HOW TO DO CONTENT ANALYSIS Like all researchmethodologies,content analysisis an effectiveprocedure only if precise hypothesesand clearly defined concepts underpin its use. Categoriesof (visual) content must be explicitly and unambiguously defined and employed consistently('reliably') to yield meaningful evidencerelevantto an hypothesis.To observeand quantift categoriesof content it is first necessaryto define relevant variablesof representationand/or salience.Then, on each variable, valuescan be distinguished to yield the categoriesof content which are to be observed and quantified. The conceptsof variablesand valuesare critical to an understandingof the procedure,so I will define thesemore completelybelow.

Variables Visual or verbal representationsdiffer from each other in many ways - on many dimensions or qualities. A content variable is any such dimension (size, colour range,position on a pageor in a newsbulletin); or any rangeof options of a similar tlpe which could be substitutedfor eachother - for example,a list of represented 'settings' such as participants (male/female;adult/child) or a number of alternative kitchen, bathroom, street,automobile,shop,and so on. Variableslike size,represented

1 5

16

HANDBOOK

OF VISUAL ANALYSIS

participants,settings,priority, duration and depictedrole consistof the setof options which are of the same classor type as defined for the purposes of the research project.Notice that in content analysisa variablerefersto aspectsof how something is represented, not to'reality'. Soif someoneis shownaselderlyin a television'soap' (by meansof make-up or clothing) it is this which the researcherobserves,judges and classifies,not the 'real' ageof the actor. This point is sometimesreferredto as judging the 'manifest content' of an image or text. It is the content as represented that is analysed,not some independentlyor 'externally' defined, and certainly not any aspectof 'reality' not actuallydepicted.So all the variablesdefined are thoseon which particular representationsdiffer from one another.

Values A variable consistsof what we will call values. These are elementswhich are of the samelogical kind. That is, elementscan be substitutedfor each other because they belong to the sameclass:theseconstitute the values on a particular variable. For example,all the occupationalroles in which people are, or could be depicted in television commercialswould constitute a variable. But such a 'role' variable would be distinct from, say,the variableof 'depictedphysicalsetting', from which it is conceptuallyindependent.Of course,somerolesusuallyoccur in particular settings,but that is an empirical contingencyand doesnot implythat the two variables arethe same(Chapter9 providesa more detaileddiscussionof this issue).To ensure that one does not confusetwo variables,a useful test is as follows: could a defined value on one variablebe substitutedfor another on that variableor not? If not, one has confusedtwo distinct variables. To summarize,then, a content analysisbeginswith the definition of relevant variablesand of the values on each.Each variable is logically or conceptuallyindependentof everyother distinguishedin a particular researchproject. The valuesdefined on each variable should also be mutually exclusiveand exhaustive(another technicalcriterion which will be discussedbelow). Schematically,the specificationand definition of variablesand the valuesof eachcanbe representedasin Table2.l.Here, four variableshavebeendistinguished. Within each,a different number of valueshavebeenspecified.In researchon gender representation,to continue one of our hpothetical examples,the table might appearasin Table2.1. Note againthe logic of sucha system:eachvariableis independentof all others. All values on a given variable are then mutually exclusive:any observedelement of representationcan only be classifiedinto one valueon eachvariable,and valuesthat are defined should be exhaustive- the values should cover all the possiblecategorizationson the respectivevariable.This may require a'miscellaneous'or'other' valueto be allowedon somevariablesasit is not alwayspossiblefor the researcherto anticipatethe completerangeof values(for instance,occupationalroles,to continue our example)that maybe found during the courseof research.So'politician' maybe a valueor role which is only infrequently found. It may be included in a 'miscellaneous'

CONTENT ANALYSIS

OF VISUAL IMAGES

value if its limited visibilify is not consideredimportant for the hypothesesbeing evaluated.However, the fact that a role is only infrequently depicted may be of great interestto the researcher,in which casesucha value should be explicitly included in the rangeof valueswhich make up the definedvariable. on four Table2.1 Genderdepictions hypotheticalvalues in magazine advertisements: variables. Variables

Fnod Male

*i

Sded;

_di

Houseduties

;

$uffisf

tr;

lfDi

Domestic

Fullpage

Public

Half full page

Nurse Values

Female

Business executive Flightattendant

More than half page

Teacher

Double page

5*

4

* These values are purely indicative; in fact there are likely to be many more values on this variable.

Definitions of variables andvalues The principal difficulty confronted by researcherswishing to conduct a comprehensive content analysisis to define unambiguouslyboth the variablesand values so 'coders') that observers(usuallycalled can classifrrepresentedcontent consistently (reliably). 'categories',I Insteadof speakingof have distinguishedvariablesand values. Eachvariable,and within it eachvalue, should be defined in terms of one principal featureof representation. This is important evenwhen a variableis physicaland seems 'duration' (in to be relatively unambiguous. The seconds)of television advertisementsis clearlyanunambiguousmeasure,butthe sizeofdisplayadvertisementsmay not be. Magazinesvary in size.So the absolutesize (areain squarecentimetresor inches) of an item such as an advertisementmay not be an appropriatevariableto allow comparison acrossmagazines.If it is not, the researcherneedsto define the variable(size)and its valuesto bestdemonstratecomparativesalienceor'noticability . In the example in Table 2.1 this was done by classifting the size of the published advertisementrelativeto a standardpageof the respectivepublication (astwo page, one page,half page,etc.). 'subjective'variables(thosewhich require codersto make subjective Other judgement) demand very clear definitions and criteria. Otherwisejudges (coders)

1 7

18

HANDBOOK

OF VISUAL ANALYSIS

may apply inconsistentcriteria, or one coder may changeher or his criteria through the courseof coding a largenumber of instances. Apart from physicalfeaturesof imagesor other visual items (such as size, duration and monochrome/colour) the overalltopic or subjectof newsitems or the product categoryof advertisements will generallybeof interestto researchers. In such cases,the questionof the appropriatescaleof analysisbecomesimportant. Doesone needto define,say,cameraanglesor shot lengthsin relation to representedparticipants (suchaspolitical figures)in analysingtelevisionnews?Or is a globaljudgementabout the focus of news items all that is required?In the latter case,the variable will be namedsomethinglike 'newsstory topic', and valueswill be definedto permit coders to classiftitems to showthe 'agenda'of issuesof one or more newsprogrammes.For instance,Bell et al. ( 1982)comparedthe agendasof differentnetworks'newscoverage during an Australian Federal (National) Election by defining values such as the following on the'news topic' variable:

(a) Celebrities & personalities: Major emphasis givento the personality(ies) per se, ratherthan their expertise or skills.Item focuses on the personasnewsworthy owing to glamour/wealth/bravery/prestige, or as embodimentof someother desirable or undesirable characteristic(s). Excludes Vice Regal,Monarchy(see ViceRegal& Monarchy).Forexample: Kirk Douglasarrivesfor AustralianFilmAwards. Formerbeautyqueengraduates from policeacademy. (b) Crime: Major emphasisgivento the manifestcriminal eventor its consequences: robbery,murder,swindlesetc.or to technicalor biographical stories of crime/criminals. For example: Inquiry into penetrationof the Paintersand DockersUnion by organized crime. Iewel thievesarrested. (c) Disasters:Major emphasisgiven to climatically induced phenomena causing human suffering (flood, heatwaves),geologicalevents of similar kind (earthquakes,avalanches).Excludesaccidents(seeAccidents & ChanceEvents).For example: Newcastledistrict bushfire. New South Wales Government aid to drought areas.

As these definitions show, coding something as complex as audio-visual news stories demands general, yet precise definitions. The kinds of data that are yielded by analysing a large corpus of material such as television news coverage during an election campaign are illustrated in Table 2.2 taken from the same research monograph.

CONTENT ANALYSIS

Table 2.2

1 2 3 4 5 6 7 B I 10 11 12 13= 13= 15 16 17 1B 19

20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35= 35= 35= 35= 35= 35=

OF VISUAL IMAGES

Duration of content emphasis categories in rank order, pre-election period, per se and as percentage of total Channel 9 broadcast time.

Sport FederalElection

112:16 81:38

24.6 17.9

1Oo/o Accidentsand chance events Crime Celebrities and personafities Disasters War Arts, entertainment and leisure Demonstrationsand protests Health and safety Investmentand shares

35:31 27:42 24:24 23:00 21:21 13:OO 11:45 1O:12 9:21

7.8 6.1 5.3 5.0 4.7 2.8 2.6 2.2 2.O

2o/o Economy Industrial relations and disputes Prisons Defence Energy Elections - other Internationalrelations Vice regal and monarchy

7:51 7:'17 7:17 6:50 6:48 6:01 5:51 5:25

1.7 1.6 1.6 1.5 1.5 1.3 1.3 1.2

3:58 3:32 3:23 3:02 2=54 2:36 2:28 1:55 1:50 'l:42

0.9 0.8

1o/o Police Legal Security Transport Science,medicineand technology Environment Minerals and resources Communications Administration Rural Trade unions Weather Socialwelfare Employment Education

1:40 1:37 1:30 O:25 O:22

o.4 o.4 o.4 o.4 o.4 0.3 0.1 0.1

Oo/o Aboriginalaffairs Animals Housing Migrant and ethnic affairs Religion Women's issues

0:OO 0:OO O:0O O:00 0:OO 0:00

0.oo 0.oo 0.oo 0.oo o.oo o.o0

Total time

7236=24

o.7 o.7 o.6 0.6 0.5

1 9

20

HANDBOOK

OF VISUAL ANALYSIS

Note that some content values are not representedin Channel 9's coverage. 'agenda' Thesemay be very significantasabsences(asissuesnot onthe broadcaster's despitetheir sociopoliticalimportance, for instance,the low priority of Aboriginal issues). 'micro'-level variablesthan does Visual content analysismay focus on more the above.Indeed, there is no limit to how preciseand finely grained an analysis may be other than the ability of the researcherto define clearly,and codersto apply reliably, the specifiedcriteria. Whether a model smiles or not, whether they look at the camera,whether they are clothed in certain ways,their skin or hair colour, can all be subjectedto content analysisand then to quantification. But whether a 'young' model is'attractive', or'American' (asdepicted)is unlikelyto be clearlyand unambiguouslydefinable.This is becauseeachof thesevariablesis a compositeof 'size' (above)eachwould require analysisinto specific more specificvariables.Like dimensions (variables)of definablevalueswhich are one-dimensionalrather than 'composite'(and thereforepotentiallyvague).

andcross-tabulations results:comparisons Quantitative Content analysisclassifiesextensivefieldsof representationin quantitativeterms.As was emphasizedearlier in this chapter,the kinds of hypotheseswhich such quantificationhelpsto testarethosewhich compareone field of representationwith another. 'men's' and'women's' The examplesreferredto aboveincludedcomparisonsbetween roles. The data of gender representations in terms ofvisual magazineadvertisements published setout in Table2.2 arenot comparative.However,that tablewasoriginally with relevantcomparisontablesshowing how other televisionchannelsprioritized their respectiveagendasduring the sameperiod (accordingto the samevariablesand values,of course).Suchtabulation of databy more than one variable(here,television station by topics) is typical of the comparativecross-tabulationswhich result from content analysis. Another exampleillustratesthis more succinctly.In Table2.3, adveftisements showing one or more femaleor male figures,or parts of the body (for example,an arm wearinga watch), or groups of malesor females,havebeen analysedaccording 'visual modality' of the respectiveadvertisementpublished in popular to the 'Modality' here refersto the 'truth value' or credibility of (linguistically magazines. realized)statementsabout the world (Kressand van Leeuwen,1996)(seeChapter7). 'representpeople,placesand things Theseauthorspoint out that visual imagesalso asthough they arereal. . . or asthough they areimaginings,fantasies,caricatures,etc.' (Kressand van Leeuwen,1996:161).The datain Table2.3 arebasedon definedvalues of modality accordingto visualfeatureswhich havethen beencross-tabulatedby the participants(persons). genderof represented Thesedata support the following conclusions(among others): 'factual'styleadvertiseI Malesare more likely than femalesto be shownin ments. (approximately two-thirds)adopta standard advertisements 2 Most magazine modality.

CONTENT ANATYSIS

OF VISUAL

IMAGES

When more than one model is shown, gender makes no difference to advertising modality. Women are depicted in approximately 70 per cent of advertisementsin the magazinesin question; men in approximately 30 per cent of comparable advertisements.

Table 2.3

Modalityand gender in magazineadvertisements.

} F*qr' ; ;; Person

Standard

Male n

139

o/o

Female n

59.7

310

Factual

Fantasy

q Total

54

40

233

23.2

17.2

100.0

54

92

456

11.8

20.2

100.0

o/o

68.0

Body part n

27

2

11

o/o

67.5

5.0

27.5

100.0

Female group n

6

52

36

10

o/o

69.2

19.2

Male group n

29

o/o

63.0

Total n o/o

541 65.4

I 19.6 129 15.6

11.5 I 17.4 157 19.0

40

to0.o 46 100.0 827 100.o

Such conclusions are relevant to a visual content analysisbased on theoretical conceptsfrom semiotics.As has been emphasizedearlier in this chapter, any theoreticallyspecificvariableand the valuesofwhich it is comprised,canbe quantified provided that it is explicitly definedand can thereforebe reliably coded.

RELIABILITY 'Reliability'

refers to the degreeof consistenryshown by one or more coders in classii'ing content accordingto definedvalueson specificvariables.Reliability can be demonstratedby assessingthe correlation between judgements of the same sampleof relevant items made by different coders ('inter-coder reliability') or by one coder on different occasions('intra-coder reliability').

Measuring reliability As content analysisclaimsto be objectiveand thereforecapableof being replicated, it must be reliableor consistentif its resultsare to be of value.Reliabilityis a simple

21

22

H A N D B O O KO F V I S U A L A N A T Y S I S

but important concept.It canbe thought of asa quantitativeindex of the consistenry or lack of measurementerror in a particular content analysis.Obviously, if a tool usedfor measuringany quality (saythe lengthsof objects)doesnot yield the same value eachtime it is applied to a given object, then it is not a reliableor consistent tool. Although content analysisis not literally a tool in the physical sense,it is a method of classificationand of quantification, so its definitions must be precise enoughto be usedreliably. To achievehigh levelsof reliabiliry, the researchermust: (a) Definethe variablesand valuesclearlyand preciselyand ensurethat all thesedefinitionsin thesameway. codersunderstand (b) Trainthecodersin applyingthedefinedcriteriafor eachvariableandvalue. (c) Measurethe inter-coderconsistency with whichtwo or morecodersapply similarto, but not part of, the criteria(definitions)usinga setof examples corpus. the research If only one coder is to be employeda pilot (trial) study should be conducted to measureintra-coderreliability' Have the coder classif'50-100 exampleson all relevantvariables.A week later, repeatthe procedure (without, of course,referring to the previousresults).Correlatethe two setsof classifications. Severalmethodshavebeendevisedfor quantitnttg reliability. The two which requirethe leastmathematicalsophisticationare per cent agreementand'pi'. Per cent agreement Simply calculatehow frequentlytwo codersagreein their judgements,ensuringthat they are both testedon the same,largenumber of items (that is, on a representative rangeof variablesand values).For the mathematical,an index can be calculatedas in the exampleof Table 2.4,which tabulates(hypothetical)data from a pilot trial of 100classificationswith four coders.

with four coders. datafrom a pilottrialof 1O0classifications Table2.4 Hypothetical *, % Agleement 80 60 90

Gffirs !9'r, 1,2 1,3 1,4

thethreepercentages: reliability- average Calculating = 76.70/o (0.767) Reliability= (80+ 60+ 90)0/o1300 If thesewere actual results,the researchermight choosenot to employ the 'aberrant' most coder in the next stagesof the researchproject, so that the average reliability (betweenthe other three coders)was higher (0.85).Even here,however, the level of reliability is below what is usually recommended,namely 0.9 or 90 per cent.

CONTENT ANALYSIS

OF VISUAT IMAGES

With four coders,such an index is rather artificial. It assumes(arbitrarily) that one coder is the norm and averagesthe others' respectiveagreementswith this norm. There are two rather technicalpoints to bear in mind. First, a high frequency of items classifiedas 'miscellaneous'or 'other' will spuriously inflate the apparent level of reliability. Lessthan ten per cent of items should fall into this categoryan any variable.Second,the fewer valuesthere are on a given variable,the more likely there is to be agreementbetweencoderswhich is basedon chancerather than on similar judgementsaccordingto the definitions.Sobinaryor tripartite classifications would needto be closeto 100per cent reliable. 'Pi': a more sensitive measure of reliability Becausethe index (above) ignores the fact that two coders may agreein their judgementspurely by chance(and that this is, of course,a function of how many valueshavebeen specifiedon particular variables),a better index hasbecomemore widely used.For example,if a coder is askedto judge whether imagesshow either a male or female, 50 per cent agreementcould occur by chance (without even looking at the images).With five valueson a variable,20 per cent agreementwould be expected.And so on. But such agreementis not really indicative of reliability. So Scott (1955) has proposed one of severalmore subtle formulae for assessing reliability by taking account of chanceagreementsbasedon the number of values distinguishedin a given content analysis:'pi'. pi = (per cent observedagreement) - (per centexpectedagreement)/(1 per cent expected agreement) However,this assumesthat the researchercan statethe expectedpercentagesfor all values on all variablesin advanceof the coders'judgements being made. This is becausethe percentageof expectedagreementis the sum of all the squaredpercentagesof all categories.So if there are six valueson a variable,and if the expected and obtainedpercentagesare asbelow, the pi coeffrcientor index would be calculated thus: Variable: occupational role Business Hospitality Sport Academic Leisure (non-sport) Other

Expected frequency

E.f. squared

20% (O.2O',) 1 O %( 0 . 1 0 ) 30% (0.30) 2OVo(O.2O) 1 5 %( 0 . 1 5 ) 5% (0.05)

o.0400 o.0100 0.0900 0.0400 o.0225 0.0025

Expectedfrequency= sum of the squaresof the e.f.values= 0.2050. If two codersclassifringa sampleof 100imagesdepicting occupationalroles in advertisements,for instance,agreedon 95 per cent of cases,then the reliability = 0.94 (the 'pi' value). index would be: (0.950-0.205)l(1.00-0.205)

23

24

HANDBOOK

OF VISUAL ANALYSIS

However, if the obtained agreementbetweenthe codersin the pilot or trial 'expected were found to be low, or if there were fewer or lessevenly distributed 'pi' frequencies',the value of the index would be reduced.As a rule of thumb, a value of at least0.80 should be obtained. If this is not achievedin a pilot trial, the researchershould re-train codersand/or redefinevaluesand re-test the degreeof reliability.

LIMITATIONSAND EXTENSIONS The main limitations of quantitative content analysisconcern the relatively un'manifest content' that it claimsto analyse theorizedconceptsof messages, texts or objectivelyand then to quantifr (seeChapter4). 'content' which are most frequently quantified in The categoriesof visual suchas'roles'depicted, media researcharisefrom commonsensesocialcategories, 'settings' shown, gender and age of representedparticipants in images. Such variablesare not defined within any particular theoretical context which analyses 'angles', scaleof visual semiotic dimensions of texts. That is, the framing, visual 'shot', are visual analysis photographic part discourse of of the and so on, that are thesein terms seldomincorporatedinto (visual)contentanalysis(Chapter7 discusses 'semiotic resources').Nor are categoriesfrom, say,Marxist or neo-Marxist theory of (to take another contentioustheoreticalparadigm) seenas appropriateto quantification. Indeed,T.-W. Adorno (the famouscultural critic) hasquipped that'culture' is, by definition, not quantifiable.Other critics of content analysispoint out that the inferencesthat are madefrom quantificationto qualitativeinterpretation (especially 'bias') thoseinvolving notions of arefraughtwith diffrculties.The cultural complexity ofvisually codedtextsmeansthat either only the most simplistic,sociallyconventional cate-goriescan be studied, or content analysisimports tendentiousor highly interpretedabstractionsinto its ostensibly'objective' definitionsof variablesand/or values. Winston (1990)arguesthat researchpurporting to demonstratetelevisualbiasagainst businessmenfails because,in part, it fails to understandthe (semiotic) codesof television - the way television means to its audience.The result is that coding categoriesusedin researchinto this questionwere frequentlymoralisticand decontextualized.For instance,whethera programmedepictinga businessmanis a comedy 'reads' that character'sbehaviour or a drama would be relevantto how an audience 'negative' and Clowns, the study criticized as or'positive'. However, Crooks,Conmen by Winston, ignored the ways in which different genresare understoodby their respectiveaudiences.(The genresand semiotic codesof tele-films are discussedin Chapter9.) 'violent In a widely influential paper,StuartHall ( 1980)makesa similar point: incidents'in cinematicgenreslike the Westernaremeaningfulonly to audienceswho know the genres'respectivecodes- the goodie/baddieopposition, the conventions relating to resolvingconflict, and so on. In short, content analysiscannot be usedas though it reflectsunproblematicallyor a-theoreticallythe socialor ideologicalworld outsidethe particularcontextofthe medium studied.Second,contentanalysiscannot

CONTENT ANALYSIS

OF VISUAL IMAGES

'bias' or be easilycomparedwith someassumed'reality'bywhich to make claimsof 'negative', these let alone'true' or'false', representation. Winston (1990)discusses 'inference'problems in convincing detail.Third, generalizingfrom content analysis results can be difficult, and claims made for the consequencesof the quantitative picture of media content may go beyond what is validly licensedby the data. For example,it is sometimesassumedthat users(viewers,audiences)understandor are affectedby texts, genresor by media content generallyin ways that reflect the kinds 'bias' in of analyticalcategoriesusedin content analysisitself.An exampleis that of televisionreporting of political issues.It is often assumedthat more coverageof one political party than its competitorsmeansthat viewerswill be more likely to favour it. But this is, obviously, an unwarranted inference.At the veryleast,suchan inference would require additional researchin order to be validated. Visual representationsraise further theoreticalproblems of analysis.Many highlycoded,conventionalgenresof imagery(the footballersavinga goal,the'cover girl' from Cleo,thepolitician engulfedby microphones,the academicexpertin front of rows of books) havebecomemedia clich6s.To quantifi' suchexamplesis to imply that the greatertheir frequency,the greatertheir significance.Yet, the easylegibility of clichdsmakes them no more than short-hand, stereotypicalelementsfor most viewers who may not understand them in the way that the codes devised by a researcher imply (see'Looking:form and meaning'in Chapter4 onpage7}).

VALIDITY: GOING BEYOND THE DATA To conduct a content analysisis to try to describesalient aspectsof how a group of texts (in our case,imagesor visualtexts) representssomekinds of people,processes, events,and/or interrelationshipsbetweenor amongstthese.However, the explicit definition and quantification that content analysisinvolves are no guarantee,in themselves,that one can make valid inferencesfrom the data yielded by such an empirical procedure.This is becauseeachcontent analysisimplicitly (or, sometimes, explicitly) breaks up the field of representationthat it analysesinto theoretically defined variables. In this way, it is like any other kind of visual or textual analysis. 'modalifr' or'represented Semioticspositsassemanticallysignificantvariablessuchas participants', or conceptualversusnarrative image elements(seeChapters4 and 7). All systematicmethods of visual analysis(evenpsychoanalyticor iconographic - seeChapters5 and 6) postulatesomefeaturesof images(and not others) as semantically significant,within the images'usual contextsof exhibition and reception. Content analysisdoes this also. What is at stakein conducting this, or any form of visual analysis,is the degreeto which the resulting statementsabout the field analysedcan be said to describefeaturesthat are, in fact, semanticallysignificant for viewers/spectators/'readers' of the images.In other words, does the analysis 'use' the yield statementsthat are meaningful to those who habitually'read' or images? This brings us back to the point at which this chapterbegan.Wheneverwe generalizeabout differencesbetween setsof images,we must implicitly define as

25

26

HANDBOOK

OF VISUAL ANALYSIS

well asobserveand quantif' the visual material in question.We might, for example, claim that the front pagesof broadsheetnewspapersshow more, although smaller, images,of more activeparticipants,than do tabloid paperspublishedconcurrently. Such a claim is a hypothesiswhich content analysiscould test.To acceptthe claim as true is to accept that something like a systematiccontent analysiscould be conductedand that it would demonstratethe differencesclaimed. The criticism that is most frequently levelledagainstcontent analysisis that 'categories') the variables/values defined(the aresomehowonly spuriouslyobjective. It is claimed that they are as subjective as any semantic variables despite being 'measured' or at leastcounted.However, such a criticism can be turned around, to point to the factthat not only contentanalysisbut all visualor verbalsemiotics,formal and informal, areonly as valid as the explicitnessand reliability of their respective theoreticalconcepts. To make inferencesfrom the findings of a content analysisor any kind of theoreticalanalysisof a group of imagesmeansthat one goes'beyondthe data'making a prediction about the salience,the social or ideological importance, the visual significance of one'sfindings.So it is bestto think of thesefindingsas'conditional''true' until and as bound by their theoretical and methodologicalcontexts,but as unlessthey are contradictedby further evidence. New theorieswill proposethat different variablesof imagesare semantically significantand posit new definitionsby which new generalizationscanbe tested.The principal virtue of content analysisis that it is explicit, systematicand open to such theoreticallymotivated,but empirically grounded critique. Thoughtful, provisional 'positivistic' 'pseudo-scientific'than competing content analysisneedbe no more or methodologies,but it is important to propose the findings of such analysesas conditional to their context.Somequestionsarenot usefrrllyanalysedby suchmethods, and many claimshavebeenmadefor content analysiswhich areimpossibleto justifr. But, asone approachamongothersto describinghowandwhatimagesmean,content analysiscan be a usefulmethodology.The questionof how one can generalizeor'go 'validity', to which I beyond the data analysed'(as I have put it) is the problem of now turn. Content analysis,by itself, doesnot demonstratehow viewersunderstandor value what they seeor hear. Still, content analysisshows what is given priority or salienceand what is not. It can showwhich imagesareconnectedwith which, who 'run'byparticular is givenpublicity and how, aswell aswhich agendasare media.Or, to put theseclaimsmore cautiously,content analysiscan demonstratesuchpatterns ofmedia representationprovidedthat one acceptsthe validityofthe categories(values and variables)definedin the research. 'Validity' refersto the (apparently circular or tautological) concept of how well a system of analysisactually measureswhat it purports to measure.Valid inferencesfrom a particular content analysis,given this definition, will reflect the degreeof reliability in the coding procedures,the precisionand clarity of definitions adoptedand (a lessobviousfactor) the adequaryofthe theoreticalconceptson which the coding criteria are based.Validity refersto the confidenceone can have in the resultsshowingthat the statedtheoreticalconceptsoffer a discriminatingdescription

CONTENT ANALYSIS

OF VISUAL IMAGES

'Images of violence' (to return to a controversialexample) of the field being analysed. will be definedin different waysby different theoreticaland pragmaticinterests.For instance,a media student might seethe televisingof accidentalcausesof physical 'violent' injury or deathasa depictionof episodes.However,an alternativetheory of televisedviolencemight excludeaccidents,arguing that'violence' should only refer to what, in their dramatic contexts,are visibly intentional actionswhich are shown to causephysical and/or psychologicalharm. The relative validity of a particular content analysis,given theoreticaldifferencesin the terms in which an hypothesis can be formulated, refers to the degreeto which inferencesare justified from the 'internal' consistenry findings to the theoreticalstatements.Whereasreliability refersto in one's method, validity refersto the external or inferential value of one's research, but it is important givenits theoreticalcontext.This cannotbe quantitativelyassessed, to askof one'sown (and,indeed,of others')researchwhetherit actuallydemonstrates what it purports to demonstrate,whether the variablesand valuesdo allow one to 'measure'the conceptsincorporatedin the hypothesesand their respectivetheories. As we haveseen,content analysisprovidesa quantified dimensionaldescripThe methodologycanbe usedto provideabackground tion of fieldsof representation. '-up'of a domain of visualrepresentation.Having conducteda content analysis,the researchercan then interpret the imagesor the imagery in qualitative ways (using semioticor someother more individual, text-orientedtheory suchasthosedescribed in the other chaptersof this volume). Typical or salient examplescan be further 'what the data mean'. So, having analysedto fill out the qualitative description of shown how frequently and in what contexts,say,imagesof passivefemalesoccur, a researchermight discussthe psychoanalyticor ideologicalsignificanceof the images in terms of metaphors,photographicstyle,historical or socialcontext (seeChapters 4and7).

AN EXAMPLE: TESTING SEMIOTIC HYPOTHESES Images carry connotations and invite individual reminiscence.They may convey a senseof duration or of nostalgiathrough codesof colour, framing and through their public context.An imagecan engagethe viewer in a fetishisticand compulsive 'owns' the image urge to look and look again,encouragingthe sensethat the viewer 'identity' (see'Looking: recognition and identity', in or that it is part of his or her Chapter 4, page 83). None of these experientialpossibilitiescan be defined and quantified very reliably. Yet dimensions of interactive' meaning (involving how 'textual' meaning (including how the viewer is invited to relateto an image) and of images are formally composed or balanced) can be defined and their frequency counted. In Chapter 7 of this book, the systemicfunctional (semiotic)approachto 'manifest content' of imagesthat imageanalysisis presented.By definition, it is the content analysisdissectsand counts, so any unambiguously definable aspectof a group of imagescan be quantified. Insofar as semioticsalso involves an empirical (observational)methodology,it is possibleto usethe analyticalconceptsderivedfrom

27

28

H A N D B O O KO F V I S U A L A N A L Y S I S

this type of theory as the basis for quantification and, hence, for comparative generalization.So to conclude this chapter, I will illustrate how it is possible to quantifi. someimportant dimensionsof what Kressand van Leeuwen(1996)call the inter-personalsemiosisof imagesfocusing on depictionsof women on women's magazinecovers.I will usetheir semiotic model to define three variables,eachwith severalvalues.Theseare meant only to illustratethe way in which a content analysis of explicitly semioticvariablescould be conducted. The specifiedvariablesand valueswill be usedto codethe two setsof twenty cover-pageimagesfrom CleomagazinereproducedasFigures2.I and2.2. Thesewill be the basisof a partial or small-scalecontent analysisto illustrate that: 1 Comparativehypothesescanbe formulated and testedusing reliablecategories relating to the semioticsof visual images. 2 Content analysiscan be conductedby giving precisedefinition to theoretical concepts(in this casefrom semiotics). material. The 3 Objective criteria need to be specifiedfor categorizing criteria must be defined unambiguously.

This small-scaleanalysisof media content will be set out in a conventional science'format, as one might find in psychologyor sociology,for example: Hypotheseswill be formulated; a Procedure(or Method) will be described;Results and discussionsectionsof a typical empirical report will be presented.Under these headings,other conventionalsub-sectionswill be named (such asvariables,values, 'worked' reliability). So this exampleof a theoreticallymotivated empirical analysis will serve as a summarizing overview of the method of visual analysis(content analysis)that hasbeenthe subjectof this chapter. Figures2.1 and 2.2 show respectivelytwenty front coversof Australian Cleo magazinefrom 1972-4and twenty from 1996-7.Thesewill be comparedby means of a content analysis. 'social

Hypotheses Following Kressand van Leeuwen(1996) it is hypothesizedthat the inter-personal meaning of Cleofront pageshas changedbetween 1972 and 1997in the following ways: t9724 The later images present models as more socially distant than covers. The modalitF of the imagesin 1996-7 is lower than in the earlierperiod. The model s' gazeat or gazeaway from the camerais not different between the two setsof images,but the poseof the models (their head,body dispositions) are less'powerful' in the more recentexamples. The 1996-7modelsare more frequentlyblonde, lessfrequentlybrunette, than in the earlierperiod. The later coversdepict modelswho look younger than do those from the earlier period. Models occludethe magazinelogo more frequently in the later period.

CONTENT ANALYSIS

OF VISUAL IMAGES

Method:definitions andcriteria Let us begin by defining eachvariableand the respectivevalueson eachbeforeusing theseas criteria to code our small samplesof images.For the sakeof exposition,I will simplifu relevant theoretical semiologicalconcepts (which are the basis for the definition of variablesand valuesin the terminology of this chapter). Variable 1: social distance Kress and van Leeuwen relate the representedsocial distance of participants 1n an image (in our examples,the depictedmodels) to Hall's (1966) discussionof 'proxemics' (the psychologyof people'suseof space): In everydayinteraction, social relationsdetermine the distance(literally and figuratively) we keep from one another. Edward Hall (e.g. 1966: 110-20) has shown that we carry with us a set of invisible boundariesbeyond which we allow only certain kinds of peopleto come. The location of theseinvisible boundariesis determinedby configurationsof sensorypotentialities- by whether or not a certain distanceallows us to smell or touch the other person, for instance,and by how much of the other personwe can seewith our peripheral (60 degree)vision. . . . With thesedifferencescorrespond different fields of vision. At intimate distance, saysHall (1964),we seethe faceor head only. At closepersonaldistancewetake in the headand the shoulders.Atfar personaldistanceweseethe other personfrom the waist up. At closesocialdistancewe seethe whole figure. At far socialdistancewe seethe whole figure.At far socialdistancewe seethe whole figure'with spacearound it'. And at public distancewe can seethe torso of at leastfour or five people.It is clear that thesefields of vision correspondcloselyto the traditional definitions of sizeof frame in film and television, in other words, that the visual systemof size af frame derives 'proxemics', from the as Hall callsit, of everydayface-to-faceinteraction. (Kressand van Leeuwen,1996:I29-3I, italics added)

On this basis a variable of visual representation, social distance, can be dMded into six values:

(a) (b) (c) (d) (e) (f)

Intimate. Closepersonal. Farpersonal. Closesocial. Farsocial. Public.

These are defined in terms of how much of the (human) participant's body is representedin the frame of the respectiveimage, as in Kress and van Leeuwen above(seeChapter 9 for a discussionof socialdistancein relation to the analysisof tele-films). Variable 2: visual modality Again, following Kressand van Leeuwen,visual modality resultsfrom

29

30

HANDBOOK OF VISUAT ANALYSIS

(colour,representational Thedegreeto whichcertainmeansof pictorialexpression detail,depth,tonalshades, etc.)areused.Eachof thesedimensions canbe seenasa scale,runningfrom theabsence of anyrenditionof detailto maximalrepresentation of detail,or fromtheabsence of anyrenditionof depthto maximallydeepperspective. And on eachof thesescales thereis a point thatrepresents thewaythegivenpictorial dimensionis usedin whatcouldbecalledstandard To thedegree naturalism. thatthe 'less useof a dimensionis reduced, it becomes, at leastin onerespect, moreabstract, 'more thanreal'.To the degreethat is amplified,it becomes thanreal',andwe associatedthiswith a 'sensorycodingorientation',an emphasis on sensory pleasure(or displeasure, asin thecaseof 'morethanreal'horrorimages), andanattemptto come (Kressand van ascloseaspossibleto a representation that involvesall the senses. Leeuwen, 1996:256) However, the authors argue that modality is context dependent.That is, modality is coded accordingto particular 'orientations' appropriateto different conventionaldomains of representation.The domains they distinguish include scientific/technological, abstract,naturalisticand sensory.A'sensory'coding orientation is appropriateto imageswhich provide sensuousand sensorypleasureto the viewer. Highly saturatedcolour conveyshigh modality in paintings such as nude or'still life' genresand in most modern displayadvertising(seeChapter7). Soavariable,modality,canbe definedasthe represented'realism'ofan image, given the sensorycoding orientation, basedon degreesof colour saturation.Three valuescould be distinguishedon this variable:'high', 'medium' and 'low'. These are relative terms and criteria may be difficult to specifr. But, as has been pointed out in the introduction to this chapter, all values must be defined clearly if reliable classifications are to be made,so, in principle, the caseof modality posesno unique methodologicaldifficulties.We might, then,setthe criteriafor the threevalues thus: (a) High sensorymodality: image useshighly saturatedcolours naturalistically. (b) Medium sensorymodality: image uses,for example,lesssaturated,'washed 'ethereal' out' or useof pastels. (c) Low sensorymodality: imageis monochrome (black-white) only.

Variable 3: behaviour Kress and van Leeuwen also discuss the way interaction between the viewer and the people shown in images is affected by the gaze of the represented participants. Simplif ing this to allow our illustrative example to be outlined, the principal 'image differences between what they call acts' relate to the ways in which images 'offers' 'demands', make or as it were, to their respective viewers (see Chapters 7 and 9): There is . . . a fundamental difference between pictures from which represented participants look directly at the viewer's eyes,and pictures in which this is not the case.When representedparticipants look at the viewer, vectors,formed by participants' eyelines,connectthe participantswith the viewer.Contact is established,even if it is only on an imaginary level.In addition there may be a further vector, formed by a gesturein the samedirection. . . .

CONTENT ANALYSIS OF VISUAL IMAGES

This visual configuration has two related functions. In the first place it createsa visual form of direct address.It acknowledgesthe viewers explicitly, addressingthem with a visual'yo.t'. In the secondplaceit constitutesan'image act'. The producer uses the image to do something to the viewer. It is for this reason that we have called this 'demand': the participant's gaze (and the gesture, of present) kind of image a demandssomethingfrom the viewer, demandsthat the viewer enter into somekind of imaginary relation with him or her. Exactly what kind of relation is then signified by other means,for instanceby the facial expressionof the representedparticipants. They may smile, in which casethe viewer is asked to enter into a relation of social affinity with them; they may stare at the viewer with cold disdain, in which casethe viewer is askedto relate to them, perhaps,as an inferior relatesto a superior; they may seductivelypout at the viewer, in which casethe viewer is askedto desire them. . . .In each casethe image wants something from the viewers - wants them to do something (come closer, stay at a distance) or to form a pseudo-socialbond of a particular kind with the representedparticipant. And in doing this, imagesdefine to some extent who the viewer is (for example, male, inferior to the represented participant, etc.), and in that way exclude other viewers. (Kressand van Leeuwen, 1996:122-3) From a different theoretical perspective, Goffman (1979) has analysed the ways in which gendered bodies are represented in advertisements, distinguishing 'head 'infantile' 'rituals bodily poses. These include several of subordination' or 'bashful knee-bending' canting' (tilting the head down or sideways while looking up), (where one knee is bent) and self-touching (as opposed to holding or performing work with one's hand(s)). He analyses these represented behaviours in terms of power: that is, that these three actions (or behaviours) signify powerlessness, and place the viewer in a position of superiority to the represented participant (model). (Chapter 8 analyses behaviour within an ethnomethodological approach.) Combining the image act analysis of Kress and van Leeuwen's semiotics and Goffman's observations, we could distinguish the following values on the variable called behaviour. (a) Offer/ideal:the model depictedoffers herself/himselfas an idealizedexemplar of a classor attribute, looking away from the viewer (for example,the statuesquepose of a femalemodel displayingclothes). (b) Demand/affiliation (equality):model looks at the viewer,directly, smiling. (c) Demand/submission:model looks down at viewer,not smiling. (d) Demand/seduction:model looks up at the viewer, head canted,smiling or 'pouting'. (e) None of the above. So in our hlpothetical

study, three variables can be defined, with several

values on each, as shown in Table 2.5. Hypotheses relating to each variable have been formulated, in Table 2.5.

Procedure To carry out the actual analysis,two coders would have been trained to classifr imagesaccordingto the specifieddefinitions. Their reliability would be calculated

31

32

HANDBOOK OF VISUAL ANALYSIS

and, if below 0.90,further training would be given until this level of reliability was attained. Coderswould be presentedwith the two setsof imageswhich they classified accordingto the definitions.Eachimageneedsto be presentedseparatelyand individually. The order of presentationwould thereforebe randomized.So that the coders brought no potentially biasing expectationsto the tasks,they should not know the hypothesisbeing tested.Presentingthe imagesgrouped chronologicallyas they are in the magazinebeing analysedwould be likely to lead coders to make similar judgementsabout eachset.(Psychologists call thesepotentiallybiasingconditions of experimentalprocedures'demandcharacteristics'. Theymay leadto codersor subjects in a psychologicalexperiment, or even the researchersthemselves,unconsciously increasingthe likelihood of a hypothesisbeing confirmed.) Table2.5 In our hypothetical studythreevariables canbe defined,withseveralvalues on each.

UaHable

Value

Social distance

1.1 1.2 t.3 1.4 t.5 t.6

intimate close personal far personal close social far social public

Modality

2.1 2.2 2.3

high medium low

Behaviour

3.1 3.2 3.3 3.4 3.5

offer - ideal demand - affiliation demand - submission demand - seduction other

Resultsandanalysis A numericalsetof resultswould be produced,as in Table2.6 (theseare,of course, hypothetical). Statisticalanalysis(such as appropriate'Chi-square'tests)would be applied to show whether the distributions of values for each variable are significantly different betweenthe two periodsbeing compared.

Discussion Acceptingthe dataaspresentedin the table,someof the hypothesescanbe confirmed without explicit definition of criteria. For example,the hypothesisthat the 1996-7

CONTENT A N A L Y S I SO F V I S U A LI M A G E S

Table 2.6 Classificationof distance,modalityand behaviourof female models on the front covers of C/eo. 1972-4 and 1996-7.

; 1s7&1%@j 1 Socialdistancel.l

2 Modality

r_hs-i

t.3 1.4 t.5 1.6

0 30 40 25 5 0

0 5 10 75 10 0

2.1 2.2 2.3

85 15 0

0 25 75

3 Behaviour :,i 3.4 3.5

o

o

40 30 30 o

40 60 20

o

modelsare more frequentlyblonde, and lessfrequentlybrunette, than in the earlier period: in 1996-7,only four of nineteenmodelsare not blonde, whereasin 1972-4, 'not ten out of twenty could be judged blonde'. (It could be arguedthat there are more redheadsin the 1972-4 period as well.) The hypothesisthat the later covers depict modelswho look younger than do those from the earlier period required judgementswhich would need to be confirmed by more than one coder or judge 'judged acting independently,perhapsusing a seriesof age-ranges'as the basisof (18 classification yearsand under, 19-25 years,26-30 years,30 yearsand over, for example).You will notice that thesejudgementscan be very difficult and that it is unlikely that highly consistentjudgementswould be madeunlessthe valueson this variablereflectedquite broad ageranges.Perhapsonly two categories could bejudged: 'less than' and'more than' 25 years. The hypothesisthat modelsoccludethe magazinelogo more frequentlyin the later period would be clearlyconfirmed byobservation,being an objectivefeatureof the images:Cleohaschangedits graphicconventionsduring the period under review. Trivial although this observationmay appear,it is a featureof the visual meaningof 'identity', its the coversthat could be significantin showinghow the magazinepresents and so could be analysedaspart of the quantitativecontent analysis. In terms of the more semiotic hypotheses,the hypothetical data show that women on the covers of Cleo magazineare representeddifferently in 1996-7 comparedwith 1972-4.Specifically,the later imagesin the publication are more distant, lessclose-up,than thoseof 25 yearsearlier.Second,modality (in terms of colour saturation)is lower in the more recentmagazines'covers.Third, the modelsdepicted in the 1990sare more likely to be shown in lesspowerful poses,even though they address(gazeat) the viewer equally frequently. The implications of thesefindings could be discussedin terms of changingroles of women, the comparctiveyouth of

33

34

HANDBOOK

OF VISUAT ANALYSIS

recently famous fashion models, the demographyof Cleo'smarket (now aimed at youngerbuyers),or possiblythe increased'sexualization' of youth culture(and/orof consumerismmore generally).

CONCLUSION Content analysisallowsresearchers to makequantitativegeneralizations aboutvisual and other forms of representation,on the basisof reliable classificationand observation. In the examplepresentedabove,schematicand simplified for the purposesof illustration, empirical results(data) could be usedto support or to refute particular sociologicalor cultural arguments.I hope to havedemonstratedhowto conduct and how to report such a content analysis,and how it could be usedto evaluatesubtle theorization of important aspectsof semiosis.Ideally, however,researchadopting this methodology should supplementand extend its findings by meansof detailed analysisof typical examples.Especiallyin the analysisofvisual texts,content analysis should be seenas only part of the methodologicalarmoury that a researchercan employ.As in all research,the significanceof the information yieldedby this method is only asvaluableasthe theoreticalprecisionof its hypothesesand henceof its defined variablesand values.

REFERENCES Bell,P., Boehringer,K. and Crofts,S. (1982)Programmed Politics:A Studyof Australian Television. Sydney:SablePublishing. Goffman,E. (1979)Gender Advertisements. London:Macmillan. Hall,E. (1966)TheHiddenDimention. NewYork:Doubleday. Hall,S.(1980)'Encoding/decoding', in S.Hall et al. (eds),Culture,Media,Language. London: Hutchinson. Kress, G.andvanLeeuwen, T. (1996)Readinglmages: TheGrammar ofVkualDesign.London: Routledge. Scott,W. (1955)'Reliabilityof contentanalysis: the caseof nominalscalecoding',Public OpinionQuarterly,17: 321-5. Winston,B. (1990)'On countingthe wrongthings',in M. Alvaradoand I.O. Thompson (eds),TheMediaReader. London:BritishFilm Institute.

Hre ffi ffi

Approaches to Analysisin Visual Anthropology MALCOLM COLLIER

INTRODUCTION Analysisof visual recordsof human experienceis a searchfor pattern and meaning, complicatedand enrichedby our inescapablerole asparticipantsin that experience. This chapterprovidesthoughts on processesof analysisthat may build on both the tangible characterof visual recordsand on our varied 'lenses'of personaland cultural identity and experience(seeChapter4). This discussionassumesthat photographsand other opticalrecordsof human experiencemay be both creationsand concretereflectionsofwhat is visiblewithin the scopeof the lensand frame.Much hasbeensaidregardingthe 'constructed'character of photographs,video and film, heighteningour awarenessof the influencesof individual, cultural, political and othervariableson the making, viewing and analysisof visual records.While this discoursehasenrichedour understandingof imagesit can also createan illusion that they contain nothing beyond their constructedcontent. Leavingasideplanned fabrications,it may be saidthat photographs,video and film are, ultimately, complex reflectionsof a relationshipbetweenmaker and subjectin which both play rolesin shapingtheir characterand content (seeChapter 8). When we usethe camerato make a visualrecord we make choicesinfluencedby our identities and intentions,choicesthat arealsoafiFected byour relationshipwith the subject.People arerarelysimply the passivesubjectscommonly assumedin much scholarlydiscussion; they, too, participatedirectly, not infrequently manipulating it for their own ends. Moreover, lfly visual image that is the product of pointing a cameraat the world alsocontainsall the optically visible aspectsof that world within the confines of the frame and the capabilitiesof the lens (seeChaptera). This visual field usually contains a complex range of phenomena,much of which is outside our awareness ascamerapersonor subject.Consequently,the content of the visual imageis rarely shapedonly by the constructiveinfluencesof recordersand subjects,afactreflected in the frequent discoveryof previouslyunseenphenomenaand relationshipsin the processofvisual analysis.Much ofthe promise and valueofvisual imagesin research is basedon this aspectof their character. All of the elementsof an image may be important sourcesof knowledge through analysis,if only we can identifr them and sort them out. The challengeis to responsibly addressthe many aspectsof images, recognrzingthat the searchfor

36

HANDBOOK

OF VISUAL ANALYSIS

'facts' 'truths'but rather or meaning and significancedoes not end in singular 'reality' producesone or more viewpoints on human circumstances,and that while 'error' may be elusive, is readily achieved.

TYPES OF ANALYSIS AND THE IMPORTANCEOF CONTEXTUAL INFORMATION A considerablerangeofvisual analyticprocesses areproductive(asthe chaptersin this volume demonstrate).In broad termsa distinction maybe madebetweenthosewhich directly examinethe content of imagesasdata (seeChapter2) and otherswhich use present imagesmore indirectly asvehiclesor triggersfor information not necessarily (see in the imagesthemselves Chapter6). While the boundary is not absolute,the distinction hasvaluebecause,while virtually all visualimagesand recordshavepotential usein indirect analysis,many are not responsiblysubjectedto direct approaches. We are frequentlyfacedwith visual recordsthat lack strong contextualinformation,that aredivorcedfrom anysystematicannotation,or areisolatedrecords.Such imagesmay be analyseddirectly only if careis taken to properly annotateand establish contextualrelationshipsand to workwithin the limitations ofthe images.Establishing such contextualbackgroundoften involvesmaking useof indirect forms of analysis as well as considerablearchival and other research.In the analysisof singleimages caution shouldbe exercised;indeeddirect analysisof singleimagesapartfrom larger collectionsis often best avoided,although such imagesmay often be very useful in indirect analysis. The processof establishinga contextfor analysismaybe illustratedbya photograph from one of my current researchprojectsshown in Figure 3.1. This powerful imageof a elderlyman apparentlyengagedin wheatharvestis part of an unannotated and fragmenteddocumentary (not research)photographic collection made in our home villagein New Mexico, USA,by -y father,Iohn Collier,lr., earlyin his photographic career.Despitethe power of this image,its analyticpotential waslimited by the lack of substantialinternal contextual information and absenceof annotation. My challengewasto developthe contextualinformation and other annotationalmost sixty yearslater so that the full researchvalue of this and other photographsin the collectionmight be realized. I startedwith the photographsthemselves,hoping that they might collectively provide somecontextfor eachother. I looked at other photos of the sameman; these showhim in otheractivitiesandwith otherpeopleassociatedwith a particularextended family, some of whom I recognizedfrom my own childhood in the community. I looked for other imagesof the wheat harvest,and determinedthat hand-cutting of wheat was not restrictedto this field or this man. Drawing on memory and other photographs,I examinedthe backgroundin the photographand wasableto identifr the precisefield whereit wasmade.I could not identifr the man, who (asI later found out) died beforemy birth. I then moved to the study of correspondence, which provided a likely datefor the photographs,and to photo elicitationsessions with neighboursand relatives.These

ANALYSIS

?i

37

IN VISUAT ANTHROPOLOGY

@;r

ffi;i

ffi"q. ffi ;

;@-l

,ry.ltu) re.

i< \i

=1-r a:t)l ii

f-ffis\)iii

€t u 1 S t u r@i"; ; 'sh;

,,1:11

= *-rt

i

='@ F :::=: 11g"rii::i + ''. # ,t- a:r$i ;',:,:. l4:i

:

@; ; ry4ry-.

?e

s:#rl

\

4

tu'{ t,k

1.-w.. , ,,11$n {'i'il

"" * -q". . "4 " "" "" "' @ " " ; " ?',lr$){ , l +,r , 'fJ .. &4.:='a:

1..:-:

ei

ffih $,t{

*^

).$

' d*d* f \,-

*& ; .-ii ,-$4': ,,,,,' ,.'' { 1.?

,

#

€ ir !r

;

ru

\ilY

..s rr .

ffi d*

,#

'f,{

scl erA

Figure 3.1 Antonio Romo in his wheat field, Rio Chiquito/Talpa, New Mexico (USA). Photograph by John G o l l i e r ,J r . , 1 9 3 9 . A 'good' photograph, its strength as a single image - simplicity of content, close-up view, semi-posed classic harvest stance - all work against its value as data without additional contextualization. Worn, patched overalls, ancient hat, rugged and worn hands, allthese reflect a life of hard work and poverty, as does the fact of hand-cutting the wheat in an era when the better-off farmers cut with machines. Use of photo elicitation provoked a wide range of responses and information, both specific and general.

38

HANDBOOK

OF V

provideda namefor the man and most ofthe other peopleseenin the largercollection, aswell asmuch additionalinformation and interestingstories.With this addedinformation, I cannow responsiblyengage in directanalysisof this photographand the rest of the collection. This exampleillustratesseveralimportant points. First, a'good'photograph is not necessarilygood data if it lacksthe necessarycontextualinformation and annotation to make it analyticallyintelligible.Second,it may be possibleto reconstruct such contextand annotationboth through comparativestudy of relatedimagesand through other forms of research.Finally, it provides a moral for responsiblephotographic recording in anthropology, best stated by an elderly member of the community, Mr Eloy Maestas,during the courseof a sessionin which I was askinghim to help identifr people and places.Frustrated,he placed the images on his kitchen table '|ohn was so careless,he should and (in Spanish)said,with considerableirritation, have written the date, the namesof the people,and the placeson the back of each photograph!'What betterstatementof the needfor systematicmethodologyin visual anthropology! Numerous images,both historical and contemporary,aresimilar to this one and mayrequire evenmore extensiveresearchto establishanydirect analyticpotential. Unfortunately, many'documentary' photographs,films and other visual imagesof cultural experiencehavelimited responsiblepotential asdata eitherbecausethey are overly constructed,aresingleisolatedimages,or becausethey lack sufficient contextual 'Good 'good' documentary' annotation.This is not a matter of or'bad' photography. 'strong' imagesand photographsareoften of limited analyticvaluefor the very reasons that makethem compelling imagesto our eyes.They areoften framed to excludecomplexity of contextor ambiguity of message and they areoften presentedto us assingle images,divorced from the larger photographic seriesfrom which they come. The demandsof a good imagefor a publication or a displayarenot the sameasthoserequired for researchanalysis.Good researchimagescontain complexiry theyrecordassociations and relationships,they are often unremarkableat first glanceand take time to read. The idealsituation is a carefullymadevisualresearchcollectionwith comprehensivetemporal,spatialand other contextualrecording,good annotation,collection of associatedinformation and maintenanceof this information in an organizeddata file. Strongvisualresearchcollectionspresentrich and intimidating potentialbecause they may be productively analysedin many different ways,both direct and indirect. It is the relative rarity of such systematiccollections that leads many scholarsto underestimatethe true potential of photography,film and video as reliablesources of cultural information. Field methodologiesfor making good visual research collections are beyond the scopeof this chapter and readersare referred to other sources(seeCollier and Collier, 1986).

DIRECT ANALYSIS When we examinethe content and the characterof imagesas data,we are engaged in direct analysis.This examination may seekinformation on the subjectsseenin

ANATYSIS

IN VISUAL ANTHROPOLOGY

the imagesor it may extract understandingregardingthe making and functions of the imagesaswell asthe perspectivesof their makers.As other chaptersin this volume examinethe analysisof the constructedcharacterof images,I will focus on handling imagesasdataregardingthe subjectstheyrecord. This discussionassumesa reasonably well-madevisual researchcollectionbut many of the techniquescan be modified to work with lessideal records. Any major analysisshould begin and end with open-endedprocesses,with more structured investigationtaking place during the mid-section of this circular journey. This approachprovidesuswith an opportunity to respondto largerpatterns within the whole that may revealthe new and unforeseen,that provide significant meaning to otherwisechaotic details.Becausemore structured analysisinevitably involves focus on already defined details or points of interest, early descentinto focusedexaminationsis likely to limit true discoveryand foster imposition of prior bias in our analysis.The following model adaptedfrom Collier and Collier (1936) outlines a structurefor working with images.

A basicmodelfor analysis First stage Observethe dataasa whole,look at, 'listen'to its overtonesand subtleties,to discover connecting contrasting patterns. Trust your feelingsand impressions,and make carefulnote of them, identifring the imageswhich they area responseto. Write down all questionsthat the imagestrigger in your mind; these may provide important direction for further analysis.Seeand respondto the photographs(or film, etc.) asstatementsof cultural drama and let thesecharacterizationsform a structure within which to placethe remainderofyour research. Second stage Make an inventory or alog of all your images.Dependingon what youarc trying to find out, designyour inventory around categoriesthat reflectand assistyour research goals. Third stage Structureyour analysis.Go through the evidencewith specificquestions- measure distance,count, compare.The statisticalinformation may be plotted on graphs, listed in tables,or enteredinto a computer for statisticalanalysis.Produce detailed descriptions. Fourth stage Searchfor meaningsignificanceby returning to the completevisualrecord.Respond againto the data in an open manner so that detailsfrom structured analysiscan be placedin a context that definestheir signifi.cance. Re-establishcontext,lay out the photographs,view imagesin entirety, then write your conclusionsas influencedby this final exposureto the whole.

39

40

HANDBOOK

OF VISUAL ANALYSIS

Openviewing Open immersion providesopportunity for the imagesand through them, we hope, 'speak' the cultural circumstancesto to us in their own terms. The procedureis not complicated.The images,be they photographs,film or video, should be viewed repeatedly,groupedinitially in an order that approximatesthe temporal, spatialand 'natural' other contextualrelationshipsof the subject-matterthey reflect.Once this setof relationshipshasbeenwell understood,theymaybe groupedin otherwaysthat allow comparisonsof similar and contrastingcircumstance.Comparativestagesof open analysisarefrequentlythemost productivephase,helpingrefineunderstanding of patternsthrough identification of recurrent and contrastiveelements. While we should begin by simply looking at the images,as we continue we can start askingbroad questionsof the visual record.Thesequestionsmay derive from our original researchconcernsor they may haveevolvedfrom what we haveseen in our initial open immersion.Our goalsarea thorough familiaritywith the character and content of the visual recordsand, we hope, identification of possiblepatterns of significancewithin the content of the images.Careful recordsshould be kept that clearly describeboth findings and questionsas they evolve,including notation of 'tells' what us what we think we see.Particular attention should be devotedto unanticipatedfindings, as thesemay be cluesto new discovery.The whole process should be seenasa foundation on which to build subsequentstagesof the analysis. This analyticevolution canbe illustratedby my analysisof cultural landscape photographs made above my home in New Mexico shown in Figure 3.2. A series of 18O-degree panoramaswere made from preciselythe samelocation,beginning in 1964and continuing on a regularbasisto the present.Printed and mounted on long folding panels,thesepanoramasprovide arcady appreciationof significantchanges over time and distinctions in how changeare manifestedin different parts of the valley,asillustratedin a selectionfrom two of the panoramas. The top imageis from a panoramamade in 1964,the lower imageshowsthe samesectionin 1996.The areaseenis alargelyHispanicl village,with most land still in handsof long time families,which contrastedsharplywith another sectionof the valley (not shown here) in which land had passedinto the handsof land speculators and is now largely occupiedby incoming Anglos2from other parts of the country. Changesin densityareimmediatelyvisiblein the illustration;lessobviousis a relative continuity in community spatialpatterns.Housing is still largelyplacedon nonagricultural lands and much of the housing is still comprised of extendedfamily compounds locatedalong ditch and road lines.The changesin the Anglo-occupied areawere different:housinghasmoved on to agriculturalland without restraint;the scaleof growth is greater;and homes are not ascloselygrouped,being spreadabout on largerresidentiallots. While theseand other patterns were readily apparent,I felt the need to be more specific.To that end, I made a carefulstudy of different sectionsof the region, counting the number of homes, noting their location and construction types, examining the distancesbetweenhomes, noting the shapeor characterof land holdings, the presenceor absenceof homes on irrigated lands and the evolution of

ANALYSIS

r:@ ':::::::=

$)), '., ),1 l

j:

IN VISUAL ANTHROPOLOGY

41

42

HANDBOOK OF VISUAL ANALYSIS

G g = . o i ( E

o)E

(/, :

.9.ll

6t

p g

.=E = : o 5

' = Y 6 F . 9

6 i 9

= . ( E

5 E = ( , ) ( J r u

85.= -ts

0)

u,

..= =

F

L

.q9;

i Fi .9E S > 6 > E : i i

E O 9 X h

F.RE E , XE 9.9tn

o g€ @ o

o

9- H; -A '= xE

t E 6

HH s

a 5 b

a o =

r E

q

a i ) O X E X ; : g

o t r o )

> 9 P * r E

Z- >E) Y; -dE-E =

O

a

?#P o 9 9 E

$

h

! ! a o

E e t = o > > otr_E

U , =E $ 5 ( E

E ;s E p 6 r = 6 E - - t o , = o x = ' t t J h o

afiE o i

9

C D O $

o H a = a i 9

,E'EE

ANALYSIS

IN VISUAL ANTHROPOLOGY

agriculturalactivity.This analysishasprovided information that allowsme to make quite precisestatementsabout the changesand their character.The particular phenomenaI studiedin detail were,however,identified by the initial open viewing.

Structuredprocesses The move to detailedexaminationof particular imagesis an engagementin'closed' 'structured' or analysis.The product of this stageis descriptivespecificitythat serves both to test initial findings and provides precision to our descriptions of those findings.While rarelyproducingnew discoveries,it can seryeto challengeor modif' an initial hypothesisand preliminary findings we have made during initial open analysis.It is in this stageof analysisthat we might count the incidenceof particular phenomena,measureor describeparticularspatialrelationshipsand comparespecific content.Most investigatorsfind this stageof analysisthe most time consuming and exhausting. The variables we examine might include precise proxemic relationships, content analysis(seeChapter 2), identification of participants,recordsof temporal shifts,tracking of behaviour,detailsof gesture,expression,posture or other kinesic detail and any number of other visiblevariables.If we areworking with recordsthat include sound aswell asimages,asis usuallythe casewith video,we might look at the relationshipsbetweenthe audible and the visible (seeChapter 9). The precisefocal points will be definedby our original researchneedsand, ideally,by the findings of our initial open investigation. An investigationmadesomeyearsagoof non-verbalfactorsaffectingstudent responseand behaviour in Cantonesebilingual classesin San Franciscoprovides examplesof structured analysis.The fieldwork phaseof the study produced a large body of film, video and still photographsof classesfrom pre-schoolthrough to fifth grade.Direct observationsand open viewing suggestedthat students'behaviourwas significantlydifferentwhen Cantonesewasspokenascomparedto thosecircumstances where only Englishwas used,with concurrent differencesin the proxemic relationshipsof studentsand instructors.As I moved to structured analysisI examinedthe film recordof eachinstructionalunit with attention to detailsof spatialrelationships, duration and evolution of student involvement in instructional activities,lengthin minutesof eachlesson,languageof communicationsand a number of other variables. This effort produced specific qualitative statementsand a considerablebody of statisticaldata which was analysedfor correlationsamong different variables.The resulting findings permitted a confident statementthat moving studentscloserto eachother and to the instructor was,in and of itself,likely to leadto more sustained student involvement in instruction activities.Analysisalso suggestedthat changes in languagetended to trigger changesin proxemic and other aspectsof non-verbal behaviour.While thesegeneralconclusionswere initially reachedwithout the aid of structured analysis,specificdetailscame from closeexamination of many images. Severalmonths were spent carefully examining spatial relationships,tracking eye behaviour, posture,attention to assignedtasks,identifring languageof instruction

43

44

H A N D B O O KO F V I S U A L A N A T Y S I S

and conversation,and a range of other specificelementsof student and teacher behaviour. This detailedanalysismade possibleresponsiblestatementsdescribing the findings and assistedthe articulation of explanationsand recommendations (Collier,1983). An illusion of numericaltangibility cantempt one to focusstructuredanalysis on the generation of statistical information but caution is advisedregarding the significanceof suchstatistics.Quite asidefrom obviousand not so obvioussampling issues,statisticsfrom microanalysiswill reflect only what we chooseto count or measureand as such should be seenas providing important descriptiveprecision rather than absoluteverification. Numerical description may be important but no more so than other, more qualitative,details. Sometimesstructured analysismodifies or contradictsthe findings of open analysis.When this occursan evaluationshouldbe madeasto the significanceof such differences.An examplecomesfrom the Cantonesebilingual study just described. Initial open analysissuggestedthe crucial distancebetweenstudentsand instructors with improved wasabout threefeet:distanceslessthan that wereconsistentlyassociated with a drop. This wasconsistentwith responses, while beyondthreefeetwasassociated other research,including that of Edward Hall ( 1966),so initial examination used three feet as the crucial distancein comparing different sessions.This produced enigmatic resultsin that a not insignificant number of sessionswith interpersonal distancesof lessthan three feethad responsepatternssimilar to thosein which these distanceswere much more than three feet.As the resultswere not fully conclusive,a rapid, open re-examinationof contrastingsessionswas carried out. This reaffirmed that distancewas important but suggestedthat my lumping of all sessionswith distancesof lessthan three feetin one group wasobscuringthe analysis.A new detailed analysiswascarried out, looking at distancesof one, two, three and five feet asseparate categories.The result was a finding that the crucial transition point was someplace befweenone and two feet.

Returnto the wholeview The volume of possiblyimportant detail generatedby detailedanalysismay overwhelm the investigator,hiding the true significanceand meaning of visual record. For this reason,analysisis best completed with open re-examination of all of the imagesbefore conclusionsare made. The myriad detailsmay now be seenin their largercontextand a coherentstatementof meaningand significancemade,supported but not obscuredby detail. Failure to engagein this final step is likely to produce conclusionsand researchreportsthat flood the readerwith particularsbut givelittle hint asto their importance. This reality canbe illustratedby two film studiesof schoolswith which I have beenassociated.The first wasa pioneeringfilm examinationof schoolingin Alaska. Twenty hours of film of classesfrom severalvillageswere brought back for analysis by a researchteam,who engagedin both open and detailedstudy of eachclass.This effort producedvolumesof detailedinformation from which a final conclusionand

ANALYSIS

IN VISUAL ANTHROPOLOGY

a report had to be completedby Iohn Collier, Jr. Initial attempts,working from the detailedfindings,wereunproductive.Was ayear-longeffortof afive-personteam of no value?In frustration,Iohn Collier,]r. returnedto a running, unstructuredviewing of all the footage.The details of analysisnow fell into a coherentwhole, providing lively detail and reality to the larger conclusions(Collier, 1973). While successful,none of us appreciatedthe significanceof the processthat had led to a successfulconclusion.The next project entaileda study of a well-known community schoolon the NavajoNation in Arizona. Building on the Alaskaproject, this project involved systematic field recording and structured analysiswhich generatedmore extensiveand refined details than those produced in the Alaska study.Again, an attempt wasmadeto write up the findings directly from the detailed analysis.An extensivemanuscript was produced, rich in specificsbut ultimately lacking cohesionor clearmessage(Collier and Collier, 1986). The return to the overviewis thereforea searchfor largerform that may move us aboveminutiae, allowing us to make full useof the power of the visual record to defineand conveymeaningful statements.

Photoelicitation in directanalysis The value of using imagesin formal and informal interviewsis under-appreciated. Commonly referred to as photo elicitation or photo interviewing, this can be an important aspectof direct analysisand is the foundation of indirect analysis. During direct analysiswe can show imagesto informants and they can identiff items, places,people,processesand activities.They can provide their own openendedreadingsand comment on ours. This processmayprovide rapid and accurate information, greatlyenhancingour largeranalysisand understanding.(For instance, considerthe panoramic view in Figures3.2 (a) and (b) discussedearlier.) Used in interviewswith a residentof the community one could obtain the namesof families and detailsofkinship, knowledgeofland ownership,namesof crops,information on agriculturalpracticesand building techniques,an understandingof land tenure and inheritance,namesof different communities and much more. In particular,visual recordsallow the participantsin an activity or processto look at and discussthose circumstancesthemselves,providing an inside viewpoint, aswasdonein myresearchinto Cantonesebilingual classrooms.Film of classsessions was shown to severalof the teachers,who then supplied detailed information on their classesand, more significantly,their own commentaryon the communication patternsseenin the footage.Suchparticipant photo elicitation would alsobe crucial to the understandingof many craft and ceremonialprocesses. Photographsoften makeinformantsmore precisein their interviewinformation. Thoric Cederstrom,while investigatingthe economicrelationshipbetweenmigrants in urban Mexico and their rural communities of origins, was frustrated by vague responsesto interviews.Rememberingthat he had made photographichousing studies,including material cultural inventoriesof home interiors, he startedusing thesein his interviews.Now informants, with photos in hand, becamespecific-'-y

45

46

HANDBOOK

OF VISUAL ANALYSIS

'I uncle sentthe money for that sofa', built that room with money from my brother 'my in Puebla', son brought the TV from Mexico tcity] when he visited last year'.3 A more completepicture of economicinterconnectionsrapidly developed. Above all, photo elicitation can often make useof visual recordslacking the contextualizationthat is a normal requirement for direct analysis.If the informant can provide the contextual information out of their own experience,even poorly annotatedimagesmaybe usedasdata.

INDIRECTANALYSIS Enlighteningasdirect examinationmaybe, it is not necessarilythe most productive form of analysis.Perhapsthe leastknown researchpotential of imagesis their useas vehiclesto knowledge and understanding via the responsesthey trigger in photo elicitation sessions.(Chapter6 discusses the useof drawing asa processof elicitation within a theraputic context.)When we useimagesin this manner we are engagedin indirect analysis.Previouslymentionedlimitations may restrictthe potentialof many imagesas data,but alternativeproceduresassociatedwith photo elicitation provide the opportunity for rich returns from a wider rangeof visual records. The richestreturns from photo elicitation often havelittle connectionto the detailsof images,which may serveonly to releasevivid memories,feelings,insight, thoughtsand memories.This potential canbe seenin the resultsof a researchexperiment carried out by SuzanneLevinewhile a graduatestudent at SanFranciscoState University.4With experiencein both visual media and anthropology shewished to apply both to an investigationof the feelingsand emotions of disaffiliatedIews.She 'found'photographs from postcards,family albums,magazines madea collectionof and other sourcesthat she hoped might evoke responsesfrom her informants. Most of thesewere unremarkableimageswith little direct analyticpotential and she with low expectations.To her beganher largelyopen-endedphoto interview sessions delight the photos triggereda flood of comments,memoriesand intenseemotional statementsof identity, relationshipsto familiesand tradition, and much more. Indeed, her delight soon turned to a new apprehension- too much data! A particular attribute of photographsis that they give birth to stories,which themselvesare important sourcesof information. A 1939photograph(Figure3.3) of wood hauling in New Mexico, for example,generatednot only information on the challengesof gettingfirewood without pick-up trucks but a host of storiesabout the elderly man in the photograph which illustrated many aspectsof community socialand economicrelationships.Other storieswererelatedto working with animals, including an enigmaticreferenceto the particular burro, as a man who would have been about nine yearsold when the photo was made said'I rememberthat burro . . . [ong pause]I rememberthatburro!', and that wasall, leavingto our imaginations a story unspoken. Another aspectof indirect analysisis that the investigatorcanusea wide range of imagesand is not limited to thosethat arepart of documentaryor researchstudies. Evensingleimageswith little annotationmaybe useful,dependingon their character.

ANALYSIS

IN VISUAL ANTHROPOLOGY

47

Manyanthropologistshavesignificantcollectionsof photographsmadeasan adjunct activity to their fieldwork that, while, perhaps,of limited direct value as data,could readilybe usedastools in interviewsbut are usuallyusedsimply to illustratelectures and publications. Countlesstimes I have looked at such photographs and asked 'Have you usedthesein interviews?'Thealmost inevitableresponseis negative- the ideahad nevercrossedtheir minds. Nor is the processlimited to documentaryimages. I havehad studentsuseadvertisingimagesin projectsthat exploredAsian-American feelingsof identity and there is no reason one could not use paintings, drawings and other art productionsin interviews,dependingon the natureof the research.The formal and informal use of images in interviews may be the most overlooked applicationof photographsto researchin the whole field of anthropology.

Figure 3.3 Unloading firewood in Romo year, Talpa, New Mexico (USA). Photograph by John Collier, Jr., circa 1939. Antonio Romo on the right, Eustacio Bomo behind burro. A common scene - few families had motorized vehicles until the late 1940s. This photograph was particularly productive when used for photo elicitation, both in this community and in others in the region, because it evoked so many memories of the past.

P R A GM A T I C I S S U E S Visual analysisis both an intellectualand a craftprocess,in which a variety of choices and procedurescan make the investigator'seffiortseasierand more productive.The following practicalcommentsare presentedtowardsthat end.

48

HANDBOOK OF VISUAL ANALYSIS

Visualdatafilesand researchrecords Photographsand other visual recordsneed clear identification that indicatestheir contextualannotation. relationshipto eachother and providesconnectionsto associated Ideally,this beginsin the field with consistentand clearrecord-keepingof locations, times and other crucial information or, alternatively, as described earlier, such information maybe reconstructedlater.In eithercase,it is bestthat simplebut crucial information be built into the identification codesattachedto eachrecord so that its 'home'is apparent.Therearemanywaysto structuresuchcodes;for example,in my own filesa photographnumbered'8523-12'is immediatelyidentifiableasbeingframe 12, roll 23, shot in 1985.Further information can then be obtainedby looking at the shootinglog for that year. (Only soft pencilsshould be usedto write on the back of photographs,since inks may bleed through and affect the image. Fine-point permanentink markerscan be usedto write on the edgesof negativesand slides.) Most video camerasand some modern still camerasallow direct printing of datesand/or times directly on to images.While it may not be desirableto havethat information on everyimage,it should go on the first imageof a roll or tapeand at the Readabletime codescanbe recordedin video, beginningof eachmajor new sequence. eitherin the field or later,and areessentialto any systematicanalysisofbehaviour and communication. The rapidly developingtechnology of digitized still imaging also providesopportunities for locking important annotation to still photographsasthey are storedin computer files. Beyond contextual annotation, it is crucial to maintain systematicresearch files as the volume of information generatedfrom visual analysiscan rapidly overwhelm our capacityto rememberits detailsand associations.Inparticular,you need This is astrue with to know what imagesgeneratedwhich information or responses. open-ended analysisas with structured examinations and is equally important in varioustypesof indirect analysis.One of the main reasonsfor having good identification codesfor the imagesis to assistin record keepingduring analysis.

Loggingand inventory After initial openviewing of a visualcollectionit is usuallydesirableto createa log or an inventory of the collection, producing an organizedlisting of content; this can often be linked to the annotation and recordkeepingdescribedabove.The inventory processis a good way to becomebetter acquaintedwith your images,asduring openended analysisyou may easily skip over portions that do not quickly catch your eye.

Stillversusmotionanalysis The basisprocessesfor analysisof still and moving imagesare similar but eachhas it is different strengths.Comparativeanalysisis easierwith still photographybecause

ANALYSIS

IN VISUAL ANTHROPOLOGY

possibleto placephotographsnext to eachother and scanback and forth between them. Rapid, open-ended searchesfor broad patterns are usually easierwith still photographsas many prints can be spreadout and more readily seenas a totality. Still photographsfacilitatedetailedanalysisof content (seeChapter 2), asin studies of material culture (seeChapter 4), the cultural geographyof a neighbourhood, identification of participantsin an event,and certaindetailsof spatialand proxemic patterns.Conversely,analysisofnon-verbalbehaviour,communication,kinesicsand other moving processesis more productive with film and video; indeed they have made analysisof suchtopics possible(seeChapters8 and 9). The comparativestrengthsof stills and film may be illustratedby the photograph of three men in conversationshown in Figure3.4.This still imageallowseasy readingof proxemic relationships,and showsdress,expressionsand gesturein a frozenmoment. It invites a readingbasedon thosevariablesand providesa glimpse of the setting,an auto repair shop.But we cannot readthe paceof movement,we do not know what camejust beforeand afterand, aboveall, our readingof the emotional quality of the interaction will derive from our projection into the sceneof what we think happened before and after. Worse yet, the reading of behaviour in a still photograph is influenced by the precise moment in which it is made. My own experimentshave shown that fractions of a seconddifferencesin exactlywhen a still photograph of interactions is made can significantly changehow it is read. Consequently,the informational content is more heavilyinfluencedbythe skillsand choices of the photographer. With film, and especiallyvideo, the continuously running record not only showsthe interaction in process(togetherwith sound) but also reducesthe likelihood that information crucial to our understandingwill be missed. It is harder to usefilm and video for indirect analysis,although many people have done so. Peopleare often more activein their responseto still images,getting more involvedin the viewing,lingeringoverthis image,skippingthat one,goingback to yet another.Presentedwithfilm orvideo theymayfall into an audiencemode and just watch. Still photographscan be viewed almost anywhere,while film and video require equipment,power and placesthat accommodateor supply these.

Repetition, stopaction,slowmotionand high-speed viewing At the risk of statingthe obvious,visual recordsmaybe viewedin waysthat areoften impossiblewhile observing'reallife,' a characteristicthat is often crucial to analysis. The potentialfor repetitiveviewingis most important becauseit allowsus to comprehend complex detailsthat might otherwiseremain unseenor fleeting.This abilityto look at exactlythe samecircumstancesagain and again is at the foundation of all direct visualanalysis.Operatingin realtime we often havebut one chanceto observe, record and comprehendwhat we see;once pastwe cannot return and we inevitably missmuch in the complexityof the circumstances. With good visualrecordswe can study a sceneat leisure,return after days,months, yearsand re-examinethem in the

49

50

HANDBOOK

OF VISUAL ANALYSIS

Figure 3.4 Conversation in Santistevan'sauto repair shop, Ranchos de Taos, New Mexico (USA). Photograph by Malcolm Collier, 1984. Guillermo Santistevan on left, Andres Martinez on right, man in centre not identified at this time. Picture is from a set recording the operation of the business, including the social functions reflected in this image. Part of an in-progress project that explores the community from 1930 to the present.

light of new knowledgeor experienceand with new questions.Almost anyonewho has engagedin this repeatedstudy of visual recordshas come awayastonishedthat there is almost alwaysmore to see. Film and video canbe viewedat different speeds;while obvious,the full significanceofthis facthasto be experiencedto be appreciated.I first realizedthispromise of the motion imagein the 1960swhen Edward T. Hall showedme super 8 footage of mundane scenesof a fiestacrowd in northern New Mexico. At normal speedthe activitywasfamiliar and unremarkablebut when projectedin slow motion it became magical,revealingthe dramatic differencesin patternsof non-verbalbehaviourand cultural styleamong Indians,Hispanicsand Anglos that I had previouslysensedbut could not articulate. The slow-motion view provided the possibility for precise descriptivestatementsregardingwhat had beforebeen intangible.

ANALYSIS

IN VISUAL ANTHROPOLOGY

It is possibleto view motion records as a seriesof still images,in frame by frame analysis(seeChapter 9). Single-frameanalysis,like that of still photographs, provides very detaileddescription of static phenomenabut hides the fluid flow of movement in studiesof communication and behaviour,for which reasonit is at best an adjunct processrather than the foundation ofsuch studies.It isverytime consuming and I havefound slow-motion analysisalmost alwaysmore productive. Conversely,and somewhatillogically,it can be productive to look at film or video at fasterthan normal speeds.I discoveredthis when showingresearchfootage back to gradeschoolchildren whoseclassesI had been filming. They loved viewing it at high speedsbecauseof the exaggeratedbehaviour,and I was struck by the fact that, while slow motion revealsthe subtletyof particular moments, fasterthan normal screeningrevealedmacro patternsthat were sometimesobscuredfrom notice by the passageof time at normal or slow speeds.In effect,high-speedviewing is somewhat similar to juxtaposingtwo contrastingstill images.

Comparisons andjuxtaposition The primary reasonfor organizingand maintaining good contextualannotation of a visual imagecollection is to allow for meaningful comparativeanalysis,one of the most productive aspectsof working with imagesand part of all stagesof research. Comparisonsaremost effectivewhen done directly,that is byplacing imagessideby side,asillustratedin the panoramic study describedearlier.This is easywith photographsbut more difficult using film and video,with which comparisonsusuallyhave to be sequential,althoughit is possibleto screenmoving imagessideby sideor on split screens.In the Cantonesebilingual study,I comparedkey segmentsby splicinga short seriestogetherand rolled them back and forth sequentiallyin an analyserprojector. In other cases,I pulled still imagesout of the film and made prints which were used for comparativestudy of detailsof spatialpatterns.A similar processcanbe followed with video, dubbing out sequencesfor motion viewing or using a digitizing device with a good computer to take offstill imagesof key frames. An interestingand sometimesproductive comparativeprocedureis to take a setof photographsand shufflethem, then lay them out in the resultingunstructured order to seeif unanticipatedrelationshipsmay be seen.

Countingand measuring Counting and measuringmay be important in structured analysis,generatingboth descriptive detail and quantitative information. The first step is to determine what phenomenaor variablesare to be examined;this decision should evolveprimarily from your initial open-endedstudy (seeChapter2). The varietyof phenomenasubject to this processis large- we can count people,objects,occurrencesof particular behaviours,and much more. We can measurespatialrelationships,sizeof objects, speedof movement,the beat and paceof actionsand communications.A key stepis

51

52

HANDBOOK

OF VISUAL ANALYSIS

to developa standardizedcode sheetthat providesconsistencyfor what is counted or measured- this facilitateslater comparativeanalysis.

Soundandimages This chapterconcentrateson visual analysisbut many aspectsof visible culture also have audible associations,as is most obvious in studiesof communication and behaviour (seeChapter9). The advent of good quality video recordinghas made it possiblefor almost anyoneto obtain synchronizedsound and image recordsof a wide range of cultural circumstances.Analysis of such records requires attention not only to the visual and audible componentsin their own terms, but alsoto their interrelationships.Many of the approachesdescribedherefor handlingvisualcontent can also be profitably adaptedto working with sound. (Adequatediscussionof combined analysisof sound and imageis beyond the scopeof this chapter;however, readersinterestedin this aspectof analysisare referredto the work of Frederick Erickson- seeEricksonand Mohatt, 1982- and Adam Kendon.)

Specialconcernsin photo elicitation Photo elicitation requiressome care.Procedurally,it is important to keep records of the order in which images are presentedto informants and, obviously, which with which images.It may be useful to experimentwith responsesare associated different combinations of imagesto seewhich are most productive. Resultsmay also vary depending on whether sessionsare with groups of people or with single individuals- againsomeexperimentationmay be in order. As with the larger analysisprocess,photo elicitation best beginswith openendedviewing, first allowing the informants to saywhateverthey wish. This approach is more likelyto produceunforeseeninformation and commentary.Ifyou immediately startwith very specificquestions,the informant is likely to getthe messagethat those arethe only topicsyou want information on and restrict their subsequentcommentary to thosepoints or other detailsthat you bring up. Thereis considerablevariation,both cultural and individual, in people'sresponse to photo elicitation.Our own work in the American Southwest,for example,suggests that both Navajo and Puebloinformants tend to begin with very specificdescriptive readingof images,moving to more projectiveresponseslater or not at all, in contrast to the more immediatelyprojectiveresponsesof Anglos,who sometimesrequire specificquestioningto producedetailedreadings.Someindividualshavestrongvisual memoriesand responseswhile othersdo not, which can affectresultswhen working with historical photographs.I once intervieweda woman who had beena practising musician in a community in northern New Mexico for over sixty years,hoping to obtain identificationsof musiciansin a collectionof excellentphotographsof festivals in which shehad participatedin the 1930s.The resultswere not as expectedas she

ANALYSIS

IN VISUAL

ANTHROPOTOGY

53

proved to havea strongmusicalmemorybut a poor visualone and I cameawaywith many interestingstoriesabout musicaltraditionsbut few identificationsofmusicians. Conversely,anotherperson,confrontedwith a photographof the audienceat a school play proceededto identif' practicallyeveryonein the crowd. Thephotograph shown in Figure3.5illustratesthe needforpatiencein photo elicitation as well as the plastic nature of memory. The picture was taken in New Mexico in the 1930sand had absolutelyno annotation. From the shapeof the hill I could identifr the location within fifty metresbut the buildings were a mystery.They were very familiar - I knew that I had passedthem countlesstimes - but I could not place them, nor could anyone else.Interviews with long-time residentsproduced comments on the high quality of the plastering but person after person failed to specificallyidenti$r the homes. Returning to work in San Francisco,I interviewed a migrant from the villagewho had moved to California over fortyyears beforeand had seldomreturned evenfor visits.When he sawthis photograph he immediately 'That's said, Tio Alberto'shouse',and proceededto givedetailedinformation on each of the buildings and their residentsin the past. Once he said it, I too recognizedthe buildings and was left wondering why identification had beenso difficult. On return to New Mexico I went to the location

i . A. '*al'

Figure 3.5 1930s.

*w

Alberto Tafoya home, Rio Chiquito/Talpa, New Mexico (USA). Photograph by John Collier, Jr.,

A simple image that proved extremely hard to identify and has yet to be precisely dated. The photograph reveals the care with which people maintained their homes, the open yards, no sign of cars, trees neatly trimmed by goats and a central placed oven. Today, the yards are filled with plants (no more livestock) and driveways, while the homes are hard plastered. Another productive image in photo elicitation.

54

HANDBOOK

OF VISUAL ANALYSIS

and studiedit: everythingwasstill therebut the detailshad changed.Treeshad grown up, a fencebisectedthe view, trim had been altered.Perhapsthose of us who had remainedin the community on a regularbasishad our perceptions,our memories, alteredby the changesthrough time, whereasthe long-absentmigrant's memorywas frozenin the earliertime period, unclutteredby the overlayof interveningyears. An area of caution in photo elicitation involves imagesof people'sprivate circumstancesand events,especiallyif picturesareshownto other peoplein the same community. The fact that your relationship to individuals may have permitted photographybyyou doesnot necessarily extendto letting otherslook at thosephotographs.It is probably best to avoid using photographsof people in private settings in interviews with others unlessyou have gained specific permission, although a significantpassageof time or distancethat provides anonymity might modi$' this advice(seeCollier and Collier, 1986:135-6).

Teamanalysis Team analysishasspecialpromise in visual analysisbecauseit providesan opportunity to exploit an important featureof visual records- the ability for multiple views of preciselythe samephenomena.As different people study the images,their various readingsmaybe comparedand linked to identifiablephenomenain the visualrecords. Equally important, discussionof team findings may help clarifr which resultsare a product of tangible elementsin the imagesand which are more clearlythe product ofthe cultural experienceswe bring to our analysis.If the analysisprocesscanbe guided team by a standardizedcode sheetor a set of common questionsand processes, images. analysiscan alsoprovide meansfor more rapid analysisof largequantitiesof

Photomapsandothermodels Still photographsmay be arrangedasphoto maps and other modelsthat assistin the discovery of information within them. If systematic photographs of an urban neighbourhood,for example,are mounted on folding boards or rolls of paper they the largervisual reality of the localeinto a can becomea photo map that compresses Figure3.6 more comprehensible space,while maintainingbasicspatialrelationships. showsa sectionof a larger photo map of a streetin SanFranciscoChinatown. The imagesacrossthe bottom of the map provide continuous, wide-angleviews of the street,while mounted abovethem aredetail photographsmadein the location above which they are placed.The full map recordseight blocks on both sidesof the street and stretchesfor over twenty feet. Thesemapscanbe arrangedon the floor in correctspatialassociationto each other. We can move back and forth among theseimages,examiningdetails,and we can also look at them as a collectivesingle image in which detailsare seenin their relationshipto eachother and to the whole. They canbe studiedthrough rapid open viewing and structured analysisto arrive at understandingofthe spatialand functional

ANALYSIS

55

IN VISUAL ANTHROPOLOGY

(€iii t:i H ,

::::::= i#i

Figure 3.6 Excerpt from photo map of Stockton Street, San Francisco Chinatown made by class in Asian American Studies at San Francisco State University. Photos by Frances Lee, Hernan Cortez, Suzanne Canieros and Mark Johnson, 1985. Wide views at bottom, details above. Map shows wide view and specifics of different parts of the neighbourhood, permitting analysis of the broad functions, visual character of the community and the specifics of each section. Subsequent mapping of the same neighbourhood provides a record of change over time, supplying data for analysis of community evolution.

56

H A N D B O O KO F V I S U A L A N A T Y S I S

ra=

6,&**

ffi

@

4

wl{o W f,@

::.::-'44' ie.#*{

f'rffi

ffi G-*-:

+.#Etu

{f_---*

Figure 3.7 A photo map spread out for analysis,Asian American Studies class, San Francisco State University.Photograph by Malcolm Collier, 1986. Large photo maps contain a considerable volume of information and are suited to team analysis. They are also good vehicles in which to engage students in examination of neighbourhoods while providing training and experience in visual observation and analysis.

organizationof the locale,the rangeof peoplewho frequent it, the stylesand content of displays,and much more (seeFigure3.7).Thesemodelscanbe temporal,functional or sociallystructured,and can be either two- or three-dimensional.

Digitalpossibilities The advent of digital camerassuggestsnew possibilitiesfor visual analysis.Some are merely technicaladvances,as in the greatereaseof constructingnicely matched panoramic photographs,but others involve more significantmethodologicalinnovations in both data gatheringand analysis.In particular, a digital camera,in conjunction with a laptop computer,providesopportunity for rapid feedbackof images. A field worker can make digital recordsof a place,an activity or other phenomena, load them into a laptop computer and immediately show them to people for quick acquisitionof annotation and analysisvia photo elicitation.This elicitedinformation can be incorporated into visual-baseddata files for further analysisat one'sleisure.

ANALYSIS

IN VISUAL ANTHROPOLOGY

In many circumstances,a good ink jet printer can be used to produce gift photographs,an important aspectof human relationsin visual research. Digitized imagescan be transmitted to other people rapidly, assumingthey havethe appropriateequipment,facilitatingteam analysisand photo elicitation.It is 'rescue'faded alsopossible,with someexperience,to and degradedphotographsvia computerizedadjustments.For instanceI havedone this with badly fadedhistorical imagesof SanFranciscoChinatown, discoveringpossiblyimportant detailsof street scenesthat raisesomequestionsabout the universalityof standardaccountsof family life in that community a century ago.

Viewsfromwithinandwithout Most discussionsof visual analysis,including this one, assumea researchprocess largelycarriedout by peoplewho arenot themselvesparticipantsin the circumstances recordedin the images.What happenswhen this assumptionis incorrect,when visual recordsof cultural processare made and analysedfrom 'within' (seeChapter4)? Such inside explorationsare becoming more common. My own work has included teaching a photographic exploration course in which Asian-American studentsrecordand expressdifferentaspectsofAsian-Americanlife, most commonly thosethat they arethemselvesassociatedwith. Fifteenyearsof experiencewith these studentshasrevealedsignificantdifferenceswhen working from within. As might be expected,the photographs (and some video) are more intimate, personalizedand contain information that is not readily availableto an outsider. From an analytic point ofview, I havefound that analysiscanproceedmore rapidly,primarilybecause the studentsbring important contextual information to their research.This allows themto makedefinitivestatementswithoutasmuch dependenceonlogging,annotation and other stepsthat arenormallytaken to ensurereliablereadingof images,although their work is usuallyenhancedby suchprocesses. A more formal exampleof researchfrom within, usingboth visual and other methodologies,hasbeencarriedout by a group ofYup'ik Eskimoeducatorsin southwesternAlaskaover the pastten years.Seekingto both understandand articulatethe skillsand needsofNative teachers,theyvideotapedtheir own classroomsand carried out their own analysis,incorporating the resultsinto their larger project (seeLipka and Mohatt, 1998).

Workingwithimperfectcollections and images Most visualrecordsof cultural circumstancesarelessthan idealfor analyticpurposes. 'documentary' Even well-made imagesare often weak researchrecords,as is true of many of the photographs anthropologistsmake for themselves.What forms of analysisare possiblein such cases? As has alreadybeen suggested,severalapproachescan be taken to make responsibleuseof suchimages.Stepscanbe takento reconstructcontextualinformation

57

58

HANDBOOK

OF VISUAL ANALYSIS

and to provide the neededannotation to directly analyseimages.Thesestepsmay include archival work, photo elicitation, a searchfor internal clueswithin images themselvesand a carefulevaluationof what is and is not possible.It is then possible to make reasonableanalysisof many types of images.Finally, most imageshave potential functions in indirect analysis,asis describedelsewherein this chapterand book. As demonstratedby the rangeof visual data analysedin this volume, visual analysisis not limited to photographs,film or video. Obviously,any visual creation can be used in studiesof cultural expressionand as a reflection of cultural values. Drawings,paintingsand other art forms areoften asreadilyproductive asphotographs in indirect analysisand may sometimesbe usedin direct analysis,providing there is adequatecontextual information and the cultural conventions of the artist are understood.

FINDINGMEANINGAND MAKING CONCLUSIONS Rich detail is of limitedvalue if it is not articulatedin meaningfulconclusions.I have suggestedthat analysisshould involve a return to the visual whole so that details may be seenas larger patterns,the discoveryof which are crucial to understanding the significanceof our research.But patternsalonedo not producemeaningand our searchfor it is complicatedby our dual role asinvestigatorsand cultural beings.The cultural lensesthrough which we operateinevitably shapeour analysis,especiallyas we seekconclusions.While this reality may limit our comprehensionof unfamiliar cultural phenomena,it is equallytrue that our existenceas cultural beingsprovides us with the ability to recognizeand respond to cultural patterns.This capability is essentialaswe reachto definethe significanceof our findings. The searchfor patterns is the first step toward meaningsand conclusions. Patterns may be found in structured descriptions and quantitative products of detailedanalysisbut the most important onesareusuallythosethat we perceiveduring the lessstructured study of our images.Both as a visually oriented speciesand as cultural beingswe arequite adeptat perceivingpatternsin the visualworld, sovisual investigatorsshouldhavefaith in their ability to seepatterns.Onceidentified,patterns canbe testedvia structured analysis,repetitiveviewing, team analysis,photo elicitation and other analyticprocesses. The reasonwe look for patternsis that the world, especiallytheculturalworld, is significantly, although not totally, made up of patterned relationships. It is reasonableto assumethat, when you discoverand confirm patternswithin a collection of images,film or video, thosepatternsmay havemeaningand significance.

Art and science A particular promise of visualrecordsin researchis the opportunity fot a productive combination of artistic and scientificprocesses. No amount of well-organizeddata

ANALYSIS

IN VISUAL ANTHROPOLOGY

or refined processesof analysisguaranteesa sound and responsibleresearchconclusion. Conclusions lie beyond information and description; as such they entail It is both necessaryand legitimateto allow ourselves creativeand artistic processes. to respondartisticallyor intuitively to visual imagesin research,both during earlier stagesof analysisand aswe articulateconclusions.However,while creativeprocesses may produceonly fictitious statementsif are essentialto discovery,artistic processes not combined with systematicand detailedanalysis.The dangeroustemptation of artistic approachesin visual anthropology is abandonmentof the responsibilityto connectfindingsto concretevisualevidence.If conclusionscannotbe supportedwith detailedinformation, descriptiveor otherwise,from anorganizedprocessof analysis, then they should be re-examined.

Presentation of findings Although changesmaybe on the horizon, one of the frustrationsofvisual researchis that we areusuallyrequiredto presentour findingsprimarily in written form. We still swim in a seaof words and more words, in which visuallybasedcommunicationsare not taken as seriousintellectualproducts.How are we to translatethe richnessand promiseof our visualdiscoveriesinto the deceptiveworld of words?Words that have often becometired and abusedthrough time and with which we must struggleto let our readersseewhat we have seenbut may not show them. The answeris not clear but, aswe awaitthe developmentof a visuallanguageof intellectualdiscourse,we can useour images,our visual analysis,to breath a bit of new life into our translations. Experiencehas shown that systematicincorporation of descriptive detail from analysisinto written accountsof the researchcan help provide clarity and validation to more generalizedconclusions.The bestway to achievethis goal is to write Procedurally,asyouwriteyou fromtheimages,notfromthe abstracteddataoranalysis. should have the imagesin front of you, drawing both description and inspiration from their constantpresence.In this manner the basisof your knowledgeremains freshand the translationdirect. This chapterhassoughtto provide principlesand ideasfrom which to develop your own plansfor research.The visualworld is rich beyond dreamsand eachvisual investigationmust definenew approaches.In the end we must play with imagesuntil they speakto us directly and from that dialoguewe draw our findings.

NOTES termusedin thiscontextto identifya peopleof mixed 1. 'Hispanic'is a highlyunsatisfactory in centraland centuries SpanishandAmericanIndianoriginswho havelivedfor several northernNewMexicoandsouthernColorado,USA. origins,broadly to referto peopleof European 2. 'Anglo'is usedin theAmericanSouthwest defined,who areneitherAmericanIndiannor Hispanic. personalcommunication. 3. ThoricCederstrom, 4. Suzanne Levine, personal communication.

59

60

H A N D B o o Ko F v t s u A r A N A L y s t s

REFERENCES Collier, 1.,h. (1973) AlaskanEskimoEducation:A Film Analysisof Cultural Confrontationin the Schools. New York: Holt, Rinehart and Winston. Collier, J.,Ir. and Collier, M. ( 1986) VisualAnthropology:Photographyasa Research Method. Albuquerque:University of New Mexico Press. Collier, M. (1983) NonverbalFactorsin the Educationof ChineseAmerican Children:A Film Study.SanFrancisco:Asian American Studies,SFSU. Erickson, F. and Mohatt, G. (1982) 'Cultural organization of participant structuresin two classroomsof Indian students',in G. Spindler (ed.), Doing the Ethnographyof Schooling: EducationalAnthropologyin Action. New York Holt, Rinehart and Winston. Hall, E.T. (1966) The Hidden Dimension.New York Doubleday. Lipka, J. and Mohatt, G. (eds) (1998) Transformingthe Culture of Schools:Yup'ik Eskimo Examples(Sociocultural,Political, and Historical Studiesin Education Series).New |ersey: LawrenceErlbaum Assoc.

Hru ffiffi w ruffi ffi

SeeingBeyondBelief:CulturalStudiesas theVisual an Approachto Analysing MARTIN TISTER AND LIZ WELLS

The relationship betweenwhat we seeand what we know is never settled. (Berger, 1972:7)

INTRODUCTION

CulturalStudies Cultural Studiescentreson the study of the forms and practicesof culture (not only its texts and artefacts),their relationshipsto socialgroups and the power relations betweenthosegroupsasthey are constructedand mediatedby forms of culture. The 'culture' in questionis not confinedto art or high culture. Culture is taken to include everydaysymbolic and expressivepractices,both thosethat takeplaceaswe live (and are not aimed at producing artefacts),suchasshopping,travellingor being a football 'textual practices'in the sensethat somekind of material artefactor supporter,and representation,image,performance,display,space,writing or narrativeis produced. As an academicfield, Cultural Studiesis interestedin the enabling and regulating institutions, and lessformal social arrangements,in and through which culture is produced,enactedand consumed.In practice,it is seldom,if ever,possibleto separate the culturesof everydaylife from practicesof representation,visual or otherwise. The focusof suchstudiesis normally on contemporaryand emergentpractices, studiedwithin their formative historical contexts.Theseare mainly those of the late 'modern' period of industrialeighteenth,nineteenth and twentieth centuries:the ization,the formation of the nation-state,the rise ofthe type of the modern individual, imperialism and colonialism,and the commodification of culture. Such contexts are now importantly extendedto include globalizationand the rangeof shiftswhich aregatheredup under the terms'post-modern'and'post-colonial',asthe legaciesand cultural forms ofthe earlierperiod areseento be radicallyrestructuredand fragmenting at the end of the twentieth century. A distinctive featureof Cultural Studiesis the searchto understandthe relationand shipsof cultural production, consumption,beliefand meaning,to socialprocesses institutions.This hasresultedin a refusalto see'society'assimplythe context,climate

62

HANDBOOK

OF VISUAT ANALYSIS

or backgroundagainstwhich to view a cultural practiceor text; rather the production of texts is seenas in itself a socialpractice.There is a similar refusalto seecultural practicesand texts as merely symptomsor documentaryreflectionsof a prior set of socialdeterminations.Instead,Cultural Studiesinsistsupon the constitutiverole of culture in sustainingand changingthe power relations enactedaround issuesof gender,sexuality,social class,race and ethnicity, colonialism and its legacies,and the geopoliticsof spaceand placewithin globalization.It examinesthesein terms of the ways of seeing,imagining, classi$'ing,narrating, and other ways of investing meaningin the world of experience,that cultural forms and practicesprovide.

MediaStudies The versionof Media Studieswhich is closelyconnectedwith Cultural Studieslargely arosewithin the samepost-war intellectualproject to comprehendthe impact of industrializationand advancingcapitalistsocialformations on new,massforms ofcommunication, representationand consumption. Part of the impetus was to do this in waysthat were more flexible and responsiveand lessvalue-ladenthan the responses to the massmediafound in the traditional canonicaldisciplinessuchasliterary studies or art history.In particularthesedisciplines'preoccupation with the ideaof individual authorshipasa sourceof meaningwascriticizedin itselfand asmanifestlyinadequate for the study of advertising,popular cinemaand television.A parallelimpetuswasto pay much closerattention to a wider rangeof expressedor representedexperience, howeverinformal, popular,sub-culturaland apparentlytrivial,than wascharacteristic of mainstream social science.Feedinginto thesecentral impulseshave been other traditions: sociologicalresearchand empirical study of audiencesfor massmedia, especiallytelevision;critical studiesof media power; the political economyof the media; studiesof media politics and the public sphere;media and communications theory; and specifichistoriesof radio, television,the press,new media and communication technologies.

Thestudyof visualculture More recently,there havebeen attemptsto define a specificfield of Visual Cultural Studies.While recognizinga formativerelationto a wider field of Cultural Studies(which alwayscontainedan interestin the visual), its proponentsdo not seethis asmerely a specialized sub-divisionor extensionof Cultural and Media Studies,but asa reworking of the whole field of concern.With the late twentieth century'sexplosionof imaging and visualizing technologies(digitization, satelliteimaging, new forms of medical imaging,virtual reality,etc.),theysuggestthat everydaylifehasbecome'visualculture'. This can be seenas an accelerationof a longer history involving photography,film, televisionand video. However,somearguethat this new visuality of culture callsfor its own field of study concernedwith all kinds of visual information, its meanings, pleasuresand consumption, including the study of all visual technologies,from 'oil

C U L T U R A LS T U D I E SA N A L Y S I S O F T H E V I S U A L

painting to the internet' (Mirzoeff, 1998:3). From this perspective,it is arguedthat the study of visual culture can not be confinedto the study of images,but should also take accountof the centrality of vision in everydayexperienceand the production of meaning.As Irit Rogoff puts it: of film and with asequence In thearenaofvisualculturethescrapof animageconnects with the cornerof a billboard or the window displayof a shopwe havepassed journeyandour by, to producea newnarrativeformedout of both our experienced suchas'documentary Imagesdo not staywithindiscrete disciplinaryfields unconscious. painting',sinceneitherthe eyenor thepsycheoperates alongor film' or'Renaissance recognizes suchdivisions.(Rogoff,1998:16) The primary purposeof this chapteris to demonstrateand criticallydiscussthe validity and usefulnessof a range of methodologieswhich have been brought into play for analysingphotographic imageswhich havebeen a major elementof visual We shallshowhowinsightsand methodsdrawn culturein modern industrialsocieties. from semiotics(seeChapters7 and 9), psychoanalyticcultural theory,art history (see Chapter 5), the socialhistory of media technology,aestheticsand the sociologyof culture are drawn upon in order to investigatehow meaning,pleasureand power are articulatedthrough specificimages.Suchimagesareproduced and consumedwithin a wide rangeof social,economicand cultural contexts,including thoseof advertising, the making of news, social documentary,medicine, the law and social control, education,the family, leisureand entertainment. First, we briefly discussCultural Studiesand methodology.The following three sectionsfocus upon distinctive questions asked of the photograph within Visual Cultural Studiesand demonstratesomeofthe key conceptsemployedwithin the field through analysesof a diverserange of photographs.Analysing examplesof photographsfrom advertisingand reportagealongwith imagesmadefor galleryexhibition, we discusscontextsofviewing, contextsofproduction, form and meaningand looking and identity.

CulturalStudies: methodologies Cultural and Media Studiesis a compound field, elementsof which are differently organizedin different institutions. It is generallyunderstoodas an interdisciplinary field,ratherthan asa discretediscipline,which appropriatesand re-purposeselements of theoreticalframeworksand methodologiesfrom other disciplines,whereverthey seemproductive in pursuing its own enquiries.lTherefore,it is hard to identify for Cultural and Media Studiesits own singular and strict set of disciplinary protocols. However,andwhile differencesof emphasisexist,most researchmethodologycourses within Cultural and Media Studiesinclude elementsof ethnographic,sociological, semiotic,psychoanalyticand critical textual methods. One way of approaching a definition of Cultural Studiesis to consider its objectsofenquiryas thewaysitunderstandsthecomplexconcept'culture'.Theseinclude, 'ordinariness' for instance,the or'everydalmess'of culture, an interestin culture as

63

64

HANDBOOK OF VISUAL ANALYSIS

the processthrough which a societyor socialgroup producesmeanings.There is a stressupon the'how' aswell asthe 'what'of culture,on productionsaswell ascontext. Cultural Studiesis, then, not only methodologicallyeclectic,but open and experimental in the ways that it frames its objects of study. While it may borrow its methodologicalresources,it seldom assumesthat it unproblematicallyhas a set of 'out objects there' or before it, about which it can then ask questionsformulated by and inherited from other disciplines. Pragmatically,its achievements haveto bejudgedin termsof the coherenceand insights of the accountsthat it givesof its objects.Its methodologicalrigour lies in the responsibleway that a researcherusesthe intellectualresourcesthat they borrow and apply. Even though an orthodox historian or sociologistmay gibe at the taking of their methodologicaltools into interdisciplinaryhands,the vitality and suggestivenessof much Cultural and Media Studieshas been widely influential on other academicdisciplinesand criticism and has had an impact upon print and television journalism. How do thesegeneralpoints inform what we attempt here?While much of our attention is given to specificphotographs,we analysethem without separating them from socialprocesses. Exceptfor the practicalpurposesof stagingour analysis - we cannot do everythingat once!- we resistrei$.ing or hypostatizingthe images. That is, we work hard not to seepictures as rigid and fixed things - beginning and ending at their frames(seeChapter8). Another way to put this is that we approach the imagesas part of what has beendescribedas 'the circuit of culture' (du Gay, 1997).Eachone can be thought of as passingthrough a number of 'moments' and its passagethrough eachmoment contributesto the meanings- plural, not singular - which it has and may have.In short, they are sociallyproduced, distributed and consumed;within this cyclethere are processesof transformation taking place and also of struggleand contest over what they mean and how they are used.To sum up we offer a check-listof the main featuresofthe analysiswhich follows.Thesewill be restatedasmore focusedquestions within the analysisitself. 1 We areinterested in animage'ssociallife andits history. 2 We look at imageswithin thecycleof production,circulationandconsumption throughwhichtheirmeanings areaccumulated andtransformed. 3 We payattentionto animage'sspecificmaterialproperties(its'artifactualness'), andto the'medium'andthetechnologies throughwhichit is realized(here,as photographs). 4 Whilerecognizing thematerialproperties of images, weseetheseasintertwined with the activesocialprocess of 'looking'andthehistoricallyspecificformsof 'visuality'in which thistakesplace. 5 Weunderstand images asrepresentations, theoutcomes of theprocess of attaching ideasto and givingmeaningto our experience of the world. With careand qualification, muchcanbegainedby thinkingof thisprocess asa language-like activity- conventionalsystems which,in the mannerof codes,conveymeaning within a signusingcommunity. 6 We temperpoint 5 with the recognitionthat our interestin imagesandother (and,indeed,livedandmaterialculturalforms)cannotbe visualexperiences reducedto thequestionof 'meaning'andtheintellectual processes involvedin

C U L T U R AS L T U D I E SA N A L Y S I SO F T H E V I S U A L

codingand decoding.As humanbeings,and asthe membersofa culture,we also interestin looking at and feeling'the world' have a sensuous,pleasure-seeking including the media that we have put in it. 'looking' 7 We recognizethat is always embodied and undertaken by someone with an identity. In this sense,there is no neutral looking. An image'sor thing's significanceis finally its significancefor some-bodyand some-one.However, aspoints I to 6 indicate,this cannot be any old significance,a matter of complete relativism.

ANALYSIS

Contextof viewing We needfirst to askwherethe imageis.What is its location (or locations)in the social and physicalworld? Our answerto this questionwill tell us much about how we meet or encounterthe image;that is,howwe attendto it. Is it, for instance,met in the public or private part of our lives?Is it something on which we can concentrateand be absorbedby in a single-mindedway or is it one contingentelementamongstothersin our busydaily transactions,in our leisuretime or aspart of our work or education?Did we deliberatelyseekout the image,in a library or a gallery?The context influenceshow we look at the imagethrough constructingcertainexpectations.For instance,the gallery addsan aura of seriousness of intellectualor aestheticintent to the picture. Second,why is the viewer looking at the photograph?\,Vhatinformation or pleasuresdo they seek?How are they intending to usethe image?Is their interestidle or purposeful?If the look is purposeful,asin, for instance,studying the imageswith which we have chosento illustrate this chapter,then it is important to know what editorial judgements have been made and how this has influenced the selection. Writing this chapter,and in particular choosingimages(which will be reproduced severalthousandtimeswithin copiesofthis book) and then discussingthem in certain ways,is a small exerciseof power. What are our reasons,our interestsor purposes, in selectingthe imageswe have?What view of the traffic in imagesarewe promoting? Uppermostin our minds hasbeenthe exemplificationof the methodologicalpointswe wish to make. Not all imageswould serveas well to do this although, in principle, if the conceptsand methods we use are of value then they should be applicableto a 'strong' examplesof the concepts wide rangeofimages.However,herewehavefavoured that we are dealinewith, in order to help elucidatepoints.2 'redneck' in a Marlboro cigaretteadvertisement The image (Figure4.1) of the is situatedon a super-sitehoarding. This is positionedto the sideof a roundabout on a major ring road in a largeprovincial city. The photographof the imagewasdifficult on foot. It is positionedwithin a major to takebecausethe hoardingis hardly accessible road complexwhereno provision is madeforpedestrians(theyareactivelydiscouraged) and is clearlyintended to be seenby passingmotorists or motorists in the frequently slow-moving traffic that is typical of this main approach to the city during peak commuting hours. The sameimagewasalsoreproducedin glossymagazinesand the magazinesectionsof major Sundaynewspapers.Literally,then, a photographicoriginal

65

66

Figure 4.1

HANDBOOK OF VISUAL ANALYSIS

Marlboro cigarette advert, circa 1995.

- alsoa digitallymanipulatedimage(seeHenning, 1995:2I7) -hasbeentranslatedinto two kinds of print of vastlydifferent scales.We canreasonablyspeculatethat one may be seenthrough the frame of a car window while on the move, while the other could be drapedacrossthe lap of a readeror browserwhile reclining on a sofaor travelling on atrain. Alternatively,it might be readat a deskwhen the readertakesa breakfrom the work they otherwisedo there. In this sensethe imagesare located,both in the physicalworld and in our everydaysocialworlds, quite differently. Taking two ofthese scenariosfurther, we canbegin to saysomethingabout how the one image is experiencedin different contexts.In the caseof the car driver or (this itselfmaybean important distinction),the imageis experiencedin time passenger or asthe spectatortravelsin space.It will loom up to confront the spectatorand then recedefrom their field ofvision. Their encounterwith the imagewill not be the result of an intention to look at iU it presentsitself to them. The image- asa publicly sited hoarding- will be seenin the contextof the (sub)urbanenvironment:the intersection of motorways stretching awayfrom the viewer, the backsof working-classhousing behind it, the sound of traffic passingin other directions,the smell of traffic on hot (or wet) tarmac,and so on. The imageis not a passiveelementin this scene.It depicts a man (who looks in the samedirection asthe viewer) who is himself besidea major road looking on at an American-styletruck. Therearehills in the backgroundasthere are in the actual location in which the image is sited.What is depictedin the image echoesor resonateswith the situation of the driver/passenger/reader of the image. On the sofa,anotherviewer looks at the imagein their domesticspace.They havesatdown, positioned themself,chosenthemagazinein which the imageis printed,

C U T T U R A LS T U D I E S A N A L Y S I S O F T H E V I S U A L

'Anobliqueglance',rc4a19. Figure4.2 RobertDoisneau.

openedit and alightedon the image.They may, in the first instance,havechosenand bought the magazinein which the advert appears.Theyhold the imageand focus on it. In a different sensefrom the first example,they too experiencethe image in time, or more precisely,within a sequenceof imagesand written words offeredby the other contents of the magazine.They might consume the magazinein a linear fashion working their waythrough from beginningto end, or more selectively,workingback and forth through its pages,in effectproducing their own juxtapositionsbetweenits various featuresand advertisements.We can askmore questionsof this situation. Is themagazine,and for somemoments the advert,their soleobjectof attention?Is the televisionon?What is being broadcast?How doesit, at somelevel of consciousness, interactwith the image?Might it be a programme about someother aspectsof life in the United States?A'road movie'?Perhapsmusic is playing - the imagemay have a soundtrack.Perhapsthe'reader'is not alonebut looks at the magazinewitha friend or partner; they discuss,judge,joke and elaboratevia anecdotesand connectionson the image. Our secondchosenexample(Figure 4.2) is an older photojournalistic image, one which still, on first encounter,may raisea smile. 'An oblique glance',by French

67

68

H A N D B O O KO F V I S U A L A N A T Y S I S

photographerRobert Doisneau,which showsa couple looking into the window of a Parisianantiqueshop,hasbeenreproducedin booksand exhibitionsabout Doisneau, the photographer,as well asbeing referencedin a number of discussionsof French humanist photographyshown in major international exhibitionson documentaryor on post-war French photography.The image was first publishedin Point de Vue (photo-magazine). In contrast to the Marlboro advert, the contemporary viewer is likely to encounterthis imagewithin the context of the work of the particular photographer, or as an examplewithin a more generaldiscussionof reportagephotography of the period.The photographmaybe reproducedin a book, or, indeed,the viewermaybe pursuinghistoricalresearchconcernedwith photojournalism,with Doisneauor with French humanist photography.Whilst the readerof Point de Vuein 1949may have come acrossthe imagein circumstancesequivalentto our putative sofareaderof the Marlboro advert (perhapswhilst listeningto the radio), contemporaryviewersofthis imagemay approachit rather differently.On the one hand, Doisneau'swork hasbeen extensivelyrecycledaspostersand postcards,and is familiar to manywho would not necessarilyknow the provenanceof the imageor the name of the photographer.On the other hand, preciselybecausethe work is now acclaimedasexemplaryof its genre and era, many viewersare likely to have in someway sought out the circumstances ofviewing,whetherborrowingorbuying abook, orvisiting an exhibition.(Thesame is true of the work of Robert Mapplethorpewhich we discussbelow.) The photographhasa title, which indicatesto the viewerits primaryhumorous focus,and it is specificallyauthored.The naming of the photographerlendsstatusto the imageasa work of art and, indeed,the context of viewing may be a galleryexhibition within which the imageis hung asa fine print, perhapsas one in a seriesof similarly carefullyreproducedphotographs.We are generallyfamiliar with the convention of the gallery,with the ritual of progressingfrom imageto image,attendingto eachone for a short spaceof time, then, perhaps,starting to make comparisonsbehveenone imageand anotherin terms of subject-matterand in terms of aestheticform. Like the Marlboro adverts,the experienceofthe encounteris inter-discursivein the sensethat situation, contemporary referencesand resonancesinform our experience,whether we merely glanceat the image or study it more intently. Questionsof socialhistory also obtain. For example,whilst viewers,both in the 1950sand now, may seethe Doisneauphotographasoffering information about Parisafter the war (for instance, the clothing worn by the boys acrossthe streetand the styleof dressof the couple), this information inevitablyholds differing implications for those consideringthe picture somefifty yearslater.

Contextof production Our next questionis: how did the imageget there?This questionshiftsour attention from how we encounterthe imageto ones about its production by others and its distribution - to the intentions and motivesof others,and the institutional and other socialcontexts,imperativesand constraintsin which they work.

C U T T U R A LS T U D I E S A N A L Y S I S O F T H E V I S U A L

Here the contrast between the Doisneau photograph and the Marlboro advertisementis instructive. Most obviously, the first was produced as a narrative Also image,asphotojournalism,whilstthe otherwasconstructedasan advertisement. the manner of production differs. Doisneau'sphotograph is about an event which has not been specificallydirected by him - although the setting up of his camera inside the shop with the painting of the naked woman placed at an angle in the window, orchestratedthe possibility of responsesfrom passersby. His conceptual approachis in linewith Henri Cartier-Bresson'sfamousdefinition of photographyas 'the simultaneousrecognitionin a fraction of a secondof a significanceof an eventas well as of a preciseorganisationof forms which give an event its proper expression' (Cartier-Bresson,1952).For the photographerthe skill is one of recognizingthe 'decisive moment', both when taking the image and in the processof selectionand editing. Here it is clearthat photography wasnot seenassomehowinherently objective but, rather,reportagewasusedbyphotographersasan opportunityfor interpretative 'photo-novels', 'telling'photos, sequences or commentary.The point was to find of imagesfor publication in illustrated magazinessuch as Life (USA), Vu (France) and Picture Post(Britain), which were popular and widely distributed from the 1930sto the 1950s.Whilst his work was more along the lines of socialobservation 'news 'hard' than news,that is the reporting of major contemporaryevents,a senseof values'will haveinformed the decisionto shoot this seriesof imagesfrom within the shop.3 Doisneaumade this image at a time which predatestelevisionasthe primary conveyorof visual information and was informed by pre-war documentaryfilm and 'humanist' focus photo movements.By the 1930sin France,therewasan established people at its upon ordinary people and everydaylife which took as subject-matter work and at leisure,depictedin streets,cafesand brothels,or at specialeventssuchas 'It fetes.DoisneaudescribedParis as a theatre of images: doesn't matter where you look, there'salwayssomethinggoing on. All you needto do is wait, and look for long enoughuntil the curtain deignsto go up' (in Hamilton, 1995:182). In our example,Doisneauhas set himself up in the shop in order to observe events;he has constructeda scenariowithin which it is likely that interestingevents may occur. By contrast, advertising imagery is overtly directorial, constructed purposefullyin line with a particularbrief and taking specificaccountof the intended meansofcommunication (hoarding,magazineadvertisement, etc.).The photographer for the Marlboro advert is not named;there is no attribution of authorship. Unlike photojournalism, the dictates are entirely commercial. In advertising the photographer or art director are named only if they are famous enough for their name to condonethe product.a The Marlboro image is an advertisementfor cigarettesand will have been designedand produced for that companyby a specialistadvertisingagencyaspart of the Philip Morris company'swider marketing and advertisingstrategies.This advert belongsto a whole seriesfor Marlboro cigaretteswhich, in part, are a responseto the early 1990sban on advertisingcigaretteson British television and the anticipation that this ban will be extendedto all forms of advertising.This accountsfor the waythat the advert (and the others in its series)contains no written or explicit referenceto

69

70

HANDBOOK

OF VISUAL ANALYSIS

either cigarettesor the Marlboro brand. Ironically, the clear referenceto smoking cigaretteswhich connectsthe imageto the product is givenin the governmenthealth warning which runs along the bottom of the image. The producersof theseMarlboro adshaveto solvea problem, that of how to referencea brand of cigarettesand how to promote a product that is widely understood to be seriouslydangerousto health. In the 1970sand 1980sMarlboro advertising associatedthe cigaretteswith the figure of the 'cowboy', itself a vehiclefor making a connectionbetweencigarettes,white masculinity, 'loner' subjectivityand the untamed nature of the mythologicalAmerican West.At that time it waspossibleto include the brand name and copy which recommended smoking (Marlboro cigarettesin particular) asnatural, pure and relaxingwithin the advert.More recentgovernment restrictionson such practiceshaveled designersand producersof the advertsto use other strategies. Theyknowthat the distinctivered colour of the Marhoro packetand logo can live on after the packetor name itself ceasesto be shown. A significantred 'the detail, sunburnt neck, a light on top of a police car, a traffic light', appearin the more recentimagesasthe only coloured elementsin what look like black and white stills from art-housemovies. Furthermore, the producersuse stills which reference movies which critically rework the myth of the West: Bagdad Cafd,Paris, Texasand Gas,Food,Lodging(Henning,1995:223-8).ThesepostmodernreworkingsofWestern mythology also allow the advertisersto shift the connotationsof smoking from the natural and relaxing toward the dangerousand the'romance of living on the edge' (Henning,1995). This advertisingimageis then a deliberateand skilfrrlresponseto legalconstraints and shiftsin the culture of health on the part of producerswhosetaskit is to maintain marketsfor their client'sproduct. The featuresof the imagewhich we havediscussed here are not arbitrary but the result of a complex interaction. This is betweenthe profit-making demandsof capitalist economics,the restrictions imposed by antismoking campaigners,health professionalsand the law, aswell asfactorssuchasthe modes of organization,division of labour and work processeswhich obtain at any specific advertising agency and the knowing semiotic practices of advertising 'creatives'.In our example,the imagetakesits form from, and dependsfor its success 'advertisement' asan on, suchfactors.Producersemploy particular strategies,which will not be the only solutionsthat could havebeenadopted,but they are outcomesof intention and'producerly'knowledgeand skill.Whetherwe,asreceiversor consumers of the image,directlytakeor acceptthe meaningstheyhaveintendedto givethe image is anotherquestion(seeChapter8).

LOOKING:FORM AND MEANING We cannote that at this point in our analysiswe still havenot begunto dealwith what we may call the image'itself or 'in-itself . This actuallyraisessomedifficult and vexed questionsaboutthe boundariesof an imageor a'text'. What is the imagein itselfrWhat are the inherent properties of a text when consideredapart from individual actsof looking at it and making senseof it?

C U L T U R A LS T U D I E S A N A L Y S I S O F T H E V I S U A L

Conventions Two usesof the conceptof a convention,understoodasa sociallyagreedway of doing something,one with literary and art historicalroots,the other sociological,play a part in the visual analysisof photographs.We start our discussionof codesand conventions with an examplefrom art history. This is usefulasa way of recognizingthattraditions of analysisemployedbythesespecialists in studyingthenarrowrangeofimageswhich makesup the history of art havealwaysoffered Cultural Studiessomething, especially the study of iconology and iconography (Panofsky,1955;seealso van Leeuwen, Chapter5 in this volume). While it is probablytrue to saythat,in Cultural and Media Studies,thesemethodologicalbranchesof art history and theory have alwaysbeen overshadowedbythe useof semioticmethodsimported initiallythrough Continental structuralism,we give them sometime here. This is becausethey immediately offer waysof talking about picturesin terms of the key conceptof 'convention'. They also allow us to startfrom noticing thingsabout imagesratherthan aboutwritten language and then seekingto apply linguistic conceptsto images.

Pictorialconventions The art historian Michael Baxandalloffers a brief but exceptionallyclear analysisof the pictorial conventionssimultaneouslyemployed in a fifteenth-centurywoodcut shownin Figure4.3:

Figure4.3 De Fluminibus. Woodcut,1483.

[Plate 15] is the representationof a river and at least two distinct representational conventionsare being used in it. The mermaids and the miniature landscapeon the

71

72

HANDBOOK

OF VISUAL ANALYSIS

left arerepresented by linesindicatingthecontoursof forms,andthepoint of viewis from a slightlyupwardangle.Thecourseof theriverandthedynamicsof its flow are andthepoint ofviewis fromvertically andgeometrically, diagrammatically registered betweenonestyleof above.A linearrippleconventionon thewatersurfacemediates relatedto what ismoreimmediately representation andtheother.Thefirstconvention - andto usnow rather we see,wherethe secondis moreabstractandconceptualised unfamiliar- but theyboth involvea skill andawillingnessto interpretmarkson paper rules:wedo not see simpliffinganaspectof realitywithin accepted asrepresentations 1988:32) by blacklines.(Baxandall, a treeasa whiteplanesurfacecircumscribed A number of points areworth drawing out of this paragraph.First, Baxandall talksabout'marks on paper' and the waysthey aresignsfor flowingwater, (imagined) bodies and land. He draws attention to a material surfaceand what it carriesas the ground (literally and metaphorically)of a pictorial representation.He is awareof the materiality of the imagethat he analyses.This is important becausea medium like a woodcut clearly offers different signifying resourcesthan, say, an oil painting or a photograph. Second,he seesthe meaningof thesemarkson a surface(that is,flowingwater) as dependenton interpretativeskill and, evenprior to that, a willingnessor interest on the part ofthe viewerto interpretthem. Sohe seesthat the imageimpliessomething about its viewer and her or his competencein looking. Third, he notes that severalkinds of marks are doing different work within the one image:some are diagrammaticand others are'more immediately relatedto whatwe see'(Baxandall,1988:32). (Lines,for example,areusedto indicateor describe the edgesof human forms. Of course,no suchlines areto be found at the edgeof our limbs but certainsituationsdo arisewhen we perceivea contrastbetweena limb and a lighter or darkersurfaceagainstwhich it is seen,the drawn line actsasa conventional equivalent for this contrast.) He seesthese different kinds of marks as lying on a spectrum of possibilitieswhich runs from the lessto the more abstract.They all simplifr reality and they are all subject(for the sensethey make) to acceptedrules,rules which may or may not be familiar to us given our membership of a historically or culturally specificsign community.

Semioticsandcodes Baxandall'sdescriptionof thesepictorial conventionscomescloseto the semiotic notion of an iconic sign, in which the signifier (the physicalmark or material thing/ object/quality)bearssomekind of resemblanceto what it signifies(what it meansor standsfor). It alsoapproaches,at the more abstractend of his spectrum,the semiotic premisethat many kinds of sign are'arbitrary';that is their meaning is not directly dependentupon some intrinsic qualitiesthat they have.Rather,within a culture, a certain kind of mark has been matched (and agreedby all who can understand the convention, the language,or the code,that it is so matched) with a certain kind of object or quality of objectsin the real world. Semioticsproposesthat there are whole rangesof visual, material, pictorial and symbolic signswhich are conventionalin the way that Baxandallidentifies,for

CULTURAL STUDIES ANALYSIS

OF THE VISUAL

examplein bodily movement and gesture,in the camerawork, narrative devicesand editing of film and television,in the weight, spacingand shapesof typographicforms. In this way the conceptof a convention is extendedto that of a 'code' - an extended systemof signswhich operateslike a language(itself a code of uttered sounds or printed marks) (seeChapter9). There have been many attempts (within Cultural and Media Studies and elsewhere)to analyseimages using elaborate and systematicsemiotic theories of codesor ofsignification(the operationof signs)basedupon the paradigmoflanguage. Over repeated attempts, often under the sway of changing intellectual fashions (structuralism,post-structuralism,deconstructionism),it hasbecomeclearthat a too rigid application of systematicmethodologiesfor visual analysis,which take written or spokenlanguageas a model, is self-defeating.There is alwaysa tendencyin such attemptsto miss the specificityof the medium, and the practicesbuilt around it in socialuse,where significationactuallytakesplace.Hence, in a good exampleof the (sometimes very productive) tensions inherent in Cultural Studies which were remarked upon at the beginning of this chapter, the application of de-coding methodologiesor the practiceof textual analysisare often challengedby insistingon the need to stressthe negotiated,dialogic and sometimesresistantand subversive dimensionsof human communication. In fact, it is this stresson the plural, rrr€ss/, contestedand evencreativenature of our discoursewith the visual and with images, the manner in which this is a site of a struggleover what somethingmeans,which often makes the Cultural Studiesanalystwary of the very term 'communication', preferringinstead'representation' or'mediation'.

Photographic conventions Convention and meaningenter into the businessof making photographsat the most basictechnicallevel.Indeed,it hasbeenpointed out that meaningis evenencodedin the design of the apparatus of photography. Very different examplesof this are providedby,for example,SnyderandAllen(1982)andSlater(1991).Theveryexistence of the rectangularframe of the cameraviewfinder and its picture plane wasdesigned into camerasat an earlystagein the history of photography.The round lensof a camera createsa circular imagewhich shadesoffinto obscurityat its circumference.Sometwo hundred yearsbefore the first successfulchemical fixing of the cameraimage, 'the portable cameraobscuraof the early nineteenthcentury was fitted with a squareor rectangularground glasswhich showedonly the centralpart of the imagemadeby the lens' (Snyderand Allen, 1982:68-9). This, as they point out, was the outcome of adjustingthe cameraimageto meetthe requirementsof 'traditional art', the rectangular easelpainting. This is a good exampleof the way in which a convention existswhile its historical origins are forgotten. In consideringthe snapshotcamerasof the midtwentiethcentury,Don Slater( 199I ) seesthat the smalliconsthat help the snapshooter setthe camera'sfocallength for a partiatlar kind of subject matter- a portrait, a family group, a landscape- already anticipate the conventionsby which such genresare recognizedand need to be produced.We will start our own hunt for photographic

73

74

Figure 4.4

HANDBOOK

OF VISUAL ANALYSIS

Robert Mapplethorpe. 'Portrait of Clifton', 1981 .

conventionsa little further on in the process,in the basicdecisionsthat a photographer takesassheor he deploystheir equipment (seeChapters3 and 9). First,however,we needto takecarenot to be misunderstood,asin what follows it could appear that we are imputing too much explicit intellectual effort to the photographer as they mobilize conventions.As for the fifteenth-century woodcut printer and his 'audience',conventionsexistwhich are both usedby photographers and understoodby contemporaryviewersof photographs.This is not to saythat photographersconsciouslychoosethe conventionsthat they will use in making a picture. Somemay of course,but we are not making the ludicrous suggestionthat they hold seminarson semioticsin order to carry out a commission(althoughit may be worth remarking how closecreativedirectors in advertising,whosejob includes briefing photographers,come to doing this). In general,the use of conventionsby photographersis a matter of assimilated'know-how', a trained senseof 'this is how to do it' gained'on thejob' and by observingwhat doesand doesnot 'work' in concrete

C U L T U R A LS T U D I E SA N A L Y S I S O F T H E V I S U A L

situations.Similarly, in looking at a photograph and finding meaning in it, we do not needto refer to a dictionary of conventions- we don't look them up. Unlike the woodcutter's 'now unfamiliar' diagrammatic representation of a river flowing, photographicconventionsareveryfamiliarto most of us:theyfall belowthe threshold of consciousattention.They are nevertheless there. It is the very degreeto which setsof conventionshavebeenassimilated,within a culture, asthe way to do something,that guaranteestheir very naturalnessrather than their evidentconventionality.With regardto photographsin particular, the fact that they arealsoproducedthrough mechanicaland chemicalprocesses tendsto persuade us that they are not the outcome of skill in handling a 'language'but are automatic, and immediate,tracesor reflectionsof what they depict (seeChapters7 andg). The portrait by Robert Mapplethorpein Figure4.4 could seemto be without convention.Both its extremesimplicity (not much scopefor artifice,arrangementor choice here) and its impact (a startled gazewhich startlesthe viewer) is surely the resultof somethingmore like spontaneityor passionthan a rule-governedactivity or 'a socially agreedway of doing something'. In fact, within its simplicity, there is a considerablecomplexity of conventionaldevicesand by identiffing them we can go a long way toward accountingfor some of its drama. The conventionaloperations that we would draw attention to are the following: framing, gaze,lighting,context, cameraposition. First, the frame (itselfl aswe sawabove,a devicewhich hasdropped well below visibility asthe conventionthat it is): considerhow tight it is setaroundthe man'sface. It excludesany signif ing context exceptthat of the deepblack background. Anything that might locate,pin down, domesticate,classifr,or otherwiseoffer us cluesas to who this personis or what he does,is ruled out. Second,the gaze;the man looks directly at us, wide eyed.In fact, what he looks at directly is the camera.Mapplethorpehasnot had him look awayto right or left, up or down, not evenslightly.It is in looking at the camerathat he appearsto be looking at us.This is a conventionknown to portrait painters(who usedlinearperspectiveand waysof highlighting the eye,rather than a camerato achievethis) and film-makers who (exceptin specialcircumstances)strictly avoid the convention in order not to break the illusion that we are looking in on anotherworld without ourselvesbeing seen. Third, the cameraposition placesus completelyon a level with the man. We are, in terms of frame, gazeand cameraposition'face to face'.Strictly speakingit is 'viewing us in our position' (not our real location in the world) who are faceto face with a man depictedin this way (which, of course,may not be how we would seehis face,whereverit is, in the real world, at work, in a shop, a nightclub, whatever) (see Chapter l). Do we not alsofeel physicallycloseto him? All of the conventional factors that we havebeenconsideringso far contribute to this senseofphysical proximity. His head fills our field of vision as it is representedby the photograph and 'he' arrestingly answers,lockson, asit where,to our gaze.Lastly,the dark backgroundand the arranged lighting of the headmay contribute to this senseof proximity. Here is a caseof significationarisingfrom the image'smateriality.We can see the dark backgroundaseither measureless depth or flat surface,the material surface

75

76

HANDBOOK

OF VISUAL ANALYSIS

'see of the picture itself.Seenin reproduction in this bookwe will tend to through'the pageto the image of an image- a poorly reproducedphotograph. Seenas a framed 'fine print' in a photographicgallerywewould be awareof the texture,lustreand grain 'stuff rather of the black areasof the photograph as a material surface- as than depicted space.Two other factors that we have already met are at work here: the location of the imageand the waywe attend to it or the sociallyappropriatemode of looking. The very sizeof the photographand the rituals of looking at photographsin galleriesare likely to distanceus from or bring us closeto the actualobject.The scale of the image alsopositions us in relationshipto it. The way in which the man's head is lit (thereis no natural light here,it is imported and arrangedby the photographer), meansthat we 'read' the head as being in front of the akeady closeblack surface. It emergesfrom the black backgroundor ground. As we worked our way through the conventionsthat are at play in this image we saw,as they operatedtogether,a photographiccode.A set of signsthat, taken together,meanssomethingto us. However, how we might expressthat meaning in spokenor written languageis another matter. There are two reasonsfor this. One is a generalmatter of what and how images'mean'andthe secondis particularto the kind of genreof art photographythat Robert Mapplethorpemakes.In general,it can be very difficult to spell out the meaningsof pictures in verbal or written language, which after all is another code.Somekind of translationis bound to takeplaceeither waybut in this casefrom imageto word. As JohnBerger(1972)hasobserved,language can never get on a level with images.Secondly,as a photographerworking as an independentartist, Mapplethorpeis not chargedwith sendingsimple,clearmessages or communicating unambiguous information to us. In fact, as a white gay photographer who frequently photographed naked black men his work is shot through with ambiguity and his interestin or desireto do this hasprovoked much debate(see Mercer, 1994:l7lff.).

Socialconventions Beforewe look further at the operation of photographic conventionsand codes, we should recognizethat photographs also work by utilizing or borrowing (by re-presentingthem) many of the visual codesthat are employedin 'lived' rather than textual forms of communication. The mimetic capacityof a photograph,the way in which the indexicalmarks which its surfacecarriescan resemblethe look of objectsand things in the realworld, means that within a photograph certain things may be depicted or represented (through photographicconventions)which arethemselvesconventionsin their own right. Theseareconventionsthat we employ in the wider socialworld, in our everyday livesand its sub-cultures.The photographwasdescribedby RolandBarthes(1977:17) 'message as being a without a code'. By this he meant somethingcloseto Susan Sontag's(1978)descriptionof the photographasa 'trace',a kind of directprint off the 'real'without any code(break-downinto units) intervening.It is, for us,much harder to seewhat the equivalent of the woodcutter's conventions are in a photograph,

CULTURALSTUDIESANATYSIS OF THE VISUAL

Figure 4.5

David Hampshire. Photograph of schoolboys, circa 1986.

although in the aboveexamplewe have,hopefully, begun to do so.However, the other sideof the coin, asit were,of this mimetic capacityof photographyto be a traceor an imprint of the real is that it canborrow and carry all of the sign systemsand codes(of dress,style,architecture,objects,bodylanguage,etc.)which, togetherwith speechand the written word, sound and smell,make the lived world meaningful. In the photograph in Figure 4.5 we can seehow this is so.We are now using the other,more sociological,conceptof convention.Much ofthe chargeofthis picture comesfrom the dressconventionsofthe boyswho arerepresentedin it. In a realsense, the photographer is using photographic codesto frame and point to this other set of signsthat are usedin everydaylife.Considerthe boy on the left of the frame.He is subvertinga dresscodeand by doing so he is signallinghis distancefrom the meanings 'respectable', of conventional,male dresscodes. A tie is an item of dressthat is designedso as to hang betweenthe collar of a shirt and the waist after it hasbeenwrapped around its wearer'sneck and tied in a certain way at the front, so that its knot sits neatly betweenthe two wings of a shirt collar. The boy in the picture hasnot failed to achievetheseconventionalrequirements of tie-wearing- he has deliberatelyusedhis tie to flout them. He has,literally, made it do something else.He has consumedthe whole length of the tie in the knot and contrived to position it someinches awayfrom its conventionalplace.The meaning of doing this will be apparent to others who also wish to resist the conventional uniformity encouragedby the institution of the school. Symbolically,at least,he is 'community' ofthe schooland his inclusion signifting his senseof exclusionfrom the 'the of his or membershipof lads'.He is marking out a difference.Clearly,the success

77

78

H A N D B O O KO F V I S U A T A N A L Y S I S

semioticwork dependsupon knowing first that there is a conventionalway to wear ties. The boy in the centre of the picture has done somethingsimilar by turning his tailored and lapelledjacket into a kind of all-envelopingtunic. If we catch echoesof genericmovie imagesof the rebel caughtin the rain or cold, we are probably not far from identiffing one of the sourcesfor this particular convention.

Powerand photographic conventions

Figure 4.6 New Year's Day, Korem camp, Wollo province, Ethiopia, 1985.

Figure 4.7 Mike Goldwater. Drought migrant, Zalazele transit centre, Tigray, 1983.

We now turn our attentionto somephotographs(Figures 4.6,4.7,4.8 and 4.9) where, given the institutions for which they were made (newspapersand magazines),we can assumesomeintention to communicateinformation wasof a high priority. Theseare imageswhich belong to the genre of photojournalism, imagesmade specificallyto report on events. Like the Mapplethorpe portrait, but for different reasons,such photographs arealsocharacteristicallymarked by their lack of apparentartifice or displayof pictorial convention.Indeed,sometimesreferredto as'straightphotography'to distinguishit from the elaboratearrangement,settingup, lighting and theatricalityof other kinds of photography (advertising,fashion,art), the very invisibility of convention seems to speakinsteadof photography'spower to provide direct evidenceof events. The credits tell us that the photographswere taken in Africa (Ethiopia and Sudan)during periods of famine in the 1980s.The original creditsalsoindicatethat the photographerswho made thesephotographswere working for picture agencies (Magnum and Bisson/Collectif),probably on a freelancebasis,and hoping to place their work with newspapers,magazinesor, possibly,famine relief charities. Thesephotographswerereproducedaspart of a special1985issueofthe photographicjournalTen: Swhich examinedthe politicsof faminerelief.It exploredthe part that photographyplayedin constructinga Eurocentricview of Africa and its peoples aseconomicallyand technologicallyweak,dependentvictims of natural disaster.The accompanyingtext arguesthe casethat photojournalists foster this view and that, togetherwiththe demandsupon aid agenciestoraisemoney,aviewofAfricaandAfricans

LTURAL STUDIES ANATYSIS OF THE VISUAT

is constructedwhich renderstheir normal self-sufficiencyand culture invisible.At the sametime the manner in which famine is representedignoresthe role of capitalism and the historyof imperialisminbringing abouta situationwherebyAfricaneconomies are crippled by long-term debt repayments,the useof fertile soil to grow cashcrops for export and dependencyon short-term emergencyaid. In the text which accompaniesthe reproduction ofthesephotographsin Ten:8, some researchis cited which indicateswhat ideasand imagesof the 'Third World' and its peoplesa group of London schoolpupils gainedfrom televisionand photoreportage.They list poverty, babiesdytrg, monsoons,disease,drought, refugees, flies,death,dirty water, beggars,malnutrition, bald children, large families,insects, poor clothing, bad teeth, kids with pot bellies,mud huts and injections (Simpson, 1985:23). These are all important factors which bear upon matters such as the tasks and commissionswhich the photographersaregiven,the purposesof the agencies that 'news they work for, the values'that they are expectedto provide and the selectionand editing of imagesby editorial stafffor publication (seethe section above,'Context of production'). However,in this sectionwe wish to concentrateupon how meaningis encoded in particular photographs.The point is to seewhat ideologicalweight conventions have,especiallywhen they operateasparts of a complex photographiccode.For the photographerwho wishesto avoid producing yet another imagewhich compounds the restrictedperceptionsof Africa noted above,there will be a struggleto encode different meaningsin a photograph,somethingthat beginswith the choiceto deploy conventionsin a singleact of imagemaking. We can seethis in the two imagesreproducedin Figures4.6 and4.7. Both imagesrepresentpeoplein obviousdistress.The main signsofthis distress arefacialexpression,the gestureofhands,bodilyposition and stance.We noted above that thesewere not, strictly speaking,photographic conventionsbut more broadly socialand cultural oneswhich photographs'cite'.They are part of the human body's expressivity.Our ability to readthe signsthrough which bodily and mental statesare expressed is a socialnot a photographicskill. The photographerexpertlyborrowsthese signsand reliesupon our abilities,learnt through our lived experience,to read them. Often to a high degreethesesignsare indexical,the classof signsfamouslyidentified by C.S.Peircewhere the material sign (the signifier) is causedby what it means(the signified).The usualexamplesin semioticprimers arethe footprint = foot, smoke= fire and knock at door = presence.They are different becausethey are lessarbitrary and symbolic than signslike'dog' or'chien' are for the animal it signifies.In theselatter cases,theseink marks on paper arenot causedby dogshaving passedthis way. In Figure 4.6 there is a certain strain in the musculatureof the man's face,in his open mouth and furrowed brow, which we can recognizeasmeaningsomething in the sphereof pain and hopelessness. Likewise,in'Drought migrant' (Figure4.7), the woman's eyesare heavy-liddedand unfocused,while she'holds' her head- they signif' somethingin the sphereof exhaustionand neryousdistraction.It is hard to be precise,but we would surelyagreeon what they do not signify and on the spectrum of conditions and feelingswhere the meaningslie. A photograph tradesin mimesis,

79

80

HANDBOOK

OF VISUAL ANALYSIS

in establishingperceivableand meaningful similaritiesbetweenone thing (a human face)and another(a setof toneson a papersurface). There are,of course,varying degreesof suchdirect, symptomaticindexicality in human and bodily expressions.Consider the'coded' smile which is not a direct expressionof pleasurebut a knowing and ironic responseto disappointmentor sadness. However,when such signsare themselvesrepresentedin a photograph,they are far from natural.Considerthat beforeand afterthe moment ofthe photographicexposure that produced theseimagesthe subject'sexpressionand position might have been different and lessculturally or symbolicallyexpressiveof distress.First,it is likely that the photographertook a number of exposuresand later chose(edited)a contactstrip ofher or his film to choosethe imagein which the signsof distress(ofwhateverorder) weremost evident.(Or selectionsmay havebeenmadeby a picture editor, anxiousto reinforceparticular inflections of the famine story.) Second,there is no direct equivalencebetweena human face seenin a photograph and in other lived situations, somethingthat waspointed out in discussingthe Mapplethorpeportrait. We will now considerthe material qualitiesof some particular photographs and how they work to add meaning to thesesocialcodesand, asthey do so, edgeus toward recognizinghow the photograph is a complex constructof signs. First,we considerthe edgesorboundariesofthe pictures:the frame.The framing 'straightest' of the picture is somethingthat must be decidedupon both at the time of offering up the camera'slensto the sceneand later when making a print in an enlarger (or, in the caseof digitizedpictures,in a computer imagemanipulation programme), or possiblywhen digitallyor physicallycroppingthe print. In 'New Year'sDay' (Figure 4.6) the frameis setwide: overtwo-thirds of the picture arearepresentsa surfacewhich we read as a baked and crackedplain of earth, somefar-distant figures,a line of low hills behind them, a strip of almost cloudlesssky.The settingof the frame in this way with the figure of the kneelingman, slightly off-centre,is the first meansbywhich he is located and isolatedin this barren space.If we imagine tightening the frame and shifting it to our left, no doubt to include a secondor more figuresin the picture, this imageof isolation disappears. Second,we considerthe depth of field, a term which refersto how much of the sceneis in sharpfocusand, dependingupon availablelight and speedof film, is a factor within the photographer'scontrol. By choosinga combination of shutter speedand aperture(or allowing an automaticcamerato setthem) depth of field canbe shallow or deep,restrictingor amplifring the information we are given.The depth of field in this imageis considerable.We seethe crackedearthon which the man sits,the texture of his worn garmentsand the delineation of the tiny distant figures and hills. This choiceof depth of field (or at leastthe choicenot to limit it) reinforcesthe senseof the man's isolation,first establishedby the frame. Third, we considerthe quality of the light that makesthe man visible and the photographpossible.Clearlythis is natural ambient light (the midday sun?)which is not directly within the photographer'scontrol. However,it is utilized by the photographerand the manner in which strong light with a high source,directly abovethe subject,describessurfacesin terms of stark contrastsis an important factor here.The deep shadowmarking the man's eyesocketsand the sharp delineationof his collar

C U L T U R A LS T U D I E S A N A L Y S I S O F T H E V I S U A T

bone and upper ribs are given emphasisby the photographer'schoice to make the exposureat a place and a time when the light is of this kind. Compare this with 'Drought migrant' (Figure4.7) wherethe light is more diffuse. The focus and the light are also reinforced by the fine grain and optical resolutionof the photograph.The skin of the man'shead,veinsstandingproud on his temple, the surfaceof his neck and chest,the rough texture of his clothes and the crackedearth are all very apparentto us. Through theseoptical meansour tactile sense is engaged. Fourth, we cannote that the frameor moment chosenby the photographerhas the man looking upwardsand out of the frame to his right. He looks up at something or someonewho is invisibleto us.A number of factorsthat we havealreadyconsidered work togetherwith this gaze(thefacial expression,the delineationofthe faceandbody, the isolation in space)to make this an imageof tragic supplication.Sucha gazealso addsa narrativeelementto the picture: somethingis happeningoutsideof the space and the moment of the photograph to which it neverthelessalludes(seeChapter7). The camera'spoint of view (the placefrom which we look) is high. He kneels;'we' stand. 'Drought Turning now to migrant' (Figure4.7) we can seehow eachof these factorsplaysa different role. The faceand upper body of the woman occupiesover half of the framed picture area.We are positionedon a levelwith her. Sheis shot in close up and,aswith the Mapplethorpephotograph,codesof spaceand personalproximity areput in play.Yet, markedlydifferent from eitherthe Mapplethorpeor the previous picture of the man on the parchedplain, her eyesdo not engageours or activelydirect our attention elsewhere.They look forward and out of the picture spacetowardsthe 'blank', spacewe occupyasviewers,but in a unfocusedway that doesnot directly answer our own gazeandwecanreadasa symptom of exhaustion.The light in this photograph is lessharsh and revealingof contrasts,softer and more diffuse, than that in 'New Year's Day' (Figure 4.6). The depth of field is shallow and we cannot read what is behind her or where sheis, exceptto seea blurred image of a secondwoman whose gestureechoesher own. Overall, eventhough we haveagaina singlemain subjectin a landscapeformat, we can conclude that this woman is not positioned and representedas an isolated victim seekingor imploring help, physicalstateunflinchingly delineated,asis the man 'NewYear's in Day'. The softnessof light and grain,the interiority of the women'sgaze together with the rhetorical quality of the way her handsare included in the frame and cradleher head,and a choreographedquality in the way this gestureis echoedin the secondfigure,approachesan aestheticizationof the horror of the woman's situation. Finally, we turn to two other images(Figures4.8 and 4.9) from the sameedition of the magazine.Theseenableus to introduce somenew points about photographic 'positioning' codesand the of the viewer.They are alsousefulbecausein comparing 'famine'we this new pair with the two imagesof have so far been discussing,we are alertedto the relativesimilarity of thosetwo imagesrather than their differences,which hasbeenthe object of our analysisso far. 'USAid' (Figure 4.8) introducesthe elementofjuxtaposition within the frame. The depth of field ensuresthat we readthe lettersUSAID on the sideof a RangeRover

81

82

Figure 4.8

HANDBOOK

OF VISUAL ANALYSIS

Chris Steele-Perkins.USAid, Sudan, 1985.

r t 'svi t t

I

u:\wttt |

I

t\Y/tt ,:/

.'@ilii@Ei@* .; -t, ''= i:::

'-

ttlt:

' : q r r

q ' ' " 4 r

+f .

:{

:

::a::a:=. ,?;-rirL

.

'a::;= ,i1::i

Figure 4.9 Mike Wells. A food convoy passing through Korem Refugee Camp on its way to Makele Camp in Tigray, February 1985.

C U L T U R A TS T U D I E S A N A L Y S I S O F T H E V I S U A L

positionedon the picture planecloseto a group ofwomen huddled in the foreground. For the European-Americanviewerthe 'ethnic' dressand swathedfacesof the women areplayedoffagainst an icon of the West'sadvancedautomobiletechnology.At first sight, like the man in 'New Year'sDay' (Figure 4.6), we seethat the attention of the group of women is directed towards another event taking placebeyond the frame, until we notice that the third woman from the left is looking acrossthe direction of the others' gazesand directly at the camera.Unlike the explicit and direct meeting of the camera's/viewer's gazeintheMapplethorpeimage(which makesno pretension to be a documentaryrecord), we havehere a rupturing of the documentaryrhetoric. Our position asvoyeurs(seeingbut not seen)and the power ofthe carnerato scrutinize (without its operation itselfbeing scrutinized)is revealed.'We'are seenand our gaze is returned. So in this imagethe photographeris, in a sense,caughtin the act of constructing,through juxtaposition, a statementabout what is beforehim. In'A food convey'(Figure4.9),for the first time in this brief analysis,the camera sharesthe viewpoint and perspectiveof the subjectsin the photograph.We standwith and behind the group who attend to their water pots and asthe child raisesits hand at the passingRedCrossconvoy.Ifwe return to think about the imageof the'redneck' in the Marlboro advertisementwith which we began this chapter,which similarly 'invites' us to look with the depicted subject(s),we can begin to seehow the more technical proposition of a 'viewing position' opens up into the fuller concept of a 'subject position'. Who are we, in eachcase?Who looks like this, with others,and at what?

L O O K I N G : R E C O G N I T I O NA N D I D E N T I T Y Who is the viewer and how are they placed to look? Being 'placed', part of having an identity, is to some extent given by the form of the image itself. In short, all or most imagesin the Westernpictorial tradition (and this includesmost photographic imageswhere the cameraand its lenseshave becomekey mechanismsin furthering and elaboratingthis tradition) are designedor structured so as to 'tell' the viewer where they are. As Bill Nichols has argued:'Renaissance painters fabricatedtextual systemsapproximatingthe cuesrelating to normal perceptionbetter than any other strategyuntil the emergence of photography'(Nichols,1981:52). As we have already noted (seep. 73), cameratechnologFwas developedand adjusted in order to take on perspectivalconventions already establishedwithin Westernart. Nichols goeson to arguethat just asthe painting standsin for thatwhich it represents,so perspective,organizedin relation to a singular imaginary point of origin, standsin for the emphasison the individual concomitantwith the emergence in the West of entrepreneurialcapitalism.Thus the viewing position constructedvia the cameracannot be seenasideologicallyneutral. Rather,particular value systems, organizedaround individualism, inform our look. That this viewingposition isvoyeuristichasbecomesomethingof a preoccupation within cultural studies.As we have alreadyremarked,in the Mapplethorpe portrait the man viewedseemsto look back at us.This is relativelyunusual.More commonly,

83

84

HANDBOOK

OF VISUAL ANALYSIS

we look at imagesof peoplewho appearunawareof the presenceof the cameraand - by extension- the possibility of becoming the object of someoneelse'slook. Thus, it has been argued,the viewer exercisesa controlling gaze.This notion of the voyeuristic gazehas been used to describethe way in which tourists look at the non-Westernworld aswell asthe way men often look at women (seeChapter3). In her well-known articleon narrativecinemaand visualpleasure,first published in 1975,Laura Mulvey drew upon Freud'semphasisupon scopophiliaas a primary human instinct and his ensuing discussionof voyeurism within her analysisof the processes and pleasuresof popular cinema spectatorship.Shearguedthat the (male) spectatorvoyeuristically gratifies his erotogenic impulses through his controlling look, which is mediatedthrough the look of the cameraand through the look of male characterswithin the world of the film, at the femalefigure on screen.The argument is complex but, in brief, she concludedthat pleasurein popular narrativecinema emergesfrom the fetishizationof the femalefigure as an object of desire.Although subsequentlycriticizedfor its narrow concernwith the male heterosexualspectator, this essaywas ground-breaking at the time as it was an early attempt to articulate psychoanalysis and feminism in order to analysevisual pleasure. Victor Burgin took up this model in relation to looking at photographs: we mayidentif,' Followingrecentwork in film theory,andadoptingits terminology, the fourbasictypesof lookin thephotograph: thelookofthecameraasit photographs 'pro-photographic' event;thelook of theviewerasheor shelooksat thephotograph; in thephotograph people(actors)depicted the'intra-diegetic'looks exchangedbetween (and/orlooksfrom actorstowardsobjects);andthe look the actormaydirectto the (Burgin,1982:148) camera. 'An

oblique glance'(Figure4.2) oflers an exemplaryopportunityto demonstrate and discussthe mobilization of the look within photography.The cameraoccupies the essentiallyvoyeuristic position of being hidden in the shop out of the sight of passers-by,thus constructinga voyeuristicposition for the viewer of the photograph 'catching whose out' of the couple cannot be acknowledgedsincethey have no idea of the presenceof the candid camera.Sincethe camerais not acknowledged,what preoccupiesus here,and,indeed,offersthe primary sourceof interestand amusement, is the exchangeof looks within the photograph.To be more precise,it is a traversing of looks,rather than an exchange,sincethe woman is contemplating,and appearsto be speakingabout, an imagewhich we cannot seewhilst the man glancesacrossher to contemplatethe fetishizednakednessof the woman depictedin the ornatelyframed painting. The humour lies in this traverse,in the neat observationof a subversive moment which is heightenedthrough the presenceof the boys in the background outside the shop acrossthe road (perhapsengagedin some mischievousactivity) - in other words, the ordinarinessof the setting. As MaryAnn Doanehasargued,this imageappearsto centreupon the woman looking, yet what makesit interestingin terms of the psychoanalyticis the scopophilic gazeof the man placedhalf out of the frame (Doane, l99I:28ff.). His gazeeffectively encasesand negateshers,not only becausethe object of her look cannot be seenand sharedby us but alsobecausethe geometryof the image is definedby the male axis

C U L T U R ASLT U D I EASN A L Y S IOSF T H EV I S U A L

of vision acrossfrom one edge of the image to the other. Her gazecentreson an imageinvisible to us and hasno part to playwithin the triangle of looks which offers complicity betweenthe man, the nude and the viewer and thus animatesthe image. Indeed,the woman, despitebeing centralwithin the picture, functions asthe butt of the joke which is what the picture is actuallyabout.This joke confirmswomen'splace as object of the look sincethe key femalepresenceis not that of the woman within the couplebut that of the nude in the painting. This photograph engagesthe viewer in complicitywith the man in wayswhich, aswith the photograph of the schoolboys discussedearlier,articulaterecognition of and, possibly,identification with a certain subversiveness. In this instance,however,the position of identification is distinctly uncomfortablefor the femaleviewer sinceit is founded on the objectificationof the nakedwoman, in phallocentricunderstandingsof desire. The bringing of psychoanalysis to bear on the imagehasbeen influential. As noted earlier,Mulvey (L975)drewupon Freud'sdiscussionof scopophilia,the instinct to look, and voyeurism,the desireto exercisea controlling gaze,in order to discuss the positioning of the femalefigure in popular cinemaasthe passivebeareror object of the malegaze.For Freudwoman was'other' and femininity wasa mysteriousriddle. As he famously remarked about women, 'you are yourselvesthe problem' (Freud, 1933:146). Mulvey drew upon psychoanalysisto investigatewaysin which, in narrative film, pleasurein looking wasconstructedaround the activemalelook. Despitecriticism for focusingon the malegazeand heterosexuallooking/desire,her essaymade a key contribution within Visual Cultural Studies as it introduced debatesabout the pleasuresof looking at imageswhich articulatedquestionsof socialpower - in this instance,patriarchy- with questionsof sexualityand the erotogenicimaginary. Within Visual Cultural Studiessuchdebatesbroadenedinitiallyto take account of the femalegaze,of homosexuallooking and of what lohn Urry has termed 'the tourist gaze'(Urry, 1990).As PatriciaHolland has remarkedin relation to ethnic otherness: Thecomingof photography gaveriseto a newsetof dilemmasaroundtheproduction of theexotic.On theonehandit displayed imagesof hithertounknownandremarkable placesandpeople,but at the sametime it had to be recognised that thesewerereal placesandpeople.The veneerof exoticismmaybe confirmedor challenged by the photographitself.(Holland,1997: ll3) Thus broader parametershave been adopted: analysisof the gazenow considersa complexity of socialpositions and power relations,and also takesinto account the implications of looking frompositionsdefinedas'other'. More recentlyissuespertaining to the fluidity of identity havebeen brought into play.Here,psychoanalyticallyinformed questionsrelatingto identity and identification processesinfluence sociologically determined questions associatedwith cultural self-location.Following Laplancheand Pontalis,we cantakeidentification as 'psychological a processwhereby the subject assimilatesan aspect,property or attribute of the other and is transformed,wholly or partially, afterthe model the other provides'(Laplancheand Pontalis,1988:205).They add that'it is bymeansof a series of identifications that the personality is constituted and specified'.This obviously

85

86

ISUAL ANALYSIS

drawsupon Freud,but it can take us into discussionsof the power of representation, and of identification processeswithin societiesacknowledgedas multi-cultural ethnically,and alsoin terms such asclassor region.As Baileyand Hall haveargued: Post-structuralist thinkingopposes thenotionthatapersonisbornwith afixedidentity - thatallblackpeople,for example, underlyingblackidentitywhich haveanessential, isthesameandunchanging. arefloating,thatmeaning It suggests instead thatidentities is not fixedanduniversally true at all timesfor all people,andthatthesubjectis constructedthroughthe unconscious in desire,fantasyand memory.Thistheoryhelps explainwhy for exampleanindividualmight shiftfrom feelingblackin onewaywhen theyareyoung,to blackin anotherwaywhentheyareolder- andnot onlyblackbut male/female, and so on. (Baileyand Hall, and not only blackbut gaylheterosexual, 1992:20) 'identities They add that are positional in relation to the discoursesaround us. That is why the notion of representationis so important - identity can only be articulated (1992:21).Here,first, identity is seenasunfixedand,second, asa setof representations' it is conceptualizedas complexly and ambiguouslycaught up within identification processes. Returning to the picturesof African peopledisempoweredeconomicallyand subjectedto famine (Figures4.6 to 4.9), how are we positionedasviewersof these pictures?Here many of us are clearly notlike the peoplepictured. Our position is asvirtual tourist, Western outsider, as onlooker. However, ambiguitiesdo enter in. We may identifr at somelevelaswomen, or in terms of ethnicity, not with the plight of the people depictedso much as in a sort of universalhumanitarian way. This identification is in accordwith the rhetoric of theseimageswhich invite compassion, and arebroadly recognizableasthe type of picturesusedby aid agenciesand charities for fund-raising.Fundamentalwithin this is the reassurance of othernessand of our safersocialand political location. By contrast,if we return to the Mapplethorpeportrait, our senseof selfmay be more ambiguouslycaught up within a set of slippagesassociatedwith sexuality and, from a white point of view, fascinationwith the ethnic other (which seemsto have motivated a number of Mapplethorpe'sportraits and hasbeenone of the key sources of offenceat, and attemptsto censor,his work). Yet a similar complexityis associated with a recent advertisementfor Pirelli tyres (Figure 4.10). Pirelli are known for producingcalendersfeaturingnakedwomenaspin-ups,which, in the 1970s,attracted disapprobationfrom feministsconcernedwithmediarepresentations ofwomen. This image of Carl Lewis,the male athlete,poised as if to sprint but wearing red stilettos (which would prevent him from running anywhere)referencesthe fetishizationof the pin-up, deliberatelyplaying upon ambiguitiesin gender,ethnicity and sexuality. As an image of a black 'hero' it challengesand disorientsassumptions.We might fantasizeourselvesas a successfulathlete, and thus in some way identifir with Carl Lewis,but the ambiguity of this image probably stops us short in our tracks.In discussingthe spectacleof the other, Stuart Hall comments that this image works 'The through acknowledgingdifference: conventional identification of Lewis with black male athletesand with a sort of "super-masculinity"is disturbedand undercut

C U L T U R A LS T U D I E S A N A T Y S I S O F T H E V I S U A L

by the invocation of his "femininity"'- and what marks this is the signifierof the red shoes'(Hall, 1997 233).The advertarrestsattention through playingwith conventional signifiersand stereotypes, confidentin our readingcompetencies, that is, in our ability to unpick and enjoy the range of intertextual referencesmobilized.

s dfwTffi rwG *; T " r'ri ,

rilfsTFil(MilJY c€r

;

Figure 4.1O

Carl Lewis photograph for a Pirelli advertisement.

Sofar all the pictureswe havediscussedhavefigured people.Finally,in Figure 4.11, we consider a landscapeempty of people but repletewith cultural references. Here a photographof a biscuit cutter, in the shapeof a sheep,is placedon top of a pile of hewn stone in the foreground of a moorland landscape.It hasbeen constructed primarily as a gallery imageand, aswith all images,relies upon the viewer bringing a set of previousknowledgesor competenciesinto play. It is one of a seriesin which domesticromanticization of the rural is wryly noted through the photographingof a toy or kitchen utensil in the location which it references.This work is culturally specific,relying as it doeson familiarity with British land, with landscapeaesthetics 'Lantra and with what we might term the Ashley- National Trust' invocation of the pastoral.The stone walls imply that this is moorland, rather than central England farmland. As with all photographs it is productive to discussthis image in immanent terms, taking account of framing, composition, tonal contrastand accordancewith the photographic convention of representingland through translatingviewsinto black and white landscapes.In formal terms, it complieswith the landscapeaestheticof horizontal divide asa one-third:two-third proportion; the land rolls awaydownwards before rising into the distance;the stonewall is foregroundedthrough sharpphotographic focus which we decodeas emphasizingits significancewithin the picture. The camerahasbeenplacedto underscorethe placing of the'sheep' in this position, overviewingthe land, exercisinga territorial gaze.Irony residesin recognizingthe incongruity of placing a biscuit cutter to stand in for the relation of actual sheepto

a7

88

H A N D B O O KO F V I S U A L A N A L Y S I S

Figure 4.1 1

Sean Bonnell. Chalk Down, from the series Groundings, 1996.

the moorlandswhich are their pasturesand which, in turn, are grasslandskept from overgrowththrough the grazingof sheep,goatsand ponies. From the point of view of the viewer there is yet more to be said.The wall on which the sheepstandsrepresentsa boundary or obstaclebeyond which we are not supposedto step.We are excludedfrom the empty, expansivehillsides.Arguably,we are reminded of our place as observer,rather than as roamer. For those of us who draw rural England into our senseof national identity, there is also reassuranceof somethingwhich may seemessentially British.Herewe might contemplatethe priority allocatedto looking, to opticality, implicit in the photographasavisual medium and, in this instance,in the photographing of land aslandscape.The image may conjure up affectivememories of other sensesheightenedin actual experienceof the rural: sound, smell,touch. 'viewing So, as we now see, position' is one componentof a more complex 'subject concept- a position' (seeChapter 9). The simplestexamplesinclude those imageswhich place us in a privileged position to seethings clearly or synoptically (someargue'panoptically').Otherscastus asvoyeurs,hidden from the view of those depicted('others') on whosebodiesor actionswe gazefrom our occludedposition. Yet other imagesbelittle us, overawingus with their scale,relativephysicalplacing and the perspectivalrenderingof the position of those who look down upon us. This particularly relatesto certain settings:church, art galleryand, aswe haveseenin the

C U L T U R A LS T U D I E SA N A L Y S I S O F T H E V I S U A L

caseof Marlboro, billboards.Many imagescentreus in a complexworld laid out for our eyein waysthat would be impossiblein reality.Issuesof proximity, intimacyand distanceare important here.However,beyond placing us, imagesalsotell us who we are in other ways; they offer us an identity. This is a transient senseof identity and consequentupon looking at the image,engagingwith and enjoyingthe messages meaningsit then givesto us.

CONCLUSION As Annette Kuhn hassuccinctlycommented: 'seen'by the In general,photographs connotetruth and authenticitywhenwhatis cameraeyeappears to beanadequate stand-infor whatisseenby thehumaneye.PhotoThetruth/authenticitypotential graphsarecoded,but usuallysoasto appearuncoded. Photography of photography drawson is tiedin with theideathatseeingis believing. (Kuhn,1985:27) anideologyof thevisibleasevidence. Nevertheless, photographsareoften treatedasif theywerea sourceof objective and disinterestedfacts, rather than as complexly coded cultural artefacts.Roland Barthes( 1984)drawsour attention to the fleetingnature of the moment capturedin alongwith limited the photographand the extentto which contemporaryexperience, knowledge of the specific context within which - and purpose for which - the photographwastaken,inform waysof seeingand introduce slippagesof meaninginto any view of the image as witness.Photography contributes to the construction of 'evidence' history; it is not a passivebystander.When photographsare presentedas of past eventsand circumstances,a setof assumptionsabout their accuracyasdocumentsis beingmade.Suchassumptionsareusuallyacknowledgedthrough statements of provenance:dates,sources,and so on. But this is to ignore wider questionsabout photographs concerning their status and processesof interpretation. As we have 'trace' to expressthe caution alreadynoted,usefully,SusanSontag( 1978)usesthe term photograph to the relation the of with which the analystneedsto think about the 'Ultimately, material world. As Allan Sekularemarks: then, when photographsare uncritically presentedas historical documents,they are transformed into aesthetic objects.Accordingly,the pretenceto historical understandingremainsalthoughthat understandinghasbeenreplacedby aestheticexperience'(Sekula,l99l: I23). As this implies,the original reasonsfor making an image,and the constraints operatingwithin the context of its making, may disappearonly to be replacedby new and substitutereferencesand expectations.Furthermore,whateveran imagedepicts or showsus,the materialmeansand medium employedto do so havea bearingupon which qualities of the depicted thing or event are foregrounded.For instance,our responses areinfluencedbythe useof glossypaperasagainstcheaper,matt, absorbent paperor by the differencesbrought about by the useof high or low resolutionVDUs when consideringimagesdownloadedfrom a websiteor storedon CD-ROM. Finally, 'what is 'how it is said'. To recognize it is not possibleto separateout said' from this is not necessarilyto go asfar asMarshall Mcluhan's edict that'the medium is the

89

90

HANDBOOK

OF VISUAL ANALYSIS

message'but it is to recognize a degree of sense in his insight. Roland Barthes used the term'the rhetoric ofthe image'to point to the way in which seductive or persuasive means are employed to make an argument or to convince us to seethings a certain way. The photographic image is, then, a complex and curious object. As we have shown, the methodological eclecticism of Cultural Studies allows the analyst to attend to the many moments within the rycle of production, circulation and consumption ofthe image through which meanings accumulate, slip and shift. This is achieved through holding in playdiverse approaches to the image which in their interaction acknowledge this complexity. This is simultaneously its strength and a point of criticism.s lndeed, in some ways Cultural Studies may seem to be a rather messy field, lacking precise boundaries and unconstrained by any single set of disciplinary protocols. But its ability to articulate a range of systematic methods of analysis in order to complexly address questions of form, production, reception and meaning while taking account of political issues,institutions and ideological discourses makes it comprehensive, significant and fascinating as a field of operation. The refusal to be prescriptive about method but rather to point to a variety of methods, and also to encourage analysts to bring into play their own experience, further underpins the strength of Cultural Studies.

NOTES I However, for the contrary argument,that Cultural Studiesshould be or is a discipline,see Bennett,T. (1998). 2 Given that historically men havebeen more activethan women asphotographers,and given the criteria mentioned above,togetherwith the copyright availability of examplesknown to us, what we have constructedis a sequencemainly of photographs of men by men. A number are alsoof black African people,photographedby white photojournalistsand now commented upon by white intellectuals.Does this matter? Cultural Studies (sometimes 'victim referred to as studies') is frequently charged with an obsessionwith political correctness(PC). We would sayit doesmatter but the answeris not to add a PC 'quota' of images.The point is to be aware,reflexively,of how the work we are doing (the academic exemplificationand exegesis of a kind of method) and the conditions in which we aredoing it meanthat somethings havebeenrenderedinvisible and othershavenot beenforegrounded. 3 Newsvaluesin Media Studiesrefersto particular dominant agendaswithin newsreporting associatedwith subject-matter,urgency,assumptionsabout the appropriatestyle of treatment, and so on. 4 An instancewould be Oliveri Toscani,the Benettonart director, whosename in itself now servesto confer extra interest in the campaign or product, recently exemplified in his leadershipof a campaignto draw attention to tourism and its implications for the survival of Venice. 5 For instance,Victor Burgin hasaccusedCultural Studiesofborrowing from psychoanalysis in ways which over-simplify and therefore misappropriate concepts and terminology (Burgin, 1996).

REFERENCES Bailey, D.A. and Hall, S. (1992)'The vertigo of displacement:shifts in black documentary practices',CriticalDecadeTen.S,2(3): Ia-8.

C U L T U R A LS T U D I E S A N A L Y S I S O F T H E V I S U A L

Barthes,R. (1977)'The photographicmessage',in S. Heath (ed,.)Image,Music, Text.London: Fontana. Barthes,R. (1984)CameraLucida.London:Fontana(origina\published in French,1980). Baxandall, M. (1988) Painting and Experiencein Fifteenth Century ltaly. Oxford: Oxford University Press,2ndedn (lst edn 1972). Bennett,T. ( 199S) Culture:A Reformer'sScience. London, ThousandOaks,New Delhi: Sage. Berger,I. 097 2) Waysof Seeing.Harmondsworth: Penguin. 'Looking Burgin, V. (1982) at photographs',in V. Burgin (ed.)ThinkingPhotography.London: Macmillan. Burgin, V. (1996) In/dffirent Spaces. Los Angeles:University of California Press. Cartier-Bresson,H.C. (1952) TheDecisiveMoment. NewYork Simon and Schuster. 'Film Doane,M.A. ( 1991) and the masquerade:theorizingthe femalespectator',in M.A. Doane (ed,.)FemmesFatales.London: Routledge. du Gay, P. (ed.) (1997) The Production of Culture/Cultures of Production. London: Sageand Oxford University Press. vol. 2 of The Freud, S. (1933) 'Femininity', in New Introductory Lectureson Psychoanlaysis, PenguinFreud Library. Harmondsworth: Penguin. Cultural Hall, S. (ed.) (1997)'The spectacleof the Other', in S. Hall (ed.) Representation: Representations and SignifuingPractices.London: Sageand Oxford University Press. Hamilton, P. (1995) RobertDoisneau,APhotographer'sLife.NewYork Abbeville Press. 'Digital Henning, M. ( 1995) encounters:mythical pastsand electronicpresence',in M. Lister (ed.), The PhotographicImage in Digital Culture. London: Routledge. 'Sweet Holland, P. (1997) it is to scan:personalphotography and popular photography', in L. Wells (ed.), Photography:A Critical Introduction.London: Routledge. Kuhn, A. ( 1985) ThePowerof theImage.London: Routledge& KeganPaul. Laplanche,J. and Pontalis,J.-8. (198S) TheLanguageof Psycho-analysis, trans. D. NicholsonSmith. London: Karnacand the Institute of Psycho-Analysis(originally published 1973). 'Reading racial fetishism:the photographsof Robert Mapplethorpe',in Mercer, K. (1994) (ed.)Welcome K. Mercer to thelungle.London: Routledge. Mirzoeff, N. (ed.) (1998) The VisualCultureReader.London: Routledge. Mulvey,L. (1975)'Visualpleasureand narrativecinema',Screen,l6(3), Autumn: 6-18. Nichols, B. (1981) Imageand ldeology.Bloomington, IN: University of Indiana Press. Panofsky,E. ( 1955) Meaning in the Visual Arts.Harmondsworth: Penguin. 'studying Rogoff, I. ( 1998) visual culture', in N. Mirzoeff (ed.), The Visual Culture Reader. London: Routledge. 'Reading Sekula,A. (1991) an archive',in Brian Wallis and Marcia Tucker (eds),Blasted Allegories.Cambridge,MA: MIT Press. Ten:8,19. Simpson,A. (ed.) (1985)Famineand Photojournalism, 'Consuming Kodak', in fo Spenceand PatriciaHolland (eds),Family Snaps: Slater,D. (1991) The Meaningsof DomesticPhotography.London: Virago. 'Photography, Snyder,I. and Allen, N.W. (1982) vision and representation',in T. Barrow, S.Armitage and W. Tydeman (eds),Readinginto Photography.Nberquerque:University of New Mexico Press(first published 1975). Sontag,S. ( 1978) On Photography.Harmondsworth: Penguin. London: Sage. Urry,I.( 1990) The TouristGaze:Leisureand Trayelin ContemporarySocieties.

91

lt w ffi

Semiotics and lconography THEO VAN LEEUWEN

INTRODUCTION This chapterdiscusses two approachesto visualanalysis:the visualsemioticsof Roland (1973,L977) Barthes and iconography.Thesetwo approachesaskthesametwo fundamental questions:the question of representation(what do imagesrepresentand how?)and the questionof the'hidden meanings'ofimages(whatideasandvaluesdo the people,placesand things representedin imagesstand for?). With respectto the imagesin Figure5.1,takenfrom a Dutch junior high-schoolgeographytextbook(Bols et a1.,1986),thesequestionswould become:(1) Who and what are the (kinds of) people,placesand thingsdepictedin thesetwo images,and how do we recognizethem as such?;and (2) what ideasand valuesdo we associatewith thesedepictedpeople, placesand things, and what is it that allows us to do so?But where Barthian visual semioticsstudiesonly the imageitself,and treatscultural meaningsasa givencurrency which is sharedby everyonewho is at all acculturatedto contemporarypopular culture, and which can then be activatedby the style and content of the image, iconographyalso paysattention to the context in which the image is produced and circulated,and to how and why cultural meaningsand their visual expressionscome about historically. The formulation 'people,placesand things' indicatesthat Barthian visual semioticsand iconographydeal,by and large,with the individual bits and pieceswithin images;in other words, they concentrateon what, in the caseof language,we would 'lexis' call or vocabulary.The social semiotic approach describedin Chapter 7 has relatively little to say about 'visual lexis' and hence one or both of the approaches describedherewould form a very usefulcomplementto it. On the other hand,Barthian 'syntax'. semioticsand iconographydo not havevery much to sayabout visual Although they do not stop at inventoriesof the meaningsof the individual people,placesand things in imagesand alsoput them togetherto show how they add up to a coherent whole, they do not usuallyidentifr specificpatternsfor this or usespecificmethods to put the meaningstogether.(Iconographysometimesdoes,but generallywithinthe confinesof a specificstyle,schoolor period.) Again, this suggests possibilitiesfor combiningsocialsemioticsandthe approaches describedhere.The formulation'people, placesand things' might also seemto exclude'abstractthings'. While it is true that Bartheshasconcentratedon figurativeimages,more specificallyphotographicimages (seeGroup e p, 1992for a structuralist approachto abstraction), and that iconography

93

SEMIOTICS AND ICONOGRAPHY

^ o o 6 -9, 6

:$ ::::g

16;

{?; r'i

o

ei;rf it '1r

'" ';

i-

t"

>

i,5.11,i:

a q

f

o ; L

cl

i : l

ix l

: a

6 b rlt

o c

o

*0} t G Gtr

w

?r

#

t

i

ssl o c I

rg

b o rf.:

rvo r

| I I

. l , l

ll

J l v

€, !

1 1

F l

P c l d E l

l t

:*l sF,l

9o Il

s

{;i =ui

>

*

*

, u E ;

F ;-

E b

s ; 1 1t s

; r t r ; r : l ; : d * i r - : - : " 3 ; ; : : i 1 E i : :

i

r

)

;

: * : i

i

:

:

-

. i : i : i ; t :

: , 2

"

- ;

J

)

i Ji : t " . : :!

q - ' r-' l: ; ! i :

; T : a ; . ;

"i '; ;' t : :

: i i : +

-

;

T : i > 4 : f !

J

'

" * E ; ; ' :

t

:

c

-

t

3:

:

= E .= o E

t|o o ED E

94

HANDBOOK

OF VISUAL ANALYSIS

hasalsoemphasizedfigurativeart, there is no reasonwhythe methodsdescribedhere cannot alsobe applied to 'abstractthings', and I will try and show this in the course of the chapter. Finally, this chaptershould not be seenasan introduction to Barthian visual semioticsand iconographyas a whole. It concentrateson the way in which 'public' imagesareanalysedwithin thesetwo approaches.It leavesaside,for instance,Barthes's later concerns with the immediate emotive impact of certain aspectsof images, with the 'punctum' of the image (Barthes,1982),or the concern of iconography with dating works of art and authenticatingtheir authorship,and focuseson concepts and methodsusefulfor contemporarystudiesof the visual representationof specific issues.

SEMIOTICS In Barthian visual semiotics,the key idea is the layering of meaning.The first layer is the layerof denotation,of 'what, or who, is beingdepictedhere?'.The secondlayeris the layer of connotation,of 'what ideas and values are expressedthrough what is represented,and through the way in which it is represented?'.

Denotation For Barthes,denotation is a relatively unproblematic issue.There is no 'encoding' into somekind of language-likecodewhich must be learnt beforethe messagecanbe deciphered.Perceivingphotographsis closelyanalogousto perceivingrealitybecause photographsprovide a point-by-point correspondenceto what was in front of the camera,despitethe fact that they reducethis reality in size,flatten it and, in the case ofblack and white, drain it of colour. In the caseof drawingsand painting the situation is not essentiallydifferent.Although the sryleof the artist providesa'supplementary message',the content is still'analogicalto reality'. Here is how Barthesdescribesthe denotationin one of his most often quoted examples:'I am at the barber'sand a copy of Paris-Matchis offered me. On the cover, a young Negro in a French uniform is saluting,with his eyesuplifted, probablyfixed on a fold of the tricolour. All this is the meaningof the picture' (1973:116).In other words, the first layer,the denotative meaning,is here constitutedby the act of recognizingwho or what kind of personis depictedthere,what he is doing, and so on. Barthesrealizesof coursethat we can only recognizewhat we alreadyknow. Describinga particular advertisementfor pasta,he writes: 'We needto know what a tomato,a string-bag,a packetof pastaare,but this is a matter of almostanthropological knowledge.This message corresponds,asit were,to the letter of the image,and we can all agreeto call it the literalmessage, asopposedto the symbolicmessage'(1977:36). Anyone who has tried to describeimagesin this way knows that such knowledgeis often lacking or existingonly at avery generallevel.We may recognizea uniform as a uniform without knowing what kind of uniform it is, or a tool asa tool without having

S E M I O T I C SA N D I C O N O G R A P H Y

the faintestideawhat it is for. Normally such a lack of knowledgeis not a problem. We are not even awareof it until we have to describewhat we see.We have mentally put it into the categoryof things about which we do not needany detailedknowledge. Clearly imagescan be perceivedat different levelsof generality,depending on the context, dependingon who the image is for, and what its purpose is. In describing denotativemeaning it may thereforebe desirableto introduce a little more context than Barthesdid, to set a plausiblelevel of generalityfor the reading.In the caseof Figure5.1 the text seeksto describe'others'foran'us'(remember the title of the 'us' Dutch 'The is teenagersin a highThird World in Our Street').This chapter, women on the left the In this context the 1980s. schoolcontext in the secondhalf of 'non-indigenous'people. are'immigrants' or, as the Dutch call them, allochtonen, Whether they areMoroccan or Turkish is not relevant.Nor is it relevantthat the three and that they arewearingthem in women arewearingdifferent kinds of headscarves for people able to recognize meanings perhaps carries different ways and that this and differentwaysofwearing them. On the other hand, differentkinds of headscarves the boy on the right will be recognizedas a Surinamer (evenif he is not) by Dutch schoolchildren. Is denotation entirely op to the beholder?Not necessarily.This too depends on the context.There are contexts(for example,certain forms of modern art) where a multiplicity of readingsis allowedor evenencouraged.But there are other contexts where the producersof the text havean interestin trying to get a particular message acrossto a particularaudience,and in suchcasestherewill be signsto point us towards the preferredlevelof generality.Evenif I were a Turkish studentof the Dutch school and if one of the three women in Figure 5.1 were my sisteror my aunt, I would still also seethat here sheis depictedas a typical'allochtoon'. Taking such pointers into account could help overcome some of the problems involved in a Barthian descriptionof visual denotation. I will list four. They can of courseoccur in various combinations: Gategorization Captions can indicate the preferredlevel of generality.But even in the absenceof a caption people can be visually representedas a specificindividual (my sister,or my aunt) or a sociattfp. ('an immigrant woman'). Typification comesabout through the useof visual stereotypes,which may either be cultural attributes (objects,dress, hairsryle,etc.) or physiognomicattributes.The more thesestereotypesovershadowa person'sindividual features(or the individual featuresof an object or a landscape), the more that person (or object,or landscape)is representedasa type. Headwearis a commonly usedcultural attribute, or asBarthescallsit'object (think of the Frenchberet,the workman's cap,etc.) and in Figure 5.1 the headsign' scarvesare clearlymore salientthan the women'sindividual features.The hairstyleof the boy on the right alsostereotypeshim to a degree,but lessso,becauseit doesnot dominate the picture to the samedegree.Again, the glamorous people in fashion magazinesand advertisementscanoften disappearasindividuals behind the hairsryles and the make-up that signifr them asdesirablesocialtypes.On the other hand, it is also possibleto de-emphasizesuchcultural attributes.JohnBerger(1972:III-12) compares

95

HANDBOOK OF VISUAL ANALYSIS

Rembrandt'searly'Self-portraitwith Saskia'to a much laterself-portrait.In the former the attributesof Rembrandt'snew-foundwealthand statusareprominently displayed. In the latter all the light fallson Rembrandt'sagedface,and everythingelseis reduced to a dark, shadowyoutline. Traditions of representationcan alsocreatephysiognomicstereotypes. To the degreethat theseare exaggerated or otherwisemadeprominent in the representation (for example,byselectingapersonor a pictureofa personin whom theyareprominent), the persondepictedwill be represented asa'tFp.'rather than asan individual.It can be argued that the pointed contrast betweenthe blonde girl and the black boy stereotypes both the girl astypicallyDutch and the boy asa typical second-generation Surinameimmigrant. Groups vs. individuals Depicting peoplein groupsrather than asindividuals canhavea similar effect,especially if similarity is enhancedby similar posesor synchronizedaction.The threewomen in Figure5.1 not onlylook similar but alsoall walk in the samedirection and are angled towards the viewer in more or lessthe sameway. This reinforcesthe'they're all the sameeffect'that constitutesgeneralization.Elsewhere(van Leeuwen,in press)I have pointed out how in pressphotographsof the Gulf War allied soldierswere usually depictedasindividuals, doing things like defusingbombs, writing lettershome, and so on, and Iraqi soldiersasgroupsinvolved in synchronizedactionslike aiming guns and surrendering. Distancing Showingpeoplefrom a distance(in a'long shot') canalsodecrease their individuality and makethem more into types,becausefrom a distancewe will be lessableto discern their individual features.Figure 5.1 is again an exampleof this. The'immigrant women' on the left are distant, the young people on the right closer,with obvious effectson the degreeto which they can be seenasunique individuals. Surrounding text As alreadymentioned, the surrounding text (or adjacentpictures) can alsoprovide pointers.Captionscangivethe nameofdepictedpeople,or describethem astypes.But pictures and words may also contradict eachother in this respect.The picture of a namedindividual may illustrate a generalizingtext, for example.British documentaries made in the 1930soften showedhighly genericshots of workers while a voice-over commentarywould somewhatpatronizingly call them by their first names.

Connotation The secondlayer of meaningis connotation,the layer of the broader concepts,ideas and valueswhich the representedpeople,placesand things 'stand for', 'are signsof . The key idea is that the denotativemeaning is alreadyestablished,that we have,for

SEMIOTICS AND ICONOGRAPHY

'allochtoon'.On this alreadyestablished instance,akeadyidentified the threewomen as layer of recognition/interpretationa secondmeaning is then superimposed,the connotation. It can come about either through the cultural associationswhich cling to the representedpeople,placesand things,or through specific'connotators',specific aspectsof the way in which they are represented,for examplespecificphotographic techniques.In Mythologies(1973) Barthesconcentratedon the former. In his essays on photographyin Image,Music, Text (1977)he addedthe latter. We alreadydiscussedthe denotativemeaningsof Barthes'ssaluting African soldier.Here is the whole quote, showinghowhis readingmovesfrom denotationto connotation: isofferedme.On thecover,ayoungNegro I amatthebarber'sandacopyof Paris-Match in a Frenchuniform is saluting,with his eyesuplifted,probablyfixedon a fold of the tricolour.All this isthemeaningofthepicture.But,whethernaivelyor not, I seevery to me:thatFranceis a greatEmpire,thatall hersons,withoutany wellwhatit signifies faithfullyserveunderherflag,andthatthereisno betteranswer colourdiscrimination, thanthezealshownbythisNegroin serving colonialism of analleged to thedetractors system: againfacedwith a greatersemiological oppressors. I amtherefore hisso-called thereisa signifier,itselfalreadyformedwith aprevioussystem(ablacksoldierisgiving mixtureof Frenchness andthereisa signified(it ishereapurposeful theFrenchsalute); (1973: 116) andmilitariness). Suchconnotativemeanings- in Mythologies(1973) Barthescalledthem'myths' - arefirst of all very broad and diffuseconceptswhich condenseeverythingassociated with the representedpeople,placesor things into a singleentity (henceBarthes'suse of terms like'Frenchness'and'militariness'to indicatethesemeanings).Secondly, they areideologicalmeanings,servingto legitimatethe statusquo and the interestsof thosewhosepower is investedin it (in this caseFrenchcolonialismand military role in Africa) - it shouldbe rememberedthat Mythologies,though translatedinto English only in 7973,datesfrom 1957.Photographsare particularly good vehiclesfor such 'finding' these meanings,becausethey naturalizethem. They canbe thought of asjust 'constructing' them. And they canalsobe meaningson the street,asit were,rather than 'in 'spellingout' their message, so many words', not sayingit thought of asnot quite 'read into it' by the viewer, rather than as so that that messagecan be construedas communicatedby a powerful socialinstitution (seeChapter4). Two elementsof the content of imagesare singledout as especiallyfrequent carriersofconnotation,posesand objects.Thereis,saysBarthes,an unwritten'dictionart' of poseswhich is known to everyonewho is at all exposedto the massmedia,and whose 'entries' againhave the kind of broad and ideologicallycoloured meaningsthat are typical of connotation: atthetimeof the distributed Kennedywidely of President Consider apressphotograph 1960election:a half-lengthprofileshot,eyeslookingupwards,handsjoinedtogether. the raisingof the signifiedsof Hereit is the veryposeof the subjectwhichprepares youthfulness, spirituality,purity.Thephotographclearlyonly signifies connotation: attitudeswhichform ready-made of a storeof stereotyped because of the existence (1977:22) (eyesraisedheavenwards, handsclasped). elements of signification

97

HANDBOOK

OF VISUAL ANALYSIS

At this point we should mention Goffman's exemplaryaccount of gendered posesand their meanings(1979),which comescloseto realizingBarthes'simagined 'lexicon', albeit for a restricted(but important) domain of meaning.Objectsare equallysignificant: Specialimportancemustbe accordedto whatcouldbe calledthe posingof objects, wherethemeaningcomesfrom theobjectsphotographed . . . Theinterestliesin thefact thatthe objectsareaccepted inducersof ideas(bookcase= intellectual) or, in a more obscure way,veritablesymbols(thedoorto thegaschamberfor Chessman's execution with its reference to thefuneralgatesof ancientmythologies). Suchobjectsconstitute excellentelementsof signification:on the one hand they are discontinuousand completein themselves . . . while on the othertheyreferto clearfamiliarsignifieds. Theyarethustheelements of a veritable lexicon.(1977:23) Theseexamplesanalysespecificpartsofimagesand areseenas'discontinuous' individual'dictionary entries',but Barthesalso readsthem togetherin a'discursive readingof object-signs'(1977:24).Inthis case,which he calls'slmtax','thesignifier of connotation is no longer to be found at the levelof any one of the fragmentsof the sequencebut at that. . . of the concatenation'(ibid.). Here is an exampleof sucha 'discursive reading': Here,for example, isa'composition'of objects: and awindowopening on to vineyards tiledroofs;in front of thewindowa photographalbum,a magnifringglass,a vaseof flowers.Consequentlywe arein thecountry,southof the Loire(vinesandtiles),in a bourgeois home(flowerson thetable)whoseowner,advanced in years(themagnifring glass), isrelivinghismemories(thephotograph album)- Frangois Mauriacin Malagar (photoin Paris-Match).The connotation whichsomehow'emerges'from all these 'captured'asthoughthescene signifringunitswhicharenevertheless wereimmediate andspontaneous, thatisto say,withoutsignification. Thetextrenderstheconnotation explicit,developing thethemeof Mauriac'stieswith theland.(1977:23) In this examplethe 'object-signs'are clearly denotative.The recognition of a place ('the country,southofthe Loire') through specificattributes('vinesandtiles')is entirely similarto the recognitionof a Frenchsoldierbyhis uniform, or an'immigrantwoman' by her headscarf.It is their concatenationwhich connotes 'myths' of the French countryside. Connotation canalsocomeabout through the styleof artwork or the techniques of photography,suchas'framing,distance,lighting, focus,speed'(1977:44).Barthes callsthis'photogenia'(1977:23):'Aninventoryneedstobemadeofthesetechniques,but only insofaraseachof them hasa correspondingsignifiedof connotation sufficiently constantto allow its incorporation in a cultural lexicon of technical"effects".' Severalof the analyticalcategoriesdiscussedin Chapters2 and 6 would fall under this heading,in particular'socialdistance','point of view'and'modality'. We cannow attemptto applythis analysismore fullyto Figure5.l,looking first 'object-signs' at the in Table5.1.Then we canlook at the 'photogenia'connotatorsin Table5.2.Looking now at 'syntax',and putting togetherthe 'object-signs'and the connotators,we get somethinglike the following story. The first generationimmigrants

SEMIOTICS AND ICONOGRAPHY

(the three women) do not adaptto 'our' culture. Theywalk through a typical Dutch street(bicycles),whereDutch women'sfashionis on salefor everyone(the shopbehind the women), but neverthelesspersistwith their own ways (headscarves). Thus it is through their own fault that they do not integrateandremain distant (long shot) and detached(profile) from' lus'autochtone(' indigenous') Dutch. By contrastthe younger generationof immigrants (the black man) adaptto our culture and eveninfluenceit as'ethnic'culture transformsinto consumerculture,includingyouth music and fashion (the plaitedAfrican hairstylewith beads).This makeslove relationshipspossible(the man's arm around the woman's shoulder)betweenautochtoon(the woman) and allochtoon(the man) and may evenhold out the promise of marriage(the ring).

Table 5.1

Denotativesignifierand signifiedin Figure5.1.

Headscarves Bicycles Dresses in shop window

lmmigrantwomen Dutch street Store with fashionable dresses

Black skin plus Afro hairstyle Blonde hair and white skin Rings Awning and corner of chair

Second-generationSurinamer Dutch girl Betrothaland marriage Caf6

Table 5.2 Connotatorand connotationin Figure5.1.

Angle: profile (3 women) semi-profile (couple) long shot (3 women) closer shot (couple)

Detachment Somewhat greater involvement Socially distant from the viewer Socially closer to the viewer

This visual'story'appearedin a schooltextbookin a time when the'asylum seekers'questionbecamecrucial in The Netherlands,and in which legitimationsfor the curtailing of immigration and the expulsion of refugeeswere construed and promulgatedby the governmentand the media- and evidentlyalsoby the education publishing industry. It is surelyno lessan ideologicalmessagethan that of Barthes's salutingAfrican soldier. Although I havegiven a qualitativeanalysisof a singlepicture, it is possibleto usethe conceptsintroducedherequantitatively(seealsoChapter 2). The chapterfrom which Figure5.1wastakenis called'TheThirdWorld in Our Street'and depictsmany 'non-indigenous'Netherlanders. In Table5.3I tabulatehow one of the'photogenia' connotators was used in the geographytextbook as a whole (VCS standsfor 'very closeshot',MCS for'medium closeshot',MS for'medium shot',LSfor'long shot'and

99

100

H A N D B O O KO F V I S U A L A N A T Y S I S

'verylong VLS for shot'). Similartabulationscould be madefor other ('object-sign'or 'photogenia' connotators. ) 'non-indigenous' people The table showsthat pictures of far outnumber picturesof indigenous'people:after all, they are'the problem'. 'Indigenous'people we know akeady.No needto show them. The main indigenouspeopleincluded are the girl from Figure5.1,a televisionreporterand a teacher.Non-indigenouspeople aremostly seenfrom somedistance,though strip cartoonsalsoshownon-indigenous peoplein closeup - but, then, cartoon drawingscancreatedistancein other ways,for instancethrough stereotypicalmodesof depiction. Table5.3 Useof 'photogenia' connotators in a chapteron 'Netherlanders'. Photographs

Drawings -i

n

3 13 10

'..', ft{$gno

2 2

g

Non',ind;qgnsus r

H*digemo*ts ;

1 6 t1

ICONOGRAPHY Iconographydistinguishesthreelayersof pictorial meaning:representational meaning, iconographicalsymbolismand iconologicalsymbolism. 'denotation'. The ideaof 'representational Panofsky meaning'iscloseto that of speaksof it as the 'primary or natural subjectmatter' (1970:53) and describesit as the recognitionof what is representedon the basisof our practicalexperience,taking into accountthe stylisticconventionsand the technicaltransformationsinvolved in the representation- for example,the fact that in medievalpaintings 'human beings,animalsand inanimateobjectsseemto hang loosein spacein violation of the law of gravity,without therebypretendingto be apparitions'(Panofsky,1970:60), or the fact that in photographythe three-dimensionalworld is reducedand flattened. The key idea is to seethis kind of recognition as separatefrom the understandingof the conventional meaningsthat may be associatedwith what is represented.As an examplehe usesa gestureof greeting:a given viewer might recognizea picture as representinga man lifting his hat, but not know that lifting your hat is a conventional form of greeting.In the analysisof contemporaryimagesthis may seeman unnecessary complication,but in studying art works from the past it is not: faces may no longer be recognized,objects,gesturesand activitiesmay have become obsoleteand establishingwhich of the people,placesand things in a picture are iconographicallysignificant(or, rather,wereat the time of its production) may require quite a bit of research. Iconographical symbolism.Atthis level,the'object-signs',to useBarthes'sterm, not only denote a particular person, thing or place,but also the ideasor concepts attachedto it. Panofskyglossesit as 'secondaryor conventionalsubjectmatter' and explainsit asfollows:

SEMIOTICS AND ICONOGRAPHY

symbolism]is apprehended by realisingthat a malefigurewith a [Iconographical kniferepresents St Bartholomew, that a femalefigurewith a peachin her handis a personification of veracity,. . . or that two figuresfightingeachotherin a certainway represent theCombatof ViceandVirtue.In doingthisweconnectartisticmotifsand (1970 54) combinations of artisticmotifs(compositions) with themesor concepts. ElsewherePanofskynotesthat suchiconographicalsymbolism alsoexistsin twentiethcentury popular art: Therearose,identifiableby standardised appearance, behaviourand attributes,the well-remembered typesof theVampandtheStraightGirl (perhaps themostconvincing modernequivalents of the Medievalpersonifications of the Vicesand Virtues),the FamilyMan andtheVillain,thelattermarkedby ablackmoustache andawalkingstick. (Quotedin Wollen,1972:146) The conventionsof the past are more easilyrecognizedas conventionsthan those of the present,but I hope that the remainder of this chapterwill show the value of studying contemporaryimageswith the tools of iconographicalanalysis. Iconologicalsymbolismiswhat, in anothercontext,would be calledideological meaning.To analyseit is, in Panofsky'swords, to 'ascertainthoseunderlying principles which revealthe basicattitudeof a nation, a period, a class,a religiousor philosophical ( 1970:55): persuasion' Whenwetry to understand person[DaVinci'sLastSupper]asadocumentof Leonardo's ality,or of the civilisationof the ItalianHigh Renaissance, or of a peculiarreligious attitude,we dealwith the work of art asa symptomof somethingelsewhichexpresses itselfin a countless and varietyof othersymptoms, andweinterpretits compositional iconographic featuresasmoreparticularised evidenceof this 'somethingelse'.The (whichareoftenunknownto discovery andinterpretation of these'symbolical'values theartisthimselfandmayevenemphatically differfrom whatheconsciously intended (1970:56) to express) is theobjectof iconology',asopposed to 'iconography'. Thereis thus a clearmove herefrom identifring generallyacceptedconventions(which the artist would alsobe awareof) to an interpretation of which the artist may not be awareandwhich maynotbe generallyaccepted, an indispenbutwhich is nevertheless '[The sablepart of the analysis: work of art] must also be understood as carrying a more-than-visualmeaning'(I97 0: 205). Unlike Barthian visual semiotics, iconography uses both textual analysis and contextual research.With respectto art works of the past it is not possibleto 'accepted 'object-signs' appealto a sharedknowledge of what stand for (Barthes's inducers of ideas'), and so iconography also usesintertextual comparison and documentaryresearchto support its interpretations.Iconographerswill'find out as much astheypossiblycanofthe circumstances underwhich the objectsoftheir studies 'readbooks on 'collect werecreated', and verifr all the availablefactualinformation', 'observe theology and mythology in order to identify the subject matter', and the interplay betweenthe influencesof literary sourcesand the effect of self-dependent

101

102

HANDBOOK

OF V

representationaltraditions to establisha history of iconographicformulae or "types"' (Panofsky,1970:41). Using this approachin studyingthe contemporaryvisualrepresentation of significant issuescan bring to light the origins of certain conventions 'genesis and undo the ideologicallyconvenient effectsof what Bourdieu has called amnesia'. In this chapter I use a contemporary study of visual racism as an example: NederveenPieterse'sWhite on Black (1992). The book resultedfrom an exhibition held in Amsterdamin 1989,and subsequentlyin severalother Europeancapitals.This exhibition wasbasedon a collectionofvisual materials(prints, drawings,magazines, books,posters,packaging,etc.)put togetherby RufusCollins, an Afro-American theatre director working in Amsterdamwho had beenastonishedat the continued existence ofdemeaningcaricatures ofblackpeopleswhich inthe USAwouldhavebeenproscribed long ago.NederveenPieterse'sresearchon the collection,which usedthe iconographical approach,formed the basisof the exhibition, and later of the book. Drawing on it here will hopefully demonstratethe relevanceof iconographyfor contemporarystudiesof visual representation. It is sometimesarguedthat iconographyfavoursthe 'original meaning' of art works from the pastwhen theseart works might mean somethingquite difFerenttoday. This is of course true. What today's tourists get out of medieval and Renaissance paintingsdiffersfrom what the artists'contemporariessawin them. Panofslcycomments 'patina that the of age' of art works is an important part of their contemporary meaning,eventhough this wasobviouslynot intendedby the artists(1970:38).And he realizesthat the're-creativeexperienceof art'dependson the'cultural equipmentofthe beholder' (1970:40).Different readingswill occur.And theywill be setin the context of differenthegemonicor counter-hegemonicsocialinstitutions (tourism, education, political movements,etc.)and servediflerent interests.Readingssuchasthoseproduced byNederveenPietersearecritical readings,setin the contextof the anti-racismmovement, and hencestronglycontestedby many. NederveenPieterseshows,for instance, how the iconographyof the 'golliwog' doll cameabout in the heydayof imperialism and colonialism,when blackpeoplein the USA and Britain wereroutinely the subject of racistmockery(andworse),andalsohow, whentheNational CommitteeonRacism in Children's Books starteda campaignbranding the golliwog asracist in the 1980s, the majority of the presscame out in favour of the golliwog and reproachedblack peoplefor their'oversensitivity'.In 1998the coverof the Guardian'sweekly'Guide' ( 11April 1998)magazinefeatureda golliwog,to highlight a programmein which the detractorsofthe golliwogweredepictedasexamplesof outdated'politicalcorrectness'. In such a context it is good to remember where the symbol comesfrom.

Representational meaning How doesiconographyestablishthat a particular imagerepresentsa particular (kind of) person (or object,or place)?Following Hermeren (1969),to whosework this chapteris much indebted,we can distinguishfive typesof answerto this questionin the work of art historians.

S E M I O T I C SA N D I C O N O G R A P H Y

The title indicates who or what is represented One possibility is that the work itself includesa written title, or some other kind of inscription or caption.This title then indicateswho or what is represented. Arguments like the following are common in the work of art historians:'A miniature of Davalos with authenticcontemporaryinscription in the collectionof Duke Ferdinandof Tirol proves that we are really confronted with a portrait of this field marshal' (Ingvar Bergstrom,quoted in Hermeren,1969 47). Many of the picturesin NederveenPieterse'sbook fall into this category.For instance,a 7947advertisementfor Chesterfieldcigarettesshowedthe boxer Ioe Louis, aswell ashis signature,underneaththe words'Chesterfield,the Champ of Cigarettes' (1992:149).A 1930sposterfor the Dutch Musicians'Union featuredthe slogan'Do not becomea musician.Dyrrg occupationfor Netherlanders'andshoweda Iewish and a blackmusicianaswell asthe horn of a gramophone.The label'buitenlandei ('foreigner') wassuperimposedoverthe faceofthe black musician.As alsoindicatedin Nederveen Pieterse'scaptions,the boxer wasrepresentedasa specificindividual and the musician asa tFpe. The identification of who or what is represented may also be done on the basis of personal experience In the caseof art works from the pastthis is obviouslyrestrictedto objects,buildings and landscapeswhich have survived relatively unchanged.And eventhen, we cannot alwaysbe sure that our recognition is at the appropriatelevel of generality.We may recognizea specificpersonwhen ageneralized'type'wasmeant to be recognized. ldentification on the basis of background research As we havealreadyseen,iconographersoften undertakecontextualresearchto establish who or what was portrayed. But this again may causethe identification to be made at an inappropriate levelof generality.Hermeren quotesthe caseof a Rembrandtdrawing called'Girl sleeping'.Thereis,apparently,good evidencethat Rembrandt'ssecondwife wasthe sitter for the portrait. But the similaritybetween the portrait and Hendride is only slight,judging by other portraits.As alsosuggested by the title, it must havebeen Rembrandt'sintention to portray a'type', rather than a specificperson. Sometimesboth the type and the personare meant to be recognizable.Edgar 'Bust Wind (1937: 138),for instance,discusses the of Commodus as Hercules'as follows: 'The Roman Emperor Commodus had his portrait sculpturedwith lifelike acclrracy,yet he surrounded his head with the skin of a lion and held a heavy club in his hand. Being thus vestedwith the emblemsof Hercules,he presentedhimself aspossessed of his virtues.' A similar kind of double identification occurs in Figure 5.2, where General Aguinaldo (1869-1964),leader of the Philippine resistanceto American colonialism, is representedas a black dancing girl (and Uncle Sam as a white old lady). The dancing girl, like the black minstrel, was an iconographicalsymbol of black people as childlike, irresponsibleand happy-go-lucky,as a simple'child of nature, and one of the most interesting,selflessand happiestof creatures'(from a late nineteenth'advertising centuryAmerican card',cf, NederveenPieterse,1992:137). As Nederveen

103

104

HANDBOOK

OF VISUAL ANALYSIS

Pietersepoints out, the cartoon is an instanceof 'niggering',the comparisonof colonized peopleswith the American minorities (black peopleand Native Americans). 'The American pressregularlyrepresentedFilipinos and other peoplesasblacks',he 'it writes, which showsthat is not ethnicity, or "race", that governsthe imageryand discourse,but rather, the nature of the political relationship between peoples' (1992: 217). The hairstyleof the dancing girl remains a potent symbol, for instance in contemporaryadvertising(Figure 5.3) and more generallyin fashion,where it is perhapsan exampleof what Halliday hascalled'anti-language',the proud useof demeaningepithets('blacks'usedto be a derogatoryterm) and marks of identification.

Lqq-;.ffi$j

l|. ir(@

\ r"t

#

':::#:!-

' -'1t =-. ,

FL**&d ;-

a. rt**.a

Figure 5.2 General Emilio Aguinaldo represented as a black dancing girl. Cartoon by Victor Giffam in Harper's WeeklV,1899.

S E M I O T I C SA N D I C O N O G R A P H Y

The contemporarystarsystemmeansthat actorsareoften to be identifiedboth asthemselvesand astheir characters.This alsoextendsto advertisements, for example 'the a Dutch advertisementshowing black actor Donald Jonesoffering a variety of fypes of coffee' (NederveenPieterse,7992:194) and adopting the stereotypedsubservientposture and smile of the'black servant'.As NederveenPietersecomments: 'Over the yearsthe role of blacksin the advertisingand packagingof cocoa,chocolate and coffeehashardlychanged.As tropicalproducts thesethings seemto bepermanently associatedwith the colour black and with black labour' (1992: I94).

#HTRA f;#h.$ tffi$Sre. ##reVffiffin##ht**_gK*{[ffi X#'i"il#. Figure 5.3 Advertisementfor a Dutch brandof crisps (TheNetherlands,1987).

1 O5

106

HANDBOOK

OF VISUAT ANALYSIS

ldentity established through reference to other pictures The identity of a people,placesor things can alsobe establishedon the basisofvisual intertextuality, of their similarity to people, placesor things in other pictures. An examplefrom Panofsky: Thereislittle doubtthattheyouthfulfigurewhich. . . 'attractstheeyesof thebeholder' in the Columbianaltarpieceis the samepersonwho appearsin Roger'sportrait of CharlestheBold.. . . Theaggressive, somewhat prognathous mouthwith itswatchfully pinchedcorners, thelarge,impatienteyes, thesombreeyebrows and,mostimportant, thelong,unrulyhairrecurin bothpictures;to judgefrom themanyotherportraitsof personages belongingto the entourage of Philipthe Good,it wouldseemthat at this mostformalof all courtsno onebut theCrownPrincecouldaffordto wearhishair as thoughtherewereno scissors andcombs.( 1953:286) In the contemporarymedia,with its plethora of photographicimagesof the famous and infamous,this type of recognition is of coursevery common. No 'title'is needed for the recognition of runner Nellie Cooman in an advertisement(seeFigure 5.8 on page 113), although with time such visual rccognizability will fade, as perhaps is alreadythecaseof JoeLouis cited above.The situationbecomesa little more complex when we alsoinclude the recognitionof 'types'.NederveenPieterse(1992:I 3 1) sums up the following characteristicsof the type of the 'black servant' (many of them of coursealsohaveiconographicalsignificance): thesmileexpressing availability; theservantdepictedassmaller,loweror in thebackground; a slightstoopin posturewhich makeshis body seemshorteror smallerand which suggests subjection; a bendin thekneeswhich,asin reverence, expresses subjection; watchingthe eyesof the personbeingserved- a look which doesnot haveto be reciprocated; on thecontrary,thepersonservedusuallylookspasttheservant; a physicaldistance betweenthe servantandservedwhichdenotessocialdistance and statusdifference. ldentification on the basis of verbal descriptions Fictitious people, placesand things may be identified on the basisof verbal intertextuality, of descriptionsin mythologicalor other literature - on condition that the description is applicableto the representationand can be consideredto have been familiar to the artist and/or patron of the work. Thus St Josephwasportrayednot on the basisof a physicalstereotype(asoften in the caseof Christ), but on the basisof the attributes of his trade, a description clearly basedon what is written about him in the Gospels.'The figure of Josephappearsin a wing besidethe Annunciation as an artisanwho fashionsmousetraps'(Schapiro,1945:182). Modern types,for instanceracialstereotypes, maybe fictitious in anothersense and havetheir origin in other kinds of literature. NederveenPieterseshowshow the physiognomicstereotfping of black peoplewas still entirely absentfrom portrayals of black people by Rubens,van der Weyden, Rembrandt and others. Before these

SEMIOTICS AND ICONOGRAPHY

stereotypescould enter into circulation they had to be constructed as meaningful signs,for instancein scientificwritings suchasthoseof GeorgesCuvier (1769-1832), the Swissscientistregardedasthe founder of paleontology: TheNegroRaceisconfinedtothesouthofmountAtlas;itis markedbyablackcomplexion, crispedor woollyhair,compressed cranium,andaflatnose.Theprojectionof thelower partsof theface,andthethicklips,evidentlyapproximate it to themonkeytribe;the hordesof whichit consists havealwaysremainedin the mostcompletestateof utter (Quotedin Nederveen barbarism. Pieterse, 1992:42) Suchwriting connectedspecificmeaningsto specificfacial features(featureswhich are of coursenot at all specificto black people).They wereto be echoedin countlessother places(cf., for example,Ineke Mok's 1999 study of racism in Dutch geography textbooksbetween1876and 1992).

lconographical symbolism A first distinction which needsto be madehereis that betweenabstractsymbols(abstract shapeswith symbolicvalues,for examplethe cross),and figurativesymbols(represented people, placesor things with symbolic value). Abstract symbolswere common in the Middle Ages.The haloeswhich indicated sainthood, for instance,had several variants.Therewerehaloeswith a cross(reservedfor Christ), triangular haloes(reserved for God) and squarehaloes(reservedfor donors who wanted to be depictedamong the saints).In the ageofthe logo, abstractsymbolsmayyet againbecomeincreasingly important. Figurativesymbolsare often seenas natural by contemporaries,as operating on the basisof transparentanalogieswith the natural world, rather than on the basis of conventions.In hindsight they may turn out to havebeenbasedon conventional ideasabout nature, rather than on nature itself. In the Middle AgesChrist was often depictedasa lion. This seemeda perfectlynatural symbolfor the resurrectionasit was believedthat lions kill their cubs after birth and then revive them after three days. Scientificaswell aspopular racism may appealto suchanalogieswith the animal world. In latenineteenth-centuryAmericablackpeoplewerecalled'coons'because raccoons havelargewhite eyesin a dark snout and areknown assly night creatures.'Coon songs' popularizedthe analogy.Their sheetmusic coverswere 'aggressively racist,showing black peoplewith lips like watermelons,eyeslike saucers,wild curly hair and elongated bodies' (NederveenPieterse,1992:135). I will now turn to the textual and contextual arguments used in iconography for justif ing the symbolic interpretationsof representedpeople,placesand things. The textual argumentsidentifr visual 'pointers' which tell the viewersthat a given motifshouldbe interpretedsymbolically.FollowingHermeren (1969),fourkinds can be distinguished: 1 The symbolic motif is representedwith more than normal care and detail comparedto similar works, or it is given an especiallyconspicuousplacein the

1o7

108

HANDBOOK

OF VISUAL ANALYSIS

composition,or madeextraconspicuousbylighting, contrastsin tone or colour, etc. Someonein the picture points at the symbolicmotifwith an expressivegesture. The symbolic motif seemsout of placein the composition. The presenceof the symbolic motif somehowcontravenesthe laws of nature. Here is an example of the third argument: It seemsstrangethat fan van Eyck aswell asGhirlandaio should be so anxiousto stress St |erome'sliking for apples.In fact,the fruit looks out of placein the saint'sstudy and this suggeststhat a symbolic interpretation is called for, or at least likely. (Ingvar Bergstrcim,quoted in Hermeren, 1969:84) And here is an example of the fourth argument,Ian van Eyck's Madonna in a Church, as interpreted by Panofsky: It seemsto haveescapednotice however,that in this painting by a masterso renowned for his naturalism . . . the sun shinesfrom the North. There is in all Christendom no Gothic church having a full-fledgedcathedralchoir with radiating chapelsthat would facethe West and not the East.And if it is hazardousto accusethe most observantof painters- and also one of the most erudite - of a mistakein scale,it would be almost sacrilegeto accusehim of a mistakeasto the simplestlaw of natureand the most familiar of ecclesiastical customs.If he decidedto reversethe laws of nature, he must havehad a reasonfor doing so. And this reasonis, simply, that the light he depicted was not 'super-essential' light intendedby him to be the light of nature but the supernaturalor which illumes the City of God, the Light Divine disguisedasthe light of day.With Ian van Eyck this light, though independentof the laws of astronomy,was subjectto the lawsof symbolism.(1953:147) Just to make sure, Ian van Eyck also included an inscription (cf. Panofsky, 1953: 148):'The Virgin Mury. . . is more beautiful than the sun and above the whole order of the stars. Being compared with the [natural] light, she is found before it. She is the brightness of eternal light and the flawless mirror of God's majesty.' Contextual arguments for symbolic interpretations are of three kinds: 1 The symbolicmotif occursoften in an artist'soeuvre(or in a certainkind of art) without any obvious historical or natural explanation. 'an 2 The symbolic motif is, to useBarthes'swords, acceptedinducer of ideas',a commonly usedsymbol (in a given period and/or kind of art). 3 There is documentaryevidencethat the artist intended the motif as a symbol, or, more generally,that he or shewas interestedin symbolictraditions. Iconographers are aware of the fact that symbolic meanings maybe intended to be understood only by a restricted audience of cognoscenti, or even be private. In 'symbolize', 'allude' 'suggest' such cases and may be more appropriate terms than 'The as in this quote from Edgar Wind: allusions [to the four elements in Raphael's 'stanzadella Segnatura' are extremely remote and reveal the playfulness of a humanist mind which rejoices in making itself understood only to a select and erudite circle' (1938:76).

SEMIOTICS AND ICONOGRAPHY

A key distinction in the work of Panofsky is the distinction between open symbolismand disguisedsymbolism.This distinction comes about with the rise of pictorial naturalismin the Renaissance. A motif is an open symbolof somethingwhen it is not representednaturalisticallyorwhenthereis no naturalisticexcusefor its presence in the image.It is a disguisedsymbolwhen it is representednaturalisticallyand when there can be both a naturalistic and a symbolic explanation for its presencein the 'objects image. In medieval painting, Panofsky explains, accepted and plainly recognisable assymbolscould mingle with realbuildings,plantsor implementson the samelevel of reality- or, rather,unreality'(1953:140).When painterswanted to representthe prophetsof the Old Testamentaswitnessesto the Crucifixion, they simply placedthem beneaththe Crossand identifiedthem with suitableattributesand scrolls. Laterpaintersbeganto find suchanachronismsunrealisticand disguisedthe prophets 'Madonna as statues.Again, in Grivelli's and Child enthroned with donor' (Figure 5.4),the Christ Child holds up a rather largeapplein a gesturewhich is clearlymeant to displaythe appleto the viewer,rather than to representthe preliminariesto eating it, and fruit alsodecoratesthe throne on which theysit. The more inconspicuousapple 'St in Ian van Eyck's Jeromein his study' (Figure 5.5) could either be interpreted 'original naturalistically(perhapsSt|erome liked apples)or asa symbol of sin' (cf. the story of Adam and Eve),aswas done, for instance,by Ingvar Bergstrom: This summaryof the importanceof the theologicalideaof medicinagivesus a clue jar of 'Tyriaca'represented in fanvanEyck's to theunderstanding of theapothecary's particularlywhenweconsiderit togetherwith theappleplacedon top of it. Stf erome, It hasdoubtless theremedyagainst to beinterpretedasa disguised symbolexpressing originalsin,acquiredsin,disease and death,whichis Christ.(Quotedin Hermeren, 1969:91) 'disguised In the caseof symbols',symbolism can be more easilydenied. Decidingwhich motifs shouldbeinterpretedsymbolicallybecomes more problematic and contestable,and the argumentswill haveto be mostly contextual,ascan be seen in the following quote: justwhichoftheobjects otherthan it isnot easytodetermine [In theMdrodealtarpiece] thepot of lilies- andof coursethe piousbookson theVirgin'stable- carrya determinablemeaning.Several of themrecurin ananalogous contextof otherworks,both by theMasterhimselfandby others,andcanthusbeshownto conformto anestablished mentionedasanindoorssubstitute tradition.Thelaverandthebasinhavealreadybeen for the'fountainof gardens' and'wellof livingwaters',oneof themostfrequentsymbols of theVirgin'spurity.Thelionson thearmrests of herbenchbringto mindtheThrone twelve of Solomondescribed in I KingsX, 18ff.,with itstwolions'besidethestays'and 'on theonesideandon theotheruponthesixsteps'.. . . Otherfeatures, such however, asthe fireplacewith its screenandthe two wall brackets. . . do not so readilylend (Panofsky, 1953:143) themselves to a symbolical interpretation. Contemponry artmay disguisesymbolism in another sense.When artistsdraw on unconsciousinspiration rather than on consciouslyknown symbolic traditions, give symbolismwill be repressedon a consciouslevel.When criticsthen nevertheless

109

110

Figure 5.4

HANDBOOK

OF VISUAL ANATYSIS

Carlo Grivelli: 'Madonna and Child enthroned with dono',

a symbolicinterpretationof suchworks, the artistswill often contestit. In the ageof post-modernisma conscioususeof symbolsand intertextualreferencesseemsto have becomemore acceptableagain. I cannot do justice to the wide range of racist iconographicalsymbols discussedin NederveenPieterse.I have aheadytouched on the iconographicalsignif'projected 'thick lower part of the lips' and the icanceof racist stereotypessuch as

SEMIOTICS AND ICONOGRAPHY

Figure 5.5

Jan van Eyck: 'St Jerome in his study'.

face' and on the figure of the'dancing girl' and her'kinky'

hairsryle (Kornelia Kinks

was a familiar figure in pre-war American advertising and had a'peaked'hair

style of

this kind). I will now give one further example: fruit. In discussing the iconographical significance of fruit in white representations of black people, Nederveen Pieterse uses many of the arguments we have just discussed, for instance references to literature such as this quote from Anthony Trollope's account of the West Indies in 1858:

[The West Indian Negro] is idle, unambitious asto worldly position, sensualand content with little. He liesunder the mango-tree,and eatsthe lusciousfruit in the sun.He sends his black urchin up for breaKastand behold the family tableis spread.He piercesa cocoanut and lo! there is the beverage.He lies on the grasssurroundedby oranges,bananas and pine-apples.(Quoted in NederveenPieterse,1992:I99) Or the argument that fruit was commonly used as symbolic of the ('irresponsibility', etc.) of black people:

'laziness'

Fruit was the classicsymbol of plenty, commonly usedto denote the natural fertility of the tropics, and hencethe'natural laziness'of blacks.For the West Indies, pumpkins and melons were the common signifiersof tropical abundance.In American folklore blacksarebesetby an uncontrollabledesirefor water-melons.It is one of the attributes ofthe child/savageimage.The water-melon suggestssloth, gluttony, lack ofself-control, childlike needs;additionally it may carry sexualovertones.Bananasand coconuts have alsobeenassociatedwithblacks.Again the connotation is tropical abundance;in addition, the bananais a classicphallic symbol. (Ibid.)

1 1 1

112

HANDBOOK

Figure 5.6

OF VISUAL ANALYSIS

'ln the good old summer time', American postcard, 19O7.

# Figure 5.7

American advertisement for grapefruit, 1930s.

SEMIOTICS AND ICONOGRAPHY

Wu# v..(@'E

=

rA= ::V,i

.*** **#

*ffiM ^"""*"* Figure5.8 Advertisement for a Dutchbrandof bananas featuring therunnerNellieCooman, 1 987. In Figures5.6-5.8,the deliberate'posedness'of the peopleand the foregroundingof the fruit could be a further argumentin favour of a symbolic interpretation. Although I haveanalysedsingleimages,it should be clearthat iconographical analysiscanbe used,not only for revealingthe history (and persistence)of suchracist symbols,but alsofor quantitativestudies,so that the approachdescribedhere could be usefullycombinedwith that of Chapter2.

1 1 3

114

Figure 5.9

HANDBOOK

OF VISUAL ANATYSIS

Titian: 'Allegory of Prudence' (1569).

With the exceptionof the lighting in fan van Eyck's'Madonna in a church', 'object-signs'.But aspects most of my exampleshavebeenof what Bartheswould call of the way in which people,placesand things are representedand compositions ('combinationsof artistic motifs') can alsobe interpretedasiconographicalsymbols. 'Meaning As Schapirosays: and artisticform arenot easilyseparatedinrepresentations; some forms that appearto be conventionsof a local or period style are not only aestheticchoicesbut alsoattributesof the representedobjects'(1970:37). He argues this in the contextofinterpreting the useof frontal and profile in medievalmanuscript illustrations representingthe story of Mosesat the battle with the Amalekites.In earlier picturesMoseswas representedfrontally (aswas usuallydone with Christ), with his

S E M I O T I C SA N D I C O N O G R A P H Y

arms extended like the arms of Christ on the Cross. Here he'pre-figured'

Christ and

all that Christ stands for. Later'he is turned in profile, and the reference to the cross is therebyblocked or weakened. Moses becomes apart of the action,like the fighting soldiers who are also shown in profile' (1970:40). The earlier rendering of Moses as if turned towards us and with arms outstretched appearsall the more clearlythen asa speciallyaccentedform suited to the reading of the episodeas a symbol . . . ; while the later profile pictures an action of which the significanceis given in the simple denotation of the words of the Bible and callsfor no deeper understandingasa symbol. (Schapiro,1970:40) In a footnote Schapiro extends this to modern art: In the painting of the later 19th and early20th century,the starkly frontal faceand the pairing of frontal and profile returned aselementsof the strong expressionisttrend, both in portraits and narrative themes.The frontal position in subjectsof sorrow, death, jealousy, anxiety, panic and despair by Munch and Ensor is a means in portraying the personin distress,self-isolatingand turned awayfrom others- he cannot 'face'the world; it is alsoa meansof engagingthe viewer'sattention to the subject'sfaceasthat of another and kindred self preoccupied with its own overpowering feelings and speakingout to the viewer. ( 1970:61)

lconological symbolism In moving from iconographicalto iconologicalsymbolismwe move from identifring acceptedconventionalmeaningsto interpretation. One such kind of interpretation interprets works on the basisof the biography of the author, as 'autobiographical' on a deep,not immediatelymanifestlevel.The following examplefrom Panofskyclearly showsthis move from identifring conventionalsymbolsto autobiographicalinterpretation.The work is Titian's 'Allegoryof prudence'(Figure5.9).It usesconventional symbolswhich werewell known to Titian's contemporaries.On the levelof 'representational meaning'we see,on the left, the profile of an old man facingleft, in the centre a frontal portrait of a middle-agedman and on the right a profile of a younger man, facing right. The iconographicalmeaningof this is asfollows: of humanlife (youth,maturityandold age),Iand] [Thethreefaces]typifr threestages symbolise thethreemodesor formsoftime in general; past,presentandfuture.Weare furtheraskedto connectthesethreemodesor formsoftime with theideaof prudence, or, morespecifically, with thethreepsychological faculties in thecombinedexercise of which this virtue consists:memory,which remembers, and learnsfrom, the past, intelligence whichjudgesof, andactsin, the present,andforesightwhichanticipates, andprovidesfor, or against, thefuture.(Panofsky, 1970:184) But Panofskygoesfurther. He notices the resemblanceof the old man to a self-portraitof Titian, and discoversthat Titian, at the time he producedthe painting, wasengagedin collectingmoneysowedto him, and managedto persuadethe Venetian authoritiesto awardhis annual stipendand tax exemptionto his son Orazio.He then searchesfor, and finds evidencethat the man in the centreindeedrepresentsOrazio,

1 1 5

116

HANDBOOK

OF VISUAL ANALYSIS

and the youngerman on the right Titian's adoptedgrandsonMarcio Venellui,born in 1545and henceZ4yearcoldat the time. The painting is autobiographical.It represents an episodefrom Titian's life, an episodein which Titian, an old man, takesmeasures to provide for the future of his son and grandson. 'somethingmore As Panofskycomments,iconologicalinterpretationrequires than a familiarity with specificthemesor conceptsas transmitted through literary 'a sources'.It requires mental faculty comparableto that of the diagnostician- a faculty which I cannot describebetter than by the rather discreditedterm "slmthetic intuition", and which maybe betterdevelopedin a talentedlaymanthan in an erudite scholar' (1970:64). But iconologicalinterpretation is also necessarilybasedon a certainprinciple of integrativeinterpretation,whetherit is autobiographical,psychoanalytical,theological,sociologicalor otherwise.Here is a differentexample,this time 'autobiographical'but 'philosophical': not Iean Seznec'sdiscussionof a miniature in a Copenhagenmanuscript. Theybetraythe Thesesmallimageshavemuchto tell us on attentiveexamination. convictionwhichhadbegunto assertitselfin the fourteenthcenturythat man is the to thisis someprisonerof theheavenly bodies,entirelyat theirmercy.But resistance aswell.In onecasewehavethefigureof a happychild,unconcernedly timesexpressed whichgravitatearound pluckingflowersandpayingno attentionto the ninespheres In their ownway,thesenaiveimagesraisethe with their dreadsymbols. him charged 127) freedomof will. (Quotedin Hermeren,1969: wholeproblemof necessityversus But, howevermuch suchinterpretationsreston'synthetic intuition', the argumentsusedto justifr them aresimilar to thoseusedin iconographicalanalysis.Again, 'Allegory of they may either be contextual,asin the caseof Panofsky'sanalysisof the prudence'(intertextualcomparisonofportraits, backgroundresearchinto documents pertaining to Titian's life and affairs,etc.),or textual,asin the examplebelow,which draws an analogy between the structure of a work and the structure of a particular religious/philosophicalinterpretation of the subject-matter: heavenandearthareno longerseparated Tomboflulius,however, In IMichelangelo's] from eachother.Thefour giganticfigureson theplatform,placedastheyarebetween andVictoriesandthecrowninggroupof thetwo angels thelowerzonewith theSlaves betweenthe terrestrialand carryingthe barawiththe Pope,serveasan intermediary not asasudden of thePopeappears, Thanksto them,theapotheosis thecelestial spheres. but asagradualandalmostnaturalrise;in otherwords, transformation, andmiraculous of theOrthodoxChristiandogma,but asanascension in thesense not asaresurrection, I45) (Panofsky, quotedin Hermeren,1969: philosophy. in thesense of theNeoplatonic Iconological analysis,then, draws together the iconographicalsymbols and srylisticfeaturesof an imageor a representationaltradition into a coherentinterpretation 'why' analysed.In the caseof Nederyeen behind the representations which providesthe 'why' is clearlysociologicaland political: Pietersethis The questionthat keepsarisingis,whatinterestsof whitesarebeingservedby these economicandpoliticalinterests Thisrefersnot only to measurable representations?

SEMIOTICS AND ICONOGRAPHY

but alsoto relationsof a subtlernaturein cultural,emotionalandpsychological spheres, andto the variouswaysin whichtheserelationsfigurein the phenomenonof subordination.Thekeythatunlockstheseimagesis whatwhiteshavemadeof blacksand why.(1992:10)

CONCLUSION In conclusion,then, Barthianvisualsemioticsand icono graphyareparticularlyuseful for investigatingthe representational('denotative')and symbolic ('connotative') meaningsof the people,placesand things (including abstract'things') included in different kinds of images.Both methodsprovide explicit argumentsfor determining whether representedelementssuch asposesand objects,and elementsof stylesuch as angle,focus and lighting, can be interpreted as symbolic, and for distinguishing between conventionally acceptedforms of symbolism and broader 'iconological' interpretations.The main differencesbetweenthe two methodslie first of all in their objectsof study- art works of the pastversusmedia imagesof the present.Hopefully the useof NederveenPieterse'shistory ofwhite peoples'representationsofblack people hasdemonstratedthat iconographycanusefullybe appliedto the analysisof contemporary imagesaswell. Another differencebetweenthe two methods is that Barthian visual semioticsremainsrestrictedto textual arguments,at leastinsofar asits explicit conceptualapparatusis concerned,whereasiconographyalsousesargumentsbased on intertextualcomparisonand archivalbackgroundresearch. Both methods recognizethat symbolism may be 'open', mingling 'objects acceptedand plainly recognisedass)'mbolswith realbuildings,plantsor implements on the samelevel of reality - or rather unreality',as Panofskysays(1953:140),or 'disguised'. In Renaissancepainting, the favoured object of study of iconography, increasingnaturalismled to an interestin presentingsymbolsin the guiseof reality. Barthes'sfavouredobject of studywasphotography.Recognizingphotographs,even documentaryphotographs,as (also) symbolic constructs,a quest in which Barthes was a pioneer,may well have playeda small part in the declineof naturalism, along with the'constructivist' power of the new digital technologies.

REFERENCES Barthes, R. (1973)Mythologie.s. London:Paladin. Barthes, R. (1977)Image,Music,Text.Lond,on: Fontana. Barthes, R. (1982)CameraLucida. London:Paladin. Berger,l.(1972)WaysofSeeing. Harmondsworth: Penguin. Bols,P.,Houppermans, M., Krijger,C.,Lentjes, T., Terlingen,M. andTeune, W., Savelkouls, P. (1986)WerkaandeWereld. Malmberg:DenBosch. Goffman,E. (1979)Gender Advertisements. London:Methuen. GroupeV Q992)Traitddu signeuisuel- Pourunerhdtorique Paris:Seuil. del'image. Hermeren,G. (1969)Representation and Meaningin the VisualArts. Lund: Scandinavian UniversityBooks. Mok, L (1999)In de ban van het ras:Aardrijkskunde tussenwetenschap en samenleving. Amsterdam: ASCAPress.

117

118

H A N D B o o Ko F v r s u A L A N A L Y s T s

Nederveen Pieterse,I. 0992) White on Black: Imagesof Africa and Blacksin WesternPopular Culture.New Haven:YaleUniversity Press. Panofsky,E. ( I 953) Early N etherlandishPainting.New York Harper and Row. Panofsky,E. ( I 970) Meaning in the VisualArts. Harmondsworth: Penguin. Schapiro,M. ( 1945)"'Muscipula Diaboli": the symbolismof the M6rode altarpiece', Art Bulletin, 27: 182-7. Schapiro,M. ( 1970) Wordsand Pictures.The Hague:Mouton. 'Visual van Leeuwen,T. (in press) racism',in R. Wodak and M. Reisigl(eds), The Semioticsof Racism.Vienna: Passagen Verlag. 'Studies Wind, E. (1937) in allegorical portraiture', Iournal of the Warburg and Courtauld Institutes,l: 138-62. 'Elements Wind, E. ( 1938) in Raphael's"stanza della Segnatura"',]ournal of the Warburgand CourtauldInstitutes,3: 7 5-9. Wollen, P. (1972) Signsand Meaningin the Cinema.London: SeckerandWarburg.

ilx ffiw ruffi

A Therapeutic Perspective: the Useof Drawingsin ChildPsychoanalysis and SocialScience

GERTRAUDDIEM.WILLE

INTRODUCTION Inthis chapterI concentrateon the communicativefunction ofdrawingsin childanalysis and their usein socialresearch.As the useof drawingsis describedin a psychoanalytical framework,I will briefly outline the psychoanalyticalapproach,consideringpictorial imagesand drawingsasexpressionsof the unconsciousemotionalaspectsofaperson. Freud'smodel of the mind will be presentedin a simplifiedway, concentratingon the topographicalframe of reference.In the secondpart of the chaptermy approachwill be primarily clinical:I will bring sequences of analyticsessions to describethe situationwhen the drawing wasdone and how it wasinterpreted.In child psychoanalysis all actions of the children, including their play and their drawings,are understood as ways of expressingtheir inner world and their relationshipto the analyst.The relationshipto the analystis mainly influencedby the transferenceneuroses.In the third part of the chapterI demonstratehow a projectivedrawing testwasadjustedfor usein a qualitative socialresearchprojectto understandunderlyingpreconsciousand unconsciousmotives.

P S Y C H O A N A L Y T I C U N D E R S T A N D I N GO F V I S U A L I M A G E S Psychoanalytictheory is not coherentand unified within a singleframework. It is a theory of normal and pathologicalmental functioning but also a theory of therapy. Freud calledhis overall theory of the mind his 'metapsychology'but nowadayswe speakof a 'psychoanalyticpsychology'(Sandleret al., L997).Although Freud described three models of the mind - the dynamic, the economic and the topographicalor structural- I will concentrateon the topographicmodel, becauseit is most relevant for understandingthe importance of drawings.This frame of referencewas usedby Freudin 1900in the Interpretationof Dreams.Fretd'slanguageis characterizedbythe rich and vivid useof metaphors,while clinical examplesfacilitatethe understanding of complicatedmatters. Pictures,drawings and metaphors show a person'semotional stateof mind much better than verbaldefinitions or descriptions.In psychoanalysis we understand

120

HANDBOOK OF VISUAL ANALYSIS

visual images,daydreams,dreams,works of art, symptoms and charactertraits as compromiseformations. Theseproducts are regardedas compromisesbetweenthe instinctualwish and all forcesthat opposeinstinctual gratification.As a compromise theyrepresenta disguisedform ofgratification,but at the sametime reflectthe defence that hadbeenmobilizedagainstit. To understandFreud'sconceptof an inner conflict 'topography' of the mind, with emphasis we want to describehis attempt to outline a on the psychologyof interrelationshipsand interactions, of qualitatively different areasof the apparatus.The three systemsthat are involved are calledthe conscious, preconsciousand unconscious.The principal referencepoint to the topographyis the psychologicalquality of the conscious,the preconsciousand unconsciouswhich are considereddeeperlayers.Boundariesfunctioning as censorshipare thought to exist betweenthesesystems.Only during periodsof conflictsare theseboundariessharply defined and separateentities;at periods of relative mental harmony thesedividing linesmust be regardedasbeing blurred. Freudgivesus a metaphoricaldescriptionon the relationshipbetweenthe three systems: hall,in whichthe to a largeentrance Letus. . . comparethesystemof theunconscious jostleoneanotherlike separate Adjoiningthis entrance individuals. mentalimpulses - in whichconsciousness hallthereisasecond, narrower,room- akind of drawing-room thesetworoomsawatchmanperforms Buton thethresholdbetween . . . Ialso]resides. actsasa censor,andwill not the differentmentalimpulses, hisfunction:heexamines him. . . . It doesnot makemuch permittheminto thedrawing-roomif theydisplease differenceif the watchmanturns awaya particularimpulseat the thresholditself ... or if hepushesit backacrossthe thresholdafterit hasenteredthe drawing-room. If theyhavealreadypushedtheirwayforwardto thethresholdandhavebeenturned we speakof backby the watchman,then they are inadmissibleto consciousness; themasrepressed. But eventhe impulseswhichthe watchmanhasallowedto cross consciousaswell; they canonly the thresholdarenot on that accountnecessarily We aretherefore in catchingthe eyeof consciousness. becomeso if they succeed (Freud,1916-17 justifiedin callingthissecondroomthesystemof thepreconscious. 2es-6) Freud'sshift from the affect-traumamodel to the topographicalmodel of the mind wasinfluencedby a shift of focus on psychologicalthinking from the influence of the externalreality to the way in which the mind dealswith spontaneousinner urges and demands.The individual wasseento be,to a largedegree,at the merry of instinctual impulses(the sexualand deathinstinct). Theseimpulses,arisingfrom the depth ofthe mental apparatuscould not alwaysbe alloweddirect expressionafter earlychildhood but, to a largeextent,could only showthemselvesindirectly in surfacemanifestations, from the systemunconsciousto the preconscious. The unconscioussystemis thought of as functioning accordingto certain specificlaws.Its contents,aslong asthey remain in this system,alwayshavethe quality of the unconscious.The term preconsciouswas introduced to refer to contentsthat are,descriptivelyspeaking,unconscious,but were capableof becoming consciousif attention wasdirectedtowardsthem. Not all contentsof the preconscioussystemare freelyaccessible to consciousness; somemay only be permitted to passin a disguised form. Equally,they may be repressedinto the unconscious.

D R A W I N G SI N C H I L D P S Y C H O A N A L Y S I S

In the unconsciousthe'characteristic' modesof functioningincludethe primary process,which is structuredon differentprinciplesthan logicalthinking. The working of the preconsciousand consciousmake useof the formal modes of thinking, what we callthe secondaryprocess.The study of the analysisof dreamsled Freudto recognize a type of mental functioning that was very different from thought-processes.The primary process,the mental functioning of the unconscious,flows freely and passes unhindered to expressunconsciouswishes(primitive hallucinations).The primary processdoesnot recognizedifferent times, suchaspast,presentor future. It alsohas no meansof expressingcontradiction and negation,which meanscontradictory elementsarequite compatibleand existsideby side.Instinctualwishesareconsidered by Freudasfollowing the pleasureprinciple and the pressuretowardsthe satisfaction of infantile instinctual wishes,however unreasonablein the present,still persist. Memories of real eventsand of imagined experiencesare not distinguishedbut are treated as if they presentconcretereality. Symbolsare treated as if they were a representationof a concrete thing. Instead of clarity the dream seeksto establish meaningin severalchainsof association,bythe psychicmechanismof condensation, ofwhat Freudcalledthe defencemechanismof condensation.The secondmechanism which is in operation is displacement,where an often apparentlyinsignificantidea is attributed with all the psychicalvalue, depth of meaning and intensity of another disguisedone.The primaryprocessfollowsthe pleasureprinciple whereasthe secondary processrepresentsthe reality principle. The secondaryprocessis the mode of functioning ofwakefrrlthinking, attention, judgement,reasoningand controlledaction.Its characteristics arethe notion of causaliry logic, a senseof time and an intoleranceof ambiguity and contradictory elements. Thereis an aim at thought identity. 'Thinking must concernitselfwith the connecting pathsbetweenideas,without being led astrayby the intensity of thoseideas'(Freud, 1900:602).He continues:'In the dream,picturesare usedin considerationof representability',which can also expressmetaphors or sayings.Freud (1900) gavemany examplesof how dream-thoughtsare transformedinto pictures.The content of the dream can expresslatent dream-thoughtswhich would not passthe censorshipof a consciousstateof mind. This mental operationwhich transformsthe raw materialof the dream (unconsciouswishes)wascalled'dream-work'. Schleiermacherdescribed 'The the characteristicof being awakeasthinking with concepts. dream,' said Freud 'thinks ( 1900:602), mainly in visual images.' 'imaginative' 'creative' products, In or such as works of art, storytelling or drawing, instinctual wish-fulfilment may be permitted, whereasthey would not otherwisebe allowedaccessto consciousness.

I N T E R P R E T A T I O N SO F D R A W I N G S I N C H I L D PSYCHOANALYSIS In child analysisthe play of children, the way they usematerialsand their freedom is understood as communication about their state of mind, their feelings in their relationshipwith the analyst.Drawing is a way of using the materialthat is provided.

121

122

HANDBOOK

OF VISUAL ANALYSIS

The aim of analysisis to understandtheseinner conflicts, anxietiesand wishes. Whether a child makesdrawings or not, when she or he does it, and in what way, differsgreatlyfrom child to child. The specialway that a child usesthe materialshows his relationshipto the analyst.I follow here the conceptof Melanie Klein (L975:7), who wrote: 'The child expresses its phantasies,its wishesand its actualexperiencesin a symbolicway through play and games.'It usesan archaic,primary process,which we know from dreams.We can understandthis languageonly if we understandit in the sameway aswe interpret dreams.In dreamswe do not isolatesinglesymbols.The sameis true for the interpretationof drawings,wherewe alwayskeepin mind the context, the situationin which theywereproduced.One singlefigurecanhavedifferentmeanings: for example,a doll can representthe child itself, or the baby the child wants to steal from its mother or the penis of its father, dependingon the context.It is sometimes surprisinghow clearly a child respondsto an interpretationby showing relief or increasedanxiety.Klein thinks that the'communication betweenthe unconsciousand the consciousis asyet comparativelyeasy,so that the way back to the unconsciousis much simplerto find' (1975:8). In the sameway aswe do in therapywith adults,we assumethat the relationship to the analystis mainly influenced by transferring emotional patterns which were establishedin earlychildhood. Betty Joseph( 1985)talks about the 'total transference situation',which enablesthe analystto get fresh impressionsof the quality of the relationship,the distortionsand fragmentations.Both children and adultstryto draw us into their familiar patterns of behaviour which they have experiencedin earlier relationships.What happensin the sessionis regardedasthe product of the interaction betweenan immediate reality and the patient'sview of the reality, which is derived from their unconsciousfantasiesrooted in her or his history.As Feldmanand Spillius 'The commented: patient'shistory, in other words, is in his inner world. It is through observingand experiencingthe pressureto live out aspectsofthis unconsciousinternal world in the immediatetransferencerelationshipthat the analystcomesto know his patient and his history' (1991:6). It is an important ability of an analystto feeland to recognizethe pressureand what is projectedon to her or him. The analystobserves how the patient reactsto an interpretation.Doeshe or sheignore it or show relief or anxiety,feelcriticizedor turn awayfrom the topic altogetherto avoidpainful feelings? As fosephfound, children expresstheir feelingsstraightforwardly: Thechildpatientusuallysoonlearnsandin somedeepwayappreciates that this is a uniquekind of relationship, andthattheanalystcanbereliedon not to bedrawninto reassuring, reprimanding, educating or playing,but will truly try to understand andhelp him facehisanxieties andpainandthedefensive manoeuwes thathehasdeveloped to try to copewith them.(1998:2) A three-year-oldboy who found a comment I had made too painful took a woodenbuilding-block and hit me on the headto getthethoughtout - he didn't want to confront himself with his feelings.Or when he was destructivebecausehe felt the separationwas too painful, and I put it into words, he was relievedand stopped.To understandand interpret thesereactionsis very helpful for the child who sometimes

D R A W I N G SI N C H I L D P S Y C H O A N A L Y S I S

feelsunderstoodon a deeplevelfor the first time in his life. The equipmentwe provide in child psychoanalysisis assimple asthe playroom itself.To enablethe child to express his or her fantasyand wisheswe use simple materialssuch as pencils,paper, glue, sellotape,crayons,scissors, woodenfigures,woodenanimals,and soon. The materials arekept in a lockedbox or drawer in order to enablethe child to keeptheir products untouchedby othersand confidential.In this way the child cankeepthe whole history of her or his therapy in a drawer or box, and can refer to the drawingsand products createdearlier. Below I offer two examplesof the way drawings are dealt with in an analytic session.A detailed description of two sessionsand the context of the origin of the drawingsis neededin order to illustrate the dynamicsof a therapeuticsession,how impressions,wishesand storiesare linked in a chain of associations.We seethat the drawingsare treatednot only asproducts but that the context is emphasized.In the interpretationswe useelementsof the drawingsand the way in which the drawing is presented.We do not isolatesinglevisualelementslike the words of a secretlanguage to be decipheredlike dream symbols;rather we concentrateon the context and the individuality of the patient. Certainlythere areprinciplesof representation:the most important person is drawn first - a way to expressconflicts is to omit this personand colour is usedto expressfeelings(Abraham, 1950:237).Theseprinciples help the 'hidden' analystto be alert to meanings(seeChapter 7).

Rupertdrawswhales First somebackgroundinformation. Rupert is a musicallygifted eleven-year-oldboy at school who hasproblems at schoolalthoughhe is very intelligent.He is aggressive and cannot adjustto the rules.He thinks he hearsvoicesgiving him orders,not allowing him to practisehis violin or do his homework.His mother had beenliving for sixyears with Rupert'sfather,though he wasmarried to anotherwoman. He promised to marry Rupert'smother and urgedher to getpregnant.He got divorcedduring the pregnancy but married anotherwoman out of the blue. He seemedto havepsychoticepisodes. Rupert'smother tried to commit suicidewhen Rupert was six months old after she found out that Rupert's father had married somebody else.Rupert destroyedher relationship,shecomplains,but now he is the only important thing in her life. Shehas a boyfriend who is not interestedin an intimate relationshipwith her. sessionhe is broughtbyhis mother. When Rupertcomesto the first assessment They aretwenty minutes late. His mother urgeshim to apologize,which he reluctantly does,pouring out a long story. But assoon asthe door to the waiting room is closed he is very shy: he standsin the middle of the room helplessly.He listensattentively when I tell him the basicrule, that he can expresswhat comesinto his mind in any way he likes and that I would try to seewhether I can help him. He goesto the chest of drawersand looksinto his drawer,then triesto openthe others.He criticizeshimself for how curious he is and walks over to the window. I tell him that his actionsshow me how interestedhe is in the other children'sdrawingsand what problemsthey have and how critical he is ofhimself. He looks round and takesa closelook at the scribbles

123

124

HANDBooK o F v t s u A LA N A L y s t s

on the wall. I saythat he might wonder what other children are doing here and what kind of problemsthey might have.He is relievedand says:'That's good that they also haveproblems.I alwaysthought I wasthe only one.' He addsthat he knows that he is not supposedto write on the wall and askswhether he could usethe paperand pencil on the table.I interpret this questionasreflectinghis disbeliefand doubt asto whether he canreallytrust my words.He becomesvery sad,and admitsthat promiseswerevery seldomreliable. As shown in Figure 6.1, Rupert ,, I '

,

iI(.Lr.aI.

:

the first whale in the middle; he explains while he drawsitwith a quickand steady hand that it represents himself.Then he draws the huge whale with the big mouth in the bottom left corner,twice

with the big teeth.Laterhe addsthe two smallerwhalesabove.When I ask him whetherhecouldmakesomecomments I on his drawing he startsto cringe. He strugglesto regain his composureand Figure 6.1 Rupert's drawing. repeatsmy question,askingwhether I meant he should tell me which whale representswhich member of his family. He repeatsthat the first one representshimself: 'I feellike a whale.Whaleshaveno words but communicatewith songsthey sing.Theyhavea distinct and articulatedcommunication.' The others he doesn'twant to explain so he asksif he is supposedto do it. I understandthat this would be too painful. He nods and looks at me gratefully.He also wantsto find out whetherI am asintrusiveashis mother, forcing him to tell me things he is not yet readyto, or if I am giving him freedom to open up asmuch ashe wants to. He givesme a little lectureon whales,the different kinds that existand the special waysof living theyhave.Then he suddenlycomparesme to a dolphin, with a dolphin that can help and savelives. Suddenlyhe tells me a story from Roman mythology about a man who helped a lion by taking out a thorn from its paw. Later,when the sameman hasto fight againstthe lions in the Roman Circus this lion jumps at him and pulls him down, only to lick his face in gratitude. I link this story with his comparing me with a helpful dolphin. If I can be helpful for him he would spareme at the Circuslater.But we areboth subjectto the cruel power of he who decidesabout life and death.At the end he hasdifficulties in leaving.He tries to extendthe session by telling me about other drawingsof whaleshe had done.At the sametime he is very scepticalabout whether anybodycan help him at all. In summary,first Rupert showsme his anxietyabout coming by arriving late. As soon as the door to the waiting-room is closedhe is helplessand anxious.His impulsenot only to look into his drawerbut alsointo other children'sdrawersbreaks through and makeshim feel ashamed.He jumps back and criticizeshimself.We can understandhis ideaof drawing asa compromise.It is not emotionallyasdangerousas his actionswhere an impulsewasactedout; he can eraselines,changethem, cantake i,

I

DRAWINGS IN CHILD PSYCHOANALYSIS

his time and think in between.Again when he is invited to saysomethingabout his drawing he retreats.He alsowantsto find out how reliableI am. His father promised severaltimes to visit him during the summer but he hasn't seenhim sincehe wasfive yearsold. His mother convincedhim not to postthe lettershe had written to his father over the yearsin order not to be disappointedagain.How deeplyhurt he is shows in his painful reaction,his sadnesswhen I addresshis mistrust and doubts about me being reliable.His deep aggressionis shown in the story about the man and the lion. Rupert identifieshimself with the dangerouslion who would spareme. But both of us are prisonersof the Roman Emperor, representinghis paranoid,hallucinatory self. In the drawing we seehow he experiencesthe close,threateningrelationship with his intrusive mother. To expressthis intenseconflict in words would havebeen very difficult at this earlystage.In the picture he canexpresstheseambivalentfeelings. In the analyticprocessit is important not to insiston a clarificationofwho represents whom but to follow the free associationsof the patient. It alsoshowsthe patient that he hasthe freedomto expresshimself the way he wantsto so that we can understand his way of dealingwith inner and outer problems.

Daphne's by drawing wayof communicating First someinformation about the case.The mother of a five-year-oldgirl, whom I call Daphne,camefor a consultationaboutwhich schoolsheshould sendher daughterto. It wasclearthat thereweremore seriousunderlying problemswhich shefound difficult to acknowledge.ShedescribedDaphne asshy at schoolbut having temper tantrums at home and beingveryjealousof her four-year-oldsister.Only afterDaphne'sanxiety about'going mad' was expressedduring the sessiondid her mother speakabout 'fits', Daphne's when shewas not herself,and her hallucinatory fever fantasieswhen she could not recognizeher mother. This way of presentingDaphne'sproblems by hiding them for aslong aspossibleshowsus the intenseanxietyin the family and the wish to concealher seriousproblems. During the sessionsthe main way for Daphne to expressherselfis by making drawings.At the first sessionsheis very anxiousand canonly comeinto the consulting room holding her mother's hand. Shequickly developsa ritual: coming in with her mother, sitting down on a chair by the table and doing drawingsmost of the time. In the first sessionshearrangesthe pencil,the ruler and the other materialsin a straight line. If sheusesa crayonshecarefullytakesone out, usesit and then puts it backmaking sureit is in a line. Only then canshetakeout anothercolour. This compulsivebehaviour showedher unconsciouswish to control her inner chaosand explosivefeelingsby putting the outer world in order. The three drawingsI discussherewere madein the third session.As her father wasvery scepticalabout therapeutichelp, during the secondmeetingwe agreedwith the parentsto bring her four more times. This would give the parents a chanceto observeat home how shereactedto the sessions.At the end of the secondsessionit remaineduncertainwhetherwe would meet again.In the third sessionsheseemedto

125

126

HANDBOOK

OF VISUAL ANATYSIS

be relievedto be back and waslessanxious.Shecameinto the playing room with her mother and satdown to pick up a blue felt-tip pen. Daphne's first drawing is shown in Figure 6.2. First shemakesa horizontal line coveringthe whole sheet of paper. In a tiring way she drawscirclesin the lower part under the horizontal line. After a quarter of an hour I commenthow important it is for her to havemyfull attention, which sheacceptswith a faint smile.When shecovers the circleswith thick lines I interprettheblue circlesasher intense and dangerousfeelFigure 6.2 Daphne's first drawing. ingswhich shewantsto hide under a calm surface.Shenods almostimperceptibly.SoI say:'You showmewithout words that you think I understoodyou.'Then shedrawsthe two figureson the right, saytngthat this is a stage,while sheaddscurtains.When I askher whether she'dlike to explain to me the drawing sheshakesher headto sayno. When I suggestthat this could be herselfand me, sheshakesher head in a way that I can seesheagrees.I put this ambiguity into words by saying:'sometimesyou shakeyour head to say no although you agreewith what I just saidbecauseyou want to find out whether I can seethat you try to disguiseit in the sameway as you did in the picture.' Shereactedwith a friendly and cheeky'Mhm?',while shedrewtwo bright lights in the cornersasif to tell me that somethingimportant was lit up. While shedraws the central figure shetells me that this is a man. After shehasdrawn the fourth figure sheaddsother detailsto the middle one and comments that this is a woman. I respondby saying:'You want to show me that sometimesyou are confusedabout the Figure 6.3 Daphne's second differencebetweenmen and women.'As an answershe drawing. makesthe seconddrawing. Daphne'sseconddrawing (Figure6.3) is of a largecakewith five lit candles.I take this asa responseto my interpretation,giving me a cakeasa present.Shelooks at me for the first time and smiles. Daphnethen makesa third drawing (Figure6.4). Shedrawsa little rabbit and saysthat this one died, in a matter-of-factvoice.As there areonly a few more minutes left I connect the death of the rabbit with her painful feelingsabout the immediate ending of her session.Sherespondsby drawing a secondrabbit and tells me that shewasaskedby friendsto take careof another rabbit. I interpret it asexpressingher

D R A W I N G SI N C H I L D P S Y C H O A N A L Y S I S

hope that I should take care of her. When I remind her of the end of the sessionshe ignoresit and carefullytakesa brown pencil to draw three swallowsand a blue sky.I acknowledgehow difficult shefinds it to leave and that sheshowsme that sheknows that shewill definitelycomebackfor threemore sessions.Sheputs her things back into her drawer and leaves. In summary, although Daphne and her parentsemphasizethat shedoesn'twant to cometo her sessions shedevelopsa strong Figure 6.4 Daphne's third drawing. attachmentto me. The secondand third week were more than five weeksapart but shebehavesasif shehad beenhere yesterday.This sequenceof drawingsshowshow sheusesthem as a way to communicatewith me. Sheusesthem as a sort of private language.Her way of making drawingsarousesstrong feelingsin me. As it is almost unbearablyslow and monotonous I get tired and feel bored, a common defenceto coveraggressive and frightening feelings.Observingmy own reactionmademe more receptiveto understandingher defenceof covering her anxiety behind monotony. The most dangerousquestionsare insertedascasualremarksto seewhether I would recognizetheir seriousness. But if I pay attention sheis very pleasedand drawsa sun or light to show me that we both face her anxieties.Then she givesme a cakeas a present.In the remark about the dead rabbit sheshowsthat sheis cut off from her feelings:shecannotbe sador desperate.If I connectit with her pain about the ending of the sessionshehasa comforting idea that shecan come back and I could take care of her anxieties.Sheis delightedto be understoodand tries to extendthe session. In the drawingswishesare expressedby adramatizationin the present,asin a dreamwherea wish for the future is expressedin the presentasif it werealreadyfi,rlfilled. Daphne'sswallowsare alreadyhere eventhough sherefersto her future sessions.

fu@,

THE USE OF A PROJECTIVEDRAWING TEST IN SOCIAL SCIENCE RESEARCH My experienceof using drawingsto accessdeepemotional conflicts made me think of using this method in socialresearchin addition to narrativeinterviews.I wanted to seewhetherdrawingsin aprojectivetestwould bring additional datato understand the inner world and the emotional experiencesin early childhood. Below I discuss two examplesof the application of this psychoanalyticmethod in a social science researchproject. The studyfocusedon identifring gender-linkedpatternsin identity formation amongsuccessful contempor ary careermen and women (Diem-WilIe, 1996a, 1996b). It was basedon data gatheredfrom interviewswith a small sampleof top managers and university professors.As I was not only interestedin demographicdata such as

127

128

HANDBOOK OF VISUAL ANALYSIS

birth order and parentalinfluencebut in their psychicinner reality,I useda projective drawing test ('the enchantedfamily', developedby Kos and Biermann,1973). This projectivetestwasdevelopedfor the assessment of the qualityof attachmentof children to their parents.It was modified in order to be used with adults.The aim of the projectivedrawing testis to visualizethe dynamicsin the family of origin. In narrative interviewswe mainly hear the consciousself-perceptionof the interviewee.The unconsciousconflictsin relationshipto the parents,for instance,areoften not shown in the content of the narrativebut more in the way the persontalks:their omissions, self-interruptions,or suddenchangesof topic. In emphasizingthe inner realityit would not be enoughto know that there were conflictswith the parents;more explicit data about the quality of the relationshipwere needed. In a psychoanalyticmodel of personality we understand that alongsidethe consciousrealm,which is consideredin sociologicalsurveys,lies the realmofthe unconscious,which influencesbehaviour.Figurativelyspefing, the consciousrepresentsthe tip of the iceberg,while the unconsciousrepresentsthe much larger submergedpart of the iceberg.Feelingstowardsother peoplearenot one-dimensionalwhen seenfrom the psychoanalyticalpoint of view. This is particularly the casein closerelationships whereloveand affectionarepresent,aswell ascontradictoryemotionssuchasaggression, jealousyand hate.An important contribution, madeby Freud, is the suggestionthat everyadult also'contains'a child that, in part, controlsand influencesbehaviour.This child part is censoredby the adult part, so it mainly appearsduring dreamsand in ways that arebeyondthe control ofthe adult part, in'Freudian slips'and unintendedactions (like forgetting something). Ifwe areto understandhow successfirl careerwomen and men relateto challenge, and what determinestheir ambition, it is not enoughto observeexternalappearances or to examinetheir personalqualitiesin the light of a one-dimensionalapproach.A psychoanalyticapproachdiffersfrom sociologicalexaminationswhichfocuson ex[ernal reality, concentrateon stability and changein social systems,and stressthe interrelationshipof socialsub-systemsin which an individual's fate is seenasan element of the whole.In a psychoanalyticstudyit is important to gain access to both conscious and unconsciousaspectsof emotional developments. In changingthe emphasisof the inner reality,my intention is not to play down the importanceof theseexternalfactors.Quite the opposite.Externalconditions,such as family background and personalabilities,play a major role. My goal, however,is to showhowtheseexternalconditionscanbe influencedbytheinner reality.Thus,in a particular family, one brother canbecomea top managerwhile the other finds great difficulties in earninga living at all. Children can seethe samepeople,for exampletheir parents,verydifferently,and parentscanalsodevelopdifFerentrelationshipswith each child. How an individual perceivesouter realitycannotbejudgedfrom the outside.The drawing of the family of origin should give the intervieweea medium for expressing their ambivalentrelationshipswith eachmember of the family, their inner picturesand perceptionsof significantfamily members.I arguethat it is not the objectivequalities of the fatheror mother that are important for a child's development,but the imageof the father constructedby the child, the so-called'father-image',or inner objectof the father.Mitscherlich makesthe point:

S2 9 D R A W I N GI N S C H I L DP S Y C H O A N A L Y S I 1

Pioneeringwork by Freudled him to discoverthat not only externaleventsand themeaningof external Fantasies canchange experiences arelaiddownin thepsyche. the events,soa newrealityis created, whichin its turn hasan effecton, andchanges perception ofexternal reality.(1985:87) In the studydiscussedhere,the detailednarrativeinterviewswith the subjects provided dataon their self-imageand self-esteem,and on their life-history, professional career and personality development. The projective drawing test was introduced during the interview when the intervieweewasdescribinghis or her family (depending onwhentheymentionedtheirfamilythiscouldbeatthe middleor endofthe interview). After they had alreadyspokenabout the atmospherein the family, how the parents dealt with conflicts,their educationalsryleconcerningtable mannersand stickingto rules,and so on, I introduced the testby taking out a white sheetof paper and saying: 'You have describedyour family situation with words. Could you be so kind as to describethem in a different medium by making a drawing. Imagine that a magician came and bewitchedyour family. Could you draw what happenedthen?'After the intervieweemadethe drawing he or shewasaskedto write down'what theyhad done' and were askedto talk about their drawing. The interviewees dealt differently with this task. Some started to draw immediately,somehesitatedand wereuncertainwhetherthey could producesomething meaningful, excusingthemselvesby saying that they were not good at drawing. I assuredthem that specialtalents were not needed,that they should just put down things or symbolsasthey cameinto their mind. If their inhibition wastoo strongthey drew lines connectingcirclesto depict the relationshipsin the family. The difference in censorshipbetweenverbalexpressionand drawingwasdemonstratedby a manager who had just describedhis family asidyllic, asliving in harmony. After thinking for 'My mother is a a few minutes, he drew his mother as a barking dog, commenting: barking dog. Sheconstantlynaggedand complainedabout us.' The principlesof interpretationdiscussedin the earlierexamplesof children's drawingsin an analyticsessionapplyto the adults'drawingsin the socialsciencestudy. The context and the explanations of the person who drew the picture are more important than singleelements.Nonethelessthere are someimportant principlesto keepin mind: for example,the subjects'useof spaceon the sheetof paper,the order of appearanceof elementsand the symbolschosen.The most important figure in a family is drawn first, the sizeof the person representingtheir dominant position in the family. The position of the drawing person in the picture showshis or her selfesteem,their perception of their position in the family. The understanding of the drawing is worked out with the interviewee drawing on data from the narrative interview.Below,I givetwo examplesof the useof the drawingsto understandingthe inner world of two interviewees. It is not possibleto giveyou a detaileddescriptionof thesetwo casestudies.I want to illustrate,however,the additional dimensionsmadevisibleby the useof the projectivedrawing test. It givesa glimpse of the hidden complexity of unconscious conflicts and ambivalent relationships.Certainly we cannot saywe understandthe complexity of eachfamily dynamic but we get an idea of what goeson behind the censoredimage of the verbal description.We seethe outline of hidden areasof the

130

HANDBOOK OF VISUAL ANALYSIS

personalityand can draw some conclusionsabout the unconsciousmotives and dispositionsfor a professionalcareer.

Hermann's drawing Hermann is the chief executiveof an international computer company.He got a job thereafterreceivinghis degreeat university 2Iyearsago.Hermann gavethe following information about his family background:he wasthe only child who was spoiledby his mother and grandmother,while his father had to return to Germany after World War II. In the drawing of Hermann (Figure 6.5),who had describedhimself as a prince Itt"\ in the family, we see the hidden negative aspectsof his specialposition. We seehim being dominated by two powerful women and the loss of the father, who had to leave r.r+the family becausehe couldn' t getcitizenship and Hermann'smother didn't want to move to Germany. Hermann draws himself as a helplessbabydependenton the two women. 3. His centralposition in the family is expressed ri.r'|. 3. bythe throne. But his mouth is closedwith a 6.5 Hermann's drawing. pacifier, and he has no arms, while his legs are in a sack.Both women are turned towards him and give him their full attention. The secondpersonhe drawsis the grandmotherwho providesfood for him, while the mother representsthe male role of the breadwinner,earninga living and dealingwith the world. He waskept dependent.Mother didn't want him to leavethe familywhen he went to university,but he did it in spite of her. His aggressiontowardsbeing kept in this dependentposition wasmainly unconscious,sincehe wasalwaysan obedient boy and a good pupil. The emotional relationshipwith his mother seemsto be ambivalent. Shetreats him as apaft of herselfbut also representsa role model for being successfuland independent,a gifted merchant. He saysthat he inherited his economictalent from his mother - it's 'in his blood'. His professionalcareerseemsto be a compromisebetweenidentificationand competitionwith his mother. He wasable to leavehis mother to study at university and to live in a different city; later working for an international companytook him to different jobs abroad.On the other hand, he works in the sameprofessionasshedid and fulfils her wish to be successful.

H

. v

a I

t

-

t

u

t

-

r

_tril

llqno8r Figure

*fr 6

Erlka'sdrawing Erika is a managerin the marketingdepartmentofa big company.Sheis an onlychild. Her father has a leadingposition in a small trading company,while her mother is a housewife.Erika wasalwaysa good pupil, but in spiteof the teacher'srecommendation

D R A W I N G SI N C H I L D P S Y C H O A N A L Y S I S

shedid not go to university,preferringto work asa secretaryin order to earnher own money. On the consciouslevel Erika describesher mother as a housewifewho has sufferedall her life from the fact that shedid not havean education and neverworked outside the home. Shewas unhappy with her tight financial situation. The father is describedby Erika in derogatorytermsasa personwithout any ambitions,a little clerk who wasonly fond of card gamesand preferredthe pub to the family home. Erika first draws her father, then her mother, 'f sltiL l*og#v and only after my question about where shewould put herselfin the picture doesshereluctantlydrawherself in the middle (seeFigure 6.6). The comments she makeswhile shedrawsarevery important asthey often contrastwith what she actuallydoes.First shewants to draw her father as 'a little, unimportant guy', she says,but she startswith him first, which meansthat emotionallyheis the most important for her. Shesays: 'Well, I would somehowseemy father asa little mouse. But I cannot draw a mouse- this looks like a little pig, oh no, this is terrible.'Erika'spicture ofher fatherdoes Figure 6.6 Erika's drawing. look like a mouse. A mouse is a weak animal. Erika puts him first but usesan inferior symbol.While she drawsher mother, shesaysit should be 'a lion, a fierceanimal', but what comesout is a cute animal which looks rather like a bear.The animalscharacterizeanantipathy suchasthat found betweena cat and a mouse.The rejectionof the father is enhanced by the pig which signifieson the one hand the maternal-nursing quality and on the other hand the low-dirty aspect.Her rejection of her father may also include a rejectionof the pleasureprinciple in Erika'slife. To Erika,her father'sway of enjoying his life is indulgenceand laziness.The more Erika talks about her father the more she soundslike her own mother, echoingher mother's reproachesand complaintsabout the fatherbeing a failure.Erika seemsto identifrwith her unhappymother, so shehas 'After to fulfil her mother's ambitions.Shesays: three or four yearsat this companyI earn more money than my father everdid.' Only reluctantlydoessheadmit that her father advancedto a leadingposition asdirector of his company,but still earnedless than shedoesnow. Erika strongly dislikesdrawing herselfin the family betweenher parents,which 'Well, she finally does, contradicting it by saying: no, I have surely never been a connectinglink betweenmy parents probablyI wassomewherethere.'And then she addsthat her father wasalwaysproud of her, proud of her excellentgradesat school. The belittling ofand contempt for Erika's fatherseemto result from a disillusionment. He might havebeena promising personto her mother, offering materialsecurityand socialacceptance, but he didn't live up to her expectations.Erika'sprofessionalcareer seemsto be emotionally a presentto her mother to fulfil her dreams. In summary I would like to clarify apossiblemisconception.I am not implying that careersuccessdependson having a mature personality, mental stability or a successfulupbringing.In fact my aim is to show that a variety of interpretationsare

1 31

132

HANDBOOK

OF VISUAL ANALYSIS

possiblein explaining careersuccess.In the caseof some subjects,involvement in a careercan be seenas a form of hyperactivity that helps to ward off a threatening depression.With other careerwomen and men, we find that personalitydevelopment has remained at the emotional level of a child, but that they are able to apply their intellectual and emotional abilities to one particular area,namely to their careers. However,in both academiaand businessthereareemotionallymaturepeopleof both sexeswho areableto sublimatetheir tensionand aggression, and exploit their abilities to the full.

CONCLUSION In this chapterI havedescribedand discussed the useofdrawingsasa path to unconscious wishesand conflicts in child psychoanalysis and (to a lesserextent) in socialscience research.The focuslay in the therapeuticcommunication of making the drawing one specialway of expressingfeelingsand asa responseto an interpretation. If we allow the dynamicsin the sessionto playthemain rolewe arelesslikelyto be'thrown offtrack' by a rigid perceptionof elementsof the drawing.It is important, therefore,to keepthe principles highlighted in this chapterin mind (what elementis drawn first, the sign and symbolicmeaning,etc.),althoughthey should neverdistractus from concentrating on the vivid interaction in the 'here and now', the unfolding of the inner world of the subject. The appliedform of psychoanalyticunderstandingin collectingdatain social researchis more complicated.A projectivedrawing test should,in my opinion, only be used as an additional method - for example, in combination with a narrative interview (seeChapter3). Using drawingsasa form to expressfeelingcan be helpful if it is combined with talking about them with the interviewee.In this way it can provide understandingon a deeperlevel and can make the subjectawareof hidden aspectsof his or her feelings.When the picture is describedby the interviewee,and questionsare askedand commentsmade by the researcher,the distinction between data collecting and interpreting beginsto blur. It is useful,therefore,for projective drawing teststo be conductedby a researcherfamiliar with psychoanalyticthinking and with sensitiveinterviewingskillsin order to maintain the integrity ofthe dataand the sensibilitiesof the interviewee.

NOTE Thenamesof thepeoplediscussed in thischapterhavebeenchanged.

REFERENCES Abraham, E. ( 1950)'Zeichnung und Farbealsseelischer Ausdruck', Psyche, a (5):237-40. Diem-Wille,G. (1996a)Karrier{rauenundKarrieremanner. Einepsychoanalytisch orientierte Untersuchungihrer Lebensgeschichte undFamiliendynamlk. Opladen: Westdeutscher Verlag.

DRAWINGS IN CHILD PSYCHOANALYSIS

'Femininity Diem-Wille, G. (1996b) and professionalism:a psychoanalyticstudy of ambition in femaleacademicsand managersin Austria', in D.F. Good, M. Grandner and M. Maynes (eds),Austrian Women in the Nineteenth and Twentieth Centuries.Providence and Oxford: BerghanBooks. 'General introduction', in B. foseph,PsychicEquilibrium Feldman,M. and Spillius,E.B.(1991) and PsychicChange.London and NewYork: Routledge. Freud, S. (1900) Die Traumdeutung, G.W.,II-III, Frankfurt: Fischer (1968), Eng.: The of Sigmund Interpretationof Dream.s,StandardEdition (SE) The CompletePsychologicalWorks Freud,2 and 3, London: Hogarth Press(1950-1974). (StandardEdition) 15-16.London: Freud,S.( 1916-17)IntroductoryLectures on Psycho-Analysis Hogarth Press( 1950-1974). 'Transference: the total situation', in PsychicEquilibrium and PsychicChange. foseph,B. ( 1985) London and NewYork: Routledge. 'Child News,2 ( 1): 1-3. psychoanalysis today', TheInstituteof Psycho-Analysis foseph,B. (1998) Klein, M. (1975) ThePsycho-Analysis of Children London: Hogarth Press. Zeichentest. Kos,M. and Biermann, G. (1973)'Die verzauberteFamilie', Ein tiefenpsychologischer Munich: Basel. UntersuchungzrAggression Mitscherlich, M. ( 1985)DiefriedfertigeFrau: Einepsychoanalytische der Geschlechter.Fr ankfurt: S. Fischer. Sandler,J., Holder, A., Dare, C. and Dreher, A.U. (1997) Freud'sModelsof the Mind: An Introductior. London: Karnac Books.

133

II ffiry T rum

VisualMeaning: a SocialSemiotic Approach CAREY JEWITT AND RUMIKO OYAMA

INTRODUCTION Socialsemioticsofvisual communication involvesthe descriptionofsemiotic resources, what can be said and done with images(and other visual meansof communication) and how the things peoplesayand do with imagescan be interpreted.

Describing semioticresources The term 'resource'marks one of the key differencesbetweensocialsemioticsand Paris schoolstructuralistsemiotics(seeChapter4). In Parisschoolsemioticsthe keyword was'code',not'resource'.It conceivedof semioticsystemsascodes,setsof rulesfor connectingsignsand meanings.Once two or more peoplehavemasteredthe samecode, it wasthought, theywould be ableto connectthe samemeaningsto the samesounds or graphicpatternsand hencebe ableto understandeachother.How thesecodescame about,who madethe rulesand how and why they might be changedwasnot considered a key issue. Someforms of visual communication actuallywork like this, for instancethe highway code,a key examplein earlyParisschoolvisual semiotics.In other forms of visual communication, for instancechildren'sdrawingsand many forms of modern art, there areno suchcodes.Like the highway codethey draw from the visual resources which Western culture has developedover the centuries,but the way they usethese resourcesis not subjectto the samekinds of rules.The highway code is governedby strict prescriptions,children's drawings and modern art by creativeinvention, the influenceof examplesand conventions,and soon. The sameappliesto the interpretation, 'take-up' the of images.Someviewersinterpret'accordingto the book' (in educational contextsyou usuallyhaveto do this ifyou want to geta good grade),othersusewhatever resourcesof interpretation and intertextualconnectionthey canlay their handson to createtheir own new interpretationsand interconnections. For socialsemioticsthis is a vital point. There are kinds of 'rules', from laws and mandatory prescriptionsto'best practice',the influence of role models, expert advice,common habits,and so on. Different kinds of rules applyin different contexts. As for breaking the rules, only people with a large amount of cultural power are

A SOCIAL SEMIOTIC APPROACH

given permission to do this, at least in public places.Most of us have to conform. In private, in the smallergroups and 'sub-cultures'we live in, we may havemore freedom, but our semiotic productions and interpretations are not likely to spread much beyond those small circles.They will remain relativelymarginal. Sometimes, however, societyneedssomething new, and then novel modes of production and interpretation will stand more of a chanceof being added to the culture's treasury ofvisual resources. Let us give an exampleof such a'resource',that of 'point of view' (Kressand van Leeuwen,1996:135-53) (seeChapters4 and 9). This resourceallowspeople, placesand things to be depictedfrom aboveor below (or at eye-level),and from the front, the sideor the back.Both thesedimensions,the verticaland the horizontal,are graded, amatter of degree.There is, for instance,a range of vertical anglesbetween 'bird's the eye'view and eye-level,and a rangeof horizontal anglesbetweenfrontality and the profile. Point of view also createsa meaning potential. This does not mean that it is possible to say what dififerent points of view will mean exactly. But it is possibleto describethe kinds of meaning they will allow image producersand viewersto create,in this case,the kinds of symbolicrelationsbetweenimageproducers/ viewersand the people,placesor things in images.In the caseof the vertical angle this relation will be one of symbolic power. If you look down on something,you look at it from a position of symbolicpower. If you look up at something,that something has some kind of symbolic power over you. At eye-levelthere is a relation of symbolicequality.In the caseof the horizontal angle,the relationwillbe one of involvement with, or detachmentfrom, what is represented.Frontality allows the creation of maximum involvement.The viewer is directly confronted with what is in the picture. If somethingis depictedfrom the side,the viewer literally and figuratively remains on the sidelines.Again, there are of course many degreesof involved or detached engagementin between. 'power', 'detachment','involvement', Two points needto be made.First, and soon) arenot'the' meaningsof theseangles.They arean attemptto describea meaning potential, a field of possiblemeanings,which need to be activatedby the producers and viewersof images.But this field of possiblemeaningsis not unlimited. If you want to expressthat somethingor someoneis impressiveand powerful, you are unlikely to choosea high angle,and ifyou seesomeonedepictedfrom a high mgle, you areunlikely to concludethat he or sheis representedasan impressiveand powerful person(although it is alwayspossible:sometimeswe saythe oppositeof what we mean and are neverthelessperfectlyunderstood).Secondly,symbolicrelationsarenot real relations,and it is preciselythis which makespoint ofview a semioticresource.It can'lie'. Photographs cans)rrnbolicallymakeus relateasan equalto peoplewho in fact haveveryconsiderable power over our lives (for example,politicians), or it can make us look in a detached way at peoplewho we are involved with (seeChapter4). To map out meaningpotentials,Kressandvan Leeuwenuse'systemnetworks', a sryleof diagrammingthat derivesfrom the work of M.A.K. Halliday (Ig7l),whose linguistic theorieshavebeen a decisiveinfluenceon this kind of visual analysis.The 'point systemnetwork below encapsulates the resourceof of view' (squarebrackets meaneither/or).

135

136

H A N D B o o Ko F v r s u A l A N A L Y s T s

power over the viewer \

equality\

Iow angle

eye-level

power of the viewer \

trigh angle

point of view

engagement\

f rontality

detachment \

Rrofile

Such resourceshave histories.They were invented in the context of specific interestsand specificpurposes.Point of view, as we have describedit here,became possibleasa resultof the invention ofperspectivein the Renaissance. Beforethat time pictures were surfaceson which to make marks, and 'the world in the picture was experiencedas a direct continuation of the observer'sown space'(Arnheim,1974: 274).Afterthatpicturesbecame'windowson theworld' (Panofsky,1953)whichallowed a particular subjectivepoint of view to be realized- and this happenedin a time in which subjectivityand individualitybecamesignificantsocialvalues.That it happened through a geometricsystemwas one of the paradoxesof the invention. As Kressand 'Socially van Leeuwensay: determinedviewpoints could, in this way, be naturalised and presentedas studiesof nature' (1996:.137).This is anotherkey emphasisof the socialsemiotic approach:semiotic resourcesare aI once the products of cultural historiesand the cognitiveresourceswe useto createmeaningin the production and interpretation of visual and other messages.

Describing andexplaining howsemioticresources areused The secondtyp. of semioticwork is describingand explaininghow semioticresources are usedin specificdomains. Socialsemioticsis not an end in itself. It is meant as a tool for usein critical research.It only becomesmeaningful once we begin to useits resourcesto ask questions.In this chapterwe draw on severalstudieswhich apply visual socialsemiotics.The first is a study exploring the visual representationof male heterosexualityin British sexualhealth materialsaimed at peopleaged13 to 19yearsold

A S O C I A L S E M I O T I CA P P R O A C H

137

$

-':

;ffffi

"t s#,

effi qlti.

ffi ffi,-t*

l,*i

I lf,ri : ,11i :tr.Zr;=l::

ffi

ffiW.

fu.:.

'Explore the possibilities' (City and East London Health promotion).

138

H A N D B O O KO F V I S U A L A N A L Y S I S

(fewitt, 1997,1999).The study revealedthat imagescan reinforcestereotypedforms of masculinitywhich in words would probablybe unacceptableto many sexualhealth workersandyoung people.It compiled a detailedand systematicdescriptionof these imagesbasedon l8 descriptivedimensionsdevelopedfrom Kressand van Leeuwen's socialsemioticapproachto the analysisof images(Iewitt, 1997).Themain dimensions willbe describedin detailincludingtheform ofrepresentation, the settingandtheprops, the actors'appearance,the composition of eachimageand the relationshipbetween the representedactorsand the viewer (encodedthrough the use of point of view, distanceand contact).Thesedescriptionswereenteredon to grid-charts(the columns representingthe dimensionsoutlined above,the rows representingeachimage - a totalof 74 imageswereanalysed).The chartswerethen analysedusingthe framework process(Ritchie and Spencer,1994),which identifiesthe patternsand relationships betweenand acrossthe descriptionsof the images(for example,the relationship betweenactorsand setting) and can thereforebe usedboth to generateideasand to systematicallytest the hypothesiswhich emergedfrom the literature on masculinity and genderand from preliminary analysisof the images(seeChapter2). 'point How was of view' used in this material?To start with the horizontal dimension,the frontal anglewasusedto increaseaudienceidentificationand involvement with representedparticipants who reduce sexualrisk. For instance,a frontal angle was used to presentthe male actor carrying condoms in Figure 7.1, while the woman was presentedin profile, at an oblique angle:this use of 'point of view' presentsthe man asa figure to identifrwith, and objectifiesthe woman asthe'other'. The useof frontal anglealsorelatedto hegemonicnorms of masculinity.Throughout the sampleof imagesthe men shown frontallywere'real men', whereasoblique angles were usedto depict men who failed to acquirethe norms of hegemonicmasculinity and thereforeshould not be identified with. For instancethe oblique anglein Figure 7.2 frames the boy on the far right of the image in a way which emphasizesthe differencebetweenhim and the other boys (all of whom are presentedfrontally), his 'otherness' 'limp being confirmed by his unbalancedposture, wrist', foppish hair and 'wimp'. glasses:he represents While the other boys in the image are representedas looking at us the boy on the far right is representedaslookingaway; like the woman in Figure7.l,he is representedasan objectfor the viewerto look at rather than engage with. As for the vertical angle,men in the samplewere portrayed aspowerful by the useof a low vertical anglein particular roles: as fathers,and when they were shown as activelyengagedin education,assertingtheir knowledge,planning sexand being sexuallyunrestrained(Jewitt, 1997). In this study visual socialsemioticsrevealedthings which were not evidentat first sight,and evenbrought to light contradictionsbetweenthe verbal and the visual message.But visual socialsemioticsby itself is not enough.To explain the resultsof the analysis,the study had to draw on other sources,on socialtheoriesof genderand masculinities(Wight,1992;Holland et al., 1993)and on earlierstudiesof genderconstruction in images(Millum, 1975;Graham, 1977;Goffinan, 1979).In studiesof the useof semioticresources,visual socialsemioticscan only everbe one elementof an interdisciplinaryequationwhich must alsoinvolve relevanttheoriesand histories.

140

HANDBOOK

OF VISUAL ANALYSIS

Expanding semioticresources Finally, there is a third contribution semioticianscan make. They can also explore how semiotic resourcesmay be expandedso as to allow more options, more tools for the production and interpretation of images and other forms of visual communication. In other words, semioticscan be a tool for design.In the past this was a by-product of semiotics.Semioticswas supposedto be 'the scienceof signs',and scienceis supposedto be about 'what is'not about'whatcouldbe' or'what might be'. Still, just as sciencecan unlock doors to new technicalpossibilities,so semiotics can unlock doors to new semioticpossibilities,whether in the form of new resources (for example,the study of sexualhealth leafletsdiscussedin this chapter informed the production of guidelineson sexualhealthresourcesfor young men (Jewitt,1998)) or of new usesof existingresources.In the caseof the representationof genderand masculinities,contemporaryartistsand performers,thosepracticalsemioticians,are well aheadof the theoristsin this respect.

SOCIAL SEMIOTICANALYSIS OF VISUAL COMMUNICATION Visual social semiotics is functionalist in the sensethat it seesvisual resources as having been developedto do specifickinds of semiotic work. It follows Halliday (for example, 1978) in recognizingthree main kinds of semiotic work, which are always performed simultaneously. Halliday calls these three kinds of work 'metafunctions', and distinguishesbetween:the ideationalmetafunction,the function of creatingrepresentations;the inter-personalmetafunction,the part languageplays in creating interactionsbetweenwriters and readersor speakersand listeners;and the textual metafunction,which brings togetherthe individual bits of representationand-interaction into the kind of wholes we recognizeas specific kinds of text or communicative event (advertisements,interviews,dinner table conversations,etc.) (seeChapter9 for discussionof filmic texts).Kressand van Leeuwen(1996)have extendedthis ideato images,using a slightlydifferent terminology: 'representational' instead of ideational'; 'interactive' insteadof inter-personal';and 'compositional' insteadof 'textual'. Ary image,they say,not only representsthe world (whether in abstractor concreteways),but alsoplays apart in some interaction and, with or without accompanying text, constitutes a recognizablekind of text (a painting, a political poster, a magazineadvertisement,etc.). The image of the young black couple in the sports car in Figure 7.1 first of all representsa particular kind of relationship(sexual)in a particularkind of setting(urban street).This is the representationalwork the photographdoes.But the woman'slook is not directedat the viewer, while the man's look is, and this shows that the picture also plays a role in an interactionfoundedon genderedpower relations.Through the man'slook and gesture (encirclingthe woman in his arms,the carbeingan'extensionofhis domain'), a health organization asksyoung men to engagewith a discourseof sexual safety.This is the interactivework the picture does.Finally, the layout, the placementand relative salienceof picture (for instance,the centralposition of the condom packet)and text,

A SOCIAL SEMIOTIC APPROACH

139

'j= ';i

o o o) .E C, c,

d

-> c l.r. v o dl

s E N N

o) f E')

lr

A SOCIAL SEMIOTIC APPROACH

and so on, do the compositionalwork that allowsus to recognizethe text asa health advertisement.

R E P R E S E N T A T I O N A LM E A N I N G 'particiRepresentational meaningis first of all conveyedbythe(abstractor concrete) pants' (people,placesor things) depicted.For example,in Figure7.2 the young man 'wimp'because, on the basisof certain visual semanticfeatures on the right signifies (longishhair, submissivepose,avertedgaze,glasses,'limpwrist') we recognizehim as 'wimp'. belonging to the classof This is the visual equivalentof lexis,of the vocabulary, and in this respectvisualsocialsemioticshasnot addedmuch to the work ofstructuralist semioticiansand iconographers(seeChapter5). Where it has contributed new ideas for the visualanalysisof representationalmeaningis in its emphasison the'syntax' of imagesas a sourceof representationalmeaning.In time-basedsemioticmodessuch aslanguageand music, 'syntax' is a matter of sequencingorder (for example,word order). In space-based semiotic modessuch asimagesand architectureit is a matter 'where of spatialrelationships,of things are'in the semiotic spaceand ofwhether or not they areconnectedthrough lines,or through visual'rhymes'of colour, shape,and 'syntax' so on. Suchforms of spatial haveof coursebeendescribedbefore,for instance by art theoristssuch asArnheim (1969,I974,1982),but for the most part only in formalist and aestheticterms (or sometimesin terms of psychologicaltheories of perception),not ascontributing to representationalmeaning (seeChapter6). Kressand van Leeuwen(1996) describevisual syntacticpatternsin terms of their function of relatingvisualparticipantsto eachother in meaningfulways.There are two kinds of pattern. Narrative representationsrelate participants in terms of 'doings' and'happenings',of the unfolding of actions,events,or processes of change. stable Conceptualpatterns,representparticipantsin terms of their more generalized, 'essences'. They do not representthem as doing something,but asbeing or timeless something,or meaningsomething,or belongingto somecategory,or having certain characteristics or components.The choiceis important, sincethe decisionto represent somethingin a narrativeor conceptualway providesa key to understandingthe discourseswhich mediatetheir representation.

Narrative structures Narrativepictures (or sceneswithin pictures) are recognizedbythe presenceof a vector.A vector is a line, often diagonal,that connectsparticipants,for instancean arrow connectingboxesin a diagram)or the outstretchedarms of the man in a leaflet 'doing' or on sexualhealth representedin Figure7.3.Thevector expresses a dynamic, 'happening'kind something of relation.In the caseof Figure7.3,the syntaxsuggests like the following meaning.The man doessomethingto the woman. What that something is may then be filled in differently by different viewers(engage,dominant, argue), but the rangeof possibleinterpretationsis not unlimited.

141

142

HANDBOOK OF VISUAL ANALYSIS

In Figure 7.3 the'actor', the 'doer' of the action is the man. More generally, 'actors' arethe participantsfrom whom or which the vector emanates,or who themselvesform the vector. Here the vector is formed by the strong downward diagonal vector of the man and woman's arms and their eyeline.The vector is bi-directional (that is, the woman's arms alsoform a vector which 'actson' the man). However,the primary'goal', the participant to whom or which the action is done,is formed bythe woman asthe upward directionalityof the vector depictingher action is more weakly articulated.The woman is representedasinvolvedin'stopping'his movementforward (echoedin the caption'If he won't usea condom he needsto be told'). In this image, as in many others in the study, imagesvisually confirmed the role of women asthe mediatorsof sexuality:besiegedby predatory male sexuality(heredepictedby'black maleness')and at the sametime depictedasableto control malesexualitybythe'threat

lf he wsn'tusea condsm he needsto be toldcondomsmakesex safer. For informatianaboutsafer sex call the NationatAID$ He*plineon 08*0 5S? 123. lt's free and canfidentiel.

Figure 7.3 Sketch of 'lf he won't use a condom' (Health Education Authority poster).

A SOCIAL SEMIOTIC APPBOACH

of withdrawal'. More generally,'goals'are the participantsat whom the vector is directed.When a picture or a scenewithin a picture hasboth an actor and a goal it is 'transactive', representingan action taking placebetweentwo parties.But it is also possibleto havea picture or scenecontaining only an actor and a vector. The eyeline,the direction of the gazeof representedparticipants(insofar asit is not directedat the viewer),is a specialkind ofvector. It createsa reactionratherthan an action. Sucha reactioncan,again,be transactiveor non-transactive.It canbe that we seeboth the person(or perhapsanimal) who is looking and the objectof his or her gaze(transactivereaction),or only the person looking and not what he or she is looking at (non-transactive);for example,the woman in Figure 7.1 looks but we do not know what at. Facialexpressionsand gesturescan then 'colour in' the nature of the reaction aspleasedor displeased,deferentialor defiant,and so on. The conceptsof narrative visual analysis(action, reaction,transactive,nontransactive)can help 'interrogate' a visual text, help to frame questionssuch aswho areplaying the activeroles of doing andlor looking and who the passiveroles ofbeing actedupon and/or being looked at in visual textswith certain kinds of participants (for example,minorities). Who are shown aspeoplewho act,who aspeoplewho reactin visualtextsabout certainissues? This kind of analysisofthe narrativeprocesses in the study of sexualhealth educationleafletsand postersrevealedthat men and women were representedasequallyactive- which is not alwaysthe casein imagesshowingmen and women.However,the natureand the occasionof men'sand women'sactionin the imagesdifferedin importantways.Overall,women wererepresentedaspassive,or less active,in the context of sexualactivities,wheretheywere shownas'reacters'rather than 'actors', or asinvolvedin non-transactiveactions,actionsthat haveno effector impact on someother entity. Men were more often shown in transactiveactions:it wasmen who were representedasinitiating sexualactivitiesand asmost sexuallyactive,literally in the driving seat,while women weremore often presentedassexuallypassive(literally 'passenger' a in the caseof Figure 7.1). The study showedthat women were shown asenforcingsexualprotection byinsisting on condom use,asin Figure7.3.The goals of male and femaleactionsalsodiffered, and women were more frequentlythe goal of actionsthan men. The goals(objects)of the femaleactors'transactiveactionscan be characterizedas risk reducers(for example,condoms,information booklets).In contrast,the cultural symbolismof the goals(objects)of the male actors'actionscan, with the exceptionof condoms,be describedasrisk enhancers(for example,contact sport or, as in Figure 7.1,a convertiblesportscar) with the associatedwealthand success.

Conceptual structures Images which do not contain vectors are 'conceptual'. They visually 'define' or 'analyse' or'classi$" people,placesand things (including againabstractthings). One kind of conceptualpatternis the classificationstructure. Classificationstructuresbring different people, placesor things together in one picture, distributing them symmetricallyacrossthe picture spaceto showthat they havesomethingin common, that

143

144

ISUAL ANALYSIS

theybelongto the sameclass.Figure7.2for instance,offerssucha classificationof the male body. Throughout the imagesvisual classificationstructureswere usedwhich consistentlyhighlighted the individuality of men. For instance,men in the images rarely looked at one another and were separated(disconnected)by the use of space on the page.In contrastwomen were visually representedas connectedboth to one another,and to men in terms of gaze,and the 'occupation of spaceon the page'. Symbolic structuresdefine the meaning or identity of a participant. In the 'carrier') symbolic attributive structure the identity or meaningof one participant (the 'symbolic is establishedby another (the attribute'). Here Kress and van Leeuwen (1996) rely on iconography (see Chapter 5). Symbolic attributes are recognized through one or more of the following characteristics:they are made salient in the representation,for exampleby their size,position, colour, use of lighting; they are pointed out by meansof a gesture;they look out of placein the whole; they are conventionally associatedwith symbolicvalues. A rangeof props conferredsymbolicmeaningor attributeson the represented participants.Carsand motorbikes,both potent cultural symbolsof male virility and sexualprowessin Westernindustrial society,were usedto confer masculinity on the men in someof the images(asin Figure7.1).Sportsequipmentand settingswereused to signalcompetitivenessbetweenmen and as signifiersof their heterosexuality. Analytical structures,finally, relate participants to each other in terms of a part-whole structure.Maps areanalyticalstructuresand so arepie charts.In all these casesa conceptor entity is definedby showinghow it is made up out of which parts. Whateverthe conceptor entity, this can alwaysbe done in more than one way.A map of Europe,for instance,may analyseEuropeasconsistingof countriesor asconsisting of differentkinds of soil, or of different altitudes.Analytical structuresalwayshavetwo 'possessive keyparticipants:the carrier (the whole) and anynumber of attributes'(the parts).Diagramsof male sexualand reproductiveorganswerethe main useofanalytical diagramsin the studyof sexualhealthleafletsand posters.In thesediagramsthe carrier was the male or femalesexualand reproductivesystem,which consistedof labelled possessive attributes(parts).The inclusionand exclusionofelementsin thesediagrams 'sexual'. servedto definethe whole,that is what waslegitimatelyconsidered It alsoserved to realizethe norms of sexuality.For example,many of the diagramsonly included a representationof the anusin the context of disease,rather than sexualpleasure,thus confirming heterosexualnorms. Kressand van Leeuwen(1996:Chs 2 and 3) distinguish further sub-typesof many ofthese'syntactic'patterns,but the onesdiscussedherearethe most important. This is not to say that they provide all that is necessaryfor studies of the visual representationof specificissuesof social significance.In the study of sexualhealth Ieafletsand posterstwo aspectswereaddedthat clearlybelongto the sphereof representational meaning:the settingof imagesand the visual appearanceof socialactors. 'level Settingswereassociated with the of malesexualcontrol' and with signifring whether or not sexualactivity had taken place.In the study, 'level of male sexual control' was defined by such factors as: who initiates and directs the action; the differencein eye-levelsof the representedparticipants(asan indication of the power relationshipbetweenthem); who vetoessexualactivity;and who possesses or provides

A SOCIAL

SEMIOTIC

APPROACH

sexualinformation. The settingsin the imagesindicatedthat women and men were representedashavingsexualcontrol in differentenvironments.Women wererepresented aspossessing sexualcontrol in the home/domesticsettings,in leisure(public) venues, in medicalsettingsand in 'nature'- outdoorswith grassand trees.Men were depicted ashaving sexualcontrol in the urban (outdoor) environment.For example,the man in Figure 7.1 is in the driving seat,literally and metaphorically- it is his car and the condomsare in his pocket.Men on the streetwere representedassexuallyin control or'predatory', but oncein domesticsettingstheywereshownto relinquishcontrol to women. The dominant representationof men in the sample,asjudged by analysisof the actors'hair,clothes,posture,facialexpression, ofparticipants the relativeeye-levels and their actions,wasone of conventionalhegemonicmasculinity.Within the images men were shown in competition in relation to masculinity and sexualprowess.The notion of a continuum of malesexualityis visualizedin the leaflet'4 Boys'(Figure7.2) which showsa row of eightboys.The dichotomybetweengladiatorandwimp is apparent astwo extremeson a continuum of masculinity,with a visual imageof a strong boy on the left of the imagethrough to a weakerlooking boy on the right. Men's individuality is emphasizedbythedifferencein their appearance, posture,age,height,ethnicityand props.The symmetryofthemen in theseimagessuggests thatwhilebeingan individual is a defining featureof being a man, masculinityis a unifring experiencewhich transcendsindividual difference.

INTERACTIVE MEANING Imagescan createparticular relationsbetweenviewersand the world inside the picture frame. In this way they interact with viewers and suggestthe attitude viewers should take towards what is being represented.Three factorsplay a key role in the realization of thesemeanings:distance,contact and point of view (seeChapters2 and 9). Together they can createcomplex and subtle relations betweenthe representedand the viewer. As will be shown, the relationship betweenthe viewer and the men representedin the sampleof sexualhealth leafletsand postersis presented as visually more demanding but more socially distant and lessinvolved than the relationshipbetweenthe viewer and the women represented.

Contact Many pictures show peoplewho, from inside the picture frame, look directly at the viewer. In this way they'make contact' with the viewers,establishan (imaginary) relationwith them. Kressand van Leeuwen( 1996)callsuchpictures'demand'pictures - the people in the picture symbolicallydemand somethingfrom the viewer. Facial expressionand gesturesthen fill in what exactlythey'demand' in this way: they can demanddeference,by unblinkingly looking down on the viewers,or piry by pleadingly looking up at them; they can addressviewerswith an ingratiating smile or unsettle

145

146

HANDBOOK

OF VISUAL ANALYSIS

them with a penetratingstare(for example,the defiant posture- handson hips, legs astride- and stareof the boy on the left in Figure 7.2). Gesturescan further modifr what is demanded,asin the famous'Your Country NeedsYOU!' recruitment poster. Without this kind of imaginarycontact'we look quite differentlyat the peopleinside the picture frame. We 'observe'them in a detachedway and impersonallyasthough they arespecimensin a displaycase.Kressand van Leeuwen(1996)call suchpictures 'offers' 'offer an of information' is made (for example,the boy on the far right of Figure7.2).The terms'offer' and 'demand'weretakenfrom Halliday ( 1985)who uses them to distinguishbetweendifferent classes of speechact,questionsand commands, which'demand',respectively,'information'and'goods and services', and statements 'ofi[er', 'information' 'goods and offers,which respectively, and and seryices'.In the study of sexualhealthleafletsand posters,contactbetweenthe viewer and the images of the men was more often in the form of a demand;in contrastwomen were more often portrayed in the form of an offer. That is, women were visually representedas more sexuallypassiveand availablethan men who werevisually representedassexually activeor demanding.

Distance Imagescan bring people,placesand things closeto the viewer or'keep them at arm's length'. In everydayinteraction the norms of socialrelationsdeterminethe distance we keepfrom eachother.This translatesinto the'sizeof frame' of shots.To seepeople closeup is to seethem in the waywe would normally only seepeoplewith whom we are more or lessintimately acquainted.Everydetail of their faceand their expression is visible. We are so closeto them we could almost touch them. They reveal their individualityand their personality.To seepeoplefrom a distanceis to seethem in the way we would normally only seestrangers,peoplewhoselivesdo touch on ours. We seethem in outline, impersonally,astypesrather than as individuals. This doesnot mean of coursethat the peoplewe seerepresentedin close-upareactuallycloseto us, or vice versa.It meansthey are representedasthough they belong or should belong 'our to group', and that the viewer is thereby addressedas a certain kind of person. Thereareof coursemanyintermediatedegreesbetweenclose-upand distance,just as there are, in everydaylife, many intermediate degreesbetween the most intimate relationsand the total absenceof a relation.To describethesethe terminology of film and televisioncanbe used.A close-up(headand shouldersor less)suggests an intimate/ personalrelationship;a medium shot (cutting offthe human figure somewherebetween the waistand the knees)suggests a socialrelationship(asin the couplein Figure7.1); 'long and a shot' (showing the full figure, whether just fitting in the frame or even more distant) suggests an impersonalrelationship(suchasin Figure 7.2).In the study of sexualhealth leafletsthe relationshipbetweenthe viewer and the representedmen waspresentedasmore intimate when the men wereaccompaniedbywomen: perhaps the presenceof women in imagesenableda more intimate portrayal of men asthey resolvedthe problematic issueof men looking at other men (seeChapter 4 for a discussionof audience' gaze').

A SOCIAT SEMIOTIC APPROACH

Pointof view The third factor is point of view (seeChapters4 and 9). We have alreadydescribed how frontal angleswere used to increaseaudienceidentification and involvement with representedparticipantswho reducesexualrisk and vertical angleswere usedto representmen aspowerful in particular roles.Detachmentwasillustratedby a malerepresentedparticipant's failure to acquire the norms of hegemonic masculinity (asin Figure7.2). One fuither aspectofinteractivemeaning,modality (seeChapter2, this volume), will be discussedin relation to the applicationofvisual socialsemioticsto explorethe rhetoricsof the scienceclassroom(Kresset al.,in press).lSo at this point it is usefulto summarizethe role of visual socialsemioticsin the study of sexualhealth promotion materialswe haveusedsofar.Visual socialsemioticshelpedilluminate how the structures ofthe imagescontributed to the representationof conceptsofmasculinity. The message of the leafletsand postersclearlywasnot only verballybut alsovisually conveyed.In partioiar the method of analysisidentified visuallysignifiedmeaningswhich, in words, would havebeenunacceptableto many professionalsand young people.It highlighted the visual oversimplificationof male sexuality,the failure of the materialsto address young people'semotional concerns,the visual polarization of men and women, and the visual legitimating of a narrow definition of male and femalesexuality.

C O M P O S I T I O N A LM E A N I N G Further resourcesof socialsemioticvisual analysiswill be exploredin relation to another example,in which we focuson how two 1l-year-old studentsgrappledwith the problem of how to make a visual and written text'scientific'. They had looked at the cellsof an onion under the microscopeand were askedto draw'what they saw' and describein writing'what they did'. The teachergaveexplicit instructionson how to composethe text, to placethe imagein the bottom part of the pageand the writing in the top part of the page.Below we first introduce some further social semiotic resourcesfor visualanalysis,and then discusstheir usein the work of Ramendeepand another student from the sameclass,Amy, who used them quite differently. They include three resourcesof compositionalmeaning:information value;framing; and salienceand modality, animportant 'interactive'resourcewe havenot yet discussed. The compositional structuresdiscussedbelow can also be applied to the layout of compositetexts suchasmagazinepages,illustratedbooks, and so on.

Information value Information valuesarc realizedby the placementof the elementsof a composition. The ideais that the role of any particularelementin the whole will dependon whether it is placedon the left or on the right, in the centreor the margin, or in the upper or the lower part of the picture spaceor page.

147

148

ISUAL ANALYSIS

In societieswhich useRoman script the direction of the readingof a text (left to right, from top to bottom) hasled to different cultural valuesbeing awardedto the left and the right. According to Kressand van Leeuwen(1996),left-right placement 'given', createsa'given-new'structure.The elementsplacedon the left arepresentedas the elementsplacedon the right as'new'.For somethingto be'given'meansthat it is presentedassomethingthe viewer or readeralreadyknows, asa familiar and agreed 'new' means that it is presented departurepoint for the message. For somethingto be assomethingnot yet known and not yet alreadyagreedupon by the viewer or reader, hence as something to which the viewer or reader must pay specialattention. The 'new'is thereforeproblematic,contestable,the information'at issue',while the'given' is presentedas commonsensicaland self-evident.Again, this is a meaning potential which will get more specificcontours in the context of specificimages. As for top and bottom, again,if some of the constituent elementsare placed on top and othersat the bottom, then what is placedon top is presentedaswhat Kress 'real'. 'ideal' and van Leeuwen( 1996)call the and what is placedat the bottom asthe 'ideal' For somethingto be meansthat it is presentedasthe idealizedor generalized essence of the information, henceusuallyalsoasits ideologicallymost salientpart. The 'real' is then opposedto this in that it is its meaningpotential to presentmore'down to earth information'. According to the context this can becomemore specificinformation (for example,details),more practicallyoriented information (for example, practicalconsequences, directionsfor action) or more real information (for example, photographsasdocumentaryevidence).In Figure 7.2,the writing in the text of men 'ideal', realizingtheir sexualfearsand concernswill be the the imageof sexualbravado the'real'.

Figure 7.4

Onion cell text by Ramendeep, aged 1 1 years.

A S O C I A T S E M I O T I CA P P B O A C H

Centrality, finally, meanswhat it is: what is placed in the centre is thereby seenaswhat holds the'marginal'elements together.The marginal elementsare then in some sensethe elementsthat are held togetherby the centre- belongingto it, subservientto it, and so on, dependingon the context. The teacher'sinstruction to placethe writing in the top part of the pageand the image in the bottom part carriedwith it a particular notion of text as primarily a written entity: the writing was the prior process,placing the image in the role of outcome.The pupils respondedto this instruction differently:Ramendeeprecast the instruction using composition to produce an different design(Figure 7.4); Amy did not transform the teacher'sdesign,but adaptedit slightly to integratethe image 'illustration'through as the partialmerging of writing and imagein her text. The questionbecomesone of whether the pupils aretreatingthe text asbeing primarily loggedin writing or image.In both casesthe texts are designedto be read top-down. In Ramendeep'stext (Figure7.4)her useof the top and the bottom of the pagegivesthe information value of abstractidealto the image (top) and the value of the empirical real to the written element.That is, the visual elementof the text, her drawing, representsthe meansof carrying the abstracttheoreticalrepresentationof 'scientificness'. In contrast,the written elementof her text is an empirical accountof 'what shedid' indicatedby the diary-like genreof the writing: a narrative,personal voice,pasttensedeclarativesand the recounting ofparticular events.The 'scientificness' is primarily logged visually in Ramendeep'stext. In Amy's text (Figure 7.5) her arrangementof elementsin the top and bottom of the pagepresentsthe writing asthe abstracttheoreticalrepresentationof 'scientificness', and her drawing asthe empirical real, the 'outcome' of the procedure.Her writing achievesthe genreof science:a proceduralaccountwith numbered steps,agentless voiceand imperative statements.Her drawing provides a specificsensoryimage of what she saw- the evidence.Amy's text nonethelessshowsa move to integratethe processand evidenceby the partial integration of the visual and written in the way title and image intersect one another.

Framing

Figure 7.5 Onioncell text by Amy,aged 11 years.

The term 'framing' indicates that elementsofa compositioncaneither begivenseparateidentities,or represented as belonging together. In other words, framing'connects'or 'disconnects' elements.Disconnection can be createdin many ways,

7 49

150

HANDBOOK

OF VISUAL ANALYSIS

through framelines(which maybe thick or thin: therearedegreesof framing),through empty spacebetweenelements,but alsothrough contrastsof colour or form, or any other visual feature- in short, through any form of discontinuity, disconnectionor contrast that can be visually signified. Connection can be achievedin exactly the oppositeway, through similaritiesand rhymes of colour and form, through vectors that connectelements,and of coursethrough the absenceof framelinesor empty space betweenelements.In everycasethe discontinuity or continuity betweenelementsin a senseexpresses what it is, that is the elementsare separatedor made to belong together.This broad meaningpotential can then be made more precisethrough the context,and alsothrough the meansof framing chosen. In Ramendeep'stext (Figure 7.4) thesecompositionalresourcesare usedin a variety of ways.The image is first and arguablythe most salientelementof the text. Imageand writing arepresentedunder different titles and the dateheadseachsection 'looking at onion cells' and the writing is entitled of the text. The image is entitled 'looking at cells'and'what I did'. Imageandwriting arepresentedastwo quite distinct parts of the sametext and their disconnectionis strengthenedby a line separating them (the factthat they areon the samepagemaintainstheir connectionasone textual unit). This separationof the visualand written elementsof the text marks a shift from the abstract(visual)to the specific'concreteactions'locatedin the pupil's personal experience(written). Two different modesareusedto realizedifferent communicative functions - the writing to conveythe personal,the visual to conveythe scientific.At another level,the framing of the visual setsscientific reality apartfrom everydayreality 'through the microscope'is only a selectionof what and alsoindicatesthat what is seen can be seen:it is a visual abstractionor generalizationof the cell. The frame of the 'open' (left) image,a rectanglewith an sidepresentsthe cell asan instancetaken out of the empirical.In Amy's text (FigureT.5) the written elementand the visualelement of the text partially merge- th ey arelessstronglyframedasseparate.The written element is stronglyframed in terms of scientificgenre- the stepsof the experimentareclear.The circular framing at the levelof the imageframesthe visualinformation as'empirically real'- what is therefor Amy. In other words,both pupils haveusedframing at the level of the text and at the levelof imageto realizetheir different interests.

Salience 'salience' The term is usedby Kressand van Leeuwen(1996)to indicatethat some elementscanbe mademore eye-catchingthan others.This againcanbe madein many differentways,through size,through colour contrasts(red is alwaysaverysalientcolour), tonal contrast- in short through anything that can make a given elementstand out from its surroundings.In Figure7.I,for instance,the car and man arethe most salient elementsof the picture becauseof their size,the way the man's faceis foregrounded (displayed)and the way the car'ssleeksmoothnesscontraststo the lesssharplydefined texturedwall in the background. The pupils'different decisionsin the positioning of, and amount of pagegiven to, the visual and written elementsof the text gavedifferent salienceto eachof these

A SOCIAL SEMIOTIC APPROACH

elementswithin their texts.The drawing is most salientin Ramendeep'stext (Figure 7.4) (itcomes first and occupies40 per cent of the page),whereasthe writing is most salientin Amy's text (Figure7.5)(it comesfirst and occupies60 per centof the page). Ramendeep'suseof a yellow-greenin her drawing (despitebeinginstructednot to use colour by the teacher)and her useof shadingaddedto its salience.

Modality Photographsare often thought of as 'imagesof the real', as imagesthat show things exactlyasthey might alsobe seenin reality with the nakedeye(seeChapters3 and4). On the other hand, graphsand diagramscan alsobe thought of astrue to reality, as imagesthat depict the world asit is, objectively,scientifically- and yet they lack all the characteristics that contributeto the photographicimpressionof realrty.They aregeneral where photos are specific,abstractwhere photos are concrete,conventionalwhere photosappearto be an 'imprint of reality'.How canthesetwo modesof representation both claim to be real when they are so very different?They can both claim to be real becausethe claimsare basedon different definitions of reality. Naturalisticmodality (modality = 'realityvalue') definesvisualreality asfollows:the greaterthe congruence betweenwhatyouseeof an objectin an imageandwhatyou canseeofit inrealitywith the nakedeye,in a specificsituation and from a specificangle,the higher the modality of that image.That, at anyrate, is the theory, for the modality of photographsis, in the end, alsobasedon convention,namelyon the conventionsbuilt into the most widely used realisticimage technologies.Modern 35mm photography,not reality, is the contemporarystandardof high naturalisticmodality, and assoonasan imagedisplays more sharpness, more colour saturation,a deeperperspective,than the averagecolour 'surreal', photo, its modality decreases againand it beginsto look'more than real', or 'fantastic' or'ghostly', dependingon the waythis particularmeaningpotentialofwhat Kressand van Leeuwen(1996)call sensorymodality i s actualizedin the specificcontext. Scientificmodality on the other hand is basednot on what things, in a specificsituation and from a specificangle,look like, but on how things are in general,or regularly, or accordingto somedeeper,'hidden'truth. The scientificimageprobesbeyond the surfaceand abstractsfrom detail.Thereoften is no background,detail is simplified or left out, colour and depthregardedassuperfluous.Preciselythose meansof expression which ensurehigh modality from the point of view of naturalismarehereregardedas unreal and irrelevantindicators of low modality. Ramendeep'simage (Figure 7,4) uses the resourcesof scientific modality (for example,the waving lines in the left sectionof the image and the lack of depth of the image) to realizean abstractedaccount of cells.The drawing appearsto be primarily concernedwith the idea of regularityand sameness: a visualsearchfor, and presentationof, scientificnessas a generalizedpattern of meaning. The distinctly different pattern of the air bubble (the circle on the left of the image) and the cells visually marks their difference.As Ramendeeplooked through the microscopein 'It the lesson,she said looks like a brick wall'. This visual analogyis also apparent in her drawing of cells.The analogyfocusedon the positive elementsof regularity

151

152

HANDBOOK OF VISUAT ANALYSIS

and uniformity of cellsand embodiesthe relationshipof the part (the cell or brick) to the whole (the onion or brick wall). A brick wall is a familiar thing in an urban environment and the familiarity implied by the visual analogy comments on its everydayness: cellsare everywhere. A-y on the other hand realizedthe key conventionsof scientificnesstext in the written elementof her work (Figure 7.5). The voice is impersonaland the agent unnamed.Imperativesgivethe readerdirectivesfor re-creatingthe experiment.These imperativeshave a generalizingeffect They convey 'what one should do' to repeat 'what I did'. The effacingof the narrator presentsthe writing asfactualand objective. The absentnarrator servesto absentthe audience:they are nowhere addressedand yet they are assumedto be completelyknown and thereforepresentin everyfacetof the language(Kress,1994).The voice of the writing servesto distanceAmy from the experienceof the experiment (in contrast to her excited exclamationsin the classroom). The visualelement,on the other hand, is more personaland lessscientific.The teacher'ssomewhatdesperatecomment on the drawing ('Did what you sawlook like "my diagram" in any way?')suggeststhat he might agreewith this assessment. AlthoughAmyusessomeofthe resourcesofvisual scientificness, shealsomakes them more personaland affective.The circular frame of the image,for instance,is a convention generallyusedto encodethe experienceof seeingthrough a microscope, making the equipment, the microscope,part of the representation.But the organic flow ofthe linesin her imagesuggestan emphasison 'the experienceoflookitg'rather 'what than on I saw'. Similarly the lack of colour and the flatnessof the imageboth suggestan abstractrepresentation,moving more towardsthe scientificthan towards the naturalisticmodality. But the frame wastentativelydrawn with a compassand so suggesteda tension betweenthe certainty of a mechanisticallyproduced circle and the hesitancyof the maker, andrealizeda sensoryor aestheticmodality focusingon the emotion and affectofthe eventrather than on scientificrealism.ltrcalizedAmy's involvement and excitement.This interestin the sensorymodality wascarriedthrough in Amy's use of the analogy'a wavy weave- in and out of eachother in our microscopes'.The imageis representedbeneaththe title 'what we saw' and the title for the 'looking writing is at cells'.Thesetitles suggestthat the agencyinvolved in the visual experienceof looking at the cellsis differentfrom the agenryinvolvedin making them visible.Through her writing Amy transformedher experienceof doing the experiment into a generalizedsetof actions,whilst through her imagesheassertedher individual experienceof seeingthe cells. Using the socialsemioticapproachto imageanalysisin the contextof this part text-analytical,part ethnographic-likeresearchshowedthat the conceptsof composition, frame and modalitywere usefulin drawing attention to how thesetextsreflectedthe consequences of pupils' differently interesteduse of genreresourcesand differently emergingexpressions of scientificness. It highlightedhowthe pupils' experiences were mediateddifferently through their use of image and writing in the texts:that is, the visual and written elementsof the texts contributed differently to the realizationof 'being scientific';and the experienceof the experimentfor eachpupil. The texts can be explainedby seeingthem asthe product of the different stanceseachpupil adopted to the recordingof their experienceoflearning and to the protocol of 'beingscientific'.

A S O C I A L S E M I O T I CA P P R O A C H

In short, the visual and the written elementsof a text appearedtoattend to different aspectsof meaning,that is, they realizeddifferentfunctions.

C R O S S . C U L T U R A LS T U D I E S U S I N G S O C I A L S E M I O T I C S Kressand van Leeuwen(1996) seetheir work asvalid for the broad domain of contemporaryWestern visual culture, as globally distributed through the hegemonyof Anglo-American massmedia in the world, albeit of course with all kinds of local accents.Our lastexampleis from a studywhich usedvisualsocialsemioticsto explore cultural differencein the realm ofthe advertisingimage,through a comparativeanalysis of British and |apaneseadvertisements(Oyama, 1999).We will focus on one aspect of the study: the cultural meaningsof the left-right, given-new type information structurepositedby Kressand van Leeuwen(1996). Figure7.6representstwo public signsfor'exit' that makeuseof a type ofvisual imagecalledisotype.A systemof isotypeswasdevisedbythe Viennesephilosopher and socialscientistOtto Von Neurath (1937,1948)to conveyinformation to the general public. Von Neurath believedthis systemto be universallycommunicative, asopposed to the opacity of verbal language,which he viewed as 'a disfiguring medium for knowledge'in that'its structure and vocabularyfail to be a consistent,logical model of objectsand relationsin the physicalworld' (Lupton, 1989:145). lry l SF (@;r; # dF

Figure 7.6

British and Japanese 'exit this way'signs.

Both signsin Figure7.6 realizethesamemeaning('exit this way ) but their visual directionalitydiffers.In the British example(first image)the orientationof the human figure is to the right, whereasin the Japanese exampleit is to the left. If, asVon Neurath suggested, the isotypeprovidesuniversalityin the manifestationof visualinformation, the questionarisesasto howthis differencecanbe explained.Clearlythelexicalaspect (content of the visual images:'exit') is independentlyvariable in relation to spatial orientation,and spatialorientation (visualsyntax)creates,implicitly, an indication of visualdirectionalitywhich is culturallydifferent. If the meaningpotentialof the given 'where 'point is,informationally, you comefrom', your of departure',and the meaning 'where 'what potential of the new, you are going to', you must attend to', then it may

1 53

154

HANDBOOK

OF VISUAL ANALYSIS

be that the visualrealizationof theseinformational meaningpotentialsis not the same in the two cultures. The key role of directionality in meaning-makingand its differencein British and Japanese culturescanbe seenin another|apaneseexample,which advertisesmale hairpiecesand appearedin a ]apanesemagazine.Therearetwo photographsabovethe lead copy, both of which show a family of three (who appearto be on holiday; there is a suitcase,on which a little girl is sitting, and the man is carryinga travel bag on his shoulder).Thesetwo photographsindicatethe differencein the man's hair: in the left photographthe man hashair, whereasin the right photographhe is bald.An advertisement for cosmeticsurgeryby a clinic taken from a British women's magazineshows partsofa patient'sbodybeforeand aftercosmeticsurgery.For example,one photograph on the top left showstwo statesof a woman's nose,before and after surgery.Clearly both exampleshavea'before and after' structure,with a problem forming the'given', the point of departurefor the message,and a'new' forming the solution. But in the fapaneseexamplethe given is on the right and the new on the left, whereasin the British example it is the other way around. Clearly, like horizontal directionality, horizontal distribution of spacealso differs in the two cultures,and along the same lines.The )apaneseexamplerealizesgivenin the right and new in the left. The British example realizesgiven in the left and new in the right. Different kinds of visual semioticsoperatein the context of fapaneseand British representations.

CONCLUSIONS Kressand van Leeuwen'smethod of visual analysisprovidesessentiallya descriptive framework. For this reasonit does not, on its own, offer all that is neededfor the sociologicalinterpretationof images.As alreadymentioned,the studyof sexualhealth promotion campaignswe usedas one of our key examplesnot only had to draw on socialtheoriesof genderand masculinitiesand earlierstudiesof genderconstruction in images,but also on methodologies allowing a more systematicanalysisof the similarities,differencesand patternsin the sampleof images(Ritchie and Spencer, 1994).Kressand van Leeuwen's(1996)method (which is simplified here)is alsoquite complex and introducesa greatdeal of new terminology which can appearpedantic to the outsider and requireselaborateexplanationeverytime the method is usedin writings that will be readby audienceswho are not familiar with it. That said,the method is effectivein bringing out hidden meanings.In the case of the sexualhealth study, it brought out that leafletsand posterswhich were meant to communicateto all young people(somespecificallyto young men) werein fact quite stronglyoriented towardsheterosexualnorms. Therewas,for instance,a visual association of analsexwith diseasewhich would not be conduciveto the aim ofthe leaflets to practisesafesex and which appealedto fear and homophobia. The method also revealedgenderrepresentationswhich were not apparentin the written text, such as over-simplificationsof male sexualityand portrayalsof men assexuallylesscomplex than women. Analysingthe action processes, the symbolicattributesand the composition of the imagesthe study found that sexwasportrayed asa skill or techniqueto

A SOCIAL SEMIOTIC APPROACH

155

be acquiredby men, comparableto driving a car or playing a sport, and a physical activity ratherthan an emotional experience.This clearlyreinforcessociety'smessage that women, not men, havesexualproblems,that men arenot sexuallyemotional, and servesto maintain the invisibility of male sexualproblems.The study alsousedsemiotic analysisto show how the imagesfailed to addressyoung men's emotional concerns and responsesto sexualexperiences.Setting,symbolic attributes,composition and framing visually assertedcompetitivenessasa key aspectof heterosexualmale sexuality and the visual display of intimacy betweenmen was restrictedto designatedsocial contextssuch as sports,businessand family groups.Men were not shown in caring roles. Finally, the study showed how the visual polarization of men and women conflicted with the verbal assertionof the need for them to negotiate.The images representedmen in a negativeway and women in a positive (or at leastmorally superior) way. This is unlikely to contribute to the desiredeffectof the campaigns.Roleswere polarized. Men were shown as taking risks, women as protecting; men as sexually knowing, women as innocent; men as simple, women as complex;men as physical, women asemotional;men asirresponsible,women asresponsible.This polarization underminedthe textualmessage of sharedsexualgoalsand emotions,and the positive potentialofthis for the negotiationofsafersex.The imagesin the samplethus reflected the traditional reliance of health promotion materials on women to mediate HIV prevention and sexualhealth messages to young men. The imagesemphasizedthe ability of women to ignoretheir own sexualdesiresand representedheterosexualmale sexuality as more potent than female sexuality,while depicting woman as sexual mediatorsand guardiansof sexualmorality. Clearlythisshowsthat in sexualhealthpromotion materials,scienceeducation and societymore generally,imagesplay arolewhich goesfar beyondthe mereillustration of what is communicatedin language,and imagescan contradict and work against spokenor written messages. If imageanalysiscanbring this out, it can help improve and changepractices(such asthe production of information leaflets,or the teaching and assessment of learning),which can then leadto that third kind of socialsemiotic work we mentioned at the outsetof this chapter- developingnew semioticresources and new usesof existingsemioticresources.

NOTE projectentitled'Therhetoricof the I Thisexampleis drawnfrom anESRC-funded research science classroom: a multimodalapproach'undertaken by C. Jewitt,G. Kress,J. Ogborn andC. Tsatsarelis for theInstituteof Education, Universityof London.

REFERENCES Arnheim,R. ( 1969)VisualThinking.Berkeley andLosAngeles: Universityof CaliforniaPress. Arnheim,R. ( 1974)Art andVisualPerception. Berkeley: Universityof CaliforniaPress. Arnheim,R.(1982) ThePoweroftheCenter.BerkeleyandLosAngeles:UniversityofCalifornia Press.

156

H A N D B o o Ko F v t s u A L A N A L y s t s

Goffman, E. (1979) GenderAdvertisements. London: Macmillan. Graham,H. (1977)'Imagesof pregnanryin antenatalliterature', in R. Dingwall, C. Heath and M. Ried (eds),Health Careand Health Knowledge.London: Croom Helm. Halliday, M.A.K. (1978) LanguageasSocialSemiotic.London: EdwardArnold. Halliday, M.A.K. ( 1985)An Introduction to FunctionalGrammar.London: Edward Arnold. Holland, J., Ramazanoglu,C. and Sharpe,S. (1993) Wimp or Gladiator:Contradictionsin AcquiringMasculineSexuality.London: Tufnel Press. ResearchOnline,2 (2). Jewitt,C. (1997)'Imagesof men', Sociological 'Howto produce inclusiveresources',in G. Lenderyouand C. Ray (eds),Let's Jewitt,C. (1998) Hear It for theBoys!London:National Children's Bureau. 'A Iewitt, C. ( 1999) socialsemiotic analysisof male heterosexualityin sexualhealth resources: the caseof images',InternationalJournalof SocialResearchMethodology: TheoryandPractice, I (4):263-80. Kress,G. (1994)LearningtoWrite,2nd edn. London:Routledge. Kress,G. and van Leeuwen,T. (1996)Readinglmages:The Grammar of VisualDesign.London: Routledge. Kress,G., Jewitt,C., Ogborn,I. and Tsatsarelis, C. (in press)MultimodalTeachingandLearning TheRhetoricsof the ScienceClassroom. London: Continuum. Lupton, E. (1989) 'Readingisotype', in M. Victor (ed,.),DesignDiscourse- History /Theory/ Criticism.Chicago:University of ChicagoPress. Millum, T. (1975) ImagesOf Women,AdvertisinglnWomen'sMagazines.London: Chatto. 'Visual Oyama,R. (1999) semioticsin a cross-cutluralperspective:a study of visual imagesin and selected British advertisements',PhD dissertation,University of London. |apenese Panofsky,E. (1953) Early NetherlandishPainting.NewYork: Harper and Row. Ritchie,f. and Spencer,L. (1994)'Qualitative data analysisfor applied policy research',in A. Bryman and R. Burgess(eds),AnalyzingQualitativeData.London: Routledge. van Leeuwen,T. (1999) Speech, Music, Sound.London:Macmillan. Von Neurath, O. (1937) International Picture Language.London: London Basic Publishing Company. Von Neurath, O. ( 1948)Basicby Isotype.London: London BasicPublishingCompany. Wight, D. (1992)'Boys' thoughts and talk about sex in a working classlocality of Glasgow', unpublishedpaper,MRCMSU, Glasgow.

H ffiw *

Kff

Practices of SeeingVisualAnalysis: Approach an Ethnomethodological CHARLES GOODWIN

INTRODUCTION A primordial sitefor the analysisof human language,cognition and action consistsof a situationin which multiple participantsareattemptingto carryout coursesof action togetherwhile attendingto eachother, the larger activitiesthat their current actions areembeddedwithin and relevantphenomenain their surround.Vision canbe central to this process.rThe visiblebodiesofparticipants provide systematic,changingdisplays about relevantaction and orientation. Seeablestructurein the environment can not only constitute a locus for sharedvisual attention, but can also contribute crucial semioticresourcesfor the organizationof current action (consider,for example,the use of graphsand chartsin a scientificdiscussion).For the past thirtyyears both conversation analysisand ethnomethodologyhaveprovided extensiveanalysisof how human vision is sociallyorganized.Both fieldsinvestigatethe practicesthat participantsuse to build and shapein concertwith eachother the structuredeventsthat constitutethe lifeworld of a community of actors.Phenomenainvestigatedin which vision playsa central role rangefrom sequencesof talk, through medical and legalencounters,to scientificknowledge. The approach taken by both ethnomethodologyand conversationanalysis to the studyofvisual phenomenais quite distinctive.At leastsinceSaussureproposed studying langueas an analyticallydistinct sub-field of a more encompassingscience of signs,different kinds of semiotic phenomena (language,visual signs,etc.) have typically been analysedin isolation from each other. However in the work to be describedhere neither vision, nor the imagesor other phenomenathat participants look at, are treated as coherent, self-containeddomains that can be subjectedto analysisin their own terms. Insteadit quickly becomesapparentthat visual phenomena can only be investigatedby taking into account a diverse set of semiotic resourcesand meaning-makingpracticesthat participantsdeploy to build the social worlds that they inhabit and constitute through ongoing processesof action (see Chapter 9). Many of these,suchasstructure provided by current talk, are not in any sensevisual,but the visiblephenomenathat the participantsare attendingto cannot be properly analysedwithout them. The focus of analysisis not thus representations or vision per se,but insteadthe part playedby visual phenomenain the production of meaningful action.

158

HANDBOOK

OF VISUAL ANALYSIS

Both the methodologyand the forms of analysisusedin this approachcanbest be demonstratedthrough specificexamples.

GAZE BETWEEN SPEAKERS AND HEARERS In formulating the distinction between competenceand performance, Chomsky ( 1965:34) arguedthat actualspeechis sofull of performanceerrors,suchassentence fragments,restartsand pauses,that both linguists and partiesfacedwith the task of acquiringa languageshouldignoreit. Investigatinga corpusofconversationrecorded onvideo,C. Goodwin (1980,1981:Chapter2)indeedfoundpreciselythe'falsestarts' 'changes and of plan in mid-course'that Chomskydescribes.In the following, instead of producing an unbroken grammaticalsentence,the speakersays:2 Cathy:En a coupleof girls- Oneothergirl from there, However,asshown in Figure 8.1,when the video is examinedit is found that the restartoccursat a specificplace:preciselyat the point wherethe speakerbrings her gazetoher addressee and finds that her addressee is looking elsewhere.Moreover,the restart actsas a requestfor the hearer'sgaze.Thus immediately after the restartthe hearerstartsto move her gazeto the speaker.Paradoxically,if the speakerhad not produced a restartat this point shecould have said somethingthat would appearto be an unbroken grammatical unit if one examined only the stream of speech(for 'En example, a coupleof girls from there . . .'), but which would in factbe interactively a sentencefragment sinceher addressee attendedto only part of it.

Restart

II

Y Pam:En a couple of girls - One other girl from there,

EN

A t t

l I

A

l

l

'

GazeArrives

I Hearer Starts MovingGaze to Speaker

Figure 8.1

Gaze between speakers and hearers: transcript of Pam and Ann.

AN ETHNOMETHODOTOGICAL APPROACH

The identitiesof speakerand hearerarethe most genericparticipantcategories relevantto the production of a strip of talk. The phenomenaexaminedhere (which occur pervasivelyin conversation)provide evidencethat the work of being a hearer in face-to-faceinteractionrequiressituateduseof the body, andgazein particular, as a way ofvisibly displayingto othersthe focusof one'sorientation.Moreover speakers not only usetheir own gazeto seerelevantaction in the body of a silent hearer,but activelychangethe structureof their emergingtalk in terms of what they see. What relevancedo processessuch as thesehave to the other issueraisedby Chomsky( 1965:3), that of determining'from the dataof performancethe underlying systemof rules masteredby the speaker-hearer'?Many repairsinvolve the repetition, with somesignificantchange,of somethingsaidelsewherein the utterance: We went- I wentto If hecould-Ifyou could Such repetition has the effect of delineatingthe boundariesand structure of many different units in the streamof speech.Thus, by analysingwhat is the sameand what is different in theseexamplesone is able to discoverthe following: first, where the streamof speechcanbe divided into significantsub-units;second,that alternativesare possiblein a particular slot; third, what someof thesealternativesare (heredifferent pronouns); and fourth, that these alternativescontrast with each other in some significantfashion,or elsethe repair would not be warranted.Repairsin other examples not only delineatebasicunits in the streamof speech(noun phrases,for example),but alsodemonstratethe different forms such units can take and the tfpes of operations that canbe performedupon them (seeGoodwin, 1981:170-3).Repairsfurther require that a listener learn to recognize that not all of the sequenceswithin the stream of within the language,for examplethat'I' doesnot follow speecharepossiblesequences 'to' in 'We went t- I went to . . .' . In order to dealwith such a repair a heareris thus required to make one of the most basicdistinctions posedfor anyoneattempting to decipherthe structure of a language:to differentiatewhat are and are not possible sequencesin the language,that is betweengrammaticaland ungrammaticalstructures. The fact that this task is posed may be crucial to any learning process.If the party attempting to learn the languagedid not haveto dealwith ungrammaticalpossibilities, shemight not havethe if for exampleshewasexposedto onlywell-formed sentences, data necessaryto determine the boundaries, or even the structure of the system. Chomsky'sargumentthat the repairsfound in natural speechso flaw it that a child is facedwith data ofvery'degeneratequality' doesnot appearwarranted.Ratherit might be arguedthat if a child grew up in an idealworld where sheheard onlywell-formed shewould lackthe herselfbecause sentences shewould not learnto producesentences analysisof their structureprovided by eventssuchasrepair. Crucial to this processis the way in which visualphenomena,suchasdispreferredgazestates,canboth leadto repair and demonstratethat the participants are in fact attending in fine detail to what might appearto be quite ephemeralstructurein the streamof speech. Whathas justbeen describedprovidesone exampleof the methodologiesand forms of analysisusedto investigatevisualphenomenawithin conversationanalysis.

159

160

HANDBOOK

OF VISUAL ANALYSTS

Severalobservationscan be made. First, the focus of analysisis not visual events in isolation, but insteadthe systematicpracticesusedby participantsin interaction to achievecoursesof collaborativeaction with eachother - in the presentcasethe interactiveconstruction of turns at talk and the utterancesthat emergewith those turns. Visual events,suchas gaze,playa centralrole in this processbut their senseand relevanceis establishedthrough their embeddedness in other meaning-makingtasks and practices,suchasthe production of a strip of talk that is in factheardand attended to by its addressee. This links vision to a host of other phenomena,including language and the visiblebody, asan unfolding locus for the displayof meaningand action. Second,what the analystseeksto do is not to provide his or her own glosson how visual phenomenamight be meaningful,but insteadto demonstratehow the participants themselvesnot only activelyorient to particular kinds of visual events(suchas statesof gaze),butusethem asa constitutivefeatureof the activitiestheyare engaged in (for example,by modif.ing their talk in terms of what they demonstrablysee). Third, in addition to the spatial dimension that is naturally associatedwith vision, theseprocesses alsohavean intrinsic temporal dimension aschangesin visual eventsare marked by, and lead to, ongoing changesin the organizationof emerging action.If onehad only a staticsnapshot,or measuredonly a singlestructuralpossibility, suchasmutual gaze,insteadof looking at the temporallyunfolding interplayof different combinationsof particip ant gaze,the type of analysisbeing pursuedhere would be impossible. Fourth, such analysisrequiresdata of a particular type, specificallya record that maintainsasmuch information aspossibleabout the setting,embodieddisplays and spatial organizationof all relevantparticipants,their talk, and how eventschange through time. In practiceno record is completelyadequate.Every cameraposition excludesother views of what is happening.The choiceof where to placethe camera is but the first in a long seriesof crucial analyticaldecisions(seeChapter 9). Despite theselimitations a video or film record doesconstitutea relevantdata source,something that can be worked with in an imperfectworld (seeChapter3). Fifth, crucial problems of transcription are posed.The task of translatingthe situated,embodiedpracticesusedbyparticipants in interaction to organizephenomena relevantto vision posesenormoustheoreticaland methodologicalproblems.Our ability to transcribetalk is built upon a processof analysingrelevantstructurein the streamof speechand marking those distinctionswith written s)rmbols,that extends back thousandsof yearsand is still being modified today (for example,the system developedby Gail Iefferson(Sackset al., 1974:731-3)for transcribingthe texture of talk-in-interaction,including phenomena,such as momentary restartsand sound stretches,that are crucial for the analysisbeing reported here).When it comesto the transcription ofvisual phenomenawe are atthe verybeginningof sucha process.The arrowsand other symbolsI've usedto mark gazeona transcript (seeGoodwin, 1981) captureonly a smallpart of a largercomplexconstitutedby bodiesinteractingtogether in a relevant setting.The decision to describe gazein terms of the speaker-hearer framework is itselfa major analyticone,and by no meanssimple,neutral description. Moreover a gazinghead is embeddedwithin a larger postural configuration, and indeeddifferent parts of the body can simultaneouslydisplayorientation to different

APPROACH AN ETHNOMETHODOLOGICAL

'participation participantsor regions(seeKendon, 1990b;Scheglofl 1998), creating frameworks'of considerablecomplexity.Thus on occasiona transcriberwants some way of indicating on the printed pageposture and alignment. In addition, not only the bodiesof the participants,but alsophenomenain their surround, can be crucial to the organization of their action. To try to make the phenomena I'm analysing independentlyaccessible to the readerso that sheor he can evaluatemy analysis,I've experimentedwith using transcription symbols,frame grabs,diagramsand movies embeddedin electronicversionsof papers.Multiple issuesareinvolvedand no method is entirely successful.On the one hand the analystneedsmaterialsthat maintain as much of the original structureof the eventsbeing analysedaspossible,and which can be easilyand repetitivelyreplayed.On the other hand,just asa raw taperecordingdoes not display the analysisof segmentalstructure in the stream of speechprovided by transcription with a phonetic or alphabeticwriting system,in itself a video, evenone that can be embeddedwithin a paper, does not provide an analysisof how visible eventsare being parsedby participants. The complexity of the phenomena involved requires multiple methods for rendering relevant distinctions (e.9. accurate transcriptionof speech,gazenotation,frame grabsand diagrams;seealsoOchs,1979). Moreover,like the two-facedRoman god fanus,anytranscriptionsystemmust attend simultaneouslyto two separatefields,looking in one direction at how to accurately recoverthrough a systematicnotation the endogenousstructure of the eventsbeing ofthe investigated,while simultaneouslykeepinganothereyeon the addressee/reader analysisby attempting to presentrelevantdescriptionsasclearlyand vividly aspossible. In many casesdifferent stagesof analysisand presentation will require multiple transcriptions. There is a recursive interplay between analysis and methods of description. Work in conversationanalysishasprovided extensivestudy of how the gazeof participantstoward eachother is consequentialfor the organizalionof action within change talk-in-interaction.Phenomenainvestigatedincludethe wayin which speakers the structureof an emergingutterance,and the sentencebeing constructedwithin it, asgazeis moved from one tFpeof recipientto another,sothat the utterancemaintains of the moment (Goodwin, 1979,1981);how its appropriatenessfor its addressee of what in terms of their hearer'svisible assessment speakersmodif,r descriptions is being said (M.H. Goodwin, 1980);how genressuchasstoriesare constructednot by a speakeralone,but insteadthrough the differentiatedvisible displaysof a range of structurally different kinds of recipients (speaker,primary addressee,principal character,etc.- seeGoodwin, 1984);the organizationof gazeand co-participationin medicalencounters(Heath, 1986;Robinson,1998);the interactiveorganizationof (Goodwinand Goodwin, I9S7);gesture(Goodwin,2000;Streeck,1993, assessments (Goodwinand Goodwin, 1986); 1994);theuseof gazeinactivitiessuchasword searches and soon. Though not strictlylodgedwithinconversationanalysistheworkof Kendon (1990a,1994,1997)on both the interactiveorganizationofbodiesasthey frame states of talk and on gestureis central to the study of visible behaviour in interaction. Haviland (1993)providesimportant analysisof the interactiveorganizationofgesture within narration (for extensiveanalysisof gesturefrom a psychologicalperspective, seeMcNeill, 1992).

1 6 1

16.2

HANDBOOK

OF VISUAL ANALYSIS

SCIENTIFICIMAGES The visible, gazingbody,and the orientation ofparticipantstoward eachother asthey co-produce states of talk, is central to the work in conversational analysisjust examined.By way of contrastmuch work within ethnomethodologyhasfocusednot on the bodiesof actors,but insteadon the images,diagrams,graphsand other visual practicesusedby scientiststo construct the crucial visual working environmentsof their disciplines.As noted by Lynch and Woolgar: Manifestly, whatscientists laboriously piecetogether, pickup in theirhands,measure, showto oneanother,argueabout,andcirculateto othersin theircommunities arenot 'naturalobjects'independent of culturalprocesses andliteraryforms. Theyareextracts, 'tissuecultures,' andresidues impressed withingraphicmatrices; ordered,shaped, and filteredsamples; carefullyalignedphotographic tracesandchartrecordings; andverbal accounts. Thesearetheproximal'things'takeninto the laboratoryandcirculatedin print andtheyarea rich repositoryof 'social'actions.(1990:5) Despiteimportant differencesin subject-matterand methodology,both fields emphasizethe importanceof focusingnot on representations or other visualphenomena asself-containedentitiesin their own right, but insteadon how they are constructed, attendedto and usedby participantsascomponentsof the endogenousactivitiesthat make up the lifeworld of a setting. Thus, in introducing their important volume Representation in ScientificPractice,Lynchand Woolgar definetheir enquiryasfollows: 'Instead of asking "what do we mean, in various contexts,by 'representation'?"the studiesbeginby asking,"What do the participants,in thiscase,treatasrepresentation?"' (1990: 11). Note that what must be investigatedis specifiedboth in terms of the orientation of the participantsand with respectto the featuresof the relevantlocal setting (for example,'in this case').This leadsto a distinctive ethnomethodological perspectiveon reflexivitf "'Reflexivity" in this usagemeans,not self-referentialnor reflectiveawareness ofrepresentationalpractice,but the inseparabilityofa "theory" of representationfrom the heterogeneoussocialcontextsin which representationsare composedand used' (Lynch and Woolgar,1990: 12). In a classicarticle Lynch formulatesthe task of analysingscientificrepresentations asthat of describingthe publicly visible 'externalizedretina' that is the sitefor the practicesimplicated in the socialconstitution of the objectsthat are the focus of scientificwork: Thisstudyis basedon the premisethatvisualdisplays aremorethana simplematter of supplyingpictorialillustrationsfor scientifictexts.Theyareessential to howscientific objectsandorderlyrelationships arerevealed andmadeanalyzable. To appreciate this, we first needto wrestthe ideaof representation from an individualisticcognitive foundation,andto replace a preoccupation with imageson theretina(or alternatively 'mentalimages'or'pictorial ideas')with a focuson the'externahzed, retina'ofthe graphicand instrumentalfieldsupon which the scientificimageis impressedand ( 1990:153-4) circulated. l/sing asdataimagesfrom scientificjournal articlesand books,Lynch describes two families of practicesused to constitute the visible scientific object: 'selection'

A N E T H N O M E T H O D O L O G I C AALP P R O A C H

and'mathematization'. Selection,illustrated through double images in which a photograph and a diagramof entitiesvisible in the photograph arepresentedsideby side,is describedasa host ofpracticesthat iterativelytransformone imageof an entity into another (for example,the photograph to the diagram) while simultaneously structuring and shapingwhat it is that is being represented.Crucial to this process practices,including filtering, uniforming, is the fact that different selective/shaping upgrading and defining, can be repetitivelyapplied creatingnot just a singleimage, but a linked, directionalchain of representations. Indeedmuch of the work of actually doing scienceconsistsin building and shaping what Latour (1986) has called 'Mathematization' inscriptionsin this fashion (seealso Latour and Woolgar,1979). refersnot simply to the use of numbers,but insteadto the host of practicesusedto transform recalcitranteventsinto mathematicallytractablevisual and graphic displays, for example graphs,charts and diagrams.Thus an image showing a map of lizard territories is assembledthrough, among other operations,driving stakesinto the lizards' environment to createa grid for measurement(and thus injecting a scientificallyrelevantCartesianspaceinto theveryhabitatbeing studied),repetitivelycapturing lizards,distinguishingthem from eachother by cutting off a different pattern of toes on eachltzard,recordingeachcaptureon a papermap of the stakedout territorf ,and finally drawinglinesaround collectionsof points to createthe map.As notedby Lynch 'is the product of thesepractices,for examplethe publishedmap, a hybrid objectthat is demonstrablymathematical,natural and literary' (tggO: 171).Note how in all of thesecasesthe focusof analysisis on the contextuallybasedpracticesofthe participants who are assemblingand usingtheseimagesto accomplishthe workthat definestheir profession. Though emergingfrom psychologicalanthropology,rather than ethnomethodology, Hutchins's (1995) ground-breakingstudy of the cognitivepracticesrequired to navigatea ship outlines a major perspectivefor the analysisof both imagesand seeingasforms of work-relevantpractice.Hutchins demonstrateshow the practices required to navigatea ship arenot situatedwithin the mental life of a singleindividual, visualtools, but are insteadembeddedwithin a distributed systemthat encompasses suchasmapsand instruments for juxtaposinga landmark and compass-bearingwithin the samevisual field and actors in structurally different positions who usealternative tools and, in part becauseof this, perform different kinds of cognitive operations, many of which havea strongvisualcomponent (e.g.locatinglandmarksand plotting positionson a map).

IMAGES IN INTERACTION All of the work discussedso far takesas its point of departurefor the investigation ofvisual phenomenathe taskof describingand analysingthe practicesusedbyparticipants to construct the actions and eventsthat make up their lifeworld. Ratherthan standingaloneasa self-containedanalyticdomain, visualphenomenaareconstituted and made meaningful through the way in which they are embeddedwithin this larger set of practices.However, within this common focus, two quite different orders of

I 63

164

HANDBOOK OF VISUAL ANALYSIS

visual practicehavebeen examined.Researchin sciencestudieshas investigatedthe imagesproducedby scientists,and the way in which theyvisuallyand mathematically structure the world that is the focus of their inquiry, without howeverlooking in much detail at how scientistsattend to eachother asliving, meaningfulbodies,or structure what they are seeingthrough the organization of talk-in-interaction. By way of contrast,studiesof the interactive organizationof vision in conversationlooked in considerabledetail at how participantstreat the visualdisplaysof eachother'sbodies asconsequential,and how this is relevantto the moment-by-moment production of talk, but did not focusmuch analysison imagesin the environment.Clearlyall of the phenomenanoted - the visible body, participation, gesture,the details of talk and languageuse,visualstructurein the surround,images,mapsand otherrepresentational practices,the public organizationofvisual practicewithin the worklife of aprofession, and so on - are relevant.The question arisesas to whether it is possibleto analyse suchdisparatephenomenawithin a coherentanalyticframework. Beforeturning to studiesthat haveprobed suchquestionsseveralissuesmust be noted. First, it is clearlynot the casethat the only acceptableanalysisis one that includesthis full rangeof all possiblevisual phenomena.Both participantsand the structuresthat provide organization for action and eventsuse visible phenomena selectively.Partiesspeakingover the telephonecanseeneithereachother'sbodiesnor eventsin a common surround. A scientificjournal can be read in the absenceof the partieswho constructedits text and diagrams.More interestinglywithin face-to-face interaction participants can continuously shift between actions that invoke, and perhapsrequire, gazetowards specificeventsin the surround and those that make relevantgazetowardsno more than eachother'sbodies.Evenin this more limited case theremay be a real issueasto whetherit is relevantto attendto everythingthat a body does;for example,somegesturesmadeby a speakermay not require gazetowardthem from an addressee. Thereis thus an essentialcontingency,not only for the analystbut more crucially for the participantsthemselves,as to what sub-setof possiblevisual eventsarein fact relevantto the organizationof the actionsofthe moment. Moreover, this meansthat in addition to investigatinghow differentkinds ofvisible phenomena are organized,the analystmust also take into account how participants show each other what kinds of eventsthey are expectedto take into account at a particular moment, for exampleto indicatethat a participant, gestureor entity in the surround should be gazedat. There is thus not only communication through vision, but also ongoingcommunicationaboutrelevantvision (Goodwin,1981,1986,in preparation; Streeck,1988). Second,visual eventsare quite heterogeneous,not only in what they make visible, but more crucially in their structure. Consider for examplethe issueof temporality. Both gesturesand the displaysof postural orientation used to build participation frameworks are performed by the body within interaction. However, while gestures,like the bits of talk they accompany, are typically brief (for example, they frequently fall within the scopeof a singleutterance)and displaysemanticcontent relevantto the topic of the moment, participation displaysframe extendedstrips of talk and typicallyprovide information about the participants'orientation ratherthan the specificsof what is being discussed.Bodily displayswith one kind of temporal

A N E T H N O M E T H O D O L O G I C AALP P R O A C H

duration (andinformation content) arethus embeddedwithin anotherclassofvisual displaysbeing made by the bodywhich havea quite different structure. Third, the structureof visualsigns,including their possibilitiesfor propagation through spaceand time, can be intimately tied to the medium used to construct sonnetsfocuseson the contrast between them. A major theme of Shakespeare's the temporally constrainedhuman body, condemned to inevitable decay,and the (limited) possibilities for transcending such corruption provided by language inscribedon the printed pagewhich can remain fresh and alive long after its author and subjecthave passedinto dust. This contrastbetweenthe temporal possibilities provided by alternativemedia (e.g.the body and documents)constitutesan ongoing resourcefor participantsin vernacularsettingsastheybuild, through interactionwith eachother, the eventsthat make up their lifeworld. In addition to the displaysmade by a fleetinggestureor local participation framework,participantsalsohaveaccessto imagesand documentswhich can encompassmultiple interactionsand quite diverse settings.This arisesin part from the specificmedia usedto constitutethe signsthey contain. Rather than being lodged within an ever-changinghuman body, such documentsconstitutewhat Latour has calledimmutable mobiles,portable material objects that can carry stable inscriptions of various types from place to place and through time ( 1987: 223). However, despitethe way in which crucial aspectsof the structure of images and documentsremainconstantin differentenvironments,theyarenot self-contained of interaction and visualartefactsthat canbe analysedin isolation from the processes work practicesthrough which they aremaderelevantand meaningful.The sameimage or document can be construed in quite different ways in alternative settings.For example,a schedulelisting all arriving and departing flights wasa major tool for almost all work groups at the airport studiedby the Xerox PARC workplaceproject (BrunCottan et al., l99I; Goodwin and Goodwin, 1996;Suchman,1992),and indeed it linked diverseworkers throughout North America into a common web of activity. Howeverwhile baggageloaderscarefullystructuredtheir work to anticipatearriving flights,sothatplanescouldbe speedilyunloaded,thesesamearrivaltimeswerealmost ignoredby gateagentslooking at the sameschedule,but concernedwith the departure of passengers. Eachwork group highlightedthe common document in waysrelevant to the specificwork tasksit faced. Similarly, on the oceanographicship reported in Goodwin (1995), a map showingwhere sampleswould be taken in the Atlantic at the mouth of the Amazon was a major document at all stagesof the researchproject. Beforethe ship sailedthe placeswheresamplescould be takenwerethe focusof intensepolitical debatebetween different groupsof scientistsand the Brazilianand American governments.After the project was completedthe map provided an infrastructurefor graphic displaysthat could be usedin publishedjournal articlesto showwhat the scientistshad found about how the watersof the Amazon and the Atlantic interactedwith eachother, that is, a wayof makingvisiblerelevantscientificphenomena.Duringthevoyage itselfthe map not only provided a common framework for the quite differentwork of variousteams of scientistsand the crew navigating the ship, but could also be looked at by lab technicians,not able to go to bed for daysat a time becauseof the map's incessant

1 65

166

HANDBOOK

OF VISUAL ANALYSIS

samplingdemands,to locateplaceswherestationswerefar apartand restwaspossible. In brief, though the material form of imagesand documentsgivesthem an extended temporal scope,and the ability to travel from settingto setting,they cannot be analysed asself-containedfieldsof visuallyorganizedmeaning,but insteadstandin a reflexive relationship to the settingsand processesof embodied human interaction through which they are constituted as meaningful entities.To explicatesuch eventsanalysis must dealsimultaneouslywiththe quite differentstructureand temporalorganization of both local embodiedpracticeand enduring graphic displays. Finally, the visual (and other) propertiesof settingsstructure environments that shape,on an historicaltime-scale,the activitiessystematicallyperformed within thosesettings.A very simple exampleis provided by the bridge of the oceanographic ship which not only had a window facing forward so the helmsmancould steerthe ship and watch for trouble, but alsoa window facingbackwards.This wasusedby a winch operatorwho had the taskof lifting heavyinstrument packagesin and out of the sea. Though beingusedhereto do science,this arrangementis in facta systematicsolution to a repetitive problem facedby sailors,such as fishermen using nets, who have to manoeuvreheavyobjectswhile at sea.Solutionsfound to thesetasks,suchasthe rearfacing window with the visual accessit provides (as well as the forward window facilitatingnavigation),arebuilt into the tools that constitutethe work environments usedbysubsequent actorsfacedwithsimilartasla.SeeHutchins (1995)for illuminating analysisof this process,including tools that visuallystructurecomplex mathematical calculations,as well as maps. Both work environments and many of the tools used within them (computer displays,etc.) structurein quite specificwaysthe embodied visual practicesof thosewho inhabit suchsettings. In an attemptto cometo termswith suchissuesGoodwin (2000)hasproposed that imagesin interaction arelodgedwithin endogenousactivity systemsconstituted through the ongoing, changing deployment of multiple 'semiotic fields' which mutually elaborateeachother. The term semioticfield is intended to focus on signsin-their-media, that is the way in which what is typically been attendedto are sign phenomenaofvarioustypes(gestures, maps,displaysofbodilyorientation, etc.)which havevariablestructuralpropertiesthat arisein part from the different kinds of materials used to make them visible (for example,the body, talk, documents,etc.). Bringing signslodgedwithin differentfieldsinto a relationshipofmutual elaborationproduces locally relevantmeaningand action that could not be accomplishedby one sign system alone.Considerfor examplea placeon a map indicatedby a pointing finger which is being construed in a specificfashion by the accompanyingtalk. Not the map as a whole,that is a self-sufficientrepresentation,nor the pointing finger in isolationfrom (a) its target (the spot on the map) and (b) the construalbeing provided by the talk, nor the talk alone, would be sufficient to constitute the action made visible by the conjoined use of the three semiotic fields, each of which provides resourcesfor specifting how to relevantlyseeand understandthe others (seethe brief discussion of the RodneyKing data below for a specificexample;seeGoodwin (in preparation) for more detailedanalysisof pointing). The particular sub-setof semiotic fields availablein a settingthat participantsorient to asrelevantto the construction of the actionsofthe moment constitutesa'contextualconfiguration'.As interactionunfolds

A N E T H N O M E T H O D O L O G I C AALP P R O A C H

contextualconfigurationscanchangeasnew fieldsareaddedto, or dropped from, the specificmix being usedto constitutethe eventsof the moment. Thus, as contextual configurations change there is both unfolding public semiotic structure and contingency(and,indeed,in somecircumstancesactionscanmisfirewhen addressees fail to take into accounta relevantsemioticfield, such asthe sequentialorganization provided by a prior unheardutterance- seeGoodwin (in preparation) for an example).

P R O F E S S I O N A LV I S I O N Work settingsprovide one environment in which the interplay between situated, embodiedinteractionandthe useofvisualimagesofdifferenttypes canbe systematically investigated.In many work settingsparticipants face the task of classifyingvisual phenomenain a way that is relevantto the work they are chargedwith performing. Frequently they must also construct different kinds of representationsof visual structure in the environment that is the focus of their professionalscrutiny. I \ rill now briefly examine how such vision is socially organized in two tasks faced by archaeologists: first, colour classificationand, second,map-making.Then I will look at how suchprofessionalvision wasboth constructedand contestedin the trial of four policemen chargedwith beating an African-American motorist, Mr Rodney King, where the key evidencewas a videotapeof the beating.

Colourclassification as historically structuredprofessional practice As part of the work involved in excavatinga site,archaeologists make maps showing relevant structure in the layers of dirt they uncover. In addition to artefacts,such as stonetools,archaeologists arealsointerestedinfeatlrres,suchasthe remainsof an old hearth or the outlines of the poststhat held up a building. Suchfeaturesare typically visible ascolour differencesin the dirt being examined(for example,the remainsof a cooking fire will be blackerthan the surrounding soil,while the holesusedfor posts thus will alsohavea different colour from the soil around them). Field archaeologists face the task of systematicallyclassifring the colour of the dirt they are excavating. The methodsthey useto accomplishthis task constitutea form of professionalvisual practice. As demonstrated by the discussion of Lynch's analysis of scientific representation,and the brief descriptionof the oceanographers, crucialwork in many different occupationstakesthe form of classi$'ingand constructing visual phenomena in ways that help shapethe objects of knowledge that are the focus of the work of a profession (for example, architects,sailors plotting courseson charts, air traffic controllersand professorsmaking graphsand overheadsfor talks and classes).Such professionalvision constitutesa perspicuoussitefor systematicstudyofhow different kinds of phenomenaintersectto organizea community's practicesof seeing. codethe colour of the dirt Goodwin (1996,2000)describeshow archaeologists theyare excavatingthrough useofa Munsell chart.Figure 8.2showstwo archaeologists

1 67

168

HANDBOOK OF VISUAL ANALYSIS

performing this task:the Munsell pagethat they areusingand the coding form where theywill record their classification.Within this sceneare a number of differentkinds of phenomenarelevant to the organization of visual practice,including tools that structure the processof seeingand classificationand documents that organize cognition and interactionin the current settingwhile linking theseprocesses to larger activitiesand other settings.Thesearchaeologistsare intently examining the colour of a tiny sampleof dirt becausethey havebeen given a coding form to fill out. That form ties their work at this site to arangeof other settings,suchasthe officesand lab of the seniorinvestigator,where the form being filled in here will eventuallybecome part of the permanent record of the excavationand a component of subsequent analysis.The multivocality of this form, the way in which it displayson a singlesurface the actions of multiple actors in structurally different positions, is shown visually in vivid fashionby the contrastbetweenthe printed coding categoriesand the handwritten entriesof the field workers.

WMffiffiffi ffiffiWffiffi

Figure 8.2 Transcript of archaeologists coding and recording the colour of the dirt they are excavating through use of a Munsell chart.

A N E T H N O M E T H O D O L O G I C AALP P R O A C H

The use of a coding form such as this to organizethe perception of nature, eventsor peoplewithin the discourseof a professioncarrieswith it an arrayofperceptual and cognitiveoperationsthat havefar-reachingimpact. Coding schemesdistributed on forms allow a senior investigator to inscribe his or her perceptual distinctions into the work practicesof the technicianswho code the data.By using such a system Of all the possibleways a worker viewsthe world from the perspectiveit establishes. that the earth could be looked at, the perceptualwork of field workersusingthis form is focusedon determining the exactcolour of a minute sampleof dirt. They engage in activecognitive work, but the parametersof that work have been establishedby the classificationsystemthat is organizing their perception. Insofar as the coding schemeestablishesan orientation toward the world, a work-relevantway of seeing, it constitutesa structure of intentionality whoseproper locus is not the isolated, Cartesianmind, but a much larger organizationalsystem,one that is characteristically mediatedthrough mundane bureaucraticdocumentssuchasthis form. Ratherthan standingaloneasself-explicatingtextualobjects,forms areembedded within websof sociallyorganized,situatedpractices.In order to make an entryin the slot provided for colour an archaeologistmust make use of another tool, the set of standardcolour samplesprovided by a Munsell chart. This chart incorporatesinto a portable physical object the results of a long history of scientific investigation of the propertiesof colour. containsnot just one but The Munsell chart being usedby the archaeologists point in the colour spaceit each three different kinds of sign systemsfor describing provides: in a grid to demonstratethe A setof carefullycontrolled coloursamplesarranged changesthat result from systematicvariation of the variablesof Hue, Chroma and Value usedto defineeachcolour (eachpagedisplaysan orderedsetofValue and Chroma variablesfor a singlehue). Numeric co-ordinatesforeachrow and column; their intersectionspecifieseach squareasa pair of numbers (suchas416 onthe 10YRHue page). 'dark yellowish brown' (thesenamesare on the Standardcolournamessuch as left-facingpagewhich is not reproducedhere).Moreover thesesystemsare not preciselyequivalentto eachother. For example,severalcolour squarescan fall within the scopeof a singlename.

Why doesthe Munsell pagecontain multiple, overlappingrepresentationsof what is apparentlythe samevisual entity (for example,a particular choicewithin a The answerseemsto lie in the waythat eachrepresentlargersetof colour categories)? ation asa semioticfield with its own distinctivepropertiesmakespossiblealternative operationsand actions,and thus fits into different kinds of activities.Both the names and numbered grid co-ordinatescan be written, and thus easilytransported from the actualexcavationto the other work sites,such aslaboratoriesand journals, that constitutearchaeologyasa profession.The numbersprovide the most precisedescription and do not require translationfrom languageto language.Howeverlocating the colour indexedby the co-ordinatesrequiresthat the classificationbe read with a Munsell book at hand. By way of contrastthe colour namescan be graspedin a way

1 69

170

HANDBOOK

OF VISUAL ANALYSIS

that is adequatefor most practicalpurposesbyanycompetent speakerof the language usedto write the report. The outcome of the activity of colour classificationinitiated by the empfFsquareon the coding form is thus a setof portablelinguistic objectsthat can easilybe incorporated into the unfolding chains of inscription that lead step by stepfrom the dirt at the site to reports in the archaeologicalliterature. However, asarbitrarylinguistic signsproducedin a medium that doesnot actuallymakevisible colour, neither the colour namesnor the numbers allow direct visual comparison betweena sampleofdirt and a referencecolour. This is preciselywhatthe colour patches and viewing holesmake possible.Thus, rather than simply specifring unique points in a larger colour space,the Munsell chart is usedin multiple overlappingactivities (comparinga referencecolour and a patch of dirt aspart of the work of classification, transporting those resultsback to the lab, comparing samples,publishing reports, etc.), and so representsthe 'same'entity, a particular colour, in multiple ways,each ofwhich makespossibledifferentkinds ofoperationsbecauseof the unique properties of eachrepresentationalsystem. In addition to its various sign systemsthe chart alsocontainsa setof circular holes,positioned so that one is adjacentto eachcolour patch. To classifrcolour the archaeologistputs a smallsampleof dirt on the tip of a trowel, puts the trowel directly under the Munsell pageand then movesit from hole to hole until the bestmatch with an adjacentcolour sampleis found. With elegantsimplicity the Munsell page,with its holes for viewing the sampleof dirt on the trowel, juxtaposesin a singlevisual field two quite different kinds of spaces:(a) actualdirt from the site at the archaeologists' feet is framed by (b) a theoreticalspacefor the rigorous, replicableclassificationof colour. The latter is both a conceptualspace,the product of considerableresearch into propertiesof colour, and an actualphysicalspaceinstantiatedin the orderly modification of variablesarrangedin a grid on the Munsell page.The pagesjuxtaposingcolour patchesand viewing holesthat allow the dirt to be seenright next to the colour sampleprovide an historicallyconstitutedarchitecturefor perception,one that encapsulates in a materialobjecttheory and solutionsdevelopedby earlierworkersat other sitesfacedwith the task of colour classification.By juxtaposing unlike spaces, but onesrelevantto the accomplishmentof a specificcognitivetask,the chart creates a new, distinctivelyhuman,kind of space.It is preciselyhere,asbits of dirt areshaped into the work-relevantcategoriesof a specificsocialgroup,that 'nature'is transformed into culture. How are the resourcesprovided by the chart madevisibleand relevantwithin talk-in-interaction?At line 17 Pam movesher hand to the spaceabovethe Munsell chart and points to a particularcolour patchwhile sayrng'En this one'.Within the field of action createdby the activity of colour classification,what Pam doeshere is not simply an indexicalgesture,but a proposalthat the indicatedcolour might be the one they are searchingfor. By virtue of such conditional relevance(Schegloff,1968) it createsa new context in which a reply from Jeffis the expectednext action. In line 19 Jeffrejectsthe proposedcolour. His move occurs after anoticeablesilencein line 18. However that silenceis not an empty space,but a placeoccupiedby its own relevant activity.Beforea competentanswerto Pam'sproposalin line 17 canbemade,the dirt being evaluatedhas to be placed under the viewing hole next to the sample she

AN ETHNOMETHODOLOGICAL

APPROACH

indicated,so that the two can be compared.During line 18 |effmoves the trowel to this position. Becauseof the spatial organization of this activity, specificactions have to be performed before a relevant task, a colour comparison, canbe competently performed. In brief, in this activitythe spatial organization of the tools being worked with and the sequentialorganizationof talk-in-interaction interactwith eachother in the production ofrelevant action (e.g.gettin gto aplacewhereone canmakean expected answerrequiresrearrangementof the visual field being scrutinizedso that the judgementbeing requestedcanbe competentlyperformed).Here sociallyorganizedvision requiresembodiedmanipulation of the environment being scrutinized. It is common to talk about structuressuchasthe Munsell chart as'representations'.Howeverexclusivefocuson the representationalpropertiesof suchstructures can seriouslydistort our understandingof how such entities are embeddedwithin the organizationof human practice.With its viewholesfor scrutinizing samples,the pageis not simply a perspicuousrepresentationof current knowledgeabout the organizationof colour, but a spacedesignedfor the ongoingproduction ofparticular kinds of action. We will now look at how a group of archaeologists make a map. This process will allow us to examinethe interfacebetweenseeing,writing practices,talk, human interaction and tool use (seeGoodwin, 1994for more detailedanalysis).

Map-making andthe practicesof seeingit requires Maps arecentralto archaeologicalpractice.The professionalseeingrequiredto produce and make useof a visual document, such asa map, encompasses not only the image itself but alsothe ability to competentlyseerelevantstructure in the territory being mapped, mastery of appropriate tools and, on occasion,the ability to analysethe work-relevantactionsof another'sbody. Thesedifferent kinds of phenomenacan be brought together within the temporally unfolding processof human interaction usedto accomplishthe activity of making a map. In the following example,two archaeologists aremaking a map to recordwhat they havefound in a profile of the dirt on the sideof one of the squareholesthey have excavated.Beforeactuallysettingpen to paper some relevanteventsin the dirt, such asthe boundarybetweentwo differentkinds of soil,arehighlightedby outlining them with the tip of a trowel. The structurevisible in the dirt is then mappedon a sheetof graph paper.Typically this taskis done by two participantsworking together.One uses a pair of rulers (one laid horizontally on the surfaceand the other a hand-held tapemeasureusedto measuredepth beneaththe surface)to measurethe length and depth co-ordinatesof the points in the dirt that are to be transferredto the map, and then speakstheseco-ordinatesaspairs of numbers(e.g.'at fifteenthreepoint two'). The secondpersonplots the points specifiedon the graph paper,and drawslinesbetween successive measurements.What we find hereis a small activity systemthat encompasses talk, writing, tools and distributed cognition as fivo parties collaborateto inscribe eventstheyseein the earth on to paper.In the transcriptin Figure8.3,Ann, the party drawing the map, is the senior archaeologistat the site,while Sue,the personmaking

171

172

HANDBOOK

OF VISUAL ANALYSIS

measurements,is her student.The sequenceto be examinedbeginswith a directive: Ann, the writer, tells Sue,the measurer,to 'Give me the ground surfaceover here to about ninety'. HoweverbeforeSuehasproducedany numbers,indeedbeforeshehas saidanythingwhatsoever,Ann in lines4 and 5 challengesher, telling her that what she is doing is wrong: 'No- No- Not af ninety. From you to about ninety'.

$$r{Ar*" f

i

h**lrh

j" Ann: 2 .l

3

4 Ann: 5 6 7 Sue: I Ann: I 10 Sue: 11" Ann: 12 13 14 Sue: 15 Ann:

h}qrr ,

G f}&

*

fe,r*fr fi*Sfr

,*

r$

px{d os

F qfusfi&*

Give rne the ground surfaceover here to about nin*y. (1.6) No- No- Not at ninety. From you to about ninef. {1.0)

"ch" Wherever therds a change in slope.

t0.6)

oMm kay. See so rf its fahty flat I'll need one fair$ f,Iat. J-whereit stapsbei:rg Lokav. Like right fhere.

Figure 8.3 Transcript of Ann (the senior archaeologist at the site) drawing a map, and Sue (her student) making measurements.

AN ETHNOMETHODOLOGICALAPPROACH173

Directives are a classicform of speechaction that sociolinguistshave used to probe the relationship betweenlanguageand social structure, and in particular issuesof power and gender.Here Sueformats both her directiveand her correction in very strong,direct 'aggravated'fashion.No forms of mitigation arefound in either utterance,and Ann is not given an opportunity to find and correctthe trouble on her own. Directives formatted in this fashion have frequently been argued to display a hierarchicalrelationship:Ann is treating Sueas someonethat shecan give direct, unmitigated ordersto. And indeedAnn is a professorand Sueis her student. Issuesof power do not howeverexhaustthe socialphenomenavisible in this sequence.Equally important are a range of cognitiveprocesses that are as socially organizedasthe relationshipsbetweenthe participants.For example,in that Suehas not produced an answerto the directive, how can Ann seethat there is something wrong with a responsethat has not even occurredyet?Crucial to this processis the phenomenon of conditionalrelevance first describedby Schegloff(1968).Basicallya first utterance createsan interpretative environment that will be used to analyse whateveroccurs after it. Here no subsequenttalk has yet been produced. However, providing an answer in this activity system encompassesmore than talk. Before speakingthe set of numbers that counts as a proper next bit of talk, Suemust first locatea relevantpoint in the dirt and measureits co-ordinates.Both her movement through space,and her useof tools suchasa tape-measure,arevisibleevents.As Ann finishesher directive,Sueis holding the tape-measureagainstthe dirt at the left or zero end of the profile. However,just after hearing'ninety' Suemovesboth her body and '90' the tape-measure to the right, stoppingnearthe mark on the upper ruler. Byvirtue of the field interpretation openedup through conditional relevance,Sue'smovement and tool usecannow be analysedby Ann aselementsof the activityshehasbeenasked to perform, and found wanting. Sue has moved immediately to ninety instead of measuringthe relevantpoints betweenzero and ninety. The sequentialframework createdby a directivein talk thus providesresourcesfor analysingand evaluatingthe visible activity of an addressee's body interactingwith a relevantenvironment. Additional elementsof the cognitiveoperationsand kinds of seeingthat Ann requiresfrom Suein order to make her measurementsare revealedas the sequence continuesto unfold. Making the relevantmeasurementspresupposesthe ability to locatewherein the dirt measurementsshouldbe made.However Sue'sresponsecalls this presuppositioninto question and leadsto Ann telling her explicitly, in several differentways,what sheshouldlook for in order to determinewhereto measure.After Ann tells Sueto measurepoints betweenzero and ninety, Suesdoesnot immediately move to points in that region but insteadhesitatesfor a full secondbeforereplying with a weak'ooh' (line 7). Ann then tellsher what sheshouldbe looking for'Wherever there'sa changein slope'(line 8). This descriptionof coursepresupposes Sue'sability to find in the dirt what will count as'a changein slope'.Sueagainmovesher tapemeasurefar to the right. At this point, instead of relying upon talk alone to make explicit the phenomenathat shewants Sueto locate,Ann movesinto the spacethat Sueis attendingto and points to one placethat should be measuredwhile describing 'See more explicitly what constitutesa changein slope: so if it'sfaifly flatl'll needone where it stopsbeingfairly flat like right there.'

174

HANDBOOK

OF VISUAL ANALYSIS

One of the things that is occurring within this sequenceis a progressive expansionof Sue'sunderstandingasthe distinctionsshemust make to carry out the task assignedto her are explicatedand elaborated.In this processof socialization through languagethere is a growth in intersubjectivityasdomains of ignorancethat preventthe successfulaccomplishmentof collaborativeaction arerevealedand transformed into practicalknowledge,a way of seeing,that is sufficient to get the job at hand done, suchthat Sueis finally ableto understandwhat Ann is askingher to do (that is to seethe scenein front of her in a manner that permits her to make an appropriate, competentresponseto the directive).It would howeverbe wrong to seethe unit within which this intersubjectivityis lodged as simply thesetwo minds coming togetherin the work at hand. Insteadthe distinction being explicated,the ability to seein the very complex perceptualfield provided by the landscapethey are attendingto, thosefew eventsthat count aspoints to be transferredto the rnap, are centralto what it means to seethe world asan archaeologistand to usethat seeingto build the artefacts,such asthis map, which are constitutiveof archaeologyasa profession.Suchseeingwould be expectedof any competentarchaeologist.It is an essentialpart of what it meansto beanarchaeologist, and it is theseprofessionalpracticesof seeingthat Sueis beingheld accountableto. The relevantunit for the analysisof the intersubjectivityat issuehere - the ability of separateindividuals to seea common scenein a congruent,work-relevant fashion- is thus not theseindividuals as isolatedentities,but insteadarchaeologyas a profession,a community of competentpractitioners,most of whom havenevermet eachother,but who nonethelessexpecteachother to be ableto seeand categorizethe world in waysthat are relevantto the work, scenes,tools and artefactsthat constitute their profession. The phenomenaexaminedso far provide somedemonstrationof howwhat is to be seenin a map, scene,human body or imagestandsin a reflexiverelationshipto other semiotic structuresthat participantsare using to constitutevisual phenomena asa relevantcomponent ofthe eventsand activitiesthat make up their lifeworld. These structures include language,the constitution of action and context provided by sequentialorganizationand waysof seeingeventsand using imagesof different types that are lodged within the practicesof particular socialcommunities,such as the professionof archaeology.

Professional visionin court Partieswho are not competentmembersof relevantsocialcommunities can lack the ability and/or the socialpositioning to seeand articulatevisualeventsin a consequential way.Theseissuesweremadedramaticallyvisiblein the trial of four LosAngelespolicemen who were recordedon videotapeadministering a beatingto an African-American motorist, Mr Rodney King, whom they had stopped after a high-speed pursuit triggeredby a traffic violation. When the tape of the beatingwas shown on national televisiontherewasoutrage,and eventhe headof the LosAngelespolice department thought that conviction of the officerswas almost automatic.However, at their first trial (they were later tried again in a federal rather than a statecourt for violating

A N E T H N O M E T H O D O T O G I G AATP P R O A C H

Mr King's civil rights) all four policemenwere acquitted,a verdict that triggeredan uprising in the city of LosAngeles,with neighbourhoodsbeingburned, federaltroops being calledin, and so on. The crucial evidenceat the trialwas avisual document:the videotapeof the beating.Ratherthan transparentlyprovingthe guilt ofthe policemen who were seenon it beatinga man lyttrg prone on the ground, the tape in fact provided the policemen'slawyerswith their evidencefor convincing the jury that their clients werenot guilty of anywrongdoing. They did this by usinglanguage,pointing and expert testimony to structure how the jury sawthe eventson the tape in a way that exonerated the policemen.In essence they usedthe tapeofthe beatingto demonstratethat Mr King wasthe aggressor,not the policemen,and that the policemenwere following proper policepracticefor subduingaviolent,dangeroussuspect(seeGoodwin,lgg4for a more wastheir useof detailedanalysisof suchprofessionalvision). Crucial to their success anotherpoliceman,SergeantDuke, asan expertwitness.It wasarguedthat laypeople could not properly seethe eventson the tape. Instead,the ability to legitimatelysee what the body of a suspectwasdoing, suchasMr King's ashe lay on the ground being beaten,and specificallywhether the suspectwas being aggressive or compliant, was lodgedwithin the work practicesof the socialgroup chargedwith arrestingsuspects: was the police.The abilityto seesucha body, and codeit in terms ofits aggressiveness, a component of the professionalpracticesthat policemenuseto codethe eventsthat are the focus of their work. Insofar assuchvision is a public component of the work practicesofa particularsocialgroup,someonewho wasn'tpresentbutwho is a member of the profession,a policeman,can make authoritativestatementsabout what canbe legitimatelyseenon the tape.However,while policemenconstitute a sociallyorganized profession,suspectsand victims of beatingsdon't. Thereforethere is no one with the socialstanding,that is membershipand masteryof the practicesof a relevantsocial group, to act as an expertwitnessto articulatewhat was happeningfrom Mr King's perspective. What was to be seenon the tape was structured through the way in which different semioticfields,suchasstructure in the streamof speech,pointingwhich highlighted specificplacesand phenomenain the imagebeing looked at and eventsin the imageitself mutually elaboratedeachother to provide a construal of eventsthat served the purposesofthe parlryarticulatingthe image.The followingtranscript (Figure8.4) provides an example.At the point where we enter this sequencethe prosecutorhas noted that Mr King appearsto be moving into a position appropriatefor handcuffing him, and that while one officer is in fact reachingfor his handcuffsthe suspectis being co-operative. By noting the submissiveelementsin Mr King's posture,and the fact that one of the officersis reachingfor his handcuffs,the prosecutorhas shown that the tape demonstratesthat Mr King is beingco-operative.Ifhe can establishthis point, hitting Mr King again would be unjustified and the officers should be found guilty of the crimesthey are chargedwith. The contestedvision being debatedhere hasvery high stakes. To rebut the vision proposedby the prosecutor,Sgt Duke usesthe semantic extremelysubtlebody movements resourcesprovided by languageto codeasaggressive of a man lytng facedown beneaththe officers (lines 7-lI). Note, for example,not

175

176

HANDBOOK

OF VISUAL ANALYSIS

only line 13's explicit placementof Mr King at the very edge,the beginning, of an aggressive spectrumintroduced in earliertestimony,but alsohowverysmall movements are made much larger by situating them within a prospectivehorizon through the repeateduseof 'startingto' (lines6, 8, I I ). The eventsvisibleon the tapearestructured, enhancedand amplified by the languageusedto describethem.

1

Prosecutor: So uh would you,

2

again consider this to be:

3

a nonagressive,movement by Mr. Klng? At this time no I wouldn't. (1.1)

4

Sgt. Duke:

5

Prosecutor: It is aggressive.

6

Sgt. Duke:

Y"". I ttist"rting

to be.l (o.g)

7

(-

This foot, is laying flat, (O.B)

I

(-

There's@tnutr

I

(-

this leg (O.4)

10

(-

in his butt (O.4)

11

(-

The buttocks area tras lGtarteOE7*e.l

(o.z)

which would put us,

12 13

t the beginning of our spectrum a g a i n . (-

Figure 8.4

(o.G)

indicates that Sgt. Duke is pointing on the screen at the body part described in his talk.

Transcript of use of video in the trial of Rodney King.

This focusing of attention organizesthe perceptual field provided by the videotapeinto a salientfigure, the aggressivesuspect,who is highlighted againstan amorphousbackgroundcontaining non-focal participants,the officersdoing the beating.Suchstructuring of the materialsprovided by the imageis accomplishednot only through talk, but alsothrough gesture.As SgtDuke speakshe brings his hand to the screenand points to the partsof Mr King's body that he is arguingdisplayaggression. The pointing gestureand the perceptualfield which it is articulating mutually elaborate eachother. The touchableeventson the televisionscreenprovide visible eyidencefor the descriptionconstructedthrough talk. What emergesfrom SgtDuke's testimony is not just a statement,a staticcategory,but a demonstrationbuiltthrough the active interplay betweencoding schemeand the imageto which it is being applied.As talk and imagemutuallyenhanceeachother a demonstrationthat is greaterthan the sum

AN ETHNOMETHODOLOGICAL

APPROACH

of its parts emerges,while simultaneouslyMr King, rather than the police officers, becomesthe focusof attention asthe expert'sfinger articulatingthe imagedelineates what is relevant within it. By virtue of the categorysystemerectedby the defence,the minute rise in Mr King's buttocks noted on the tape unleashesa cascadeof perceptualinferences that havethe efFectofexoneratingthe officers.A risein Mr King'sbodybecomesinterpreted asaggression, which in turn justifiesthe escalationof force.Like other parties, such asthe archaeologists, facedwith the task of coding a visual scene,the jury was led to engagein intense,minute cognitive scrutiny as they looked at the tape of the beating to decidethe issuesat stakein the case.However, once the defencecoding schemeis acceptedas a relevant framework for looking at the tape, the operative perspectivefor viewing it is no longer a layperson'sreaction to a man lying on the ground being beaten,but insteada micro-analysisof the movementsbeing made by that man's body to seeif it is exhibiting aggression. In the first trial, though the prosecutiondisputedthe analysisof specificbody movementsasdisplaysof aggression,the relevanceof looking at the tape in terms of sucha categorysystemwasnot challenged.A key differencein the secondtrial, which led to the conviction oftwo ofthe officers,wasthat therethe prosecutiongavethejury alternativeframeworksfor interpreting the eventson the tape.Theseincluded ways of seeingthe movementsof Mr King's body that Sgt Duke highlighted as normal reactionsof a man to a beatingrather than asdisplaysof incipient aggression.In the prosecution'sargument Mr King'cocks his leg', not in preparationfor a charge,but becausehis musclesnaturallyjerk after being hit with a metal club. The study of the practicesused to structure relevantvision in scientificand workplaceenvironments,what Hutchins ( 1995)hascalled'Cognitionin the Wild', has becomethe focus of considerableresearch.A major initiative for such studieswas provided bylucy Suchmanin the early1990swhen sheinitiated the workplaceproject while at Xerox PARC.The site chosenfor researchwas ground operationsat a midsizedairport. Documentsand imagesof many differenttypes,and the ability of actors in alternativestructural positions to seeand analyseeventsin relevantways,were crucial to the work of the airport. Phenomenathat receivedextensivestudy included work-relevantseeingof documents,aeroplanesand events(Goodwin and Goodwin, 1996),the constitution of sharedworkspaces(Suchman,1996),the study of how a common document co-ordinateddifferent kinds of work in different work settings, and the practicesinvolved in seeingand shapingphenomenain collaborativework (Brun-Cottan,1991;Suchmanand Trigg, 1993).In part becauseof the centralrole playedbyvisual phenomenain the workbeing analysed,the project'sfinal report was submitted as a videotape(Brun-Cottan et a1.,1991).Subsequentanalysisgrowing from this project has focused on the organization of both documents and visual phenomenain a range of occupationalsettings,such as law firms and the work of architects.In England Christian Heath and his collaboratorshave investigatedthe structuring of vision within interaction in a range of settingsincluding the control room for the London Underground,centresfor the production of electronicnews,art classes, and so on (Heath and Luff, 1992b,1996;Heath and Nicholls, 1997).In much of this researchthere is a focus on how core practicesfor the organization of talk,

177

17f3

H A N D B O O KO F V I S U A T A N A T Y S I S

reference,gestureand other phenomenacentral to the production of action within human interaction can encompassnot only talk but also embodiment in a world populatedby work-relevantobjects. Hindmarsh and Heath (in press)investigatereferencewithin such a framework. LeBaron(1998),Streeck(1996)and LeBaronand Streeck(in press)examine how gestureemergesfrom the interaction of working hands acting in the world in settingssuch as architects'meetingsand auto body shops.Robinson (1998) has provided analysisofhow participantsin medicalinterviewsorganizetheirinteraction by attendingto how gazeis shiftedfrom other participantsto relevantvisualmaterials in the setting,such as medical records.Whalen (1995) analysedhow the talk of operatorsrespondingto emergency911 callswasorganizedin part by the taskof filling in required information on a computer screenwith a specificvisual organization. RogersHall and ReedStevenshaveinvestigatedvisual practicesin a rangeof school, scientificand occupationalsettings(Hall and Stevens,1995;Stevensand Hall, in press). Researchin computer-supportedco-operativework hasfocusedspecificallyon new forms of visualaccesscreatedby electronicmedia.Heath and Luff ( I 992b,1993) have doneconsiderableresearchon interactionmediatedthrough video,demonstratingthe crucial waysin which resourcesavailableto partieswho are actually co-presentto each other arenot availablein media spaces.Yamazakiand his colleagueshaveexploredthe systematicproblemsthat arisewhen particularkinds of directives,suchasinstructions for how to useCPRto start a heart attackvictim's heart,aregiventhrough talk alone, for exampleover the phone,without accessto a relevantvisualenvironment.Patients usually die sincethe novice is not able to placehis or her hands at the appropriate spot on the patient's body. To remedy some of these issuestechnologiesthat incorporatebasicresourcesavailablefor doing referencein face-to-faceinteraction, such as pointing, havebeen developed.Theseinclude a remote-controlledcar with a laser that has the ability to move while clearly marking the specificplacesbeing pointed at in a remote environment (Yamazakietal., 1999).Nishizaka(in press)has investigatedhow participants co-ordinate gazeboth spatially and temporally on electronicdocumentssuchascomputer screens.Kawatoko(in press)hasinvestigated how lathe workers organize perceptual fields so to make visible the invisible movementsoftheir cutting tools.Both Kawatokoand Ueno (in press)haveexamined the organization of vision on many different levels(from documentsto systematic placementof objectson the warehousefloor as part of its work flow) in the work practicesof a large warehouse.In all of this, work practicesfor seeingrelevant phenomenaare systematicallyembeddedwithin processesof socialorganization, structuresof mutual accountabilityand the organizationof activity.

CONCLUSION Within both conversationanalysisand ethnomethodologyvisual phenomenahave been analysedby investigatinghow they are made meaningful by being embedded within the practicesthat participantsin a varietyof settingsuseto constructthe events and actionsthat make up their lifeworld. This hasled to the detailedstudy of a range

AN ETHNOMETHODOLOGICALAPPROACH179

of quite different kinds of phenomena,from the interplay betweengaze,restartsand grammar in the building of utteranceswithin conversation,to the construction and useofvisual representations in scientificpractice,to howthe abilityoflawyersto shape what can be seenin the videotapeof policemenbeating a suspectcan contribute to disruption of the body politic that leavesa city in flames,to the part playedby visual practicesin both traditional and electronicworkplaces.Visual phenomenathat have receivedparticular attention include: The body asa visiblelocus for displaysof intentional orientation through both gazeandposture. The body as a locus for a variety of different kinds of gesture,from iconic elaboration on what is being said in the stream of speech,to pointing, to the hand asan agentengagedwith the world around it. Visual documentsof many different types used in both scientificpractice and the workplace, for example, maps, graphs, Munsell charts, coding forms, schedules,television screensproviding accessto distant sites, architectural drawingsand computer screens. Material structure in the environment where action and interaction are situated.

This perspectivebrings togetherwithin a common analyticframework both the detailsof how the visiblebody is usedto build talk and action in moment-to-moment interaction and the way in which historically structured visual imagesand features of a settingparticipatein that process(seeChapter4). Ratherthan standingaloneas self-contained,self-explicatingimages,visualphenomenabecomemeaningful through the way in which they help elaborate,and are elaboratedby, a rangeof other semiotic fields - sequentialorganization, structure in the stream of speech,encompassing activities,etc.- that arebeing usedby participantsto both constructand makevisible to eachother relevantactions.The focusof analysisis alwayson how the participants in a settingthemselvesdisplaya consequentialorientation to visual phenomena(for example,by shifting gaze after a restart,focusing their work on a Munsell chart or building imagesas a core component of the practicesusedto make visible scientific phenomena).A variety of different methodologiesare employed.However a basic component of many researchprojectsincludesgoing to the site where the activities being investigatedare actuallyperformed and examining what the participantsare doing thereascarefi.rllyaspossible.Videotaperecordsarefrequentlymost usefulbecause of the way in which they preservelimited but crucial aspectsof the spatialand environmental featuresof a setting, the temporal unfolding organization of talk, the visible displaysof participants'bodiesand changesin relevantphenomenain the settingas relevantcoursesof action unfold (seeChapter3). Analysistypicallyrequiresnot only viewing the tape,ethnographicrecordsand documentscollectedin a setting,but also the construction of new visual representationssuch astranscriptsof many different types(note how somein this paperincorporateboth detailedtranscription of the talk and avariety of different kinds of graphicrepresentations).While this analysissheds much important newlight on howvisual phenomenaareorganizedthrough systematic discursivepractice,it is not restrictedto vision per sebutis insteadinvestigatingthe more generalpracticesusedto build action within situatedhuman interaction.

180

HANDBooK oF vrsuAL ANALYSTS

NOTES 1 Vision is not, however,essentialasboth the competenceof the blind and telephoneconversationsdemonstrate.Below it will be arguedthat situatedaction is accomplishedthrough the juxtaposition of multiple semioticfields,only someof which make vision relevant. 2 Talk is transcribedusingthe systemdevelopedby Gail Jefferson(seeSacksetaL,I974:731-3). Talk receMngsomeform of emphasisis markedwith underliningorbold italics.Punctuation is usedto transcribeintonation: a period indicatesfalling pitch, a questionmark risingpitch and a comma a falling contour, aswould be found for exampleafter a non-terminal item in a list. A colon indicateslengtheningof the current sound.A dashmarks the suddencut-offof the current sound (in English it is frequently realized as glottal stop). Comments (e.g. descriptionsof relevantnon-vocalbehaviour) areprinted in italicswithin double parentheses. Numberswithin singleparentheses mark silencesin secondsand tenthsof a second.A degree sign (") indicatesthat the talk that follows is being spokenwith low volume. Left brackets connectingtalk by different speakersmark the point where overlapbegins.

REFERENCES 'Talk on Language Brun-Cottan, F. ( 199I ) in the work place:occupationalrelevance', ReseArch and SocialInteraction,24l.277-95. Brun-Cottan,F., Forbes,F., Goodwin, K., Goodwin, C., Harness,M., Jordan,B., Suchman, L., Randall, L. and Randall, T. ( 1991) The WorkplaceProject:Designingfor Diversity and Change,video producedby Xerox PaloAlto ResearchCenter. Chomsky,N. ( 1965)Aspectsof the Theoryof Syntax.Cambridge,MA: MIT Press. Goodwin, C. (1979)'The interactive construction of a sentencein natural conversation',in New York: Irvington G. Psathas(ed.), EverydayLanguage:Studiesin Ethnomethodology. Publishers. 'Restarts, pauses,and the achievementof mutual gazeatturn-beginning', Goodwin, C. ( 1980) SociologicalInquiry, 50(3-a) : 272-302. Goodwin, C. (1981) ConversationalOrganization:InteractionBetweenSpeakersand Hearers. New York: AcademicPress. 'Notes on story structure and the organization of participation', in Goodwin, C. (1984) Max Atkinson and John Heritage (eds),Structuresof SocialAction. Cambridge:Cambridge University Press. 'Gesture Goodwin,C. ( l936) asa resourcefor the organizationof mutual orientation', Semioticao 62 (rl2):29-49. 'Professional (3):606-33. vision',AmericanAnthropologist,96 Goodwin,C. (1994) 'Seeing in depth', SocialStudiesof Science,25237-74. Goodwin,C. (1995) 'Practices Ninchi Kagaka(CognitiveStudies:Bulletin Goodwin,C. (1996) of color classification', of the fapaneseCognitiveScienceSociety),3 (2):62-82. 'Action Goodwin, C. (2000) and embodiment within situatedhuman interaction', lournal of Pragmatics,32. 'Pointing Goodwin, C. (in preparation) as situatedpractice',in SotaroKita (ed.),Pointing Were Language,Culture and Cognition Meet. 'Concurrent Goodwin, C. and Goodwin, M.H. ( 1987) operations on talk: notes on the interactiveorganizationof assessments', /PRA Papersin PragmaticsI, no. I:I-52. 'Seeing Goodwin, C. and Goodwin, M.H. (1996) asa situatedactivity: formulating planes',in Y. Engestromand D. Middleton (eds), Cognitionand Communicationat Work. Cambridge: CambridgeUniversity Press. Goodwin, M.H.(19S0)'Processesof mutual monitoring implicated in the production of descriptionsequences',SociologicalInquiry, 50 303-17.

AN ETHNOMETHODOTOGICAL APPROACH

181

Goodwin, M.H. and Goodwin, C. ( 1936)'Gestureand coparticipation in the activity of searching (ll2):51-75. for a word' , Semiotica,62 Hall, R. and Stevens,R. (1995) 'Making space:a comparisonof mathematicalwork in school and professionaldesignpractices',in S.L.Star (ed.), TheCulturesof Computing.pp.l18-145. Haviland, J.B. (1993) 'Anchoring, iconicity, and orientation in Guugu Yimidhirr pointing gestures',lournal of LinguisticAnthropology,3(1): 3-45. Heath, C. (1986) Body Movement and Speechin Medical Interaction. Cambridge: Cambridge University Press. Heath, C. and Luff, P. (I992a)'Media spaceand communicative asymmetries',Human Computer Interaction,7: 3 | 5-46. Heath, C. and Luff, P. (199b) 'Crisis and control: collaborativework in London Underground Control Rooms',Journalof ComputerSupportedCooperativeWorh I Q):2a-a8. Heath, C. and Lufl P. (1993) 'Disembodiedconduct: interactionalasymmetriesin videomediatedcommunication', in G. Button (ed.), Technologltin Working Order.London and NewYork Routledge. Heath, C. and Luff, P. (1996) 'Convergentactivities:line control and passengerinformation on the London Underground', in Y. Engestromand D. Middleton (eds), Cognitionand Communicationat Work. Cambridge:CambridgeUniversity Press. Heath, C. and Nicholls, G. (1997) 'Animating texts: selectivereadingsof news stories',in L.B. Resnick,R. Salj6,C. Pontecorvoand B. Burge (eds), Discourse,Toolsand Reasonsing: Essayson SituatedCognition.Berlin, Heidelberg,NewYork: Springer. Hindmarsh, |. and Heath, C. (in press)'Embodied reference:a study of deixis in workplace interaction', I ournal of Pragmatics. Hutchins,E. (1995)CognitionintheWild.Cambridge,MA: MIT Press. 'Organizing Kawatoko,Y. (in press) multiple vision', Mind, Cultureand Activity,T (l-2). (1990a) Kendon, A. Conducting Interaction: Patterns of Behavior in FocusedEncounters. Cambridge:CambridgeUniversity Press. Kendon, A. (1990b) 'Spatial organization in social encounters:the f-formation system',in A.Kendon (ed.),Conductinglnteraction:PatternsofBehaviorinFocusedEncounters.Cambridge: CambridgeUniversity Press. Kendon, A. (1994) 'Introduction to the specialissue:gestureand understandingin interaction',Research on Languageand SocialInteraction,2T(3):1714. Kendon,A. (1997)'Gesture', Annual Revinv of Anthropology,26 109-28. Latour, B. (1986) 'Visualizationand cognition: thinking with eyesand hands',Knowledgeand Society:Studiesin the Sociologyof Culture Pastand Present,6; l-40. Latour, B. (1987) Sciencein Action: How to Follow Scientistsand Engineersthrough Society. Cambridge,MA: Harvard University Press. Latour, B. and Woolgar, S. (1979) LaboratoryLife: The SocialConstructionof ScientificFacts. London: Sage. LeBaron,C. ( 1998)'Building communication: architecturalgesturesand the embodiment of ideas',PhD dissertation,University of Texas,Austin. LeBaron,C. and Streeck,f . (in press)'Gestures,knowledge,and the world', in D. McNeill (ed.), Gesturesin Action, Language,and Culture. Cambridge: Cambridge University Press. Lynch, M. ( 1990)'The externalizedretina: selectionand mathematizationin the visual documentation of objectsin the life sciences',in M. Lynch and S.Woolgar (eds),Representation in ScientificPractice.Cambridge,MA: MIT Press. Lynch,M. and Woolgar, S.( 1990)'Introduction: sociologicalcontributionsto representational practicein science',in M. Lynch and S.Woolgar (eds),Representation in ScientificPractice. Cambridge,MA: MIT Press. McNeill, D. (1992)Hand (t Mind: Wat Gestures Reuealabout Thought.Chicago:University of ChicagoPress. 'seeing Nishizaka,A. (in press) what one sees:perception,emotion and activity', Mind, Culture andActivity,T (I*2).

18.2

H A N D B o o Ko F v r s u A l A N A L Y s t s

'Transcription Ochs,E. (1979) astheory', in E. Ochsand B.B. Schieffelin(eds),Developmental Pragmatics.New York: AcademicPress. 'Getting Robinson,J.D.( 1998) down to business:talk, gaze,and body orientation during openings (I):97-123. of doctor-patientconsultations',Human CommunicationResearch,25 'A Sacks,H., Schegloff,E.A. and Jefferson,G. (I974) simplestsystematicsfor the organization of turn-taking for conversation', Language,50l.696-7 35. 'Sequencing Schegloff,E.A. (1968) in conversationalopenings',AmericanAnthropologist,T0: r075-9s. Schegloff,E.A. ( I 998)'Body torque', SocialResearch, 65 (3): 535-96. 'Disciplined perception: learning to seein technoscience', Stevens,R. and Hall, R. (1998) in M. Lampert and M. Blunk (eds),MathematicalTalk and ClassroomLearning:Wat, Why and How. pp. 10749. Cambridge:CambridgeUniversity Press. 'The Streeck,J. ( 1988) significanceof gesture:how it is established' , IPRAPapersin Pragmatics, 2 (1):60-83. Streeck,I. (1993)'Gesture as communication I: its coordination with gazeand speech', CommunicationMonographs,60 (4): 275-99. Streeck,I. 0994)'Gesture as communication II: the audienceas co-author', Researchon Langaugeand Social Interaction,27 (3) : 223-38. Streeck,I. 0996)'How to do things with things', Human Studies,19:365-84. 'Technologies Suchman,L. (1992) of accountability: of lizards and airplanes',in G. Button (ed.), Technologyin Working Order: Studiesof Work, Interaction and Technology.London: Routledge. 'Constituting Suchman,L. ( 1996) sharedworkspaces',in Y. Engestromand D. Middleton (eds), Cognitionand Communicationat Work. Cambridge:CambridgeUniversity Press. 'Artificial Suchman,L. and Trigg, R. ( 1993) intelligenceascraftwork', in S.Chaiklin and J.Lave (eds), UnderstandingPractice:Perspectives on Activity and Contexr.Cambridge: Cambridge University Press. 'Technologies lJeno, N. (in press) of mutual accountabilityof society,socialorganization,and activity for collaborativeactivity', Mind, Cultureand Activity,T (I-Z). 'A Whalen,J. ( 1995) technologyof order production: computer-aideddispatchin public safety communications', in P. Have and G. Psathas(eds), Situated Order: Studiesin the Social Organizationof Talk and EmbodiedAction.Washington,DC: University Pressof America. Yamazaki,K., YamazaH,A., Kuzuoka,H., Oyama,S.,Kato, H., Suzuki,H. and Miki, H. (1999) 'Gesturelaser and GesturelaserCar: developmentof an embodiedspaceto support remote instruction', in S. Bodker, M. Kyng and K. Schmidt (eds), Proceedingsof the European Conferenceon Computer-SupportedCooperativeWork, 12-16 September1999, Copenhagen, D enmark.Dordrecht/Boston/London:Kluwer Academic.'

ilE ffi% K ruff

Analysing FilmandTelevision: a SocialSemioticAccountof Hospital: an UnhealthyBusiness BICK IEDEMA

INTRODUCTION The following still (Figure 9.1) is taken from a documentaryabout a Melbourne hospital. It showsa frame from a scene(about 7 minutes into the documentary)in which a clinician walks through a hospital corridor while speakingover her shoulder

Unit

tf

Figure 9.1 Frameone: frame from a sceneabout 7 minutesinto the documentary. to an interviewer who follows her with a cameta. While negotiating the maze of corridors which connect offices and wards, the clinician says: It's easyfor some people to say,you know, this is the fine line, you don't step over it. Whereasfor us,we actuallyseethe patient, and I'd be more inclined to look for lessurgent or emotive placesto savemoney. I mean cardiology is, uh, a life threatening and an acute situation and I'd prefer not to savemoney there,you know, . . . . You don't treat public patientslike secondclasscitizens,if we can afford it. They get the besttreatment in the private hospital.Why don't they get the sametreatment in a public hospital?I mean, they're still somebody'smother or father, they're still deservingthe respectto be treated . . . [passesthrough door; cut]

144

HANDBOOK

OF VISUAL ANALYSIS

The sceneis highly dynamic as severalthings are going on: the talking and walking, which involveslooking wherewe'regoing,turning cornersand openingdoors,against a backgroundof hustleand bustleof the hospital.The impressionit leavesis that this clinician'sworld is an activeand busy one. The documentaryis about hospitalwaiting lists,budgetblow-outs and rising costsof healthand agedcare.The dilemmawhich the programmeposesis:we havethe know-how and the technologyto treat peopleand makethem better,but accordingto the presentAustralian governmentwe can't afford it. The managementof hospitals is nowadaysmore concernedto curb doctors'useof resourcesneededto curepatients than with ensuringproper standardsof clinical care. Justbefore the clinician scene,we seethe Head of the CardiothoracicUnit, Dr Keith Stockman,and the Managerof Clinical Services,David Hillis, talking about how cardiothoracicservicesarean exceptionaldrain on the hospital'sbudget.We are alsoshownpatientsand their reactionsto illness.And thereis the chaosin the hospital where patientssometimeshave to wait for up to two daysfor a bed on a ward. We seehow 80-year-oldMrs Biggsendsup waiting for over a dayin emergencyafter she suffers a debilitating heart attack, and how, after her acquittal from hospital, her bewildered83-year-oldhusbandwandersaround their old house in a dazeupon leavinghis wife behind in a distant nursing home. We arelikely to be overwhelmedwith grief at being confronted with the humiliations to which thesepatients are subjected.After Mrs Biggsleavesthe hospital her patient record goesup to administration where sheis enteredinto the computer and filed awayas a $5400loss.As the keyboardtaps out this cost-benefit analysisacross the screen,its rattle evokesthe steadyadvanceof steelyindustrial mechanizationand cold economicrationalization. Could therepossiblybeanothersideto this story?Is it possiblethat the means of representationhave been exploited to favour one viewpoint and render all other viewpoints irrelevant?How many television-watchingpeoplewould think of asking questionslike this when confrontedwith sucha moving accountabout the deplorable stateof the current healthcaresystemand the money-centredmentality of its hospital managers?Onlythose who havesomeknowledgeabout the politics of hospital administration and healthpolicywill knowwhose viewsthe documentarytakesasgiven and what (other) questionscould be askedhere. But let's not worry for now about the politics of the viewspresentedin this documentary. Let's first look at how they are represented.A systematiclook at how the documentaryvisuallyrepresentsthe various stakeholdersrevealsimportant diflerences. First, both patientsand clinicians (that is, doctors and nurses)occupy a spacemuch closerto the camerathan administratorsand managers.Moreover, cliniciansare shown in action (arrangingbedsover the phone, wheelingpatientsabout, talking to patientsand family, doing operations),while administratorsmerelyaddressmeetings or stareintently at computer monitors. Further, cliniciansappeartogetherin shots (sometimesas many as five or six of them), whereasadministratorsusually appear alonein shots. 'patterns' There are more worth commenting on. Rarely do clinicians and administratorsappearin the sameshot: either a cut or a swervewill separatethem.

A N A L Y S I N GF I L M A N D T E L E V I S I O N

The administratorsare often filmed from low angles(suggestinga degreeof power), while cliniciansare generallylevelwith the camera.Lastbut not least,the doctors are consistentlyshown on the left of the screen(constructedaswhat is familiar: the Given), and the administrators are consistentlyshown on the right (as the unfamiliar: the New) (seeChapter 7).1The patternsjust listed suggestthat the documentary'sfilmic structureand editing conventionsposition cliniciansand manager-administratorsas different, and that the clinicians are favoured. There are still other things that support this reading.Apart from the different meansby which they arevisuallyrepresented,stakeholdersare alsomadeto perform 'subject very different roles'.Cliniciansareinterviewedon camera,while the administrators are only shown speakingto clinicians and writing on whiteboards during meetings.Clinicianshavelonger'visual turns': we seeand hear more of them than of the administrators.The camerawill follow clinicians around the hospital,while aforay by one of the younger administratorsinto the surgeryward is shown in steadylong shot, reinforcing the contrastbetweenthe young bureaucrat'sdiffidenceand the bustle of the pulsing surgicalward. So far we havenot saidmuch about how the documentaryunfolds overall.At this broader level,the timespanwhich framesthe documentaryincludesthe last few months of the financial year.The narrative centreson management'sfailure to balance the books.This main thread is punctuated,if you like, by three sub-narrativesabout patients,one of whom getscured,one of whom diesand the last ofwhom endsup in a distantold-agenursinghome.Eachsub-narrativeoffersan instanceof somemedical heroism or other, and invites the inference that curbing medical expenditure is tantamount to letting peopledie.At the end of the documentarywelearn that David Hillis (the Manager) has accepteda job at another hospital, and that the hospital's budget problem is no longer his. We are encouraged,again,to infer the difference 'mere between administrators',who canwalk awayfrom their jobs, and doctors,who are committed to the continuity of unconditional patient care and cure ('we could not havedone more'). In short, what this documentaryfavoursis the medical-clinicalview on hospital 'we organizationand health policy: are doing our bestto carefor the suffering;don't limit us in the work we are doing or cut the resourcesneededto do it; increasethe national health budget if hospitalscontinue to havebudget overruns.' What the documentarydoesnot show, and what it is structured to suppress, areissueswhich callclinicians'demandfor clinicalautonomy and unlimited expenditure seriouslyin question.Suchissuesare clinical practicevariation (which is to do with, for example,a specificoperationrateof 10per 100,000in Sydneyat a costof say$1000, which might contrastwith a rateof only 3 per 100,000in WaggaWaggaat a costof only say$225),wastefulpractices(for example,eachtime a patient shiftsto anotherward more tests- often onessheor he hasalreadyhad - are orderedby different doctors), and the widespreadoccurrenceof clinical practiceswhich are not basedon medicalscientificevidencebut on personalor hospital-specificexperiences, andwhich at times (Walton, perpetuatequestionableand sometimesevenfatal treatments 1998). It is obvious now that the scenewith which this chapterbeganbearsa significancethat goesfar beyond what the woman clinician has to say.It is a continuous

1 85

186

'ISUAL ANALYSIS

5l-secondshot,which representsa very long'turn at talk'. It is a hand-heldcamera shot, which infusesthe portrayal with immediacy, urgency and dynamism. It is a medium-closeshot which at times becomesa closeshot, suggestingcloseness and intimacy (seeChapters2 andT).It accordsthe clinician direct address:she,and not the administrator, is given the opportunity to reflect on the health caresituation at length,on her terms and in her own territory. It givesthe clinician the opportunityto interpersonalizeheropinion: 'cardiology is . . . a life threateningand an acutesituation' and'they'restill somebody'smother or father'.In sum, closeanalysisofthis shot alone bearsout how televisiondocumentaryis a prime sitefor creatingsubtleand powerful kinds of discursiveimpact. My commentsabout this documentaryso far havebeenrather ad hoc. It will be useful to look in more detail at the range of filmic deviceswhich are used to constructargumentativeperspectiveor, ifyou like, discursiveimpact. In what follows I will thereforepresentsomeof the main categoriesand tools that play a role in a kind of filmic analysisthat aims for both deconstructiveand socialsignificance.

ANALYSING FILM AND TELEVISION Tele-filmic analysiscanbe done in a wide variety of ways,including thematic, 'auteur'centred,psycho-analyticalor symptomatic, structuralist and semiotic perspectives (seeBordwell, 1989,for a history of trends in film theory and modes of analysis). While eachof theseperspectivesprovidesinterestingcontributions, they all tend to be concernedwith establishingwhat the tele-film is ultimately'really about' (that is, their content).This may not givedue attention to issuesregardingthe sociopoliticsof readingposition ('abovecontent'),nor to the structuralimportanceof subjectselection, framing and editing ('below form'). In contrast,the analyticalstrategiespresentedhere aim to link the tele-film's sociopoliticalintertextualitiesto the waysin which it hangstogetherfrom one second to the next. In that regard,this approachis ambitious. The approachtaken here derivesfrom what hasbeentermed socialsemiotics (Hodge and Kress,1988;Lemke, L995;Thibault,1997).Socialsemioticsis concerned with the political understandings,the readingpositionsand the practicalpossibilities which analysismakesavailable(seeChapter 7). Socialsemioticspromotes detailed analysis,but its startingand end point is about situatedpraxis (Iedemaand Degeling, forthcoming). It acknowledgesthat the analyst'sown readingposition is likelyto guide her or his interpretations,but it seesthat asa strengthrather than a failing. Analysis is a sociopoliticalrelevance,not sometheoreticalabstraction. In fact, my reasonfor choosinga hospital documentaryfor analysisis that I know that the story is not assimple asthe documentarywould like to haveus believe. I want to find out why the documentaryis so powerful, rather than just be content with being saddenedby Mrs Biggsor uplifted by heart-patientMick's recovery.This documentaryis - besidesbeing a moving account - a political statement,and it demandsan informed readingand reply.

A N A L Y S I N GF I L M A N D T E L E V I S I O N

Socialsemiotics Socialsemioticscentreson the issueof how I, the viewer, am positionedby the telefilm in question,and how I seecertainsocialallegiances and valuesasbeing promoted over others.In that sense,socialsemioticsdeniesthereis a gapbetweentext or product and audience.Myanalysisis a readinglargelyinfluencedbymysocial, ethnic,economic, genderand knowledgebackgroundsor'coding orientations'(Bernstein,1990).I make no truth claimsfor the resultsof my analysis,but I do claim to be ableto support my claimswith systematicevidenceand basemy political argumentson them.2 A social semiotic analysisaims to enableus to question the ways in which 'social the tele-cinematictext presents reality', and should provide meansfor me to 'talk back' (Nichols, 1981:95) rather than be overwhelmedby what is, ultimately, a very sophisticatedand powerful medium. It should enableus, in other words, to see the barebonesof the documentary,its nuts and bolts, enablingus to engagewith the medium in a systematicand informed way. Finally, in contrast to traditional semiotics(seeChapter 4), socialsemiotics 'signs', does not focus on but on sociallymeaningful and entire processes('texts'). The signis an analyticalcategory;text,bycontrast,is a socialcategory.Texts,then, are definedasbeing the semioticmanifestationof material socialprocesses. Texts are marked off by sociallyrecognized(that is, sanctionedand perhaps policed) beginningsand ends.In that sense,books, films and televisionshowsare texts,but so are birthday parties,interviews,telephonecallsand football games.But while birthdaypartiesand telephoneconversationsand the like takeplacein ordinary or'real'time and spaceand can be calledpresentations, films and televisionproducts constructtimes and spaceswhich obeyand bearout thosemedia'slogic(s),and which are thereforere-presentations. Our concernhere is with re-presentations.

Socialsemioticanalysis of tele-film Tele-filmsare abstractfrom time and space:they only show asmuch asis considered artisticallyor logicallynecessary. This is what (following EtienneSouriau) Christian Metz calls'diegesis'(1974:98; seealsoNichols, 1981:81).Diegesispoints to the 'real' differencebetweenwhat the tele-film implies happenedin time and what the film itself actually shows.What the tele-film shows is alwaysand necessarilyless and differentfrom that which imputedlywent on in'real' time.3The hospitaldocumentary for example,spansthe last five months of the financial year,but lastsitself only half '[in] an hour. That is, it compresses time and space,or, in Metz's words, the chain of imagesthe number of units liableto occur is limited' (tgl+:gg). Tele-film constructsits own time(s) and space(s)usingspecializedtechniques. One of thesetechniquesis continuity, or continuity-editing (Nichols, 1981:85). Continuity-editing aims to smoothenout the gaps('cuts') which mark diegeticshifts 'shape' of focus.It thus naturalizesthe ofthe edited sequence.While spatialcontinuity

187

188

HANDBOOK

OF VISUAL ANALYSIS

is constructedby making visual fieldsoverlap,temporal continuity is often achieved by maintaining sound continuity acrossvisual cuts.The issueof continuity becomes important when we want to decidewhether a tele-film is constitutedpredominantly (realisingabsence of scenes(realisingspatialand temporalcontinuity) or sequences of such continuity). The construction of continuity has important implications for how we see(read:construct!)reality(seebelow). Socialsemioticsfocuseson thesetechniquesto highlight not only what was edited in and how, but also to show what was left out and thus constructedas unimportant or asnatural and taken-for-granted.Next to this structuraland intertextual analysis,socialsemioticsis alsocruciallyconcernedwith reasoningabout the choices film and televisionproducersmake in relation to the socioculturalfieldswhich they decideto hone in on. In what follows, I will first proposesix levelsof tele-film analysis.Then I will showhow visual,linguistic and soundfeaturesof tele-film areanalysedand what such analysiscan reveal.Finally, I will point out that there is never a simple way of determining what specifictele-filmic featuresmean,suchashand-heldcameramovability, low cameraanglesor fastcuts.Featuresarealwaysembeddedin continuouslychanging cultural trendsand tastes,or what havebeencalled'modalities'(Lotman,1977:59; Kressand van Leeuwen,1996:l59ff.). It is thesemodalitieswhich will ultimately determinewhat meaningsspecificfeaturesarelikely to attractin the contextsin which they aremobilizedor read (seeChapter4).

SIX LEVELS OF ANALYSIS Table9.1 setsout the six levelsof analysisproposedhere.Thesesixlevelsbring together analyticalcategoriesoften usedin film theory (frame, shot, scene,sequence)with categories usedin genreanalysis(stage,genre). At the lowestlevelof analysis( I ), the frame is what the analysttakesto be the salientaspectof a shot.With long shots,you may decidethereis more than one salient framewhich needsto be included in the analysis.The 51-secondshot discussedat the beginningof this chapter,for example,could be representedby severalsalientstillsor frames,showing how the clinician is shown from different anglesand distancesand in different locales. At the next levelup (2), shotsareuncut cameraactions.Shotsarelike 'cells'or 'distinct spacesthe succession ofwhich . . . reconstituteshomogeneousspace'(Mitry, cited in Heath, 1986:395). The clinician'sspeechquoted at the beginning of this chapter,for example,is filmed asone uncut, continuous shot.a One level up again (3), scenescomprisemore than one shot. The defining characteristicof scenesis their continuity of time and place:'the scenereconstructsa unit still experiencedas being "concrete":a place,a moment in time, an action, compact and specific'(Metz, 1974:127-8). For instance,after the clinician fi.nishes speakingsheopensa door and entersa room. The scenecutsto the insideof the room and pansto show her crouchedin front of a cupboardfull of surgicaldevices.All this takesplacein the sametime and spaceand is part of the samescene.

ANALYSING

Table 9.1

FILM AND TELEVISION

Six levelsof tele-filmanalysis.

1 Frame

A frame is a salient or representativestillof a shot

2 Shot

In a shot the camera movementis unedited(uncut);if the camera's positionchangesthis may be due to panning,tracking,zooming,and so on, but not editing cuts

3 Scene

In a scene the camera remains in one time-space, but is at the same time made up of more than one shot (otherwise it would be a shot)

4 Sequence In a sequence the camera moves with specific character(s) or subtopic across time-spaces; when it is hard to decide whether you're dealingwith a scene (1 time-space)or a sequence(multiple time-spaces), this is because editors may render time-space breaks as either more obvious (-> sequence boundary) or less obvious (-> scene boundary) Generic stage

6 Work as a whole

Roughly,stages are beginnings,middlesand endings;each genre has a specific set of stages: narrativestend to have an orientation, a complication,a resolutionand maybe a coda; factualor expository genres may have an introduction, a set of arguments or facts and a conclusion, or an introduction and a series of facts or procedures Dependingon the lower levels,the work will be more or less classifiable as a particular genre; the primary distinction is between 'narrative' (fictional,dramatic genres)and 'factual' (expository,thematic, issueoriented genres); genres are predictable relations between socialcultural,industrial-economicand symbolic-mythicorders

'alternating slmtagmas' Scenesareoften structuredusingwhat Metz refersto as (1974:103):theseareshotsof differentpeopleor objectsparticipatingin one and the sameinteraction - a game,a pursuit or a dialogue.For example,atrypicaldialogue sceneis often structuredasfollowss: 1.Establishing shot. . .;2. Longshot(mastershot);3. Medium-twoshot;4.Reverse 6. Cut-away(or angles(over-the-shoulder shots);5. AlternatingMediumclose-ups; shot(usuallya reverse insert);7. AlternatingMedium close-ups; 8. Re-establishing angleor a two-shot)(Branigan,1975:75) Our decisionasto whether different shotsbelong to one and the samescene will depend on the degreeof temporal and spatial continuity or discontinuity we perceiveto link the shots,and this is rarelyclear-cut(Nichols, 1981:219dealswith this issue).The tele-film asa whole will be structuredaccordingto certainrhythms at each oftheselevels,and analysisofrhythmicboundaries mayhelp guideour decisions(van Leeuwen,1985;Martinec,1997;seebelow). At the fourth levelup (4), sequences will comprisea rangeof contiguousscenes which are linked not on the basisof spaceand time continuity, but on the basisof a thematic or logical continuity. For example,we seeMrs Biggsin emergencywaiting for a bed; we seeher husbandpacing around in confusion, and later he staresat his sickwife in despairashis daughtertriesto wakeher up. Then follows a sceneof several

189

190

HANDBOOK

OF VISUAL ANALYSIS

minutes in which a doctor and a socialworker convincethe family that Mrs Biggsis unfit to go home and must go to a nursing home. Finally,we seeMr Biggswandering around his old house,sayingamongstother things: 'I only carefor her to comeback.' All thesetakeplacein different placesand times,but centreon the sameissue- Mrs Biggs - and thereforebelong to the samesequence. In brief, then: 'The shotis a sectionof film exposedduring a singletake.A scene is comprisedof one or more shotsoccurringwithin one time and place.A sequenceis composedof a group of sceneshavingdramaticunity' (Oumano, 1985:160). Moving further up the'hierarchy', sequencescombine into genericstages (5), which in turn realizethe specificgenrethey belong to. Stageboundariesmark significantshiftsin the narrativeor documentary.For example,charactersmaydisplay a shift from resistanceto acceptance, from pursuit to confrontation, or from struggle to abandonment.In a documentary,a new argument may be signalledby a change in topic, time or place.Suchnarrativeor documentaryshiftsor'stage boundaries' are often further highlightedby one or more editing features(a changein camerause or effect,or a changein the music or sound track). A stageboundary may alsoinvolve a changein how the charactersarepositionedin relation to one another (especiallyin fiction film), aswell asa changein how I am positionedasviewer by the film. In short, stagesare those elementswhich tell us where we are in the overall filmic text: 'this is wherethings arestartingto go wrong', 'this is wherewe enter another argument', 'this is where things come to a kind of conclusion',and so on. Stagesstructurethe tele-film or'genre' (5) asawhole. Broadlyspeaking,we can distinguishfactualor 'expositorygenres'and'narrativegenres'(afterNichols, 1981, 1991).6Other labelshave been used to capture this distinction, like van Leeuwen's ( 1985: 223)' thematic'versus'dramatic'filmic texts. Narrativesturn on the resolution of somekind of contradiction, problem or (Nichols, lack 1981:7a). This resolutionis cruciallyachievedthroughtime andon the basisof characters'involvement.Inthe classicalHollvwood fiction film. the narrative presents psychologically definedindividualswhostruggle to solvea clear-cutproblem or to attainspecific goals.In thecourseof thisstruggle, thecharacters enterinto conflict with othersor with externalcircumstances. Thestoryendswith a decisive victoryor defeat,a resolutionof theproblemanda clearachievement or non-achievement of the goals.(Bordwell, 1989:18) Bordwellconcludesthat the 'principal agenryis thus the character,a distinctiveindividual endowedwith an evident,consistentbatch of traits, qualitiesand behaviours'( 1989: 18). In short, fiction films centre around, on the one hand, a problematicwhich requiressomeform of resolution,and, on the other hand, actionsby characterswhich structurethesefilms'causallogic and progress(Nichols,l98l:76). Often narrativespromote a liberal-humanistethoswhich constructsindividual charactersaseither manipulating a calculativerationality or asat the mercy of a fatal flaw, or both. Suchconstructionslimit the causesbehind what peopledo, the consequencesofwhat they do and the changesthey undergo.In reality,of course,what people do and what happensis alwaysand necessarily over-determinedby all kinds of factors,

A N A L Y S I N GF I L M A N D T E T E V I S I O N

albeitto varying degrees.So narrativeshaveto simplifuwhat goeson or would go on. They do this to be ableto imbue individuals' actionsor specificeventswith narrative significanceand logic. Thus, it is in both structureand purposethat narrativesdiffer 'A radicallyfrom life aswe experienceit: narrativehas a beginning and an ending, a fact that simultaneouslydistinguishesit from the rest of the world' (Metz, 1974:l7). If the tele-film appearsto makea truth-claim for its content (not its message!) 'expository it is, after Nichols ( 1981),an genre'- and this is alsosubjectof courseto 'imagined'. 'argument Expositionsturn on it being'realistic'ratherthan or description' (Nichols, 1981:81), and thus derivefrom the principle of how certain issuesor facts link to others.TExpositionscohereinsofar asthey centreon specificissuesor individuals.But, unlike in narratives,this coherenceis a logicalcontinuity, in contrastto that which links specificindividuals'actions and the times and placesof their enactment. The broad distinction betweenexpositionsand narrativeshasbeenproblematizedrecentlysincedocumentarymakers,in doubt as to whether they can portray anybodyor anything objectively(cf. Clifford and Marcus, 1986;Marcus and Fisher, 1986),havestartedto liberallyusefictional devices.In otherwords, factualityhasnow 1991).At beenacknowledgedasbeingafilmicconstructionjustasisfiction(Nichols, the sametime, fiction films havestartedto becomemore and more'documentary-like' in the way that they presentrealistic portrayals of the life of heroin addicts,of the plight of the working class,or of the inhumane practicesof the nation's bureaucracy. 'Third (world) films' (Gabriel, 1989; PinesandWillemen, 1989)alsooften challenge receivedWesterndistinctionsbetweennarrativeand docum entary(think, for example, of the Iranian frIm TheRunner,Iedema,1998a).

T H E M U L T I . F U N C T I O N A LN A T U R E O F M E A N I N G . M A K I N G Asidefrom the six levelsdiscussedabove,socialsemioticsworks with yet anothertool: the hypothesisthat all meaning-making alwaysdoes three things simultaneously. According to this hypothesis,all meaning-making,whetherthe imagesin fiction film or documentary,the music, the actual talk and even the noise-sound track, always (Halliday, 1973,l97B).These performsthreeoverarchingfunctions,or metafunctions 'representation','orientation' 'organization',8 three metafunctionsare and Representationconsidersmeaninginsofarasit tellsus about the world in some way.Dependingon wherewe standand who we are,we can saythat the documentary we startedthis chapterwith is about a hospital with budget problems,or about the problemsdoctor managershavein balancingbudgets,or about hospital administrators imposing economicrationalism on patient care. Thus, under the headingrepresentation,we talk about what meaningsrepresent visually,verbally,musicallyor sound-wise.Accordingly,we canaskwhatis the subject which the shotsportray and what it, he, sheor they is/aredoing. For example,do we dealwith a doctor at work on a patient or an administrator addressinga meeting,or a computer screenshowingthe inside of a man's heart or a financial spreadsheet? We can askrelatedquestionsabout the music track.eIs it etherealchurch-like music suggestingspirituality and'higher values'(like during the openingshotsof the

191

192

'ISUAL

ANALYSIS

hospital documentary),or is it electronicand avant-gardecreatingassociationswith technologyand science?Is the soundtrackabout office noises,such as the shuffling of paper or the creakingof chairs,or is it about ward noises,the squeakingwheelsof hospitalbeds,footstepsechoingthrough corridors, or the tinkle of surgicaltools and the bleepingof heart monitors?And is the verbalor speechtrack about patients,their sufferingand their well-being,or is it about money, budgetsand schemesaimed at balancingthe books?All thesequestionsaddressissuesof representation. Second,orientationis to do with howmeaningspositioncharacters and readersviewers.The waysin which the administratorsarepositionedin the hospitaldocumentary are distant,isolatedand static.Dependingon other aspectsof meaningthat make up the documentary,this may discourageus from identifring with them and give the impressionadministratorsare cold and calculating.It is in contrastwith how the cliniciansarepositioned:intimate, socialand dynamic.Theyinteractwith others:they are shown in the sameshot with other cliniciansand with patients.Their interactions suggestthey are social,they do 'real'work and they are concernedand caring. Specificissuesthat are important hereinclude the following. Doesthe camera usea high angle,a low angle,a side-onor oblique angle,or is it positionedbehind the subjectin question?Do we seepeoplein closeup or long shot?Doesit movewith the subject and in that way construedynamism,urgencyand immediacy,or doesit keep its distanceand stand steady?All theseissueshave important consequences for how (see subjectsarepositioned Chapter7). We can also ask what inter-personalizingmeaningsthe soundtrack makes. In van Leeuwen'sterms (1999:28),suchquestionsincludethe following.What degree of socialdistanceis construedbetweensound and the listener/viewer?Are we more likely to be impressedby the hustle and bustle of a busyhospitalward, by the swooshing pulsingsof a patient'sblood rateand by the technicalbleepsof a heartmonitor, than by the more domesticatednoisescoming out of the meetingroom - the rustle of papers, the squeakingof a felt pen on a whiteboard, the measuredspeech,the creaking of officechairs,or the rattle oftyping on a computer keyboard?The busynoisesaremore likely to inter-personallyappealto us than the controlled soundscoming from the meetingroom. Third and last,organisationconcernshow meaningsaresequenced and integrated into dynamictext. The visualediting of the documentaryinevitablyimposesa peculiar semiotic structure and rhythm: beginning, middle and ending; problem-solution; argument-in-favour,argument-against;and so on. Suchsequencings haveto do with how meaningsarelinked together,in what order and in what kinds of rhythmic units. Rhythm andhowit interweavesspeech,sound,movement,imageeditingandmacrotextual structuring is an important featureof the organizationof filmic text. Moreover, the rhythmic cadencesof thesesequencingshave significant consequences for how 'read' we the text (van Leeuwen,1985;seealso,vanLeeuwen1986,1989,1991,1996). We will look at this in relation to specificexamplesin the next section. The point of asking all thesemetafunctional questionsis that their answers will giveus a senseof what kinds of patternsprevail.l0More importantly, they may give cluesabout how the various patternsenrich eachother. For example,we know that the documentaryis about doctorsand hospitaladministrators.We alsoknow that the

A N A L Y S I N GF I L M A N D T E T E V I S I O N

orientation of the documentaryis to favour the clinicians,with their longerexposures, their more dynamic appearances,and so on. In consonancewith this, the textual organrzation of the documentary is such that each of the administrative-financial sequences(whose orientation is formal and rationalizing) is both precededand followed by a highly dramatic and inter-personalizing sequenceabout one of the patients.The point is that these organizational,orientational and representational patterns and choicesenhanceand reinforce each other. In the caseof the hospital documentary thesepatternssubtly downplay the signifi canceof the formal administrativeand managementmeetingsand highlight that of the patient storiesand their hospital experiences.

S T R U C T U R I N GT H E T E L E . F I L M : E D I T O R I A L R H Y T H M Considerthefollowingextractfromthe documentary(Figure9.2).The extractrepresents the first minute or so of the documentary.The numbers under the framesare taken from the video counter, and are meant to give a senseof how long shots are on the screenfor. The extractcontainssevenshots:the hospital exterior (lalb); the admissions officeon the phone (2); a nurseon the phone on a ward (3); a femalepatientin bed (4); two nurseswalking through a corridor discussingbed issues(salb); the female patient again (6), and finally a nurse making a bed (7). As the counter numbers indicate,eachof thesesevenshotslastsaround eight digits; that is, they are more or lessequalin duration. Shots2,3, 5 and 7 show hospital staff engagedin locating sparebeds,not addressingthe camera.Shots4 and 6 involve the patient speakingto us, the viewers, about how desperatethe situation is. The shot that follows marks the beginning of the voice-over,announcingthat the hospital is over budget.Thereis thus a rhythmic alternationbetweenthe work itself presentedthrough indirect addressshots(2, 3, 5, 7) andthe commentson the hospital situation through direct addressshots(4,6). It is in the rhythmic cadencesof the text as a whole that speech and sound, movement, music and moving image interweave.In fact, and referring to film, van Leeuwen arguesthat rhythm is the prime organizerof filmic meaning: 'rhythmic grouping segmentsthe text, at the level of perception,into units which are not only rhythmically but also semantically coherent.Without meaning anything in itself, rhythm is neverthelessa necessarycondition for meaning'(1985:223). In this view, rhythm is the prime meansfor creatingsignificancesin text:

Figure 9.2 Opening scene of Hospital: An Unhealthy Busrness.

193

194

HANDBOOK

8tr] trltrl E o otf a E E E

.'DE

loc Itrtr lI E o tcr

trtr otr -j n '1

OF VISUAL ANATYSIS

trE

Melbourne - a i - -

t-1

1(b)

5(a)

|

|

I

ANALYSING

FILM AND TELEVISION

conditionfor for [. . .] anelementto occuron thebeatof therhythmremainssufficient in thechain. . . whatis it to beperceived asmoreprominentthanthe otherelements asmoreimportant,more rhythmicallymadeto appearprominentwill be perceived of whetherit is moreimportantin an worthyof attentionthanwhatis not - regardless a sound,a cameramovementon a objective sense. Thus,by placinga word,a gesture, momentrhythmicallyprivileged. . . , the editorcanmakeit salient,drawtheviewer's attentionto it. (vanLeeuwen, 1985:222) The argumentheredrawson speechresearch,wherepsycholinguisticexperiment has shown that subjectsreact more to and better remember accentedsyllablesthan unaccentedones.On this basisvan Leeuwenarguesthat'the segmentationof film texts for the purposeof structural analysisshould be basedon the rhythmic structure of film, into shots' (van Leeuwen,1985:223). rather than on ils ddcoupagef"segmentation"] The best way to explain how this aspect of the analysisworks is by way of a further example.Crucially, the rhythmic alternation describedabovepervadesthe restof the documentaryaswell. Looking at how the documentaryfurther unfolds,we 'dialogue' betweenthe two notice how the editing and cutting construesa kind of voicesin this debate:the imperativeof patient careversusthe constraintsof hospital funding. Table 9.2 setsthis alternation out in broad terms. This section of the documentary is structured around Mick's heart operation and the excessivecosts 'stenting' (inserting of a metal tube into one of his cloggedarteriesto open it up again). The longer extracttabulatedin Table 9.2 showsa rhythmic alternation between 'patient care'voice the two main voicesin the debate(markedA and B in the table).The (A) centresaround Mick's illnessand successfulheart treatment thanks to stenting. 'balancing The the budgetvoice' (B) centresaround administrativemanagerstelling cliniciansthey need to cut back the number of stentsthey usebecausethey are too expensive. In the introduction to this chapterwe commented on the waysin which the documentaryasa whole positionsthe cliniciansdifferently from the administrators. The cameraand the editing constructthe doctors asactors:they do operations,they wield surgicaltools, they install the stents,they cure patients,and so on. The administrators,by contrast,are constructednot asdoersbut astalkers.Following an almost predictablerhythm, this'doing-talking' contra-distinction reappearsthrough the documentary. In fact, it organizesthe documentary. But, as van Leeuwensays,that which is allowed to occupy or representthe salientrhythmic beats'willbe perceivedas more important' (1985:222).To get a senseof how this doing-talking contrast stacksthe casein favour of the clinicians, considerthe following extract(rowA4 in Table9.2) from one of the operationscenes (Figure 9.3). In this scene,there is very little talking. When someonedoestalk, the commentsrefer to the progressof the operation or to the movementsof the inserted deviceshown on the computer screen.In other words, medical talk relatesdirectly to the'reality' of the clinical work. This extractshowsthat the clinicians(both doctorsand nurses)arepredominantly actors:that is, they engagemost often in doing processesrather than saying (asdo the administrators).As argued,when they do engagein sayingprocesses, processes

195

196

H A N D B o o Ko F v t s u A L A N A L y s t s

Table 9.2 The sectionon 'stenting'. Step in the argumqnt

Scene

Vi

:

fu"

u=* tr#r,9 -

r,i(i:r:;..@,.. lq = rt!4:: @t-ix@;.Ks.i!@,

A1

Introduction of the patient: Mick

Mick driving a car delivering mail; Mick entering the hospital

Voice-over: 'To balance the books the hospital must treat more patients. Those who will benefit most of this plan are people sitting on waiting lists, people like Mick, the local post man . . .

81

Meeting where David Hillis (manager) gets Keith Stockman (clinician)to

DH at whiteboard; KS seated; DH and KS never in the same shot

DH: 'We can't afford stenting for the rest of the year'; voice-over: 'Keith Stockman is on the line.'

explain his budget overrun A2

Operation: doctor speaking to Mick

Mick in bed, doctor and nurse talk to him

'We'lltry and make you run on six cylinders.'

82

KS and DH on their own; KS proposes 25% operation level

KS and DH seated in DH's office sharing desk space

DH:'2OVois where l'll try and make it sit for the rest of the financialyear'; 'what l'll do is just go bang and stop the lot.'

A3

Female clinician (SG) monologue

Camera follows clinician through corridor

'They're stillsomebody's

83

Clinician points out cost of stents in cupboard

Clinician pointing at boxes in cupboard

A4

Operation procedure; doctor arguing that Mick needs stent

Bloody hands manipulating tools, computer screen showing vascular details

84

Meeting between KS and administrator who argues that non-stenting is better for the hospital budget; the more often a patient comes back the more treatment units the hospital can mark up

Clinician on the left (Given) listening to administrator on the right (New)

A5

The operation

85

DH tells cardiologists that he needs to put a stop on their unit's expenditure

AG

Operation: Mick speaks from his bed; Mick says goodbye to others in his ward

mother or father, they're still deserving the respect to be treated.' 'So there's an awful lot of money in this cupboard, well over a hundred thousand dollars I'd sav.'

'lf we leave this now, this man will come back' Glinicians gathered in office; DH never in same shot as them Doctor: 'You realize you're costing the hospitalan arm and a leg here'; nurse says: 'lt went very well' [churchy music starts againl

A N A L Y S I N GF I L M A N D T E L E V I S I O N

B6

woman walking through room full of files, picks file, then shown seated in front of computer typing in numbers

doctors are reported to have achieved agreement on 25% operation (rather than DH's 20%)

Table 9,3

Voice-over:'Calculate his worth': Mick filed away as $60O0 cost; 'a numbers game','coding patients has become an obsession'

Summarizing the analysis.

lr

t;; Representation (ideation): what is it about?

A lot of the shots that include doctors also show action or dynamic vectors (arms connecting doctor and patient, etc.); administrators -manager shots tend to lack vectors; doctors are often shown interacting with multiple others (in contrast to administrators)

Doctors enact temporally organized scenes, such as operations or patient consultations;administrators-managers are shown in less logically connected snippets of interaction (sequences)

The doctors enact curing and caring narratives centring around specific people, while the administratorsenact financial management narrativescentring around'abstractideas' and numbers

Orientation (interpersonality): how does it enact the social?

Doctors are often shown closer up, in more colourful and socially significant surroundings than the administrators -managers (who are often portrayed using a low camera angle, with relativedistance, and lackingthe colour and social intimacy of the doctor-centred interactions

Doctors show their sense of social responsibility by curing patients or facilitating their death; administratorsmanagers enact a thematic consistency (meetingfinancial deadlines)but are not implicated in social relations bevond their official administrative role

The doctors are positionedas those who deal with and face what is real - life and death - and they thereby monopolizea host of inter-personalizing resources.The administrators are positioned as those who deal with what is imaginaryand 'unreal'- finances, mobilizingcontrol ('mustness', see ledema, 1998b)

Organisation (textuality): how is it put

lnitiallyadministratorsmanagers are positioned as given (left of screen) and doctors as New (right of screen); once into the documentary this pattern reverses, positioningthe doctor consistentlvas Given

While the doctors engage in long scenes (which temporally unfold)the administrators engage in short sequences (which have no temporal but a thematic logic). Doctor scenes are constructed on the basis of less abstract ('closer-tohome') thematics (time), while administrator sequences are constructed on the basis of a conceptual thematics (finances,overruns,etc.)

The doctor-centred segments punctuate the administrator-centred ones in such a way to suggest (but not state explicitly!)that, whatever cost-overrunsthey incur, they have 'bloody'good reasons. This implicitargument structure favours the doctors'view: 'don't

together as semiotic construct?

bother me with your bean-countingproblems'

197

198

HANDBOOK

OF VISUAL ANALYSIS

ln Unhealttu Iusiness

1

,1\ Figure 9.3

An operation scene.

thesetend to be linked to what is going on with the patient or on the computer screen, showing the progressof the operation on the artery.The doctors' sayings,therefore, are shown to be grounded in the reality of the patient'sbody, a reality enhancedby both the blood on the doctors'glovesand the pulsing arterieson the computer screen. In sceneswhere the clinicians interact with the administrators, the camera stopstracking people and becomesstatic. Its shots separatepeople out rather than showingthem working together.The contextsnow aresilentboardroomsand offices dominated by the lifelesswhite of the walls,the furniture and people'sshirts.Finally, the reality around which everythingcentresis the computer screenwith its rattling sumsand totals.Here, reality is not the pulsing heart of a sick patient, or the urgency or medical care,but the mean and spurious logic of accountancyand economic rationalism. We could delvefurther into the detailsof image and sound and 'prove' that what we've exemplified so far is valid for the rest of the documentary. But social semiotic analysisis an interpretativeexercise,and not a searchfor 'scientificproof . Its purposeis to describehow textsconstruct'realities',and to arguethe sociohistorical nature of their assumptionsand claims (seeChapters4 and7). So rather than search for further proof, we should think about how the filmic mode hasbeen exploitedto

1 99

A N A L Y S I N GF I L M A N D T E L E V I S I O N

-__."+\

**R-i f f i r ; L N,-D

\

K t) ,I,=

HANDBOOK

OF VISUAL ANALYSIS

servespecificinterests.In this specificcase,documentaryfilm making hasservedthe interestsof a professionalminority; medical clinicians. Table 9.3 summarizesthe things I havesaidabout levelsof filmic analysisand about their metafunctionalaspects.

LIMITATIONSOF SOCIAL SEMIOTIC ANALYSIS Socialsemioticsdoesnot acceptthe claim that'how somethingis said/shown/done only relatesarbitrarily to what is said/shown/done'.For that reason,the analysispresented above placed the representationof the health debate (the 'what') in direct relationwith its filmic realization(the'how'). The analysisshowedthat the arguments put forward by the documentarywere significantly'helpedalong'by how they were realized. But there are distinct drawbacksto this form of analysis.First, the analysis presentedabovewaslaborious(seeChapter3). It takesa lot of time and concentration to do transcriptsof televisionor film material.Second,the analysiscanbecomequite technical: we were working with six levels of analysisas well as with three metafunctions. Third, while the analysis,though complex, may seemstraightforwardor evenmechanical,thereis a stronginterpretativecomponentat work. This is especially the casewhen the formal featuresof the material have been identified, and they then needto be made senseof. An interpretativeprinciple I applied without being explicit about it is that of 'redundancy'(Bateson,1973;Lemke, 1995).This principlehelpedme determinehow patterns 'reinforced' one another. For example,there is a commonality among the following featuresin the documentary: clinicians appear in groups; clinicians do operationson patients;and cliniciansrefer to pulsing arterieson a computer screen. Each of thesefeaturescontributesto clinicians' claim to relevanceand importance, emphasizing:their social relevance,their clinical relevanceand their scientific relevance.We could equallylist the featureswhich createredundanciesin the caseof the managersand administrators: their white shirts, their concern with talk over action, their filmic isolation,to name but three. A fourth limitation is that socialsemioticsis primarily concernedwith and is 'textual' explicit about structuresand not with categoriesofviewersand their readings of the texts.While emergingkinds ofWeb designmakethe distinction betweentext and readingtenuous(because viewersconstructtheir own'hypertext'(Snyder,1996,1998)), traditional tele-films (much like books) make it still possibleto do a hypothetically 'ideal' reading(Kress,1985),and to analysethe text in its linear entirety.Actual reading practices,of course,rarely conform to such idealizedanalyticalreadings(McHoul, 1991)(seeChapter4). The claim socialsemioticswould make,however,is that the kind of analytical readingpresentedabovegivesus a meansto understandand manipulatewhat might otherwiseremain at the levelofvague suspicionand intuitive response.It alsoclaims thattheabilitytosystematicallyanalysetexts,whetherliterature(Hasan,1985),diagrams, maps (Kressand van Leeuwen,1.996),paintings, sculpture, architecture (O'Toole,

A N A L Y S I N GF I L M A N D T E L E V I S I O N

1994),orevenchildren'sbedroom arrangementsand toydesign(van Leeuwen,1998), providesthe possibility for renegotiatingthe meaningsinherent in such constructs, rather than seeingtheseasfixed, irrevocableand natural. A final limitation of socialsemiotic analysisis that it readsthe tele-film text irrespectiveof the specificindividuals who were involved in its creation.This is not necessarily and alwaysthe case(Butt, 1998),but the analysispresentedabovemakesno referenceto the socialcircumstancesof the documentary'smaking: the conflictsand constraintsthe film-makersmayhaveexperiencedwhilefilming, the pressuresexerted on them by specificstakeholders,or the technicaland practical circumstancessurrounding its making. Someof thesedetailscould shedlight on specificcamera,script or sound choicesin the text and could well inspire different or richer interpretations. Having pointed to the shortcomingsof this kind of analysis,I would alsowant to point out why I think the analysisis powerful and important. Socialsemioticsdoes 'by not acceptthat textsare made accident':eachaspectof tele-film contributesto its meaningpotential in a meaningfulway.Also, it helpsus seethat the meaningsprojected 'natural' and self-evidentthan they might by the varioussemioticslistedaboveareless seemjudging from the way their materialitiesblend in with the other materialities and meaningsof everydaylife. Socialsemiotics,in other words, declaresanything 'fair game'.It setsno boundariesaround humansuseto exchangeand communicateas what is text and what is not, around what canbe analysedand what not. In that sense, socialsemioticsis a meansto'making strange'(Shklovskiiin O'Toole and Shukman, 1977),to consciousness-raising and to informed socialaction (Lemke, 1995;Iedema and Degeling,forthcoming).

CONCLUSION This chapterhasproposedwaysof dealingwith moving imagesin televisionand film. It hasclaimedthat the analysisof micro-aspectscan contribute to our understanding of how narrativesunfold, or how argumentsare put together and reinforced. The tools put forward aretwofold. First,there aresix levelsof analysis:frame,shot, scene, sequence, stageand genre.Second,therearethreemetafunctionsgoverningmeaning: representation,orientation and organization.Lessa tool than an interpretativeprinciple, redundancy,orthe questionasto howpatterns amplif'one anotherthroughlikeness, enablesus to make inferencesabout specificcameraand editing choices. With regardto the documentaryusedfor exemplificationof this kind of analysis,it was arguedthat its organizationfavoursthe clinicians' point of view in a host of different ways,most of which were touched on above.As documentary,the text exploitsresourcestypical of genreswhich aim to construean'objectiveview', suchas 'the omniscientnarrator' (Leahy,1996:45)whosecommentarystands'at a remove, abovethe fray' (Nichols, l99l: 197).Yet, and in tension with this, the programme relies heavily on editing style, camera use and sound editing to differentiate and calibratethe variousviewpointsthat areat stake. Importantly, when watchingprogrammeslike Hospital:an UnhealthyBusiness 'sounded'. we may be critically attunedto what is said,but lessso to what is shown or

2O1

2o.2

H A N D B O O KO F V I S U A T A N A L Y S I S

If we turn our attention to imagesor sound, we often have no other resourcesfor dealingwith them than intuition and commonsense.But if we cannot deconstruct editorial and camerastrategies,or visualsand sounds,a whole universeof meanings escapescritical notice. To mount a penetratingcritique of the kind of documentary looked at here, we need, in addition to a critical linguistic literacy, a critical visual-audialliteracy enabledby tools such asmade availableby socialsemiotics.

NOTES The conceptsGiven (left sideof the imageor frame) and New (right side)were adoptedby Kressand van Leeuwen(from Halliday's systemicfunctional grammar - seeHalliday, 1994) to distinguish the different values of the opposite sidesof visual space(Kress and van Leeuwen,1990,1996). 2 To locatethis claim within the self-conscioussphereof current film theory,socialsemiotics 'declares' its theoreticaltools, methodsand objectives,and aims for politically meaningful findings and arguments.Socialsemioticsdoes not exploit theoretical reflexivity to turn analyticallyvacuousor hedgeabout its conclusionsto such an extent that no one but the most determinedcan take issuewith them (Eco, 1990,1992). 3 Warhol's Empire,an eight-hour film of night engulfingthe Empire StateBuilding, is setin 'real'time, and getsus to questionour own (naturalized)expectationthat film-makingwill involve diegeticediting ('we only get the important bits'). 4 The averageshot length of classicalHollywood films was around 9-10 seconds.Frederick Wiseman'sHospitaldocumentary averages32 secondsper shot (Nichols, l98I:2Il; also Heath, 1986:fn. 36).Van Leeuwennotesthat'shotsgenerallylastbetween2 and40seconds, although shotsasshort asan eighth of a secondand aslong as 10minutes do occur' (1985: I

220-r). 5 Scenesaregenerallysubjectto two camerarules.First,the 180-degree rule ensuresthat 'one will alwaysfind the samecharactersin the samepartsofthe screen'(Heath, 1986:395);this is to prevent confusion about who is where in the two-dimensional filmic space.The 30degreerule prevents'jumps in space',ensuringa smooth line in from shot to shot (Heath, 1 9 8 63: 9 8 ) . 6 For the sakeof brevity and simplicity I will not addressthose experimental-poeticfilmic textswhich fall outsidethis classification. 7 Bill Nichols ( 1991:32ff.) distinguishesfour expositorygenres:classic'voice-of-God' commentaries;observations(minimizing the film-maker'spresence);interactives(wherefilmmaker and socialactorsovertly acknowledgeone another);and reflexives(whereattention is drawn to the form of the work itself). 8 Halliday'soriginal labelsare ideational,inter-personaland textual and appliedto language only ( 1994:135).Lemkeproposedthe parallelterms usedhereto talk about semioticsother than language(1989, 1992). 9 Van Leeuwenarguesthat in the caseof music, about-nessor ideation'rides on the back of inter-personalmeaning,while in visual representationit is the other way around (1999: 154). Hence the perfect complementarity of coursebetween film and music, and their combined force of impact. 10 Bill Nichols, a renowned film theorist, also uses 'metafunctions' to analysefilm and television,but talksabout them slightlydifferently:aesthetic(inter-personal),logic(textual) and diegetic (ideational) temporalities. He remarks on the parallel between these 'temporalities' and his three genericcategoriesof film and television:poetic (aesthetic), expository (logic) and narrative (diegetic).Those familiar with Halliday's metafunctions will appreciatethe similaritiesbetweenHalliday's and Nichols's categories.

F I L MA N D T E L E V I S I O N 2 O 3 ANALYSING

11

'Balloon'

here refers to an alternativebut outdated treatment of thesekinds of arterial problems.

REFERENCES Bateson,G. ( 1973)Stepsto an Ecologyof Mind. New York Granada. Bernstein, B. (1990) The Structure of PedagogicDiscourse:Class,Codesand Control Vol. VI. London: Routledge. Bordwell, D. (1989) Making Meaning Inferenceand Rhetoricin the Interpretationof Cinema. Cambridge,MA: Harvard University Press. (2): rca. Branigan,E. (1975)'The spaceof Equinox Flower',Screen,17 Butt, D. ( 1998)'Developinga map for Englishstudies',kqmote addresspresentedat the Australian SystemicFunctional LinguisticsAssociationConference,Adelaide,2B-30 September. Clifford, |. and Marcus, G. (eds) ( 1986) Writing Culture: ThePoeticsand Politicsof Ethnography. Berkeley:University of California Press. Eco, U. (1990) The Limits of Interpretation.Bloomington/Indianapolis: Indiana University Press. Eco, U. (1992) Interpretationand Overinterpretation,ed. S. Collini. Cambridge: Cambridge University Press. 'Towards Gabriel,T. ( 1989) a critical theory of Third World films', in J.Pinesand P. Willemen (eds),Questionsof Third Cinema.London: British Film Institute. Halliday, M.A.K. (1973) Explorationsin the Functionsof Language.London: Edward Arnold. Halliday, M.A.K. ( 1978)LanguageasSocialSemiotic.London: Edward Arnold. Halliday, M.A.K. (1994 [1985]) An Introduction to FunctionalGrammar (2nd edn). London: Edward Arnold. Hasan, R. (1985) Linguistics,Languageand VerbalArt. Geelong,Victoria: Deaking University Press. Heath, S. (1986).'Narrativespace',in P. Rosen (ed.), Narrative, Apparatus,Ideology:A Film TheoryReader.NewYork Columbia University Press. Hodge, B. and Kress,G. (1988) SocialSemiotics. Cambridge:Polity Press. 'Comparing Iedema,R. ( 1998a) filmic constructionsof confli ct:Bonnieand Clyde(USA) versus The Runner (Iran)', paper given at the 25th International SystemicFunctional Congress, Cardiff, 13-18 July. Iedema,R. ( 199Sb)'Appraising the health debatein televisiondocumentary, or how to objectifr bias',paper given at the SydneyInternational Workshop on Stance,17-18 December. Iedema,R. and Degeling,P. (forthcoming) 'Workplace research:putting discourseanalysisto work in organisational settings', Palmer Centrefor Clinical Management Researchand OrganisationalInnovation. University of New South Wales, Sydney,Australia. Kress,G. (1985) Linguistic Processes in Socio-culturalPractices.Victoria: Deakin University Press. Kress,G. and van Leeuwen,T. (1990) ReadingImages.Geelong,Victoria: Deakin University Press. Kress,G.andvanLeeuwen,T.(1996) London: Readinglmages:TheGrammarofVisualDesign Routledge. Leahy,G. (1996) 'Faith, fidelity and openness:rescuingobservationaldocumentary',in Media InternationalAustralia,no. 82, November:40-7. Lemke,J. (1989) 'Semanticsand socialvalues',Word,40 (I-Z),April-August: 37-50. Lemke, l. (1992)'semantics, semioticsand grammatics:an ecosocialview', paper presented at the International Congressof SystematicFunctional Linguistics, Sydney: Maquarie University. Lemke,J. (1995) TextualPolitics:Discourseand SocialDynamics.London: Taylor & Francis.

2O4

H A N D B o o Ko F v t s u A L A N A L y s t s

Lotman, I. (1977) Sdmiotiqueet Esthdtiquedu Cindma (trans. S. Breuillard), Collection 'Ouvertures'. Paris:EditionsSociales. 'ReadingS', McHoul, A. (1991) in C. Bakerand A. Luke (eds),Towardsa CriticalSociology of ReadingPedagogy: Papersof the XII World Congresson Reading.Amsterdam: Benjamins. Marcus, G. and Fisher,M. (1986)AnthropologyasCultural Critique:An ExperimentalMoment in theHuman Sciences. Chicago:University of ChicagoPress. Martinec, R. (1997) 'Rhythm in multimodal texts',mimeo. London: The London Institute. Metz, C. (1974) Film Language: A Semioticsof the Cinema.New York: Oxford University Press. Nichols, B. ( 1981)Ideologyand theImage.Bloomington: Indiana University Press. Nichols, B. ( 1991) Representing Reality:Issuesand Conceptsin Documentary.Bloomington: Indiana University Press. O'Toole, M. ( 1994) TheLanguageof DisplayedArt. London: LeicesterUniversity Press. O'Toole, M. and Shukman,A. (1977)'A contextualglossaryof formalist critical terminology', in M. O'Toole and A. Shukman (eds), RussianPoeticsin Translation(Volume 4). Oxford: Holdan Books. Oumano, E. (ed.) (1985) Film Forum: Thirty-Five Top FilmmakersDiscussTheir Craft.New York: St Martin's Press. Pines,f. and Willemen, P. ( 1989)Questionsof Third Cinema.London: British Film Institute. Snyder,I. ( 1996)Hypertext.Melbourne: Melbourne University Press. 'Digitial Snyder,L ( 1998) literacies:moving from pageto screen',papergivenat the University of SydneyLiterary Symposium'ChangingVisual and Verbal Literaciesin ClassroomPractice K-12', 19September. Thibault, P. (1997)Re-readingSaussure: theDynamicsof Signsin SocialLife.London: Routledge. 'Rhythmic van Leeuwen,T. (1985) structureof the film text', in Teun van Dijk (ed.), Discourse and Communication - New Approachesto the Analysis of Mass Media Discourseand Communication.Berlin: de Gruyter. 'The van Leeuwen,T. (1986) consumer,the producer and the state:analysisof a televisionnews item', in Elenor A. Grosz, Terry Threadgold, Gunther Kressand Michael Halliday (eds), Semiotics,Ideology,Language.Sydney:Pathfinder Press. van Leeuwen,T. ( 1989)'Changedtimes, changedtunes:music and the ideologyof the news', in lohn Tulloch and Graeme Turner (eds),Australian Television:Programs,Pleasuresand Politics.Sydney:Allen & Unwin. van Leeuwen,T. ( 1991)'Conjunctivestructurein documentaryfilm and television',Continuum, 5 0):76-114. van Leeuwen,T. (1996) 'Moving English: the visual languageof film', in S. Goodman and D. Graddol (eds),RedesigningEnglish: New Texts,New ldentirles.London: Routledge. van Leeuwen,T. (1998)'Multimodality', plenary paper given at the Australian Systemic Functional LinguisticsAssociationConference,Adelaide,28-30 September. van Leeuwen,T. (1999) Speech, Sound,Music. London: Macmillan. Walton,M.(1998) TheTroublewithMedicine:PreservingtheTrustBetweenPatientsandDoctors. Sydney:Allen & Unwin.

VisualCommunication is an excitingnewjournal beinglaunchedto providean internationalforum for the growing body of work in visual communication. Its definitionof the visualwill be broad and inc lu d e : . stilland movingimages . graphicdesigns . v i s u a l p h e n o m e n as u c h a s f a s h i o n , professional vision,postureand interaction . the builtand landscaped environment . roleof the visualin relationto language, music, soundand action Articles in Wsuaf Communicatfon will: . criticallyinvestigatehow the social world is constructed,representedand contestedin visualdiscourse . examinethe use of the visual in a rangeof sociological, anthropological, historicaland scientifi c researchareas . explorethe structuresand historiesof the l a n g u a g e sa n d t e c h n o l o g i e so f v i s u a l communication, and their relationto thoseof othermodesof communication . describeand contextualize (socially, culturally

Editors Theo van Leeuwen Universityof Wales,Cardiff, UK Carey Jewitt /nstitute of Education. UK Ron Scollon Georgetown University,USA Teal Triggs London College of Printing, UK

First lssue February2002 Three times a year February,June, October fSSN:1470-3572 Pricesavailableon request

an d h i s to ri c a l l yth ) e u s e o f th e s e vi sual lan g u a g e sa n d te c h n o l o g i e si n v i sualand multimodalgenres,textsand communicative events . searchfor waysof expandingthe resourcesof visualcommunication and theiruses

SAGEPublications,6 Bonhillstreet,LondonEC2A4PU,UK SubscriptionHotline + 44 (O)2O7330 L266l Website:www.sagepub.co. uk

Index The letter n after a number indicates an entry in the notes Abraham, E., 123 actor in narrative processes,742, 743 Adorno, T. W.,24 Allen, N.,73 analyticalprocesses,144 arbitrary signs,72, 199 Arnheim, R., 136,141 art history, 63, 7 l, 100-17 attributes,95, 98, 101,103,108-9, 111-13,144, 1 5 4 ,1 5 5 Bailey,D.A.,86 B a r t h e sR, . , 2 , 7 6 , 8 9 , 9 0 , 9 2 - 1 0 01,0 1 ,1 1 4 ,1 1 7 Bateson,G.,202 Baxandall,M.,71-2,73 B e l l ,P . , l - 2 , 7 , l 8 Berger,1.,76,95 Bergstrcim,I., 103, 108 Bernstein,8,787 Biermann, G., 128 body, 157,160,161, 162,164,165,179 Boehringer,C., 18 Bonnell,S.,88 Bordwell,D., 186,190 Branigan,E., 189 Brun-Cottan, F., 165, I77 Burgin,Y.,84,90n Butt, D., 201 cameraangle, 135-6 cameraposition, 75, 160 Cartier-Bresson,H., 7, 69 categorizationin images,75,95-6 Cederstrom,T.,45 centraliry 140,149 choice of methodology, 7, 63-5 Chomsky,N., 158,159 classification processes,143-4 Cliffiord,J., 191 closeshot, 146, 186,192 codes,7 l, 72, 75-6, 79, 94, 134 coding procedures(content analysis),32 coding reliability (content analysis),21-4 coding schemes,169,seealsoprofessionalvision Collier, l. lr, 38, 39, 44, 45,54 Collier,M., 2, 4,5, 6, 8, 38, 39, 44, 45,54 Collins, R., 102

colour, l5I, I52, 168-71 comparativecross-tabulation,20-1 comparative hlpotheses, 2, I3-l 4, 28 comparison and juxtaposition method, 51 compositional meaning, 138, 740, 747, 747-9, 154, 155,seealsoorganization compromise formations, 120 conceptualvisual processes,25, 143-5 connotation,2, 3, 27, 96-8, lI7 contact, 138, 145-6 content analysis,l-2, 6-8, 10-34, 43, 99*100 context of production, 68-70, 202 context ofviewing, 7,65-8,95, 102, 164-6, see a/soprofessionalvision contextualinformation, 2, 6-7, 36-8, 10 1, I 02, 107-8, 116,123,125,130-1, 157,I59, 162, 1644 continuity in film and video editing, 187-8 conventions,7 l-83, 101,134 photographic convention s, 7 3-6 photographic conventions and power, 7 8-83 social conventions as representedin images, 76-8 conversationanalysis,3, 157, I 58-61 Croft, S., 18 cultural stereotypes,96 cultural studies, seevisualcultural studies Cuvier, G., 107 Degeling,P.,186,202 demand pictures,30-1, 145 denotation,2,94-6,100, 117 depth offield, 80 descriptive dimensions in social semiotics, 138 detachment, 735,747 diegesis,84, 187,203n Diem-Wille, G.,3 digital technology (use in visual analysis),56-7, 178 direct image analysis,36,3846 disguised symbolism, 109, II7 distance, seesocial distance D o a n e ,M . 4 . , 8 4 D o i s n e a uR , .,68,69 du Gay,P.,64 editing film and video, 185, 197-8 Ensor,J., 115 Erickson,F.,52 t62-79 ethnomethodology,3, 5, 8,

20,4

INDEX

expectation,2, 13-14 expositorygenre, I9l, 192 eyeline, I43, seealsogaze facialexpression, 43, 79,80,143, L45 Feldman,M,122 Fisher,M., 191 focus,8l frame,51, 188, 189,202 framework analysis,138 framing,75,80,81, 149-50,I52, I55 Freud,S.,84, 85, 86, 119-21, 128,L29 frontal ang)e,75, 114-15, I38, I47 Gabriel,T., 191 gaze,34, 43, 75, 81, 83, 84-5, 143, 144, 145-6, 158-61,164,179 generalizationin images,95-6, 103, 151 genre,188,189,190,202 gesture,43,79, 143, 145-6, 16l, 164 Ghirlandaio, D., 108 given, 145,153-4, 185 goal (in narrative processes),142,143 Goffman,E., 31, 97,I38 Goldwater.M..78 G o o d w i n ,C . , 3 , 6 , 1 5 8 ,1 5 9 ,1 6 0 ,l 6 l , 1 6 4 ,1 6 5 , 1 6 6 ,1 6 7 ,I 7 I , 1 7 4 , 1 7 7 Goodwin,M.H., 161, 165,177 Graham,H., 138 Grivelli,C., 110 Group v individual portraits, 96, 184 Groupe p,92 Hall, E., 29,44,50 Hall, R., 178 H a l l , S . ,2 4 , 8 6 , 8 7 Halliday,M.A.K., 104,135,140,146,191,203n Hamilton, P.,69 Hampshire,D.,77 handheld camera,l8l H a s a n ,R . , 2 0 1 Haviland,J.B.,161 Heath,C., 161,177, I78 H e a t h ,S . ,1 8 8 , 2 0 3 n H e n n i n g ,M . , 6 6 , 7 0 HermerenG , . , 1 0 2 ,L 0 3 ,I 0 7 , 1 0 8 ,1 0 9 ,I 1 6 high angle, 135-6, 138, 192 Hindmarsh, 1., 178 Hodge,R., 186 Holland, J., 138 H o l l a n d ,P . , 8 5 Hutchins,E., 163, 166,177 iconic signs,72 iconographicalsymbolism, 100, 107-15, 116 iconography,2-3, 92, 100-17, I4l, I44 iconological symbolism, | | 5-17 ideal, 148

ideational metafunction, 140, 197,203n, identification (of viewer with image), 85-6,147 identifring representedparticipants, 43, 103, 107 identity, 65, 85-6, 89, 159 Iedema,R., 1,4, 186,I91,202 image act, 30, seealso demand pictures, offer pictures imagesin dreams,121 indexical signs,79-80 indirect image analysis, 46-7 information v alue, | 47-9 interactivemeaning 27,28, I40, 145-7, 197 interpersonal metafuncti on, I 40, 197 interpreting analytical results, 32-4, 58-9 interview methods in visual analysis,45-7,524, r27-32 intuition in visual analysis,58-9, 116 involvement,I35, 147 Iefferson,G., 160,l80n Jewitt,C., 1,3, 138,140,155n foseph,8'122 Kawakoto,Y., 178 Kendon,A., 52, 161 I(ein, M., 122 Kos, M., 128 Kress,G., 20,2840, 135,136,138,140,l4l,144, r45, 146,148,150,r5r, r52, 153,154,155n, 1 8 6 ,1 8 8 , 2 0 1 , 2 0 3 n K u h n ,A . , 8 9 Laplanche,|., 85 Latour,B. 163,165 Leahy,G.,202 LeBaron,C, 178 Lemke,J., 186,200,202,203n Levine,S.,46 lexis (visual),92, 97, 98-9, 141 light(ing),75,80 Lipka,J,57 L i s t e rM , .,2,5 long shot, 146, I89,192 Lotman, J., 188 low angle, 135-6, I38, L85, I92 Luff, P., 177, I78 Lupton, E., 153 L y n c h ,M . , 5 , 1 6 2 ,1 6 3 , 1 6 7 M a p p l e t h o r p eR, . , 6 8 , 7 4 , 7 5 , 7 6 ,8 1 ,8 3 ,8 6 Marcus,G., 191 Martinec,R., 189 materialityof the image,64,76,8I mathematizationof images,163 McHoul, A.,20I Mcluhan, M,96 McNeill, D., 161 media studies,62,63,90n, seea/socultural studies

TNDEX

medium shot, 146, 189 Mercer, K., 76 M e t z ,C . , 1 8 7 ,1 8 9 ,l 9 l Michelangelo,116 Millum, T., 138 Mirzoeff,N.,63 Mitry,I., 188 Mitscherlich, M., 128 modality, 20-1, 25, 151-3, 188 M o h a t t ,G . , 5 2 , 5 7 Mulvey, L.,84-5 M u n c h ,8 . , 1 1 5 narrative structure in (documentary) film, 185, 190.l9l,192 narrativevisual processes, 25, l4l-3,154 naturalistic modaliry I5l, I52 Nederveen-Pieterse, 1.,3, 102,103,105,106,107, 1 1 0 , 1 i 1 .t t 6 - t 7 new, 148,153-4 Nicholls,G.,I77 N i c h o l s ,B . , 8 3 , 1 8 7 ,1 8 9 ,1 9 0 ,1 9 1 ,2 0 2 , 2 0 3 n Nishizaka,A., 178 non-transactivenarrativeprocesses,143 non-verbal communication, 3, 43,47,79, seealso body, facial expression,gesture,pose Ochs,E., 161 offer pictures,30-1, 146 Ogborn,1.,l55n open symbolism,109,117 open viewing, 39, 43, 44-5 organization, 191, 193-5, 197,202 seealsotextual metafunction orientation, 791, 192, 197,seea/sointerpersonal metafunction O'Toole,M,201,202 Oumano,E., 189 Oyama,R., 3, 153 P a n o f s k yE, . , 2 , 7 1 , 1 0 0 ,1 0 1 , 1 0 2 ,1 0 6 ,1 0 8 ,1 0 9 , 1 1 5 - 1 6 t, t 7 , 1 3 6 P e i r c eC , .S.,79 photo elicitation, 36, 45-7,52 photographic records as data,2, 4-5, 49, 89-90 physiognomic stereotypes,96, I10 Pines,J.,191 pleasure,63, 65,84, 151 point ofview, 83, 85, 135-6, 138,147 pointing, 34, 166,175,176,178 Pontalis,J. 8., 85 p o s e , 4 37, 9 , 9 7 , 1 3 81, 4 1 ,1 4 5 ,1 6 1 posture, seepose power (in socialsemiotics),63,135 practice,34, 157,160,162,164,168,

20r professional vision, 167-7 I p r o f i l e ,l l 4 - 5 , 1 3 8 , 1 8 9

projectivedrawing test, 119, 127-8,132 proxemics, 29,43, 44, seealso socialdistance psychoanalytic cultural theory, 63 psychoanalytic image analysis,3-4, 84, lI9-33 Quantitative analysis,204, 434, 51-2, 99-100 Raphael,108 reaction processes,143 real, 148 redundanry,200 reflexivity, 162,174 Rembrandt,95-6, 103, 106 representationalmeaning,27, l0O, 102-7, 116, 140,141-5, l9l,197,20A, seea/sodenotation representedparticipants, 25, l4I, I43 researchquestionsin content analysis,14 rhythm in film and video, 192,193-9 Ritchie,J., 138,154 Robinson,1.D., 16l, L78 Rogoff,L,63 Rubens,P.P.,106 Sacks,H., 160,l80n salience,140, 150-l sampling, 14 Sandler,J., I 19 Saussure,F. de, 157 scene,188, 189,190,202 Schapiro,M.,2, 106, 114-15 Schlegoff,E., 160,161,170,173,180n scientific modality, l5l-3, 162-3 Scott,W., 23 segmentationanalysis(film), 194 Sekula,A., 5, 89 semiotic field, 166, 180 semiotic resources,13U0,155, 157 semiotics,2, 63,7 l, 72-3, 92-100, Il7, 157, 186, 187 seealsosocialsemiotics sensorymodality, 151, 757 sequence,188,189,202 setting,138,144-5, 155,161 Seznec,1., 116 Shklovsky,Y.,202 s h o t , 1 8 8 ,I 8 9 , 1 9 0 , 2 0 2 Shukman,A.,202 sign,72, 79, 80, 157, 165, 166, 187 Simpson,A., 79 sizeof frame, L46,seealsosocialdistance Slater,D., 73 S n y d e r1, . , 7 3 , 2 0 I s o c i ad l i s t a n c e , 2 9 , 8 1 , 8 9 , 9 61,0 6 ,1 3 8 ,1 4 6 ,1 8 4 , 192 seea/soproxemics social practic e, seepractice socialsemiotics,3, 8,25, 134-56, 186-205 S o n t a gS , .,76,89 sound and image relations in film and video, 43, 52.192

2O9

21O

rNDEx

Souriau,8., 187 Spencer,L., 138,154 S p i l J i u sE, . 8 . , 1 2 2 stage,188, 189, I90,202 stageboundary,190 Steele-Perkins, C.,82 stereotypes,95-6, 97, 106, i l0 Stevens,R., 178 Streeck,J., 161, 164,I78 structured analysis,43-4 style,94, 98 symbolic meaning, 2-3, 100,I07-I7, 144,seealso connotation syntax(visual),73,92,97,98-9, ll4=15, l4I, 153

Urry,I.,85 values(in content analysis),15-20,29-31 Van der Weyden, R., 106 Van Eyck,I., 10, 109,1I 1, I 14 Van Leeuwen,T., 2, 20, 2840, 7 I, 135, 136, 138, r40, I4l, 144,145,146,L4g,150,151,r53, 154, 188,189,rg0, lg2, lg3, lg4, rg5, 2ol, 203n variables,13-20, 29-3I, 43 vector, l4l,143 video recordsasdata,2,4-5,48-51, 160 viewer positioning, 81, 83-9, 135-6, 138, 145-7, I87,I92, seealsocontext of viewing viewer research,5,7, 16I,201, seealso profess-ionalvision visual anthropology, 2, 35-60 visual cultural studies,2, 8, 6I-9I Von Neurath,O., 153

technology(image-),56,62-3, 64, 73,83, t5l telefilmic analysis,183-205 text (visual),6, 15, 70, 169, 187, 193, 194,201 textual metafirnction.2T, 140,197,seealso compositionalmeaning therapeuticuseof imageanalysis,4,5 ll9-27 Thibault, P., 186 T i t i a n , 1 1 4 ,1 1 5 - 1 6 T o s c a n iO , .,90n transactivenarrative processes,143 transcription of video data, 160-1, 200 Trigg,R'177 Trollope,A., 111 Tsatsarelis,C., l55n

Warhol, A.,203n WellsL , .,2,5 W e l l s ,M . , 8 2 W h a l e n ,J . , 1 7 8 Wight, D., 138 Willemen, P., l9l W i n d , 8 . , 2 , 1 0 3 ,1 0 8 W i n s t o n ,8 . , 2 4 , 2 5 Woolgar,S., 162,163 word and image relations,6-7, 96, I03, 138, I49, 155

Ueno, N., 178

Yamazaki,K., 178