Cognitive Biases

COGNITIVE BIASES

ADVANCES IN PSYCHOLOGY

68 Editors:

G. E. STELMACH

P. A. VROON

AMSTERDAM

-

N O RTtl-HOLL AN D NEW YOKK OXFORD TOKYO

COGNITIVE BIASES

Edited by Jean-Paul CAVERN1 Ceiiter jbr Rescurch iii Cogiiitiw P.sykobgy (CREPCO)

Uiiir*cr.siry of Prowrice and Nurioiiul Cciirer.fiw .Scicnrijic Heseurch (CNRS) Ah- eii -Proi w i i v , Fru tice

Michel GONZALEZ Ccnrcr f b r Rcscuru*k iri Cogtiitire Psyc*liolo,qy (CREPCO) U 11i\*cr.siry($Pro wrce

urrd Nurioircil Ccirfcrfi)r Sc-icririjicRcscurrh (CNRS) Aix-crr- Prorcrrcc. Fruricc

AMSlEKDAM

NORTH-HOLLAND NEW YORK OXFORD TOKYO

NORTH-HOLLAND ELSEVIER SCIENCE PUBLISHERS B V Saia Burgerhartstraat 25 P O Box 21 I, 1000 A E Amstcrddin, The Netherlands Distributors for the United States and Canada: ELSEVIER SCIENCE PUBLISHING COMPANY. INC 65.5 Avenue of the Americas New York, N.Y. 10010. U.S.A.

L i b r a r y of C o n g r e s s C a t a l o g i n g - i n - P u b l i c a t i o n

Data

C o g n i t i v e b i a s e s / e d i t e d by J e a n - P a u l C a v e r n i . J e a n - M a r c F a b r e . Michel Gonzalez. p. crn. -- ( A d v a n c e s in p s y c h o l o g y , 6 8 ) Includes bibliographical references. I S B N 0-444-88413-0 1 . C o g n i t i o n . 2. S e l e c t i v i t y ( P s y c h o l o g y ) I. C a v e r n i . J e a n - P a u l . 11. F a b r e , J e a n - M a r c , 1 9 4 5 - 1 1 1 . G o n z a l e z . M i c h e l . I V . S e r i e s A d v a n c e s in p s y c h o l o g y ( A m s t e r d a m . N e t h e r l a n d s ) ; 68. BF311.C5512 1990 153.4--dC20 90-36435 CIP

1SBN:O 444 884130 ELSEVIER SCIENCE PUBLISHERS B.V., 1990 All rights reserved. No part of this publication may be reproduced. stored in a retrieval system. or transmitted, in any form or by any means. electronic. mechanical, photocopying, recording or otherwise, without the prior written permission of the publisher, Elsevier Science Publishers B.V./ Physical Sciences and Engineering Division, P.O. Box 1991, 1000 BZ Amsterdam. The Netherlands. Special regulations for readers in the U.S.A. - This publication has been registered with the Copyright Clearance Center Inc. (CCC), Salem, Massachusetts. Information can be obtained from the CCC about conditions under which photocopies of parts of this publication may be made in the U.S.A. All other copyright questions, including photocopying outside of the U.S.A., should be referred to the copyright owner. Elsevier Science Publishers B.V., unless otherwise specified. No responsibility is assumed by the Publisher for any injury and/or damage to persons or pi-operty as ti niutter of products liability. negligence or otherwise, or from any use or operation of any methods. products. instructions or ideas contained in the material herein. Printed in The Netherlands

TABLE OF CONTENTS

Introduction ........................................................................................................

7

Section 1: Biases relative to the external structure of information Arie W. Kruglanski

Conditions for accuracy: General or specific?.................................

15

Jean-Paul Caverni and Jean-Luc Peris The anchoring-adjustment heuristic in an "information rich, real world setting": knowledge assessment by experts ..............35 Klaus Fiedler and Roman Graf Grouping and categorization in judgments of contingency ....... 47 Jo H u y s , Gerry Evers-Kieboorns, and Gery d'Ydewalle Framing biases in genetic risk perception ......................................

59

Annick Weil-Barais and Gerard Vergnaud Students' conceptions in physics and mathematics: biases and helps ..........................................................................................

69

Section 2: Biases in reasoning pragmatics Vittorio Girotto and Guy Politzer Conversational and world knowledge constraints on deductive reasoning ............................................................................

87

Ruth M. J. Byrne & P. N. Johnson-Laird Remembering conclusions we have inferred: what biases reveal....................................................................................

109

Jean Costermans and Veronique Heuschen Syllogistic reasoning with probabilities and continuous truth values ................................................................................................

121

Jonathan St. B. T. Evans and Paul Pollard Belief bias and problem complexity in deductive reasoning ....................................................................................................

131

Vittorio Girotto Biases in children's conditional reasoning....................................

155

Mark T.G. Keane Are there biases in analogical reasoning?.. ....................................

169

Henry Markovits and Christiane Lesaye Pragmatic reasoning schemas for conditional promises: context and representation ...............................................................

183

Guy Polltzer Non-logical solving of categorical syllogisms ...............................

193

Section 3: Response biases and context effects Allen Parducci Response bias and contextual effects: When biased? ................207 Claude Bonnet Psychophysical approaches, contextual effects and response bias.............................................................................................

221

Anne-Caroline Schreiber, Stkphane Rousset, and Guy Tiberghien Context effects in face recognition: below response bias. The contribution of a simulation ....................................................... 243 Harald G. Wallbott The relative importance of facial expression and context information in emotion attributions - Biases, influence factors, and paradigms ......................................................

275

Douglas H. Wedell Methods for determining the locus of context effects in judgment.....................................................................................................

285

Section 4 Biases relative to the categorization activity Willem Doise Social biases in categorization processes ......................................

305

C. Neil Macrae and Miles R. C. Hewstone

Cognitive biases in social categorization: process and consequences ..................................................................

325

Edith A. Das-Smaal Biases in categorization.........................................................................

349

Jean-Paul Cod01 Studies on self-centered assimilation processes. .......................

387

Jean-Marc Fabre Specificity and categorization in judgment: a cognitive approach to stereotypes ................................................

401

Section 5: Biases in pmbabiusticjudgment George Wright and Peter Ayton Biases in probabilistic judgment: a historical perspective ...... 425 Carl Martin Allwood and Carl-GustavBjBrhag Are two judges better than one? On the realism in confidence judgments by pairs and individuals .....................

443

Peter Ayton and George Wright Uncertain memories: evaluating the competence of probabilistic cognition .............. 465 Michael H. Bimbaum, Carolyn J. Anderson, and Linda G. Hynan Theories of bias in probability judgement ...................................... 4 7 7 Nigel Harvey Overconfidence in self-assessment of motor skill performance ....................................................................................

499

Alastair G. R. McClelland, Andrew S . Coulson and Sarah E. Icke Bias in meta-memory performance and its implications for models of memory structure .......................................................

5 11

Section 6 Biases and cognitive aids Gideon Keren Cognitive aids and debiasing methods: C a n cognitive piUs cure cognitive ills? ........................................................................

523

David W. Green Confirmation bias, problem-solving and cognitive models ..... 553 Marie-Paule Lecoutre, Jean-Luc Durand, and Jean Cordier A study of two biases in probabilistic judgments: Representativeness and equiprobability.........................................

563

This Page Intentionally Left Blank

ACKNOWLEDGEMENTS

The editors would like to thank the following persons who joined them as members of the Editorial Board: B. Brehmer R. Eiser V. Girotto M. Hewstone S.E. Newstead J.-L. PCris

J. Pynte P. Slovic J.A. Sniezek G. Tiberghien A. Tversky

We are also grateful to V. Lamongie who translated or verified the English of all chapters written by non-native English speakers, and to V. Lamongie and Y. Saborit for their devoted and conscientious preparation of t h e camera-ready copy for this publication.

This Page Intentionally Left Blank

Cognitive Biases J. - P . C a v e r n i , J. - M . , F a b r e and M. Gonzalez ( E d s . ) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

INTRODUCTION

COGNITIVE BIASES: THEIR CONTRIBUTION FOR UNDERSTANDING HUMAN COGNITIVE PROCESSES Jean-Paul Cavemi Jean-Marc Fabre and Michel Gonzalez CREPCO ( C e n t e r f o r R e s e a r c h i n C o g n i t i v e P s y c h o l o g y ) Aix-en-Provence, France

Research in psychology currently offers a n ever-growing number of empirical findings interpreted in terms of cognitive b i a s e s a n d their effects on a wide variety of tasks including inference, categorization, assessment, and comparison. With the rising number of research fields in which the notion of bias is being applied, we find not only a n increase in the diversity of the models set forth to account for such biases, b u t also a n extension of the very meaning of the concept. This extension appears to be occurring despite the independent development of the fields in which the notion is being called upon. The present work answers to a need to a s s e s s the contribution of the notion of cognitive bias to our understanding of the processes a t play in the various realms of cognitive activity. A recent book by Evans (1989) illustrates the utility of s u c h a n endeavor in studies on logical problem solving and in research into behavioral decision making and statistical inference. What phenomena can be qualified as cognitive biases?

It does not appear possible today to group all of the phenomena that have been qualified as cognitive biases under one and the same definition. However, the presence of a cognitive bias has generally been invoked in reference to what is thought to be the absence of bias. As such, a bias is detected when deviation from a norm is observed. The more systematic a n d frequent s u c h normative deviation is found to be, the more legitimate the

8

J.P.

Caverni,

J.M.

Fabre,

&

M. G o n z a l e z

assertion that a bias is indeed present. Occasional and accidental errors are obviously not part of the issue of cognitive biases. The norm serving as the standard for detecting the presence of a bias is generally not arbitrary or based on convention. It reflects what should be the outcome of a task carried out rationally or in a manner suitable to the situation at hand. All of the work presented in this book falls with the framework of general cognitive psychology, and represents a n attempt to find out what processes are responsible for certain systematic deviations from expectations in a specific theoretical situation, regardless of the domain serving to illustrate or actualize the outcomes. For a given task, the presence of a cognitive bias is relatively easy to assert if there is a normative model describing how the situation should be processed, and thus j u s t exactly what the "correct" response is. For example, logic provides u s with rules for determining the validity of inferences. If a subject systematically makes inferences in a given situation that are not logically valid, we may thus legitimately conclude that a cognitive bias is present. The assertion that a cognitive bias is present is not so easy to make when there is no normative model specifylng the expected response in the reference situation. On the other hand, the presence of a cognitive bias can sometimes be invoked to explain response variability in situations that only differ - at a given level of description - in respects postulated t o be irrelevant t o the concerned task. This point of view can be adopted in particular in regards to the effects of context (of which the task is assumed to be independent) or to the effects triggered by the way data is presented in a problem (of which the solving process is assumed to be independent). Let u s talk for a minute about the different standpoints taken to approach the notion of bias in various research fields. In what seems to be the most appropriate case for defining this notion, the researcher has at his or her disposal a model describing response accuracy, or more broadly speaking, describing the coherency of a set of responses. This state of affairs exists in the area of inference and decision making. Note that models, as elements of the theoretical framework of a study, are subject to discussion. They are not necessarily intangible. What is called a bias today may very well lose that s t a t u s tomorrow if, say, the current framework

Introduction

9

appears too simplistic, naive, or based on some superficial apprehension of the situation and the involved processing. In such a situation, the notion of bias loses its relevance. This does not mean that it does not retain its role as a useful frame of reference for providing evidence of the existence of different levels for describing results. In a r e a s where traditional research techniques have continuously shown that response variability occurs regularly, it is clear that attempting to find some order in that variability, and then to model it, are research essentials. This is indeed what is happening in the study of categorization. Here, the notion of bias can either not be crucial or reflect a pseudo-rational state deemed to correspond to that of unknowledgeable individuals. In truth, it may represent a prior research stage, the stage preceding the one where the researcher discovers that a certain amount of systematic variability is the rule in that field. Let us now come back to the case where a normative model does exist, and a fundamental role is granted to the notion of cognitive bias. For those whose main purpose is to describe the actual activity of subjects, the problem becomes determining what characteristics of the task and situation render possible the suitability, vis-a-vis the concerned norms, of how a subject performs a cognitive activity, whether or not that subject considers those norms to be justified. A subject's state of knowledge and cognitive abilities will obviously determine how he or she will perform a task. In certain situations, the normative model may require operations t h a t obviously do not correspond to the subject's cognitive capabilities or state of knowledge. How can we expect a subject to produce responses that are consistent with the model, when he or she is unable to function in the manner it advocates? The relevance of the notion of bias is t h u s relative, and depends on the specification of the task.

What does the study of Cognitive biases have to offer? Biases, like other distortions such as perceptual deformations, have provided the grounds for new lines of research. Apparent dysfunctioning, revealed by non-conformity with certain statistical norms, certain conceptions of optimal behavior, or certain axiom

10

J.P.

Caverni, J.M. F a b r e ,

& M.

Gonzalez

sets, can be a research object p e r se - the motive for accumulating observations aimed at enumerating the different manifestations of such dysfunctioning. I t can also be taken as an indicator or a warning. This approach is all the more justified knowing that cognitive biases can have undesirable, costly effects. In this event, focusing on biases should contribute to the development of cognitive aids or correction procedures that bring together the necessary conditions for reducing if not canceling their effects. Ascertaining the presence of processing biases in certain situations obviously does not suffice for providing remedies, however, and it is legitimate to assert that in order to conceive of effective aids we must understand the concerned bias-generating cognitive processes. Research must be conducted with two simultaneous objectives in mind: (1) to better our understanding of the nature of the cognitive functioning indicated by the presence of a bias, and (2) to conceive of bias-correction procedures based on what is already known about that cognitive functioning. Providing evidence of a cognitive bias naturally leads us to raise the question of the nature of the process (i.e. how the situation is processed cognitively) in which it originates. A given process in itself is not biased. I t can only be said to generate bias in regards to the situation in which it was implemented. The notion of cognitive bias does not have an explicative status, and it cannot be associated to any one category of cognitive process model. Listing all cognitive biases and the conditions under which each emerges is nevertheless a n essential research step. Knowledge can be drawn from the observation of variability and the questioning of the validity of expected results. In this sense, the issue of cognitive biases above all leads us to reassess, for example, models which were accepted outright due to the common confusion between descriptive model and normative model, but which have turned out to be inconsistent with observed data. Concluding that a cognitive bias exists implies concluding that a normative model (if there is one) is invalid from a descriptive point of view. But a t the same time, the researcher must indeed assume that the concerned cognitive activity possesses some degree of functional coherence, and thus that a descriptive model can be found to account for the observed bias. Alternative models replacing the initial normative ones must then be devised. To do so, the discerned properties of

Introduction

11

the biases are often the only cues to the characteristics of the descriptive model. A striking example of this approach can be found in the work done by Kahneman and Tversky (see for example Kahneman, Slovic, & Tversky, 1982) whose empirical results consist almost exclusively of biased probability assessments. Their work h a s h a d a strong impact on current conceptions in probabilistic judgment. Biases have thus been incorporated into a positive conception of cognitive activity, one that grants them the status of cues, of dependent variables. Biases should indeed be used as indicators likely to clarify our understanding of cognitive functioning, and thus of what is deemed to be normal functioning on the basis of certain criteria.

Cognitive bias research fields presented in this book The diversity of the fields in which cognitive biases have been investigated h a s led to a wide variety of approaches to the phenomenon. Each investigation field h a s its own specific problematics, which ordinarily do not revolve around the notion of bias. Thus, the way in which the notion has been applied depends to a significant extent on the issues debated in each field. To account for the cognitive activities involved in different types of tasks, specific research traditions have emerged. Tasks in which the cognitive activity involves reasoning and problem solving, categorization, assessment, and judgments of probability and confidence, have each been granted their own particular theoretical approach and perspective. In each case, cognitive biases of a different nature have been found. These are dealt with to differing extents in this book. Another way of considering the contribution of a given study is to look a t the kind of determinant affected by cognitive bias. The characteristics of the situation in which the task is performed is most certainly the most widely studied factor t h u s far. Two situational determinants are examined in this book: context (in its various acceptations and with its various effects) and the external structure of information. A special section h a s been devoted to studies on the correction of biases and cognitive aids. Their connection to

12

J.P.

Caverni, J . M .

Fabre,

&

M. G o n z a l e z

research that more directly focuses on the characteristics of biasgenerating cognitive functioning is discussed. The applied research approach taken in these areas is the natural extension of the issue of cognitive biases. Aix-en-Provence, 25 January 1990

REFERENCES Evans, J.St.B.T. (1989). pias in Human Reasoninc: Causes and Conseauences. Hfllsdale. N J : Laurence Erlbaum Associates. Kahneman, D., Slovfc, P.. & 'hersky. A. (Eds.) (1982).Judgment under uncertaintv: Heuristics and biases. Cambridge: Cambridge University Press.

Concerning the publication of this book The idea for is book originated in a n international conference held in Aixen-Provence. France, in September of 1988. Some of the chapters herein are derived from papers presented at the conference, while others are answers to requests made of other specialists in the field. The problem that arose was the inclusion of texts that did not focus on cognitive biases per se. but whose results suggested the presence of bias. This is the case when the concerned bias is not defined relative to the notion of accuracy or conformity with a certain set of rules, but rather is brought to bear in the interpretation of the systematic fluctuation of responses observed in situations deemed to be identical. The problem of the relevance of some to the papers to the theme of this book was posed in particular when their authors did not explicitly propose criteria for delineating the range of application of the notion of bias in their research field. Should we nevertheless have excluded findings that only reflect distortions for readers who are unaware of certain models now known to be valid? One reason for not applying too strict a rule to the selection of texts is that we hoped to supply the reader with a collection of studies dealing either with neighboring research objects or sharing a common theoretical orientation. Readers may note dmerences in the way the notion of bias is presented, i.e. differences in definition, in the value attributed to it. in its relevance to the results discussed. Comparative reflection of this type would in our mind be beneficial to our readers. Thus, the studies included in this book contribute to the advancement of the issue of cognitive biases in at least two respects. Above all, they are evidence of the wide variety of fields to which the notion of bias is currently being applied. In addition, they demonstrate how diverse the definitions and conceptions of this phenomenon are.

SECTION 1 Biases relative to the external structure of information

This Page Intentionally Left Blank

Cognitive Biases J. - P . Caverni, J. -M. Fabre and M. Gonzalez ( E d s . ) Q E l s e v i e r Science P u b l i s h e r s B . V . (North-Holland),

1990

CONDITIONS FOR ACCURACY GENERAL OR SPECIFIC? Arie W.Kruglanski University of Maryland, College Park, USA

Abstract. This chapter discusses past trends and possible future directions for research on the problem of accuracy in social perception and cognition. It is argued that previous claims about the general accuracy or inaccuracy of human judgment suffer from serious difficulties a s do attempts to delineate the conditions for accurate or inaccurate judgments, or to characterize the process whereby accurate or inaccurate Judgments are made. However, whereas general questions about human accuracy may be difficult to answer, it should be possible to assess or vary accuracy in specific settings. Such a situation-specific approach defines accuracy as a metajudgment comprising a "target judgment". a "criterion judgment" and a Judgment of the correspondence between the two. From this perspective two paradigms for the study of accuracy are discussed. In the realist paradigm a n external criterion is tentatively assumed and the question revolves around factors that determine the distance between a subject's judgment and that criterion. In the phenomenal paradigm both the criterion and the target judgments are made by the subject. and the question revolves around the subject's perception of accuracy.

The issue of accuracy in social perception and cognition h a s remained a constant source of fascination to researchers, although the way the topic has been approached has tended to vary over the years. In this chapter, I would like to discuss some of the historical background and contemporary concerns of accuracy research in social cognitive psychology. My historical commentary will be brief. Rather than concentrating on the Dast I should like to deal with the p r e s e n t and particularly with the f u t u r e . Thus, the gist of my discussion will center on (1) contemporary issues in the study of accuracy, and (2) possible paradigms for further accuracy research based on the conclusions that current analyses seem to offer. HISTORICAL BACKGROUND

It is of interest to note that the topic of perceptual accuracy has been a major area of social cognitive research in the 1950s. For instance, numerous studies investigated the personal characteristics of accurate and inaccurate judges, or attempted to assess the proportions of accurate judgments in given situations (for a review see Funder. 1987). This early accuracy work came to a virtual halt

16

A . Kruglanski

following the publication of a set of critiques by Cronbach (1955, 1958; Gage & Cronbach, 1955; Gage, Leavitt, & Stone, 1956). The gist of Cronbach's critiques was aimed at the discrepancv score used to measure accuracy. Specifically, Cronbach has argued that a global discrepancy score between a subject's judgment and a criterion may include several spurious elements. For instance, a correspondence between the subject's judgment and the criterion may arise because of a response set, irrespective of the external stimulus to which the subject is reacting, or it may arise because of a prejudicial tendency associated with a given domain of' judgment. According to Cronbach's argument those confounds need to be taken out of the global discrepancy score if "true accuracy" is to be assessed. At the time no appropriate methodologies existed to carry out the recommended partitioning of the accuracy score into its various components (for new methodological developments in this regard see Kenny & Albright, 1987). Thus, the effect of Cronbach's critique was t o eliminate accuracy research altogether. Instead, emphasis shifted to the study of the judgmental process without much attention paid to the issue of whether the outcomes of such a process are accurate or inaccurate. For instance, in the social cognitive psychology of the 1960s considerable interest revolved about the cognitive consistency theories (Festinger, 1957; Heider, 1958); those theories address the ways whereby motivational and cognitive factors interact to produce judgments, attitudes and opinions; thus, they deal with the cognitive process rather than with issues of accuracy. Similarly, attribution theories address the process of causal judgments without according much attention to the accuracy of those judgments. The process models, however, while initially descriptive soon acquired prescriptive or "normative" overtones (Funder, 1987). Accordingly, researchers' interest has come to center on persons' tendency to stray from optimal modes of judgment (e.g. as embodied in models of attribution or of statistical inference), and the emphasis shifted from the study of process per se, to the study of bias or inaccuracy (see Tversky & Kahneman, 1974; Nisbett & Ross, 1980). For over a decade, research on judgmental biases and errors has exerted a dominant influence on views of the human cognitive process. A major implication of such research has been that persons' judgments and inferences are fraught with errors and mistakes of various sorts. Recently, however, several authors took

Conditions for accuracy

11

an issue with the conclusion that lay persons are incorrigibly inaccurate and error prone (Einhorn & Hogarth, 1981; Funder, 1987; Hastie & Rasinsky, 1988; Kenny & Albright, 1987; Kruglanski & Ajzen, 1983; McArthur & Baron, 1983: Nisbett, Krantz. Jepson, & Kunda, 1983; Swann, 1984). Moreover, recent discussions made a n attempt to go beyond the issue of whether people are generally accurate or biased, and proceeded to delineate some external boundary conditions for accuracy, and to hint at the (internal) process underlying accurate versus inaccurate judgments. I would like now to review some such attempts and to evaluate the progress (if any) toward answering the "million franc" question of how to turn people into more accurate judges. Let me first consider some proposals concerning the external conditions under which accurate versus inaccurate judgments may be reached. BOUNDARY CONDITIONS FOR ACCURATE JUDGMENTS: NATURAL VERSUS ARTIFICIAL SETTINGS

McArthur and Baron's (1983)ecological perspective on human knowledge emphasizes the accuracy of perception in natural environments. Errors may be committed in artificial settings and are often indicative or inappropriate over-generalizations of highly adaptive attunings. For instance, a shark might react to an electric field inappropriately as if it were prey. However, in natural environments electricity would be correlated with food. Hence, responding to it would hardly count a s a n error. Note, however, that the existence of some adaptive attunings does not logically imply that all or even most perceptions in natural settings are accurate. For instance, the restricted visual capacity of bats may result in failures to make various discriminations in their natural environments. While not critical to their physical survival such failures could still be considered to represent perceptual errors. The notion that persons are more accurate in "natural" versus "artificial" settings is seconded in papers by Swann (1984) and Funder (1987). According to Swann " ... the errors that people make in the laborato ry... occur only rarely in everyday contexts ... " (1984, p. 460). One reason for this could be that in everyday settings persons may exercise considerably greater control over their social reality than in laboratory settings. In a sense, they have a degree of control over the "criterion" (e.g. the target's behavior) as well as

18

A.

Kruglanski

over the judgment (prediction of the behavior). For instance, a mother might perceive her son to be very successful and responsible: in turn, not wishing to disappoint the mother the son might work hard to indeed become successful and responsible, rendering the mother's perceptions accurate. Perceivers' influence over targets based on the targets' motivation to uphold the perceivers' expectancies may thus enhance the accuracy of social perception (Swann, 1984). Note, however, that a target's desire to uphold the perceivers' expectancies by creating a corresponding reality is unlikely t o constitute the sole, naturally occurring motivation relevant t o accuracy. Occasionally targets may be motivated to uphold the perceivers' expectancies contrary to what they themselves believe. For instance, a sales person may agree with the customer's expressed preference irrespective of his/her own tastes or inclinations. In yet other instances a n individual may be motivated to deceive his or her interaction partners altogether, hence promoting disparate rather than shared realities. Spies, undercover agents, or confidence tricksters may work hard to disguise their "true" identities. If they succeed, perceivers may end up with grossly inaccurate predictions. On occasion, interaction partners may have motivations that "pull" their respective beliefs in opposite directions. For instance, antagonists in a bitter feud may be motivated to hold negative opinions of each other: Such unfavorable views might conveniently legitimize otherwise unacceptable hostilities against one's rival (cf. Bar-Tal, 1985). For the most part, the negative view held by a person's antagonist is unlikely to be shared by the relevant target. In turn, a n absence of shared reality may result in predictive inaccuracies. For instance, if B's antagonist believes him to be cruel and callous, whereas B believed instead that (s)he is warm and kind, the antagonist may err in predicting B's behavior toward neutral C. I t t h u s appears that several motivational factors may determine whether interacting persons in real world situations subscribe to a commonly shared reality, hence whether they are accurate in predicting each other's behavior. The diversity of possible motivations highlights the difficulty of estimating people's general accuracy in natural settings. In conclusion, the notion that persons are more accurate in natural versus artificial settings is open to various exceptions, and on close examination seems less than totally compelling.

Conditions f o r a c c u r a c y

19

CIRCUMSCRIBED VERSUS GLOBAL ACCURACIES Recently, Swann (1984) proposed that the "circumscribed", or situation-specific form of accuracy is generally easier to attain than . forms of "global" or trans-situational accuracy. Specifically, circumscribed accuracy usually require only that perceivers predict target behaviors in a limited range of social situations ... in which perceivers themselves may often be present ... This not only reduces the number of potential influences on target behavior that perceivers must consider, it also raises the probability t h a t perceivers will be familiar with these influences and with how they are apt to affect the behavior of targets ... " (p. 462). Note, however, that from a social-cognitive, constructionist perspective (cf. Fiske & Taylor, 1984; Kruglanski, 1989) the "number of potential influences" that perceivers may consider is not necessarily constrained by the specificity or generality of the relevant judgment. Such "potential influences" represent the perceiver's hypotheses about the determinants of the target's behavior. In turn, the number of hypotheses generated by the perceiver may depend on a variety of cognitive and motivational factors "internal" to the individual, and not be "externally" determined by the topic of the hypotheses, or their generality versus specificity. Thus, a person could entertain numerous interpretative hypotheses about a "specific" situation and only a few hypotheses about a "general" situation depending on (1) the number of relevant constructs momentarily accessible to this individual (cf. Higgins & King, 19811, (2) his or her stable pool of available constructs on the topic (cf. Higgins, King, & Mavin, 1982). (3) his or her (epistemic) motivation to generate or search for further relevant notions (cf. Kruglanski, in press), and (4) his or her creativity or constructive ability (cf, Amabile. 1983). In sum, the notion that some stimulus situations (e.g. "circumscribed" ones) generally invite fewer interpretative notions than other situations, hence that they afford a greater likelihood of accuracy, may be debated from the constructivist perspective. Of unique social psychological interest is Swann's (1984) argument that circumscribed accuracy is often higher (or easier to attain) than global accuracy because of "identity negotiation" processes in social perception ... Two separate processes of this 'I..

20

A . Krugl anski

type a r e distinguished, "behavioral confirmation" and "selfverification". Behavioral confirmation refers to targets' tendency to behave in ways that confirm the expectancies of perceivers (cf. Snyder, Tanke, & Berscheid, 1977) Self-verification refers to targets' tendency to behave in ways that confirm their self-concepts. According to Swann. "if target self-conceptions and perceiver expectancies differ, both global and circumscribed forms of accuracy are likely to be higher if self-verification occurs, because in such circumstances "... perceivers are induced to bring their actions and beliefs into harmony with target self-conceptions (hence) they will be equipped to predict how targets will behave in the future because targets will theoretically continue to behave in accordance with their self-conceptions ... " (p. 466). By contrast, processes of behavioral confirmation could contribute to circumscribed accuracy while detracting from global accuracy as " ... targets may behave quite differently once they escape the constraining influence of the perceiver ... " (ibid.). While Swann's conclusions may often hold, behavioral confirmation processes need not invariablv lower global accuracy. First, the target could occasionally use his or her own behavior as a cue to his or her properties (attitudes, traits) and come t o internalize perceiver expectancies via dissonance reduction or selfperception processes (cf. Bem, 1972). If that occurred, a perceiver who expected the target to behave similarly across situations could be right. Secondly, a t least on some occasions perceiver could take into account the situational constraints (including his or her own influence) on target actions and not expect such behaviors t o generalize to alternative contexts. For instance, the psychoanalyst could recognize t h a t a client's aggressiveness represents a transference process that is uniquely engendered by the therapeutic setting. Similarly, the powerful boss may identify a n employee's friendliness as ingratiating behavior and not expect it to replicate in more egalitarian situations. Such perceiver sophistication might help him/her avoid pitfalls to global accuracy. To conclude, the argument that "circumscribed" judgments are generally more accurate than "global" judgments is open to debate. The number of interpretative constructs the perceiver considers need not be more restricted in circumscribed versus global settings, nor need behavior confirmation processes invariably lower global accuracy. ~

Conditions f o r a c c u r a c y

21

THE ACCURACY MEDIATING PROCESS As we have seen, the debate about persons' general accuracy h a s led to the consideration of putative boundary conditions for accuracy (that is, of naturalistic versus artificial conditions, or of circumscribed versus global conditions). In turn, discussion of boundary conditions typically hinted at the general process assumed conducive to accurate judgments. Aspects of such an implied process will now be examined in some detail. Amount of information considered

Several authors imply that the more information a perceiver takes into account the greater the likelihood of his/her being accurate. McArthur's and Baron's (1983) suggestion that active perceivers are typically more accurate than passive ones could be interpreted in terms of the greater amount of information active exploration might afford. Similarly, these authors' notion of "sins of omission" refers to cases where the perceiver misses part of what is afforded by the external stimulus because of attentional selectivity, or because the stimulus array is impoverished. In both cases errors are, therefore, traced to informational deficits. Einhorn's and Hogarth's (1981) suggestion that feedback from the environment enhances accuracy implies that the ability to receive sufficient amounts of relevant information improves accuracy. Finally, Swann's (1984, p. 462) suggestion t h a t "circumscribed accuracy" is easier to attain than "global accuracy" is premised in part on the assumption that in the former case perceivers have more relevant information at their disposal. On close examination, however, the relation between the amount of information and accuracy seems complex. First, the terms informational "sufficiency" or "insufficiency" often used to suggest that more information means better accuracy are circular. Such terms typically fail to specify in advance a definite informational quantity, and are used after the fact, that is, after a n accurate or inaccurate judgment has occurred. Secondly, as several authors have stressed (cf. Campbell, 1969; Weimer, 1979) any amount of information is compatible with multiple alternate hypotheses. Thus, one may continue to hold on to a n inaccurate judgment despite considerable amounts of information that, while consistent with the "correct" alternative

22

A . Krugl a n s k i

(that the individual may have failed to entertain), are equally consistent with t h e "incorrect" hypothesis. In other words, considerable amounts of information could be non-diagnostic (cf. Trope & Bassok, 1983) with respect to the "correct" and "incorrect" hypotheses. Thirdly, as Cantor and Kihlstrom (1987) have noted, extensive information processing may instill in t h e judge a sense of "expertise" and overconfidence that may dull his or her vigilance to inconsistent information, occasionally leading to error. Finally, extensive informational search may undermine correct judgments if the information received was itself "biased" or "distorted". For instance, the military commander who believes in the high likelihood of a surprise attack may modify his opinion on the basis of new information. Yet he could have been correct in the first place, and the new information may have been fabricated by foreign agents, and intended to deliberately mislead him. All of this suggests t h a t processing increasingly more seemingly relevant information need not improve one's chances of reaching a correct judgment: (1) Even if "correct", the information could be non-diagnostic between the erroneous hypothesis and the correct alternative. (2) A broad informational base for one's schemata could instill a false sense of security and "expertise", leading to an inappropriate assimilation of inconsistent information. (3) Occasionally, the information processed could be "incorrect", or more compatible in fact with the erroneous than the correct alternative, leading the perceiver astray. Motivational factors Beyond informational deficiencies, inaccurate judgments have been occasionally linked to motivational deficits. Thus, McArthur and Baron (1983, p. 230) suggest that perceivers' inability to detect deception may have been due to a lack of sufficient motivation. Again, the term "sufficiency" in this context could be question begging. The issue, however, is whether motivation may be linked to accuracy in any simple way. Consider the distinction between degree of motivation and type of motivation. Recent research (cf. Kruglanski & Freund. 1983; Freund, Kruglanski, & Schpitzajzen, 1985) suggests that some motivations, e.g. a need for cognitive closure, may facilitate "freezing" on particular judgments. To the extent that such

Conditions for accuracy

23

judgments deviate from the criterion, a high degree of "motivation" could result in error. Even a high degree of the motivation to avoid closure (prompted, e.g. by a strong fear of invalidity) need not increase judgmental accuracy: As noted earlier, one's initial hunch could be correct: thus, the tendency to abandon it on the basis of further, possibly invalid information might increase the likelihood of error. The foregoing arguments suggest that increasing incentives (for correct judgments) need not reduce the incidence of error. Reviews of the relevant empirical literature concerning the relation between motivation and accuracy (e.g. Einhorn & Hogarth. 1981) corroborate this conclusion.

Normative models of judgment Some of the literature on judgmental heuristics implies that the process whereby accuracy judgments are rendered is represented by the "normative" models of inference (cf. Kahneman, Slovic, & Tversky, 1982; Nisbett & Ross, 1980). However, normative models too can be conceived of as constructed, hence potentially fallible, representations of reality (cf. Kruglanski & Ajzen, 1983). As Einhorn & Hogarth (1981) noted a normative model may be inapplicable to given circumstances (e.g. the model's assumptions may not hold). When this occurs judgments derived from the model will be inaccurate (i,e., will deviate from the criterion). For instance, the finding that people are insufficiently regressive in their predictions, hence that they stray from values derived from the normative model, assumes that the model itself is applicable in a given situation. This may or may not be the case. If a fluctuation in outcomes represents random variability around a stable parameter, the regressive model may indeed apply. However, fluctuation may represent systematic shifts rather than random variability. For instance, " ... if you think that Chrysler's losses are being generated by a stable process, you should predict that losses will regress up to their mean level. However, if you take ... them as indicating a deteriorating quality of management, you should be predicting even more extreme losses ... " (Einhorn & Hogarth, 1981, p. 56). Thus, normative models do not appear to safeguard accuracy judgments, a t least not in an absolute sense. In sum, accuracy literature has identified several process variables implied to generally improve accuracy. However, close examination of those variables indicates serious difficulties. In

24

A.

Krugl a n s k i

particular, neither high degrees of motivation, nor large amounts of information, nor the use of "normative" models may reliably improve accuracy. I t seems fair to conclude that as of now, no compelling analysis exists concerning the general process whereby accuracy is obtained. RECENT TRENDS IN ACCURACY RESEARCH: SOME INTERIM CONCLUSIONS

To summarize, no compelling arguments or evidence exists for the notion of boundary conditions separating more from less accurate judgments. Specifically, the issues are moot whether "naturalistic" judgments are generally superior to judgments in "artificial" settings, and whether situationally specific judgments are typically superior to global judgments. Finally, suggestions for process variables that may generally improve accuracy have been found wanting and open to exceptions. Thus, processing large bodies of information, or having high motivational involvement in a topic need not improve accuracy and could even detract from accuracy in some circumstances. The difficulties in identifylng general boundary conditions or process variables tied to judgmental accuracy could relate to the fact that the criterion of accurate judgments in social perception and cognition is situation specific. What is considered true in situation A may be considered false in situation B. For instance, identifying a target person as a woman may be considered true in one situation and false in another situation. Hence, the very same process that yielded a correct judgment in situation A, could foster an inaccurate judgment in situation B. For example the rule "if a person wears a skirt s h e is a woman" may work very well in some places (e.g. Provence) b u t lead to judgmental errors in other places (e.g. Scotland or New Guinea). Secondly, it is important to note that the criterion is after all someone's (notably, the standard setter's) judgment. I t may be a highly compelling judgment with which many intelligent people would agree. But intelligent people can change their mind, and history is replete with examples of their having done s o in regard to most important judgments (e.g. the shape of the planet Earth). Thus, as with all judgments the criterion should be capable of revision given the appropriate arguments, and the motivation to seriously consider them. I t follows that the same judgment that was

Conditions f o r a c c u r a c y

25

previously discrepant from the criterion, hence incorrect, could now coincide with the (revised) criterion, hence be correct. Similarly, a previously correct judgment would become incorrect in reference to the new criterion. In short, the very same combination of variables could give rise to correct and incorrect judgments. Thus, it does not seem feasible to hope that a general set of either external conditions or internal processes that mediate accurate judgments would be uncovered. Nor does it seem feasible to hope for a meaningful answer to the question whether persons' judgments are generally accurate or inaccurate. Admitting the possibility of criteria1 shifts, different answers to this question could be appropriate a t different times. The discussion so far might seem to paint a gloomy picture for the future of accuracy research. Actually, such a negative outlook befits only general questions about human accuracy. In contrast, situationallv-specific questions about accuracy appear more readily tractable. Indeed, in the remainder of this paper I should like to explore several possible directions for accuracy research within situationally-specific paradigms. I t will be suggested that in such cases accuracy issues are profitably approached in terms of the process whereby all judgments are reached. Thus, whereas historically the study of the judgmental process s u q d a n t e d the study of accuracy in social perception (Jones, 1985; Funder, 1987) according to the present analysis the two perspectives may be fruitfully combined. A PROCESS-BASED ANALYSIS OF SITUATIONAL ACCURACY

I should like now to sketch two possible approaches to the study of accuracy in social perception and cognition. Both approaches are based on the definition of accuracy as a correspondence between judgment and some criterion or standard. The realist paradigm addresses the study of accuracy from the "external" standpoint of the standard setter. The p h e n o m e n a l paradigm addresses accuracy from the "internal" standpoint of the judge. Those notions will now be considered in greater detail. The Accuracy Framework

Consider the definition of accuracy as a correspondence between judgment and criterion. As I already noted, such a

26

A.

Krugl anski

criterion is in principle somebody else's (notably, the standard setter's) judgment. As such it should be subject to all the cognitive and motivational factors that affect all judgments. The realistic approach to accuracy ignores the standard setter's judgmental process, focusing instead on the subject's judgment and on its degree of correspondence to the criterion, or the externally defined "reality". Most social cognitive research on error and bias falls into this category, for example the study of cognitive heuristics (for reviews see Kahneman, Slovic, & Tversky, 1982) in which lay judgments are compared with a priori assumed normative criteria. In short, the realistic approach to accuracy sets a criterion of a given content and asks how that content is mirrored by subjects' judgments. Many theories of judgment imply that judgments are inferred from relevant evidence. This requires: (1) prior formation of a connective schema, i.e. a "rule" that links a given category of evidence and a given judgment, (2) momentary accessibility of the rule as well a s the evidence, and (3) appropriate motivational conditions that may induce the tendency to accept or reject the evidence and/or search for alternative types of evidence. Rule learning Prior formation of connective schemata may involve learning particular rules in various ways. For instance, some people may have acquired various judgmental heuristics (Kahneman, Slovic, & Tversky, 1982) that imply judgments a t variance with "normative" statistical predictions. To the extent that the latter are assumed to represent the accuracy criterion, use of heuristics may be said to propagate errors (Nisbett & Ross, 1980). Recent work by Nisbett and his colleagues (Nisbett, Krantz, Jepson, & Kunda, 1983; Fong. Krantz, & Nisbett, 1986; Jepson, Krantz, & Nisbett, 1983; Nisbett, Fong, Cheng, & Lehman, 1987) suggests that the teaching of statistical rules can increase the likelihood of statistical reasoning, hence of the rendition of statistically accurate judgments. For instance, Fong et al. (1986) found that subjects' likelihood of giving statistical answers to simple problems increased with their previous amount of statistical education. Similarly, subjects' tendency to give statistical answers was higher at the end of a course in statistics than at the beginning. Some inferential rules may relate to persons' conceptions of their own selves, or to their self-schemata (Markus, 1977). Depending on the situational criterion such self-schematic rules

Conditions f o r accuracy

21

may increase or decrease accuracy. Thus, some individuals may utilize negative self-schemata in reference to given classes of judgment whereas others may use more positive schemata in those domains. According to Beck's theory of depression (Beck, Rush, Shaw, & Emery, 1979). for example, negative self-schemata may systematically influence the judgments of depressives. Grouch0 Marks' famous quip "I will never join a club that accepts me as member" is a n example of a self-schematic rule that may lead to derogatory inference concerning an external entity, in this instance a club. On the other hand, if the "objective" value of the club was in fact low, the very same rule may contribute to accuracy. In the same way that (situationally) appropriate inferential rules may enhance judgmental accuracy, inappropriate rules may hinder accuracy. I t is thus important that such inferential "stumbling blocks" be identified and eliminated from the individual's conceptual repertory. Thus, inappropriate assumptions about sample representativeness (Nisbett & Ross, 1980), or insufficient appreciation of chance factors in observed outcomes may have to be attacked "heads on" if persons' statistical accuracy is to improve. Similarly, to enhance the accuracy of self-judgments (e.g. by bringing them in line with the criterion of social consensus) one's overly negative or positive self-schemata may have to be modified.

Accessibility In order that it be utilized in a judgmental situation, a rule must not only be available in an individual's long term memory but also be momentarily accessible (Higgins & King, 1981; Higgins, King, & Mavin, 1982; Higgins & Bargh, 1987). Specifically, accessibility of an "appropriate" rule will help accuracy whereas that of an "inappropriate" rule will hinder accuracy. Recent research suggests that accessibility is determined by frequency as well as recency of activation (Wyer & Srull, 1982; Higgins, Bargh, & Lombardi, 1985). For example, in Luchins' (1942) work on cognitive sets an inappropriate judgmental rule may have become highly accessible because of activation recency, and may have, therefore, led to erroneous solutions on subsequent judgmental tasks. The same research, however, suggests that heightened rule accessibility facilitated performance accuracy on a previous task to which the rule was appropriate. Accessibility of relevant constructs is possibly involved in the encoding of situational evidence in ways that highlight the

28

A . Kruql a n s k i

atmlicabilitv of given inferential rules. Support for this notion comes from studies in which specific "packaging" of the evidence may have primed the appropriate encoding categories, hence increased subjects' tendency to employ otherwise under-utilized principles. Thus, in research by Ajzen (1977) base rate information was more likely to be utilized when it was interpreted to possess causal significance. In research by Kruglanski, Friedland, and Farkash (1984) statistically correct use of the 'regression to the mean' rule increased where such evidence was couched in familiar everyday examples. Presumably such examples served to activate the appropriate constructs ("chance factors", "variability") that rendered the 'regression' logic more apparent to subjects. Similarly, research on logical reasoning (Griggs & Cox, 1982) suggests that subjects' erroneous failure to falsify hypotheses (via the modus tollens) is much reduced when the examples are couched in familiar terms. Motivation An individual's tendency to make a given inference may depend also on their motivational condition. To the extent that the evidence supported an undesirable conclusion (s)he may downgrade the value of the evidence and base his/her judgments instead on alternate evidence (Lord, Lepper, & Ross, 1979; Lord, Lepper. & Preston, 1984). I t is noteworthy that motivational biases due to desirability of specific conclusions may hinder or help accuracy depending on whether or not the judgment they promoted coincided with the situational criterion. For instance, the parent of an aspiring pianist may be motivationally biased, hence judge her performance as superior to that of her competitors. Such a judgment may happen to correspond with the referees' assessment, hence be correct under the circumstances. However, a similar motivational bias on the part of the other contestants' parents would hinder the accuracy of their judgments. A motivation to have cognitive closure (Kruglanski, 1989) may increase persons' tendency to encode the stimulus information in terms of momentarily accessible categories and decrease the tendency to search for alternative relevant constructs. To the extent that the accessible categories did not coincide with the criterion, need-for-closure effects could detract from accuracy. For example, in research by Kruglanski and Freund (1983, study 1) primacy effects in impression formation were more pronounced

Conditions f o r a c c u r a c y

29

under a high versus low need for closure (manipulated via time pressure). Presumably, primacy effects reflect subjects' tendency to encode the stimulus information on the basis of (positive or negative) categories activated by early items in the series. Similarly, the tendency to anchor judgments in early assessments (Kahneman, Slovic, & Tversky, 1982) could reflect a n inclination to base judgments on initially accessible estimates. Indeed, Kruglanski and Freund (1983, study 3) found s u c h erroneous anchoring tendencies to be stronger under a high versus low need for closure. Note that in some circumstances need for closure effects could contribute to rather than detract from accuracy. This would happen when initially accessible categories coincided with the criterion whereas subsequently generated categories did not. A motivation to avoid or postpone cognitive closure may contribute to judgmental accuracy where initially accessible "hunches" diverged from the criterion. Accordingly, Kruglanski and Freund (1983, study 1 and 3) found weaker primacy and anchoring effects under a high (vs. low) need to avoid closure (manipulated through evaluation apprehension). A need to avoid closure could hinder accuracy in situations where initially accessible categories coincided with the criterion whereas subsequently suggested ones did not. Initially accessible categories may represent "chronically" rather than momentarily active constructs (Higgins & King, 198 1). For instance, prejudiced subjects may have chronically accessible stereotypes of particular social groups. Under a high need for closure, those prejudicial constructs may drive subjects' judgments to a greater extent than individuating information about targets, occasionally resulting in inaccuracies. Thus, Kruglanski and Freund (1983, study 2) found that Israeli subjects under a high (vs. low) need for closure exhibited a stronger tendency to assign a higher grade to the same composition where the author was identified as a member of a positively stereotyped group (Ashkenazi) versus a negatively stereotyped one (Sepharadi). Similarly, Bechtold, Naccarato, and Zanna (1986) found that subjects prejudiced against women in management discriminated against female candidates for a position only under a high (but not under a low) need for closure. As the male and female candidates were equally qualified for the job (their resumes were rated as equally attractive by judges blind to the

30

A . Kruylanski

candidates' gender), subjects' discriminatory judgments may be considered erroneous. SUMMARY ACCURACY RESEARCH IN THE "REALIST'PARADIGM

The preceding discussion suggests that psychological factors affecting the judgmental process in general are relevant to accuracy research in the "realist" paradigm. Such factors determine the person's tendency to render a judgment of a particular content, hence be accurate or inaccurate depending on the content of the criterion. Assuming further that the accuracy criteria in social perception are tentative and potentially modifiable it follows that the very same combination of factors that once fostered "accuracy" could now foster "inaccuracy" if the judgment has shifted. In short, accuracy research in the realist paradigm should be both informed by notions of the judgmental process and treated as relative to tentative situational criteria. But the accuracy issue can also be investigated from a different perspective, notably that of the judging subject. In such a case, the research problem revolves around factors t h a t induce t h e perception of accuracv. This topic is discussed in what follows.

Phenomenal Accuracy In the phenomenal approach to accuracy the comparison of judgment with a criterion is carried out by the subject. Thus, a subject's perception of accuracy is likely to be affected not only by (1) what (s)he perceives as the case (subject's criterion) b u t also by (2) what (s)he perceives as a given judgment (e.g. someone else's opinion), and by (3) his or her perception of the degree of correspondence between judgment and criterion. More specifically, in order that the accuracy inference be made it is first necessary that the target and criterion judgments be formed and available in the person's memory. Secondly, both judgments should be jointly accessed and compared with each other. Probably, such comparison would not be carried out without at least some degree of interest in its outcome, i.e., a given degree of motivation to make a n accuracy assessment. Finally, in order that an inference of accuracy or inaccuracy be made it is essential that the evidence (for accuracy or inaccuracy) be appropriately interpreted.

C o n d i t i o n s for accuracy

31

Consider, for instance, the question of whether target and criterion judgments would be even compared with each other. This may be largely determined by the individual's concomitant awareness of target and criterion, or their joint momentary accessibility. In turn, a possible determinant of joint accessibility may be the degree to which the judgmental topic is involving or self-relevant. If it is, the person might possess highly accessible opinions readily recalled by mere consideration of the issue. Thus, any attempt to determine another person's (or one's own previous) judgment on such a topic is likely to prime one's own current opinion (the criterion) hence increase the likelihood that the two will be compared. These issues could be profitably explored in further research. Or consider the possible motivational determinants of comparing the target and the criterion judgments. Thus, a n individual may have a high need for closure on the topic of a given judgment's accuracy (a need to "know" whether the judgment is or is not accurate), and this may increase the likelihood of the comparison. In contrast, a person motivated to avoid closure on the accuracy issue would be less likely to carry out the comparison. Also the person may find some possible outcomes of a comparison more desirable than others and this may affect whether a comparison would occur. An individual may wish to prove that he or she was right: if such a comparison outcome appears likely this may enhance the tendency to make the comparison, whereas expectancy of the opposite outcome may reduce such a tendency. Perception of the comparison outcome as such would also be affected by various cognitive and motivational factors. For instance, an individual may strongly expect to agree with another person, e.g. a longtime friend known to generally share his or her attitudes and opinions. Such an expectancy may raise the individual's perceptual threshold for discrepancies, hence lower his or her tendency to judge that error has occurred. Finally, even if a discrepancy between target and criterion judgments h a s been perceived it is not obvious how it will be interpreted. The interpretation I have been implying so f a r is that in such a case the target judgment would be deemed in error. However, in some circumstances one could change his mind about the criterion and consider it as mistaken and the target judgment as correct. In other cases no cut and dry decision of which judgment is correct and which is mistaken may be made. Rather both the

32

A . Krugl anski

target and the criterion judgment might be considered as partially correct, that is, correct within certain specifiable boundaries. All of these issues could be systematically investigated by further research in the phenomenal accuracy paradigm. RECAPITULATION AND CONCLUSION

To summarize, in this chapter I have attempted to address past, present and possible future concerns of research on accuracy in social perception and cognition. I have suggested that several major questions typically posed in the accuracy domain have not been satisfactorily resolved thus far, nor do they seem capable of resolution in the foreseeable future. To reiterate, it has not been possible to meaningfully answer the question of whether people are generally accurate or inaccurate, or to reliably estimate the proportion of times a t which they may be. Furthermore, it has not been possible to delineate general classes of boundary conditions for accurate and inaccurate judgments. Finally, it has not been possible to elucidate the process (or "method") for reaching generally accurate judgments. Instead of attempting to answer general questions about the accuracy of social perception a situationally-specific approach to issues of accuracy could be more fruitful. Such an approach defines accuracy in terms of the correspondence between a judgment and a criterion, and assumes that the criterion for accuracy represents the standard setter's perceptions, rather than being absolute in any strong sense of the term. The present interpretation suggests that, despite historical separation, a study of the judgmental process is highly relevant to the study of accuracy. According to this analysis, determination of accuracy constitutes a meta-judgment comprising three components: the simple judgment, t h e criterion, a n d the correspondence between them. Each of these components is a judgment governed by various cognitive and motivational factors that affect the judgment process in general. The foregoing analysis suggests two general paradigms for accuracy-related research. In the realist approach the situational criterion (the reality) is assumed to be known, and the research addresses the conditions under which subjects' judgments approximate the criterion. In the phenomenal approach the focus is on subjects' compound perception comprising the judgment, the

Conditions for a c c u r a c y

33

criterion and their correspondence. Both research paradigms raise interesting issues for future study. By systematically outlining their judgmental underpinnings, these lines of work could be extended in new and fruitful directions.

REFERENCES Ajzen, I. (1977). Intuitive theories of events and the effects of base rate information on prediction. Journal of Personalitv and Social Psvcholom. s,303-314. Amabile, T.M. (1983). The social Dsvcholom of creativity. New York SprlngerVerlag. Bar-Tal. D. (in press). GrouD Beliefs. New York Springer-Verlag. Beck, A.T.. Rush, A.J.. Shaw, B.F.. & Emery, G. (1979). Coanitive theraDv of derxessios. New York: Guilford. Bem. D . J . (1972). Self-perception theory. In L. Berkowitz (Ed.). Advances in , vol. 6, New York: Academic Press. Campbell, D.T. (1969). Prospective: Artifact and control. In R. Fbsenthal & R.L. Rosnow (Eds.) mifact in Behavioral Research. New York Academic Press. Cantor, N. & Kihlstrom, J.F. (1987). Personalitv and social intellipencg. Englewood Cliffs. NJ: Prentice Hall. Cronbach, L.J. (1955). Processes affecting scores on "understanding of others" and "assumed similarity". Psvcholoeical Bulletin. 52. 177-193. Cronbach. L.J. (1958). Proposals leading to analytic treatment of social perception scores. In R. Tagiuri and L. Petrullo (Eds.). Person PerceDtion and Intemersonal Behavior. Stanford, CA: Stanford University Press. Einhorn. H.J., & Hogarth, R.M. (1981).Behavioral decision theory: Processes of judgment and choice. Annual Review of Psvcholoey. 32. 52-88. Fiske. S.T. & Taylor, S.E. (1984). Social Coenition. Reading, MA: Addison-Wesley. Fong. G.T.. Krantz. D.H.. & Nisbett. RE. (1986).The effects of statistical training on thinking about everyday problems. Cognitive Psvcholom. 253-292. Freund. T., Kruglanski. A.W., & Schpitzajzen, A. (1985). The freezing and unfreezing of impressional primacy: Effects of the need for structure and the fear of invalidity. Personalitv and Social Psvcholoe BulletLn. 11.479-487. Funder. D.C. (1987). Errors and mistakes: Evaluating the accuracy of social judgment. Psvcholoeical Bulletin, 75-91. Gage, N.L., Leavitt, G.S., & Stone, G.C. (1956).The intermediary key in the analysis of interpersonal perception. Psvcholoeical Bulletin, s,258-266. Griggs. R.A. & Cox, J.R. (1982). The elusive thematic-materials effect in Wason's selection task. British Journal of Pmcholoa. 73, 07-42. Hastie, R. & Rasinsky. K.A. (1988).The concept of accuracy in social judgment. In D. Bar-Tal & A.W. Kruglanski (Eds.). The Social Psvcholom of Knowled&. Cambridge, England: Cambridge University Press. Higgins. E.T. & Bargh. J.A. (1987). Social cognition and social perception. Annual Review of PsvcholoPy, B,369-425. Higgins, E.T. & King, G.A. (1981). Accessibility of social constructs: Information processing consequences of individual and contextual variability. In N. Cantor & J. Kihlstrom (Eds.),Personalitv. Cognition and Social Interaction. Hillsdale, NJ: Erlbaum. Higgins. E.T.. King, G A . . & Mavin, G.H. (1982). Individual construct accessibiltty and subjective impressions and recall. Journal of Personalitv and SOC ial PsvcholoQ~, 43.35-47. Jepson. E.E., Krantz. D.H.. & Nisbett. RE. (1983). Inductive reasoning: Competence or skill? Behavioral and Brain Sciences, 6,94-501.

u.

m,

A . Kruglanski

34

Jones, E.E. (1985).Major developments in social psychology during the past five decades. In G. Lindzey & E. Aronson (Eds.). The Handbook of Soc ial New York Random House. pmcholom (3rd ed.. vol. 1, pp. 47-107). Kahneman. D., Slovic. P. & Rersky. A (Eds.) (1982).Judament Under Uncertaintv; Heuristics and Biases. Cambridge: Cambridge University Press. Kenny. D.A.. & Albright. L. (1987).Accuracy in interpersonal perception: A social relations analysis. psvcholoaical Bul1etin. 390-403. KruglansM. A.W.. & Ajzen, I. (1983).Bias and error in human judgment. E u r o D a Journal of Social Psvcholoq, 19.448-468. Kruglanski. A.W. & Freund. T. (1983). The freezing and unfreezing of lay inferences: Effects on impressional primacy, ethnic stereotyping and numerical anchoring. Journal of Emerimental Soclal Psvcholom, 448-468. Kruglanski. A.W., Friedland, N., & Farkash, E. (1984).Lay persons' sensitivity to statistical information: the case of high perceived relevance. J m f 503-518. personalitv and Social Psvcholo@. Kruglanski. A.W. (1989).Lav eDistemics and h uman knowledae: Coanitive a n d motivational bases. New York: Plenum. Lord, C . G . , Ross, L., & Lepper. M.R. (1979).Biased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence. Journal of Personalitv and Social Psvcholoa, z,2098-2109. Lord. C.G.. Lepper. M.R., & Preston, E. (1984). Considering the opposite: A corrective strategy for social judgment. Journal of Perso nalitv and Social Psychology, 47.1231-1243. LuchincA.S. (1942). Mechanization in problem solving: the effects of einstellung. In J.F. Dashiell (Ed.). Psvcholoaical MonoaraDhS, vol. 54.whole number 248. Markus. H. (1977).Self-schemata and processing of information about the self. Journal of Personalitv and Social Psvchology, s,63-78. McArthur. L.Z., & Baron, R.M. (1983).Toward a n ecological theory of social perception. Psychological Review. 90,215-238. Nisbett. R.E.. & Ross, L. (1980). Human Inference: Strategies and Shortcominas of Social Judgment. New York Prentice Hall. Nisbett. R.E.. Krantz, D.H.. Jepson. C . . & Kunda. Z. (1983). The use of inductive heuristics in everyday inductive reasoning. , - P 90.339363. Nisbett. R.E.. Cheng, P.W.. Fong. G.T., & Lehman, D. (1987). Teaching reasoning. Unpublished manuscript. University of Michigan. Snyder, M.. Tanke, E.D., & Berscheid, E. (1977). Social perception and interpersonal behavior. On the self-fulfilling nature of social stereotypes. Journal of Personalitv and Social Psvcholcgy. 35,656-666. Swann, W.B. (1984). Quest for accuracy in person perception: A matter of pragmatics. Psvcholoaical Review, 91, 457-477. Trope, Y. & Bassok, M. (1983).Information gathering strategies in hypothesis testing. Journal of Ernerimental Social Psvcholo~y, 560-576. Weimer, W.B. (1979).psvcholoav and the ConceDtual Foundations of Science. Hillsdale. NJ: Erlbaum.

m,

u,

s,

u.

Cognitive Biases J. -P. C a v e m i , J. -M. F a b r e and M. Gonzalez (Eds.) Q E l s e v i e r S c i e n c e P u b l i s h e r s 8. V. (North-Holland),

1990

THE ANCHORING-ADJUSTEMENT HEURISTIC IN AN "INFORIMATIONRICH, REAL WORLD SETTING" : KNOWLEDGE ASSESSMENT BY EXPERTS Jean-Paul Cavemi and Jean-Luc P h i s CREPCO ( C e n t e r f o r R e s e a r c h i n C o g n i t i v e Psychology) U n i v e r s i t y of Provence, Aix-en-Provence, F r a n c e

Abstract. Through the manipulation of the external structure of the information available to subjects, our experiments enabled us to more accurately describe the anchoring-adjustment heuristic and its use in the assessment of knowledge acquisition by experts. In Experiment 1, the effects of assimilation to a n anchor were placed in opposition to the effects triggered by a contrast with the other productions to be assessed. When in conflict, only the anchor assimilation effect seems to occur. In Experiment 2. the anchor effect was studied in a situation where an unspecified anchor was to be extracted by subjects. The results showed that only the dimension isolated and manipulated beforehand, i.e. the previous mean, turned out to be determinant. These findings clearly argue in favor of the validation and generalization or prior interpretations. Even when opposed to other types of information. the information traditionally used to provide evidence of anchor effects in the assessment of knowledge acquisition is what counts.

Tversky & Kahneman (1974) initially described the anchoring-adjustment heuristic as one of three strategies frequently used in probabilistic judgement tasks. When using this heuristic, " ... people make estimates by starting from a n initial value that is adjusted to yield the final answer. The initial value, or starting point, may be suggested by the formulation of the problem, or it may be the result of a partial computation. In either case, adjustments are typically insufficient (Slovic & Lichtenstein, 1971). That is, different starting points yield different estimates, which are biased toward the initial values" (Tversky & Kahneman, 1981, p. 14). The scope of application of this heuristic is not restricted to probabilistic judgement: anchoring-adjustment has been reported in papers dealing with inference (Einhorn & Hogarth, 1982), gambling evaluation (Lopes & Ekberg, 1980) and information integration (Lopes, 1982). In fact, anchoring-adjustment may occur in any task requiring a numerical response, provided a n initial estimate is available (Pitz & Sachs. 1984). The psychological reality of such cognitive biases h a s often been questioned, some authors pointing out that this kind of phenomenon might be induced by experimental laboratory settings

36

J.P.

C a v e r n i & J.L.

Peris

(Berkeley & Humphreys, 1982; Hogarth, 1981; Winkler & Murphy, 1973; Ebbesen & Konecni. 1980). According to this view, there is a strong possibility that the bias might vanish when subjects are placed in the information rich context in which the actual decisions are made, just as perceptual illusions disappear when subjects are allowed to interact with the perceived object. The criticism is a serious one, to which a recent paper by Northcraft & Neale (1987) answered by investigating the biasing potential of an initial estimate in a n "information rich, real world setting". In order to assess the generalizability of laboratory demonstrations of the anchoring bias, students and real estate agents were asked to make pricing evaluations about real estate properties, after touring them, and after being given a n experimentally manipulated listing price. Results show that price estimation is biased toward the proposed listing price, as predicted by t h e "anchoring-adjustment" hypothesis. The following experiments are based on the same rationale, and provide a broader base for claims of generality by demonstrating the effect on yet another task: knowledge assessment by experts, or more specifically, grading of papers by teachers (for a review on this topic, see Caverni, 1987). Furthermore, each of these experiments addressed another question. - Studies of anchoring-adjustment situations usually involve only one anchor, and no contextual information. In the real world, however, situations such as knowledge assessment by experts often do involve contextual information. and this contextual information may interact with anchoring effects. In Experiment 1, two factors were manipulated in order to study the effect of antagonistic potential biases. Conceivable outcomes are reciprocal annulment or preponderance of one effect over the other, and any theoretical description of the underlying cognitive processes should account for the data collected in this kind of situation. - Tversky & Kahneman (1974) explicitly stated t h a t a n initial estimate does not have to be given directly in order to work as an anchor. Data is scarce, however, on situations where subjects have a n opportunity to perform a "partial computation" to obtain an anchor. Experiment 2, is such a situation, where the anchor is not directly given to subjects but has to be extracted from a set of information. This extraction process is studied by manipulating the information available to subjects before the evaluation task,

Anchoring and knowledge assessment

31

and results are indicative of what aspects of this information are taken into account. Numerical evaluation can be conveniently described as a function of the object being evaluated. In the case of paper grading, the function could be called G, and the grade given to paper y would be noted G(y). Evaluation, however, is a complex cognitive process, and experimental research h a s demonstrated that a number of factors must be taken into account in order to properly describe the data. Thus, a more powerful notation must be used, and the ' I ' symbol can be borrowed from conditional probability notation in order to represent the grade given to paper y when evaluation takes place in modality a of factor F G(y I Fa). When more than one factor is involved, the symbol for the logical "and" ('A') can be used to specify the considered modality of each factor. Thus, G(y I F a A Gg A HJ is the grade given to paper y when levels u, 6, and a of factors F, G and H are used. EXPERIMENT 1

Paper grading is prone to a well-established bias whereby poorer grades are attributed to a given paper when it is evaluated after one (or several) good paper(s) than when it is evaluated after one (or several) poor paper(s) (for a review, see Caverni, 1987). If this "contextual setting" factor is labeled C. we can write: G(y I Cd c G(y I Cd where ch and C1 stand for the conditions where y was evaluated after high quality and low quality papers, respectively. Grades given to a student's previous productions, if they are known to the evaluator, might act as a n anchor, resulting in the relation: G(y I A+) > G(y I A-1 where A+ and A- stand for conditions where the paper's author is known to have previously earned high or low grades, respectively, on similar work. What predictions can be made concerning situations where both factor C and factor A are present? When the predicted effects are antagonistic, their combination may result in either the prevalence of one over the other (C > A or C c A) or in reciprocal annulment (C = A). Table 1 summarizes the expected results in each condition in terms of a comparison between the grade given to paper y in this condition and the grade given to this same paper in a condition where neither C nor A are present (i.e. when the evaluator

38

J.P.

Caverni

&

J.L.

Peris

has no knowledge about previous productions of this source, and only one paper is to be evaluated). Table 1. Predicted results for the three hypotheses.

A>C

AcC

A=C

As can be seen in Table 1, only those conditions where C and A have antagonistic predicted effects (G(y I A+ A Ch) and G(y I AA Cd) are relevant to the present discussion, since predicted results are the same for all three hypotheses in conditions C(y I A+ A Cd and G(y I A- A Cd. For this reason, only the conditions with antagonistic effects will be used in Experiment 1.

Method Ninety-six teachers in French secondary schools were asked to rate seven texts on a 0 to 20 scale. These texts were based on actual French language papers produced by 6th graders, but were edited in order t o standardize their length and the number of included errors. For a fourth of the subjects, positive information about the authors was given before evaluation ("These papers were written by good pupils"). Another fourth of the subjects received negative information about the authors ("These papers were written by poor pupils"), while the remaining subjects received no information about the authors (control condition). This between-subject factor thus had three levels noted A+, A- and Ao, respectively, Three of the seven papers were experimentally manipulated by introducing a number of errors, resulting in two different versions of each text:

A n c h o r i n g and knowledge assessment

-

39

a "high level" version, with a single style error: a "low level" version, with 8 style errors and 10 spelling errors.

These three papers were always presented in the same order, in 3rd, 4th and 5th position, and for a given subject, were of the same quality. This manipulation resulted in a two-modality, between-subject factor. This factor will hereafter be called the C factor, these three papers serving as the "contextual setting" for later evaluation. As explained before, since only antagonistic-effect conditions are of interest, the C factor was not fully crossed with the A factor, and all subjects in the A+condition were presented high level contextual papers (A+ A C h ) , while all subjects in the Acondition saw low level contextual papers (A- A Cd. Subjects in the A0 control condition, however, were divided into two groups (A0 A C h and A0 A Cc). The four other papers were the test items. Only one version of these texts was used, each of them including 4 style errors and 4 spelling errors (with these errors, the test items were of intermediate quality, when compared with the high and low quality contextual papers). The four test items were presented in 1st 2nd. 6th and 7th position, so that two of them were evaluated before the contextual papers, while two were evaluated after. The grade attributed to the last two papers was the dependent variable of the experiment. The evaluation order of the test papers was controlled by using the following four sequences: 1) TI T2 - - - T3 T4 2) T2 T1 - - - T4 T3 3) T3 T4 - - - TI T2 4) T4 T3 - - - T2 T1 where T i , T2, T3 and T4 are the test papers, and the dashes are the contextual papers. The contextual manipulation is based on the use of either good or bad papers as context. If a subject did not consider that these papers were actually of the expected quality, then the contextual manipulation failed, and this subject's data cannot be used to study the effect of this manipulation on later evaluations. For this reason, the mean grades for the first two papers and for the three contextual papers were computed for each subject, and only those subjects for whom these two means differed by more than 1.5 were considered for further analysis. This criterion was reached by only

Caverni

J.P.

40

&

J.L. Pkris

45 subjects (46.9 YO),and 18 additional subjects had to undergo the experiment in order to have a roughly equivalent number of subjects in each condition. This relative inefficiency of our quality manipulation does not, however, prevent us from studying the effect of this manipulation when it actually is effective. Results Table 2 presents the mean grade and standard deviation for the two last test items (i.e. for papers evaluated after contextual papers), for each of the four experimental conditions. The first comparison of interest is G(y I A0 A Cd vs. G(y I A0 A Cd, which tests for a pure context effect in the control group. It turns out that, as expected, a low quality context leads to better grades on following papers (10.95 vs. 9.53; t(30)= 2.12; p < .05).

Table 2. Data from Experiment 1 (see text). Condition A+

ch

Mean

2.60

11.75 ~

A-

Standard deviation

~

~~

Cc

9.83

1.88

Ch

9.53

1.81

CC

10.95

1.97

The anchoring bias can be tested by comparing G(y I A+ A C d vs. G(y I A O ACd, which reveals the effect of positive information about the author (in a good context) and G ( y I A- A C d vs. G(y I A0 A Cd, which reveals the effect of negative information about the author (in a poor context). In the former case, the expected effect was reliable (11.75 vs. 9.53: t(30) = 2.83; pe.05). but the

Anchoring and knowledge assessment

41

second comparison fell short of significance (9.81 vs. 10.95: t(29) = 1.62; p > .05). Finally, the critical comparison is G ( y I A + A Ch) vs. G(y I A- A C1). I t turns out that positive information about the author leads to better grades than negative information, even when papers supposedly written by good pupils were evaluated in a good context and papers supposedly written by poor pupils were evaluated in a poor context (11.75 vs. 9.83; t(29) = 2.37; p < .05).

Discussion Information about the author, as well as the quality of the preceding contextual papers, proved to affect the grade attributed to a given paper. Even though the G(y I A- A Cd vs. G(y I A g A Cd comparison was not conclusive, it must be pointed out that G(y I A-A Cd led to a higher grade, as predicted by the anchoringadjustment hypothesis. An even stronger point supporting the the assertion that information about the author has an effect is the fact that, when this factor is in conflict with the contextual effect, it is strong enough to override its rival, as demonstrated by the G ( y I A + A Ch) vs. G(y I A- A Cd comparison. The conclusion of Experiment 1 is thus that under conditions where both an anchoring bias (information about the author) and a contextual bias (quality of preceding contextual papers) are in effect, evaluators give precedence to the anchor over the context. EXPERIMENT 2

The purpose of the following experiment was to have subjects deal with a set rather than a single piece of information about the author of the to-be-evaluated paper. Experiment 1 demonstrated that this kind of information is taken into account by subjects, and Experiment 2 was a n attempt to determine exactly what aspects of the available information are actually used. The principle was to experimentally manipulate various features of the available information: if a given feature is important for evaluators, then its manipulation should have an effect on the grades given. The information about the author was presented to the subjects as a set of grades given to that author's previous work. This kind of information was chosen because it is both close to the

42

J.P.

Caverni & J.L. P e r i s

manipulation used in Experiment 1 and realistic, since in real life, teachers quite often keep records of the past achievement of pupils.

Method Forty-eight teachers were asked to grade (on a 0 to 20 scale) twelve texts similar to those used in Experiment 1. Subjects were contacted by mail, each receiving: a) a letter introducing the research topic and purpose: b) 12 papers, labeled from A to L; c) instructions requiring them to grade the papers: d) a grade record, where each letter was associated with a set of five numbers, under the heading "Grades attributed to pupils for previous works". Each five-number set included one grade that was equal to the mean of the set. These sets, however, differed in other respects: - Their mean was either 7 (condition Mi) or 13 (condition Mh). - The grade equal to the mean was in 2nd (condition MP2), 3rd (condition MP3) or 4th (MP4) position in the set. - The standard deviation in the set was either fi (condition SD1) or f i ( c o n d i t i o n SDh). - The set was either globally increasing (last grade higher than the first one, condition Gi) or globally decreasing (last grade lower than the first one, condition Gd). - The highest and lowest grades in the set were either in central positions (2nd and 4th in the set, condition Ec) or in outer positions (1st and 5th in the set, condition Eo). Factors G and E are between-subject factors and define 4 experimental groups, while factors M, MP and SD are within-subject factors and define 12 possible experimental conditions for a given paper. In each experimental group, each paper was associated equally often with a given set of grades.

Results A n analysis of variance taking the five experimental factors into account was run on the grades given to papers by subjects. Only three reliable effects were found: - The average grade was higher when the mean of the corresponding set was 13 than when it was 7 (11.08 vs. 10.12: F(1,44) = 24.58; p < ,001).

A n c h o r i n g and knowledge a s s e s s m e n t

43

- This effect of the set's mean interacted with the position of the highest and lowest grades in the set (F(1.44) = 2.47: p c .05). When the highest and lowest grades were in outer positions, a mean of 13 gave an advantage of 1.35 over a mean of 7 (11.51 vs. l O . 1 6 ) , while when the highest and lowest grades were in central positions in the set, this advantage decreased to 0.76 (10.64 vs. 9.88). - The effect of the set's mean also interacted with the position of the grade equal to that mean (F(1.44) = 3.93; p < .05). When that grade was in 3rd position (i.e. in the middle of the set), a mean of 13 gave an advantage of 1.65 over a mean of 7 (11.30 vs. 9.65), while this advantage was only 0.83 when the set's mean was placed in 2nd or 4th position (10.81vs. 9.98 and 11.12 vs. 10.29, respectively) .

Discussion The mean of the set of grades given to previous work by the same pupil had a biasing effect on the evaluation of a paper. The effect of the mean was stronger (a) when a grade equal to this mean was at the center in the set. and (b) when the highest and lowest grades in the set were in outer positions. On the other hand, presenting a globally increasing or decreasing grade record does not seem to have affected evaluation. A possible overall interpretation of these findings is that subjects do extract the set's mean and use it, and only it, as a n anchor. The order of the grades in the set that seems to make it easiest for subjects to extract the mean (i.e. a value equal to this mean in the center, and maximum and minimum grades in outer positions) is the order that led to the strongest effect of this mean on evaluation. A potentially meaningful factor (whether the grades are getting better or worse from one paper to the next) seems to be discarded altogether. These results also suggest that this "anchor extraction" process is not completely deliberate: scrambling the order of five positive integers bellow twenty should not, presumably, prevent high-school teachers from consciously computing the mean. The fact that this scrambling does affect subjects' judgement thus has a double meaning: subjects use the mean of previous grades as an anchor, and they are most likely unaware of this fact.

44

J.P.

C a v e r n i & J.L.

Peris

CONCLUSION

In any attempt to account for general human cognitive processes, it is legitimate to raise the question of the potential specificity of their implementation in experimental situations as compared to natural situations or situations designed for motives other than research. However, the paradoxical nature of the answer to this question is evident: it is only by incorporating the properties of natural situations into experimental situations that we can test the generality of the processes initially found to occur in a prior experimental situation. Through the manipulation of the external structure of the information available to subjects, our experiments enabled us to more accurately describe the anchoring-adjustment heuristic and its use in the assessment of knowledge acquisition by experts. In Experiment 1 , the effects of assimilation to an anchor were placed in opposition to the effects triggered by a contrast with the other productions to be assessed. Until now, effects of this type have been found in situations where one or the other has been studied, but not both. When in conflict, only the anchor assimilation effect appears to occur. In Experiment 2. the anchor effect was studied in a situation where a n unspecified anchor was to be extracted by the subjects. This could be done in several ways. The results showed that only the dimension isolated and manipulated beforehand, i.e. the previous mean, turned out to be determinant. These findings clearly argue in favor of the validation and generalization or prior interpretations. Even when opposed to other types of information, the information traditionally used to provide evidence of anchor effects in the assessment of knowledge acquisition is what counts. Thus, manipulation of the external structure of information is one way of testing hypotheses concerning the internal structure of cognitive functioning. Past findings lead us to hypothesize that the privileged role granted to the anchor is based on the shared structure of the information that is chosen and the response that is to be given, in our case, the expression of a given state of knowledge available to the subject at a given time. Other potential information is neglected whenever it is part of a set in which the degree of commonality between the elements can be differentiated: this differentiation may explain why context effects no longer occur

A n c h o r i n g and k n o w l e d g e a s s e s s m e n t

45

when in conflict with information that is relative to the level to be expressed in the response. Similarly, we can expect the fact that a series of grades occurs in random or increasing order to be a potential anchor whenever the mean is not available.

REFERENCES Berkeley, D. , & Humphreys. P. (1982). Structuring decision problems and the bias heuristic. Acta Psvcholocica, 50. 201-252. Caverni. J.P. (1987).Knowledge acquisition assessment by experts: Effects and models of the cognitive functioning of evaluators. EuroDean Journal of psvcholom of Education, 2. 119-131. Ebbesen E.B.. & Konecni, V.J. (1980)On the external validity of decision-making Research: What do we know about decisions in the real world?. In T. S. Wallsten (Ed.) Cocnftive Drocesses in choice and decision behavior, Hillsdale. NJ: Laurence Erlbaum Associates. Hogarth. R.M. (1981).Beyond discrete biases: Functional and dysfunctional aspects 197-217. of judgemental heuristics. Psvcholoaical Bulletin. Lopes. L.L. & Ekberg. P.H.S. (1980). Test of an ordering hypothesis in risky decision making. Acta Psvchologica. 45,161-167. Slovic. P.. & Lichtenstein. S. (1971).Comparison of Bayesian and regression approaches to the study of information processing in judgement. Oreanbational Behavior and Human Performance. 6, 649-744. Tversky. A.. & Kahneman. D. (1974).Judgement under uncertainty: Heuristics and 1124-1131. biases. Science, B. Tversky. A.. & Kahneman. D. (1981). The framing of decision and the rationality of choice. Science, 21 1, 453-458. Winkler, RL., & Murphy, A.H. (1973).Experiments in the laboratory and the real 252-270. world. w n i z a t i o n a l Behavior and Human Perfomancc.

a,

m.

This Page Intentionally Left Blank

Cogni ti ve B i a s e s J.-P. Caverni, J.-M. Fabre and M . Gonzalez ( E d s . ) 8 E l s e v i e r Science P u b l i s h e r s B . V . (North-Holland), 1990

GROUPING AND CATEGORIZATION IN JUDGMENTS OF CONTINGENCY Klaus Fiedler and Roman Graf U n i v e r s i t y of G i e s s e n , West Germany

Abstract.

The cognitive process and performance of contingency detection is examined. The same statistical contingency may be easier to detect or experienced to be stronger when the stimulus events can be grouped and categorized in a consistent and intelligible fashion. Three experiments are reported to Illustrate this point. First it is shown that observers are more sensitive to event contingencies when observations can be encoded in terms of meaningful categories (i.e.. when the bivariate distribution of a disease and a virus over different countries is consistent within geographic categories of countries). Another experiment addresses redundancy among correlated attributes and context variables. The perceived contingency between and y is shown to depend on the redundancy created by an irrelevant context variable which can increase the consistency in the multivariate system. Finally. the possibility is considered that many socially significant attributes (e.g.. honesty, attraction) cannot be observed immediately but have to be inferred from more p r o m a 1 cues. Framing and grouping of these mediating cues can also influence the ease with which contingencies are detected.

x

INTRODUCTION

The ability to abstract contingencies from a stream of multiple observations over time is presupposed as a basic cognitive skill in virtually all conceptualizations of human intelligence (Bindra, 1976; Bolles, 1972; Heider, 1958; Inhelder & Piaget. 1958; Rotter, 1966). Learning theories, attribution theories, and approaches to the study of decision making, concept formation, reading, or behavior therapy are all based on the fundamental assumption that higher organisms possess the necessary devices for detecting contingencies between stimuli, behaviors, and consequences. Inspired by the pioneering work of Smedslund (1963) and Inhelder and Piaget (1958). numerous experiments conducted during the following two decades revolved around the issue of performance and shortcomings in contingency judgment, the search for biases being the most prominent objective (cf. Alloy and Tabachnik, 1984; Crocker, 1981; Nisbett and Ross, 1980). In the light of more recent research, however, the overly pessimistic conclusion that human beings are insensitive to contingencies had

40

K. F i e d l e r

to be modified in several respects. According to the present state of the art, humans are in principle capable of capturing event contingencies -- provided the parameters of the observation task facilitate inductive reasoning (Alloy & Tabachnik. 1984; Bettman, Roedder John, & Scott, 1986; Fiedler, 1985) and observers are not misled by too strong preconceptions or stereotypical beliefs about the contingencies to be detected. When, on the other hand, prior expectancies are a t variance with the contingency actually observed, "theory-driven'' forces usually override "data-driven'' processes. This is perhaps the best established message that the literature can offer at present (Crocker, 1981; Fiedler, 1985; Wright & Murphy, 1984). However, top-down influences are by no means confined t o shared beliefs or empirical expectations about the sign and strength of the contingency itself. Indeed, there are more subtle ways in which prior knowledge can affect the outcome of a n observation task even when observers do not expect a certain outcome. To the extent that the structure of prior knowledge and its organization in memory determine the encoding and representation of observed events, the abstraction of contingencies from new information will be a function of older information in long-term memory. In particular, the present article addresses three different facets of information structure that may affect the grouping and organization of observations in an inductive reasoning task. These three facets pertain to (a) the structuring of multiple observations in a stimulus series, (b) the relations among several attributes that covary across observations, and (c) the rule by which the distal meaning of an observation is inferred on the basis of more proximal cues. The common denominator underlying all three aspects is that an observation task requires the observer's ability to reduce and efficiently encode the complex information and that success on such a task depends on the grouping and categorization of data regarding (a), (b), and (c). Considered from this perspective, contingency assessment means, in the first place, to make sense of the observations rather than to provide an accurate statistical account. This is best illustrated by example. Imagine a consumer survey intended to assess people's perception of the relation between prices and quality of products. Three kinds of grouping or categorization processes may be involved. The first variant refers to the depth of cues mediating the perception of single events. Many distal attributes (e.g., product quality) cannot be observed sensorily but have to be inferred from

Contingency judgements

49

more proximal cues. Thus, the quality of a hi-fi deck cannot be perceived like the colour of a car, but has to be "construed" or "computed" on the basis of physical and esthetical cue^ of the hi-fi deck. Likewise, the categorization of the hi-fi deck as either expensive or cheap involves a comparative judgment against the background of general knowledge about other products and the current state of technology. This raises another interesting point, namely, that the cues which help to infer the product's quality and price may partly overlap so that inferences regarding the two attributes may not be independent. This may artificially create or bolster u p a n apparent contingency if perceived quality and perceived expensiveness are inferred from overlapping sets of cues. Secondly, categorical inferences originate in the need to organize the width of multiple observations. Consumers may be informed about product prices on one occasion and about quality on a different occasion, the corresponding observations being often uncoordinated and unsynchronized. Before price can be related to quality in memory, the two univariate distributions have to be coordinated -- a process that relies heavily on the encoding of the carrier events (i.e., the products). An efficient and often utilized encoding strategy (cf. Cohen, 1966) relies on higher-order product categories (e.g., food, clothes, electronics, furniture). Contingency assessment will therefore depend on how attributes are related a t the categorical level. For instance, some unexpected cooccurrences of, say, low prices and high quality should be more apparent when they occur regularly for certain salient product categories t h a n when the unexpected events are distributed unsystematically across all product categories. The third and final aspect pertains to the breadth dimension of different stimulus attributes including the context attributes which make up a n observation task. Context variables can create redundancy and raise the overall balance in the system of covarying attributes s o that a relation between x and y may become more apparent, if y is highly correlated with some extraneous attribute z. For instance, the price-quality relation may be easier to detect if prices are regularly correlated with different shops or brands that render the price differences more salient or meaningful. Such a reinforcing effect of irrelevant, correlated variables is commonly known as an accentuation effect (Eiser & Stroebe, 1972). Figure 1 summarizes the assumptions developed t h u s far concerning the three different facets of information structure

K. F i e d l e r

50

supposed to mediate the cognitive process of contingency assessment. The vertical or depth dimension refers to a set of cues -a,b,c, - ...,f from which the meaning of singular observations has to be abstracted. The horizontal axis denotes the breadth of different stimulus attributes or variables u,v_,w_, ...,z yielding a pattern of contingencies, and the width of multiple observations (over time) L,i& ,...,n is represented by the diagonal axis. Grouping and organization in all three dimensions can help to make sense of observations, to understand the contingency, judge the validity of the data, and organize the empirical information in memory. Depth

c

r

Breadth

Figure 1. Three facets of grouping and categorization in contingency assessment.

To illustrate the operation of all three aspects of cognitive structure, three empirical studies are reported in the remainder of this article, one pertaining to each aspect. The stimulus materials and t a s k situations vary between studies, b u t all three demonstrations share, as a common denominator, the impact of grouping and categorization on contingency assessment. While the studies provide nothing but qualitative demonstrations based on

51

Contingency judgements

simple research methods, it is worth noting that these findings are completely new in the relevant literature. ORGANIZING MULTIPLE OBSERVATIONS

To demonstrate the dependence of contingency judgments on the way in which multiple observations are organized in memory (i.e.. the diagonal in Figure 11, an experimental task was chosen for which it seems safe to assume a categorical memory structure. Participants had to observe the distribution of a fictitious disease and a fictitious virus across countries from different geographical parts of the world (i.e., African countries, Asian countries, South American countries). The information on the occurrence of the disease and of the virus was presented in a n unsynchronized fashion, that is, in two separate runs. The coordination problem arising from such an order of presentation is illustrated in Figure 2 along with the stimulus frequencies actually used. Present Disease Virus

16 8

Absent

1

8 16

Disease present disease absent Virus present

Virus absent

Virus present

Virus absent

Figure 2. The same univariate distributions of a virus and a disease over countries (a) can be coordinated such that the correlation between virus and disease is either positive (b) or negative (c). The same univariate distributions of the disease and the virus (Figure 2a) are compatible with quite different b i v a r i a t e distributions (Figure 2b.c) when the 24 observation pairs are coordinated. Depending on the experimental condition, the association of countries with the occurrence of the virus and disease

52

K. F i e d l e r

was manipulated to yield either a 0 = +0.50 correlation as in Figure 2b or a 0 = -0.25 correlation as in 2c, although the univariate distributions of the virus and the disease were the same as in Figure 2a in all conditions. Since information on the virus and the disease was given in separate runs, the detection of the (sign of the) contingency will obviously depend on memory for the associations between particular countries and the occurrence of the virus and the disease. At this point, categorical coding comes into play. When the categorical organization of countries (by geographic regions) facilitates the recognition of the relationship -- because both virus and disease are consistently present for certain categories and consistently absent for others -- contingency detection should be facilitated. Performance should be impaired if the categorical structure does not help organize observations. The central prediction is therefore that of an interaction between the direction of the contingency presented (0= +0.50 vs. -0.25) and categorical structure: When confirming observations (virus present/disease present and virus absent/disease absent) and disconfirming observations (virus present/disease absent and virus absent/disease present) consistently belong to separate geographic categories, the direction of the contingency should be easier to detect than when the same @-correlation is distributed unsystematically across categories. This prediction is actually borne out by the data. Only when the observations make sense in terms of the geographic grouping of countries are the positive and negative contingencies discriminated successfully (see Figure 3). THE IWACT OF AN IRRELEVANT CONTEXT VARIABLE

The next experiment addresses the breadth dimension referring to multiple attributes that covary in an observation task. Does the perceived contingency between two focal variables depend on the redundancy created by other, task-irrelevant, context variables? To answer this question, we consider the most elementary case of one single context variable z which is redundant with one of two focal variables x and y. Any redundancy between x and the irrelevant context variable z may facilitate the observation task and thereby help to assess, or disguise, the contingency of x_ and y. For instance, a moderate correlation between political arguments and different newspapers may be easier to detect when

53

Contingency judgements

political arguments are strongly correlated with different sources (e.g., parties).

M e a n contingency judgment

r = - 0.25

r =

+

0.50

Stimulus contingency Figure 3. Mean contingency judgments as a function of actual contingency and categorical structure.

Lonely-hearts advertisements were used as stimuli because this material represents multiple correlations among diverse person attributes yet is compatible with the experimental requirement to control the serial order of the information. The observers' attention was directed at the contingency between two focal variables in the series of ads, namely, sex and education. A task-irrelevant variable, income, was manipulated to be either redundant (i.e.. highly correlated) with education, or not. A strong positive correlation between related variables such as education and income should raise overall consistency and help organize the multivariate system into meaningful clusters. Thus, a n interaction was hypothesized between the direction of the contingency presented (the proportion of male

54

K. F i e d l e r

and female cases with higher and lower education being either 8:4 and 4:8 or vice versa) and the redundancy of the context variable (education-income correlation being Q, = 0.67 vs. 0.00). Since subject sex did not correlate with any of the dependent measures, the data were pooled across male and female observers. Judgments of the relation between education and the context variable, income, did not reflect the redundancy manipulation, suggesting that participants were not aware of the task-irrelevant covariate. Several control judgments were biased (uniformly in all experimental conditions) in the direction of common stereotypes, women being associated with lower income, high education with high income, high education with likeability, and prosperity with the prospect of getting many responses to a n ad. In particular, there was a highly significant illusory tendency to associate male people with more education. Most importantly, however, the central prediction regarding the judged contingency between sex and education was supported. Apart from a direction of contingency main effect, F(1,36) = 9.31, p c 0.01, indicating that observers were in general sensitive to the manipulated contingency, a significant direction x redundancy interaction was obtained, F(1.36) = 4.29, p c 0.05. Accordingly, the correct discrimination of opposite contingencies is largely confined to the redundant condition b u t almost disappears in the nonredundant condition (Figure 4). INFERRING DISTAL ATTRIBUTES FROM CORRELATED CUES

To examine the third aspect of grouping and categorization, referring to the cues mediating the perception of distal attributes (Le., the depth dimension in Figure l), participants were asked to assess the contingency between two distal personality attributes, satisfaction and vigor. The degree to which stimulus persons possessed these attributes had to be actively inferred on the basis of six cues: Humor, Intelligence, Health, Social Contact, Courage, and Self-Adequacy. Although these cues do not refer to basic physical attributes, they are less complex than the two distal attributes and more amenable to public judgments by other people. One subset of cues (Intelligence, Courage, Health) was assumed to be more suitable for inferring vigor whereas the other subset (Social Contact, Humor, Self-Adequacy) appears to be more closely related to satisfaction. The manipulation of interest pertains

55

Contingency judgements

to these two subsets of cues. In the common cue condition, judges had to infer satisfaction and vigor from the same common pool of mixed cues, with no mention of the division into subsets. In the SeDarable cue condition, by contrast, the two subsets of cues were grouped together so that the graphical presentation encouraged the formation of independent inferences based on separate cues.

M e a n contingency j u d g m e n t 1.2 1.o

0.8

0.6 0.4 0.2

0.0 ~n3

r = - 0.33

r =

+

0.33

Stimulus contingency Figure 4. Mean contingency judgments as a function of actual contingency and a task-irrelevant context variable. Although the cue-intercorrelations built into the stimulus series (using a n algorithm by Kaiser & Dickman, 1962) were held constant (r = 0.40 within subsets and r = 0.10 between subsets), the inferred contingency between vigor and satisfaction was predicted to be higher when both attributes were "construed" from the same common pool than when separable cue subsets were suggested. Forty stimulus person were described in terms of the six cue dimensions, represented as graphical profiles and the judges' task was to infer the stimulus persons' vigor and satisfaction from the

56

K. F i e d l e r

profiles, These inferences were then analyzed according to Hursch, Hammond and Hursch's (1964) formalization of the lens model, which amounts to a twofold multiple regression of the two inferences on the same set of cues. The regression weights confirm the premise that judges in the separable cue condition discriminate between cue subsets more than judges in the common cue condition. Of major interest, however, are the "perceived" correlations between the distal attributes. Consistent with the expectation, inferences of vigor and satisfaction correlated higher in the common cue condition than in the separable cue condition (median r = +0.49 versus +0.21; z = 3.10, p < 0.005, according to a Mann-Whitney U-test). Thus, the categorical grouping of the cues mediating the perception of distal attributes can also be a crucial factor in contingency assessment under natural conditions. CONCLUDING REMARK

These empirical demonstrations confirm and illustrate the theoretical argument that subjective assessments of contingencies depend on the framing and organization of three facets of the underlying data array: the width of multiple observations, the breadth of covarying stimulus attributes, and the deDth of cues mediating the perception of complex attributes. Taken together, all three reported experiments show that a (statistically invariant) contingency is more easily detected or experienced as stronger when observations can be meaningfully grouped or organized in these three dimensions. The first experiment indicated that people are more sensitive to an observed relation between a fictitious virus and a disease across different countries when the multiple observations make sense in terms of geographic categories. Secondly, the perceived correlation of two focal attributes (sex and education) was shown to depend on the context in which it is embedded. In particular, the focal correlation was accentuated when a n irrelevant context attribute (income) was correlated in a redundant fashion with education, rendering the pattern of interrelations more meaningful and intelligible. Finally, the third experiment demonstrated that even the grouping and framing of the cues that mediate the assessment of more abstract attributes can affect subjectively experienced contingencies. The findings from all three experiments converge in showing t h a t detecting contingencies means, in the first place, to make sense of

57

Contingency judgements

observations by organizing events, attributes, and cues in terms of consistent patterns, in accordance with prior knowledge. At the same time, it becomes apparent t h a t modelling contingency assessment as a purely statistical problem provides an impoverished and inadequate picture of the cognitive operations involved in the processing of contingency information.

REFERENCES Alloy, L.B.. & Tabachnik, N. (1984).Assessment of covariation by humans and animals: the joint influence of prior expectations and current situational information. Psvcholoaical Review, 112-149. Bettman, J.R.. Roedder John, D.. & Scott, C.A. (1986).Covariation assessment by consumers. m e J ournal of Consumer Research, 316-326 Bindra. D. (1976). A theorv of intelligent behavior. New York: Wfley. Bolles, RC. (1972).Reinforcement, expectancy, and learning. Psvcholoeical Review. 29.394-409. Cohen, B.H. (1966).Some-or-none characteristics of coding behavior. &puma1 of Verbal Learning and Verbal Behavlor. 5.182-187. Crocker. J. (1981). Judgment of covariation by social perceivers. Psvcholoeical Bulletin. 90.272-292. Eiser. J.R. & Stroebe. W. (1972).Categorization and social i u d m e n t . New York: Academic Press. Fiedler. K. (1985).Foan itive Strukturieruna der sozialen Umwell. Goettingen: Hogrefe. Heider. F. (1958). ~ D S V holom C of interpersonal relations. New York: Wfley. Hursch. C.. Hammond. K.R.. & Hursch. J. (1964).Some methodological considerations in multiple-cue probability studies. Psvcholodca Review,

a,

u,

a.

42-60. Inhelder. B. & Piaget. J. (1958).The arowth of logical thinking from childhood

tQ

&lescencc. New York Basic Books. Kaiser. H.F., 81 Dlckman, K. (1962).Sample and population score matrices and sample correlation matrices from a n arbitrary population correlation matrix. Psvchometrika. 27. 179-182. Nisbett, R.. & Ross, L. (1980). Human inference: Strategies and shortcomings of social judgments. Englewood Cliffs, NJ: Prentice Hall. Rotter. J.B. (1966).Generalized expectancies for internal versus external control of reinforcement. Psvcholoaical M o n o a r m . (1, Whole Nr. 609). Smedslund. J. (1963). The concept of correlation in adults. Scandinavian Journal of PWcholo&, 4. 165-173. Wright. J.C.. & Murphy, G.L. (1984). The utility of theories in intuitive statistics: The robustness of theory-based judgments. Journal of ExDerimental Psvcholoa General. 113,301-322.

an,

This Page Intentionally Left Blank

Cognitive Biases F a b r e a n d M. Gonzalez ( E d s . ) J.-P. Caverni, J.-M. 0 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

FRAMING BIASES IN GENETIC RISK PERCEPTION Jo Huys*, Gerry Evers-Kiebooms**. and G&ry d'Ydewalle* * **

Department o f Psychology, U n i v e r s i t y of Leuven,

Belgium

C e n t e r f o r Human G e n e t i c s , U n i v e r s i t y of Leuven, Belgium

Abstract. In order to investigate the effect of framing (Tversky & Kahneman.

1981)

in the context of genetic risk, a description of the symptoms of Down's syndrome was given to 120 women. The recurrence risk was presented as the probability of having an affected child or as the probability of havLng a healthy child. Prenatal diagnosis was framed a s a way of reducing the overall risk of having an affected child or a s a way of eliminating the risk of having a child with Down's syndrome. We found that when the risk was framed in terms of a healthy child. some people reformulated it in terms of a n affected child. We also found that more than half of the women gave an incorrect answer. This was probably due to the difficulty of the description, involving a number of probabilities.

INTRODUCTION

The technology for the diagnosis of an ever increasing number of inheritable diseases and traits is developing rapidly. While the contribution of gross chromosomal abnormalities such as trisomies to genetic pathology h a s long been recognized, small structural chromosome defects are now being detected by high resolution banding techniques in syndromes previously considered to be due to single-gene defects. The diagnosis and carrier detection of many diseases are becoming realities. Consequently, a growing number of individuals and parents are confronted with genetic information. Not only does this increase their control and responsibiIity, it also forces them to make difficult and often painful decisions. For example, do they want a pregnancy if there is a n increased risk of having a child with a hereditary disease or, if available, do they want prenatal diagnosis (Evers-Kiebooms et al., 1987)? The sophisticated development of genetic technology contrasts sharply with the knowledge and understanding of the general public on these matters. The need for appropriate information and for support in decision making about genetic problems h a s become critical. One of the purposes of genetic counseling is to meet this challenge. In 1975, the aims of genetic

60

J . Huys,

G. Evers-Kieboorns,

& G.

d'Ydewalle

counseling were defined by the Committee on Genetic Counseling as follows: "Genetic counseling is a communication process which deals with the human problems associated with the occurrence, or the risk of a genetic disorder in a family. This process involves an attempt by one or more appropriately trained persons to help the individual or family: (1) comprehend the medical facts, including the diagnosis, the probable course of the disorder, and the available management: (2) appreciate the way heredity contributes to this disorder, and the risk of recurrence in specified relatives: (3) understand the options for dealing with the risk of recurrence: (4) choose the course of action which seems appropriate to them in view of their risk and their family goals and act in accordance with that decision: (5) make the best possible adjustment to the disorder in a n affected family member and/or to the risk of recurrence of that disorder." Evers-Kiebooms and Van den Berghe (1979) reviewed the follow-up studies on the impact of genetic counseling published since 1970. They concluded that although genetic counseling improves the knowledge of the recurrence risk, many parents of children with a genetic disorder fail to understand the genetic implications of the disease adequately. Second, the design of a number of published studies does not permit the assessment of the impact of genetic counseling on knowledge, on reproductive plans, and on reproduction. These and other findings raise the question of if and to what extent concepts, theories, and models from the field of risk perception and decision making could expand our insight into the subject. We are not looking for what is the "right" way to present or perceive a risk figure, or for what is a "good" decision to make, but if parents are to decide themselves on whether, for example, to have another child or not, we must find methods for presenting them with the relevant data. A review of the literature on risk perception and decision making (Billen, Evers-Kiebooms, & d'Ydewalle, 1987; Huys, Evers-Kiebooms, & d'Ydewalle, 1989) revealed that the perception of the presented information is not as uniform as one might expect. Framing (Tversky & Kahneman, 1981) or formulation of the problem influences the evaluation of the different options.

Framing b i a s e s i n g e n e t i c r i s k p e r c e p t i o n

61

GENETIC RISK PERCEPTION AND FRAMING

The communication of a recurrence risk is a n important aspect of genetic counseling. Up until a few years ago, it was generally assumed that reproductive decisions are influenced by risk figures in predictable ways: the higher the risk level, the stronger the determent from further childbearing (Emery et al., 1972). This assumption fits the concept of rational decision making perfectly. However, Lippman-Hand and Frazer (1979) found that parents tend to transform the risk into binary form: the two alternatives, the child will be normal or the child will be affected, are given equal consideration. Moreover, Evers-Kiebooms and Van den Berghe (1979). and Swerts (1987) found that a significant number of parents have problems recalling the correct recurrence risk. And even when it is recalled, the understanding of hereditary transmission is still rather superficial (Evers-Kiebooms, 1987). With these complexities in mind, we set out to investigate the communication of risk in the context of genetic counseling. Although one of the main goals of genetic counseling is to provide information about the hereditary disease, the information is not always presented the same way. Studies of judgment and choice have revealed that the preference for one of the options is readily influenced by the formulation of the problem. Indeed, differences in presentation may well affect the way people perceive their risk and the decisions they make, contrary to the invariance principle of normative decision theory. There are two framing aspects in which we are particularly interested. First, a problem can be formulated in terms of gains or in terms of losses. The public health problem (Tversky & Kahneman, 1981) illustrates the effect of this formulation. A change in wording from "lives saved" to "lives lost" induces a shift in preference from risk aversion to risk seeking. More similar to the situation of genetic risk is the situation studied by McNeil, Pauker, Sox, and Tversky (1982) concerning how people use statistical information on the possible outcomes of alternative therapies, surgery or radiation therapy, for operable lung cancer. Surgery has a n average operative mortality rate of 1 0 9 ' 0 , and a five-year survival rate of 34%. For radiation therapy there is no treatment mortality, and the five-year survival rate is 22%. One of their manipulations was the characterization or framing of the outcomes in terms of mortality or in terms of survival. As a result,

62

J. Huys, G. Evers-Kiebooms,

& G.

d'Ydewalle

the preferences of physicians and patients between therapies varied markedly with the formulation of the outcomes. Surgery was relatively more attractive in the survival frame than in the mortality frame. Kahneman and Tversky (1986) argue that framing is a powerful determinant of choice, stating that "the failure of invariance is both pervasive and robust. I t is as common among sophisticated respondents as among naive ones, and it is not eliminated even when the same respondents answer both questions within a few minutes" (p. 198). However, in the same paper they state that "it is worthy of note that in other contexts people automatically transform equivalent messages into the same representation" (p. 204). The second aspect of framing in which we are interested is that undertaking a certain action appears more attractive if it is framed as the elimination of risk than when it is described as a reduction of risk (Kahneman & Tversky, 1979). Outcomes that are merely probable are underweighted in comparison with outcomes that are obtained with certainty. This pseudo-certainty effect is illustrated by Slavic, Fischhoff, and Lichtenstein (1982). who found that a hypothetical vaccine that reduces the probability of contracting a disease from 20% to 10% is less attractive if it is described as effective in half of the cases than if it is presented as fully effective against one of two exclusive and equally probable virus strains that produce identical symptoms. The formulation of the recurrence risks in genetic counseling may influence the person's perception of probability and provides a framework in which meaning is assigned to the risk figures. Kessler and Levine (1987) investigated the effect of linguistic framing of probabilities on their assessment. Subjects were asked to evaluate pairs of numbers that they were told were mathematically equivalent. In each pair one number was framed as a proportion and the other number as its equivalent percent. In Studies 1 and 2, the numbers represented nothing but numbers: in Studies 3 and 4, the subjects were asked to imagine themselves as prospective parents, and the contrasts were presented as if they were genetic risks. The most intriguing finding was that a new strategy of responding emerged when numbers were presented a s genetic risks. In Studies 1 and 2, everyone justified the evaluation with numerical reasoning (e.g. since 50 is greater than either 1 or 2 then 50% is greater than 1/21. In Studies 3 and 4, a number of subjects

Framing biases in genetic risk perception

63

reported a person-reasoning strategy in which they imagined groups of faces or people (e.g. a one out of four risk is associated with a n image of four persons, one of which was seen as affected). Thus presenting probabilities as recurrence risks rather than as abstract numbers altered the information and meaning of the numbers and the cognitive processes used to evaluate them. Although this study is not immediately comparable to our study, it does demonstrate that the framework in which probabilities are presented influences their evaluation. This is consistent with Pearn (1979) who, in his discussion of the subjective assessment of recurrence risks, suggested that how a recurrence risk is presented or expressed may influence the cognitive processing of such information. AN EXPERIMENTAL STUDY OF FRAMING IN GENETIC

RISK

The two aspects of framing, presentation in terms of gains or in terms of losses and the formulation of an action as leading to a n elimination or to a reduction of risk, can easily be applied in the context of hereditary disease. First, the recurrence risk can be formulated as the probability of having a n affected child or it can be formulated as the probability of having a healthy child. Second, prenatal diagnosis can be framed as reducing the overall risk of having an affected child or it can be presented as eliminating the risk of having a child with the hereditary disease for which the risk was specifically increased. Subjects

The subjects were 120 women between 2 5 and 35 years of age. This population was chosen because it is the age group that is most involved in reproductive decisions. The women were recruited from a number of women's organizations. Participation was voluntary. Method

All the subjects received a description of the symptoms of Down's syndrome. We chose this disorder because its prenatal diagnosis is readily available and because it is the most common serious chromosome abnormality. Therefore, it is quite well known. Parents who already have a child with Down's syndrome have a 1%

64

J. Huys, G. Evers-Kiebooms,

&

G. d'Ydewalle

risk of having another child with Down's syndrome, and as there is a general 2% risk of having a seriously affected child, these parents have an overall risk of 3%. In the second part of the text, the two elements of framing were manipulated. For one half of the subjects, the recurrence risk was formulated in terms of the probability of having an affected child (1% for Down's syndrome and a n overall risk of 3%). while for the other half the risk was presented in terms of the probability of having a healthy child (99% chance of having a child without Down's syndrome, and 97% chance of having a normal child). To investigate the influence of the pseudo-certainty effect, prenatal diagnosis was described as eliminating the risk of having a child with Down's syndrome (from 1% to O%, or from 99% to 100%), or it was presented as reducing the risk of having a child with a serious disorder (from 3% to 2%, or from 97% to 98%), or prenatal diagnosis was just explained without explicitly mentioning reduction or elimination. Each woman was assigned to one of the six conditions of the 2x3 factorial design. After reading the text, they all had t o answer a number of questions. The main question, "Parents who already have a child with Down's syndrome have .......% chance of having ............ on a subsequent pregnancy," checked whether people used the same formulation to describe the probability as was used in the text. The next question measured on a seven-point scale the perception of the percentage that was given a s a n answer in the first question. Obviously, this could have two meanings because some people answered the first question in terms of "% chance of having a normal child", while others answered it in terms of "Yo chance of having an affected child". The other questions measured the perception of the seriousness of the disease, the importance of prenatal diagnosis, and the decisions that the women would make about pregnancy and about making use of prenatal diagnosis should they have an increased risk of Down's syndrome. Results

For the main question, the frequencies of the answers within each category of formulation are presented in Table 1. The results reveal two main things. First, when the risk was framed as a probability of having a n affected child everyone (35/35),excluding the women who gave a n incorrect answer, adopted this description.

Framing b i a s e s i n g e n e t i c r i s k p e r c e p t i o n

65

On the other hand, when the risk was framed as a probability of having a healthy child, a third of the women (8/27) transformed this formulation into a probability of having a n affected child. Second, half of the women (58/120) gave a n incorrect answer. One condition (risk framed as probability of having an affected child, and prenatal diagnosis explained without mentioning elimination or reduction of risk) is noteworthy: only three women gave a n incorrect answer. The large number of incorrect answers casts doubt upon the meaning of the answers to the other questions. Therefore, we will further restrict our attention to the pattern of findings for the main question.

Table 1. Frequencies of the different answers to the main question.

CONDITIONS % affected child

observed answers

risk elimination

risk reduction

not mentioned

% normal c h i l d

risk elimination

risk reduction

not mentioned

6

8

10

17

12

9

syndrome

6

5

8

3

3

3% s e r i o u s l y affected

2

5

9

1

1

8

5

CORRECT ANSWERS

1%Down's

97% normal 99% w i t h o u t Down I s syndrome

INCORRECT ANSWERS

5

1

12

10

3

8

11

14

66

J. Huys, G. Evers-Kiebooms,

&

G. d'Ydewalle

DISCUSSION

We expected, first, that people would adopt the formulation of the given description, and second, that this formulation would influence the evaluations of the probability and consequently the decisions made. As stated above (see Table 1). the first expectation was only met in the conditions where the occurrence risk was formulated in terms of having a n affected child. When the probability was presented in terms of having a healthy child, some people (8/27) recoded it in a representation that abstracted from the actual formulation. Framing, as proposed by Tversky and Kahneman (1981). assumes that decision makers are only using the information that is explicitly displayed in the formulation of the problem. Information that has to be inferred from the display or created by some mental transformation tends to be ignored. Our data suggest that people do not always ignore implicit information in a context of genetic counseling. When a genetic risk is framed as a probability of having a healthy child, some women recode it as a probability of having an affected child. This reformulation is probably because information about a healthy child is much more unusual than information about an affected child in the context of genetic risk. Moreover, we can assume that when people hear about a hereditary disease, they have in mind a genetic risk, and not the probability of having a healthy child. When the formulation is not consistent with the representation they have in mind, they probably will adhere to their representation. In this sense, the studies of Kessler and Levine (1987) are applicable. They showed that when subjects role-played parents a t risk for a child with a genetic disorder, they tended to use a n alternative form of reasoning. This suggests that the context in which probabilities are presented can be a n important factor in determining the way the probabilities are perceived, The second intriguing finding was that half of the women gave an incorrect answer, which might have been due to the degree of difficulty of the description. Each text contained probabilities about the genetic disorder and about the effect of prenatal diagnosis, and a s Edwards (1983) noted, people are not used to working with probabilities. This explanation is supported by the finding that only three women gave a n incorrect answer in one condition, the one in

Framing biases in genetic r i s k p e r c e p t i o n

61

fact where the description was the easiest to understand. First, since the risk was formulated as the probability of having an affected child, there was no need to reformulate it. Second, prenatal diagnosis had just been explained, without mentioning its effect on the reduction or elimination of risk. So, this condition contained the fewest probabilities. To answer to some of the observed difficulties in this study, we are planning another experiment in which we will use another genetic disease with a higher recurrence risk: in the previous study, a bottom effect may also have been at play. Since the genetic risk for Down's syndrome was perceived as rather small and prenatal diagnosis was available, most women were inclined to choose another pregnancy. Second, we will try to explain the probabilities in a more comprehensible way. To minimize the textual difficulties we will separate the manipulation of framing in terms of gains or losses and the manipulation of framing in terms of reduction or elimination of risk. In this way, we hope to reduce the number of incorrect answers.

REFERENCES Ad hoc committee on genetic counseling (1975).Genetic counseling. Ame rican Journal of Human Genetics, 27. 240-242. Billen. A.. Evers-Kiebooms, G.. & dYdewalle. G. (1987).Risicoperceptie en erfeliikheid: Een coenitieve benadering. (Risk perception and genetics: A cognitive approach). Lisse: Swets & Zeitlinger. Human cognitive capabilities, representativeness, and ground Edwards, W. (1983). rules for research. In P. Humphreys, 0. Svenson, & A. Vari (Eds.). malvsing and aiding decision Drocesses (pp. 507-513).Amsterdam: North-Holland. Emery, A. E.. Watt, M. S . . & Clack, E. R. (1972). The effects of genetic counseling in Duchenne Muscular Dystrophy. Clinical Genetics. 3. 147-150. Evers-Kiebooms. G. (1987).Decision making in Huntington's disease and cystic fibrosis. In G. Evers-Kiebooms, J. Cassiman. H. Van den Berghe, & G . d'Ydewalle (Eds.), Genetic risk. risk DerceDtion and decision making (pp. 115149).New York: Alan R Liss. Evers-Kiebooms. G.. Cassiman, J.,Van den Berghe. H., & dydewalle. G. (Eds.) (1987). Genetic risk. risk DerceDtion and decision making. New York Alan R. Liss. Evers-Kiebooms. G.. & Van den Berghe. H. (1979). Impact of genetic counseling: A review of published follow-up studies. Clinical Genetics, 465-474. Huys, J., Evers-Kiebooms. G.. & d'Ydewalle, G. (1989). Een theoretisch ovenicht van beslissingsmodellen. (A theoretical review of models of decision making). Psvcholoeica Bekica. 29.25-50. Kahneman, D., & Tversky. A. (1979). Prospect theow: An analvsis of decision under risk Econometric& 47. 263-291. Kahneman. D.. & Tversky. A. (1986). Choices, values, and frames. In H. Arkes. & K. Hammond (Eds.). Judgment and decision making (pp. 194-210).Cambridge, MA: Cambridge University Press.

a,

68

J. Huys, G. Evers-Kiebooms,

&

G. d'Ydewalle

Kessler. S.. & Levine. E. K. (1987). Psychological aspects of genetic counseling: The subjective assessment of probability. American Journal of Medical Genetics. 28. 13-22. Lippman-Hand. A.. & Frazer, F. C. (1979). Genetic counseling: Parents' responses to uncertainty. In C. Epstein. C. Curry. S. P a c h a n , S. Sherman, & B. Hall (Eds.1. Risk. communication, and decision making in genetic counseling (pp. 289324).New York Alan R Liss. McNeil. B. J.. Pauker. S. G., Sox. H. C.. & Tversky, A. (1982).On the elicitation of preferences for alternative therapies. The New Eneland Journal of Medicine, 1259-1262. Pearn. J. (1979). Decision making and reproductive choice. In Y. Hsia, K. Hirschhom. R. Silverberg, & L. Godmilow (Eds.),Counseling in genetics (pp. 223-238).New York Alan R Liss. Slovic, P., Fischhoff. B., & Lichtenstein, S. (1982).Response mode, framing, and information-processing effects in risk assessment. In R Hogarth (Ed.). Directions for Methodolorn of Social and Behavioral Science, U. (pp. 21-36). San Francisco: Jossey- Bass. Swerts, A. (1987).Impact of genetic counseling and prenatal diagnosis for Down Syndrome and neural tube defects. In G . Evers-Kiebooms,J. Cassiman, H. Van den Berghe. & G . d'Ydewalle (Eds.), Genetic risk. risk DerceDtion. and decision paklng (pp. 61-83). New York Alan R. Liss. Tversky. A.. & Kahneman. D. (1981). The framing of decisions and the psychology of choice. Science, 211.1453-1458.

m,

Acknowledgements. We greatly acknowledge the contribution of N. Michiels in carrying out the experiment in partial fulfillment of her licence degree at the University of Leuven.

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e and M. G o n z a l e z ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B . V. ( N o r t h - H o l l a n d ) ,

1990

STUDENTS' CONCEPTIONS IN PHYSICS AND MATHEMATICS: BIASES AND HELPS Annick Weil-Barais*and Gkrard Vergnaud** * **

Inter-university Laboratory for Research on Scientific and Technological Education, University of Paris I , France

Laboratory of the Psychology of Child Development and Education University of Paris 5, France

Abstract. The thought modes that children and adolescents develop to solve problems encountered in everyday living are analyzed from two points of view: (1) sources of systematic error in solving mathematics and physics problems, and (2) the possible factors of knowledge development in these fields. Taking into account the gaps in thought modes necessarily involved in gaining access to scientific knowledge. a n d considering the possible links between the initial conceptions of individuals and the conceptions of experts, the authors discuss some potential means for providing cognitive guidance to foster the development of knowledge.

Since the early seventies, there has been a surge of interest in research on learning in physics and mathematics. The increased need for education in scientific fields occurring a t a time when our industrialized societies are undergoing a period of crisis and change h a s led to the development of new research programs in these areas, for the most part interdisciplinary. This trend has provided psychologists with privileged grounds for studying problem solving and the development of rational thought. Among the many studies, those dealing with the "conceptions" of students have been s o widely developed t h a t a new research paradigm h a s emerged (Gilbert & Swift, 1985). Research on conceptions has generally been conducted in conjunction with the study of the transfer of knowledge. Such studies have very often led researchers to stress the differences between the thinking of novices (students) from those of experts in the field (teachers, mathematicians, or physicists, conceived of as the actors of mathematical or physical thought). In comparison to the forms of thought developed in these disciplines, the thought processes of subjects (children or adults) often appear to be deficient. Accordingly, they are characterized by their lacks and insufficiencies. The conceptions of subjects are viewed as catalogs of errors, labeled by certain authors as "misconceptions" or "naive conceptions". This view of conceptions amounts to conceiving of

70

A.

Weil-Barais

d G.

Vergnaud

the child, student, or learning adult as unfinished, imperfect, or deficient in comparison to expert adults. In our minds, this framework is inadequate for approaching the issues in cognitive development and learning which are of interest to us here. It would appear more fruitful to consider the subject, as Piaget did, to be a dynamic system endowed with regulatory mechanisms capable of ensuring his or her progress. In this framework, subjects' conceptions may be viewed as assimilating structures with which other conceptions (such as those taught in school) may conflict. The latter are liable to create imbalance in cognitive structures, thus triggering accomodation processes. This is the frame of mind used here as we study student conceptions in a n attempt (a) to point out the difficulties and discontinuities in thought with which children are confronted as they approach the conceptions developed in fields reputed t o be difficult such a s mathematics and physics, and (b) to determine to what extent students' prior conceptions can be used to generate new conceptions. I. STUDENTS' CONCEPTIONS VIEWED AS COGNITIVE BIASES

Let u s recall that the term "conception" was coined in the research trend aimed a t describing the "signified concepts" (properties, relational invariants, operative invariants) and the "signifiers" students are apt to activate (or evoke) when they approach a knowledge domain. For example: How do children go about enumerating, or estimating a n area? What representation tools do they use? How do they understand the Cartesian product? What ideas do they have about matter, phenomena (changes in state, chemical reactions, light, magnetic or electrical phenomena), physical systems and their interactions, physical entities such as heat, force, work, energy? Among the methodologies developed to study conceptions (questionnaires, verbal associations, conceptual networks, ...), those dealing with the way in which individuals solve problems seem to be the most prevalent. This line of research enables u s to define cognitive invariants, of which subjects may or may not be aware. A subject's statements, i.e. what he or she is capable of explaining, are indeed insufficient for understanding their knowledge. Student conceptions may be considered to constitute a bias if they lead the students to give systematic responses that differ from the expected ones (the ones generally required in the subject

S t u d e n t s ’ c o n c e p t i o n s i n physics and mathematics

71

matter being taught). Both in mathematics and physics, such biases are indeed observed. They are manifested by certain regularities in the types of responses given for certain classes of problems. Let u s illustrate with a few examples. In mathematics, the idea that addition is accompanied by an increase, and subtraction by a decrease, is often held by young children in the first few years of elementary school (Gelman & Gallistel, 1978). Thus, problems such as those involving searching for the initial state when the transformation and the final state are known, in which case the pupil must subtract the increase or add the decrease (see 2 below) are solved much later than those in which the initial state and the transformation are known (1 below). (1)Peter has 5 marbles. He plays a game of marbles with John and wins 7 marbles. How many marbles does he have now‘? +7

Symbolic representation:

+

0

(2) Robert has j u s t played marbles with Celia. He lost 7 marbles in the game. He now has 5 marbles. How many marbles did he have before playing’? -7 + Symbolic representation:

0

151

Problem 2 is solved one or two years after problem 1 . Similarly, the primitive conception children have of multiplication, thought of as the iterated addition of the same term, causes them to believe that the product of a multiplication is always greater than the multiplicand. Division is associated with a smaller result, since it is viewed as the iterated subtraction of the same term. Moreover, seeing multiplication as iteration means that the multiplier is a whole number: likewise for the divisor. When children begin studying algebra, their prior conception of numbers is another obstacle. They consider numbers as magnitudes (cardinal numbers, lengths, areas, money, physical quantities) and not as relationships between magnitudes. Consequently, a result cannot be negative. Moreover, as they read expressions from left to right, they often consider that algebraic expressions model situations which progress in time from the initial state on the left to the final state on the right. With such a conception, the equal sign does not mean the symmetric and transitive equality relation that is required to understand equations,

12

A.

Weil-Barais

& G.

Verqnaud

but rather a relation linking a production process to a n outcome (Vergnaud, 1988). In addition to the errors t h a t stem from conceptual difficulties, other errors can be attributed to students' ideas about the problems and about their teacher's expectations. In traditional schooling, the required answer can generally be found by performing operations on the numbers given in the problem statement; on the basis of this fact, students tend to proceed in that manner regardless of what numbers are stated in the problem. For example, in the following problem: In a class, there are 12 girls and 13 boys. How old is the teacher? Answer: 12 + 13 = 25 years old. The effect of breaking the "didactic contract" by asking a question that cannot be answered (Brousseau, 1986), although strong, seems to disappear relatively easily when children are taught to solve problems of diverse natures (Brissiaud. 19881. In the field of phvsics, other sources of difficulty may exist. We will essentially discuss difficulties related to the conceptions students have of physical entities and difficulties stemming from experiment comprehension mode.

Conceptions pertaining to physical entities Most terms used in physics to refer to physical entities already belong to the everyday vocabulary of students. In physics, however, these terms have a very different meaning than they do in everyday life. For example, it is common in everyday French to say that an individual "est fort" (is forceful) or "a de la force" (has force), or that an object "est chaud" (is hot) or "a de la chaleur" (has heat). Thus "force" and "heat" refer to characteristics of objects, whereas in physics, they are entities t h a t describe interactions between systems (A and B. for instance). The physicist will thus speak of the force exerted by A on B, and will talk about heat as the transfer of energy from system A to system B. It is also a common belief that energy is used up, is lost, which is contradictory to one of the fundamental properties of energy, namely, its conservation. Many frequently observed "errors" thus stem from the fact that students attribute the same meaning to the words used in physics as they do in daily life. If cognitive bias exists in this case, it is caused by an extremely general thought process consisting of conserving the associative links that pertain to a given element, perceived a s a n invariant.

Students' conceptions in physics a n d mathematics

73

Conceptions pertaining to experiment interpretation The way students interpret experiments is linked to their causal conceptions. Such conceptions may develop when predictions are to be made concerning the evolution of a situation (in order to answer questions like "What will happen if ...3") or when explanations are to be given concerning a given phenomenology (in order to respond to requests such as "explain", "interpret", "justify"). Although prior research (cf. reviews and studies by Driver & Erikson, 1983; Gilbert & Swift, 1985: West & Pines, 1985; Tiberghien & Weil-Barais, 1987) has always dealt with branches of physics (mechanics, optics, electrokinetics, ...) and the explanations given by students are specific to the area under consideration, all such explanations have certain points in common: - They focus on the perceptual characteristics (often the most salient ones) and event-related characteristics of the situation: actions, movements, changes in aspect (deformations, modifications in color, appearance of bubbles, steam, smoke, ...1. For instance, during a heating operation, students (ages 12 to 14 in the study mentioned above) focus on the presence of flames, the appearance of bubbles in the liquid, the vapor released, or a black deposit on the tube, rather than taking into account the variations in temperature visible on a thermometer (Tiberghien, 1985). - The various elements of the situation are essentially described in terms of their properties and functions. In students' causal explanations, such descriptions prevail over those accounting for the interactions between the elements. For example, students (age 12 to 14 in the study mentioned) think that a balloon rises up because it is light (Sere, 1985). - When students conceive of the interactions between the elements of a situation, they generally think of these interactions as dissymmetrical. For example, students (age 13 and 14 in the study mentioned) believe that a book placed on a table presses on the table, but that the table supports it without effort; that a ball hanging by a string and dipped into a liquid is pushed by the liquid, but that the ball does not do anything to the liquid since the string holds the ball back (Kraibani, 1984). - The event history, i.e. what happens sequentially in time and space, is also a preferred means of comprehending experiments. This conception leads students to relate the actions taken upon objects to the observed "effects", or to adopt a sequential processing

I4

A . Weil-Barais & G. V e r g n a u d

method in which the elements in the situation are taken into account one after another. Thus, in regards to the processing of electric circuits (when the various elements of the circuit are represented graphically), Closset states "it all looks as though the battery was a reservoir from which current escapes at a constant rate and adventures out into the circuit, only changing when it encounters obstacles, without the downstream affecting the upstream" (Closset, 1988; our translation). Such comprehension modes, which are essentially of the "event-related" type, unlike the "conceptual" type comprehension done in the field of physics Neil-Barais, Lemeignan & Sere, 1990) are inadequate in that they often lead to erroneous predictions. For example, when considering the modifications in an air-filled container subject to various actions (exposure to the sun, contact with fire, ice, application of pressure ...), students who associate the modifications with variations in the quantity of matter (Sere & WeilBarais, 1988) are "wrong" vis-a-vis the conceptions developed by physicists, who postulate on the basis of the study of interactions and exchanges between systems, that in such situations, the quantity of matter is invariable. In the various branches of physics, and in mechanics in particular (studied by Champagne, Klopfer & Anderson, 1980; Clement, 1982; Di Sessa, 1981; Lemeignan & Weil-Barais, 1987; McKloskey, 1983; Ogborn, 1985; Trowbridge & McDermott. 1981; Viennot, 1979), intuitive conceptions have been shown to persist, even in more advanced university students taking physics courses. Different explanations have been proposed to account for the persistence of such intuitive conceptions relative to the physics models being taught. According to McCloskey and Kaiser (1984), the problem originates in the encoding capacities of individuals. The fact that individuals preferably take velocity into account, at the expense of acceleration, is thought to be due to the fact that the human visual system may be less effective in discriminating differences in acceleration than differences in velocity. In a very Darwinian perspective, the authors point out the fact that in phylogeny, these two dimensions do not have the same impact on the survival of the species. Attention-related factors have also been considered to play a part, in particular by Siegler (1983) in his studies on the principle of a scale. In all of these interpretations, the visual perception system plays a major role in the construction of knowledge in subjects. In our mind, not enough attention has

Students' conceptions in physics and mathematics

I5

been paid to the functional role of these conceptions, and not enough investigation h a s been done on the problems subjects encounter when constructing them. Indeed, subjects may be assumed to first develop conceptions during interaction with their environment. For anticipating and controlling actions in the environment, the observed conceptions appear to be satisfactory. Their range of validity covers practical living. The "errors" reported in some studies often result from the fact that subjects are faced with questions they have never before had to ask themselves, or questions that involve unusual values for the variables in that particular situation. Accordingly, in our study on the conservation of gaseous matter (Weil-Barais et al., 1987; Sere et al., 1988),we asked children to compare quantities of air before and after subjecting the containers filled with that air to certain actions. To answer this uncommon question, the children often mentioned everyday situations in which the quantity of air in fact decreases (for example, their deflated ball that had been in the sun). By analogy, and without taking into account the fact that the time interval between the tobe-compared states in the test situation was not the same as in the situation evoked, children incorrectly concluded that the amount of air had changed. It should be noted that while subjects can control the functional value of their intuitive conceptions of the world a t will, they cannot do so with physics models. McKloskey and Kaiser (1984) justly state that the physics of Galileo and Newton describe an idealized world (experiments without friction, isolated systems, uniformly accelerating movement, etc.) over which individuals only have very indirect means of control. Most often, Galileo and Newton themselves only imagined their experiments through reasoning of the going-beyond-the-limits type, by varying the assumptions accepted at that time. The models used in physics call for entities that generally cannot be accessed by sensory receptors. Remember indeed that there are a certain number of physical entities which can be calculated but not measured (for example, heat can only be calculated by hypothesizing the conservation of energy). Physical entities thus appear to be "hidden" entities (Martinand et al., 1987). Furthermore, the controls performed by thought processes operate on formal expressions, which are already the result of the conceptualization of situations. In physics, students thus come across a difficulty that also exists in algebra: verification of the meaning of symbolic representations does not depend upon one's

16

A.

Weil-Barais

& G.

Vergnaud

potential ability to concretely represent the entities and relationships expressed, but upon the conceptual elements taken into account (concepts like system, state, interaction, transfer, conservation, only to mention some of the important concepts in mechanics). In algebra, the meaning of computations is verified by taking into account the conservation of equalities following certain transformations, and by carefully sequencing the steps so as to progress towards the goal: isolating the unknown on one side of the equal sign, eliminating one or more unknowns, ... (Vergnaud, 1988). We are dealing here with detour behaviors, which in the light of their complexity, take time to become efficient and automatic enough for subjects to no longer need to rely upon their habitual methods of controlling reality. This kind of analysis enables u s to understand why students in the face of difficulty voluntarily resort to their habitual methods even when they have acquired a certain degree of expertise on the subject. Thus, the implementation of inappropriate means for controlling one's cognitive activity may be a source of error, even when subjects have begun to acquire more suitable conceptualization tools. To illustrate this fact, we shall discuss a study conducted with tenth graders (first year of high school in France) on momentum (p) (Lemeignan & Weil-Barais, 1987). The following situation was used and was aimed at assessing the conceptions the students had acquired after a teaching series lasting approximately 12 hours. Two small train cars, each equipped with an identical magnet on one end, were placed with their magnet ends facing each other on rectilinear, horizontal tracks. They were held in place at the onset of the experiment. Once released, they moved towards each other by the simple attraction of the magnets until they stuck together.

(vl

= 0)

(v2 = 0)

After

Figure 1. Symbolic representation of the situation.

Students' conceptions in physics and mathematics

I1

In this situation, the students' intuitive conceptions of the world led them to predict that the joined cars would move in the direction of the heaviest of the two. The students were then asked to analyze this situation in terms of the conservation of the momentum (p). On the basis of the observed events, the correct prediction is t h a t whatever the respective m a s s ( m ) of the two cars (whether the same or different), their velocity (v)when joined together will always be null. Indeed, since the momentum of the two-car system is null before they come together, it is also null afterwards (application of the principle of the conservation of the quantity of motion). The students distorted the analysis in two ways: (1) By concentrating on the event which for them was "the movement of the cars once released", they took the "during" into account (the time interval during which the cars moved) to the detriment of the "before" event (when the trains were not moving), even though they did have information about this event (the velocity of the cars was equal to zero). This led them either to ignore the principle of conservation for lack of knowledge of the velocities of the cars, which varied in the course of their displacement, or to make false inferences in order to apply the conservation principle. (2) By making mistakes (usually sign errors) in transforming the algebraic expressions in order to make the result of their analysis fit their intuition of the real world. I t appears here that the predictions subjects are capable of making on the basis of their personal conceptions act as a means of controlling a formal system (algebra) which, although available, is not yet sufficiently autonomous for them to be able to apply the control means internal to that formalized system. Although the examples considered in this chapter are necessarily limited, we can see that the sources of error are varied. We have pointed out some of the kinds of errors that stem from the use of signifiers that have acquired a given meaning in one setting (daily living, or sometimes learned at a lower school level, or in courses other than mathematics or physics) and a different meaning in another setting. Many researchers dealing with this issue have often treated it as a vocabulary-related or definitional problem, or as resulting from difficulty in acquiring formalized graphic or mathematical systems. This amounts to reducing the formation of

ia

A.

Weil-Barais

& G.

Vergnaud

concepts to a "game" of symbols. The conceptual framework we developed to deal with the problem of knowledge acquisition (Vergnaud, 1987) leads u s to consider the formation of concepts a t three levels: - The set of situations that make the concept meaningful in a variety of ways. - The set of operational invariants (properties, relationships, objects, theorems-in-action, etc.) that are progressively grasped by students, in a hierarchical fashion. - The set of linguistic and non-linguistic symbols that represent those invariants and are used to indicate them, to communicate about them, and to discuss them, and therefore to represent situations and procedures. In this framework, it is understandable why the situations chosen to introduce a concept (at the "signified" level) play a major role, since they are responsible for the meaning granted to that concept. The question of the inappropriateness of some of the situations used should therefore be raised. For example, if used exclusively, the "tiling" situations used to teach students how to calculate area may indeed contribute to reinforcing the primitive and highly persistent conception that multiplication is the iteration of addition, and in doing so, may prolong the one-dimensional view of area. The second source of error we observed concerns the problem comprehension modes preferred by subjects. On an overall basis, whether in mathematics or physics, the perspective generally used by students for comprehending problems is the action and event domain. Although one of the goals of studying mathematics and physics is to acquire conceptual tools for thinking about and understanding the real world, this goal is only achieved by means of detours which are far-removed from the kind of processing performed on the information subjects extract from their environment, detours which involve highly specific concepts and thought control modes (Rostand, 1985). There is no reason to believe that students can access the thought processes which have gradually developed in the course of human history, unless they are specifically taught how to do so. Our task as psychologists thus consists of pointing out where the thought processes of students differ from those developed in the disciplines being taught. Such analyses will make it possible to

Students

'

conceptions in physics and mathematics

79

determine the types of changes in thought processes students need to undergo, and to study the various possible ways in which transition from one thought mode to the next might be achieved. 11.

STUDENTS' CONCEPTIONS VIEWED AS COGNITIVE STEPS AND HELPS

If we consider the two processes which appear to be fundamental to learning: the analogy process (That's the same as ...) and the discrimination process (That's different from ...). we might ask ourselves to what extent can the initial conceptions of students be used to foster their conceptual progress. Two cases should be distinguished here: one in which the students' conceptions are compatible with the concepts introduced by the teacher, and one in which they are not. The most favorable case is obviously student-teacher conception compatibility. Here, the students' conceptions can be considered as the "precursors" of the to-be-acquired concepts. For example, the fact that students have already discovered that adding 6 to 3 is the same thing as adding 3 to 6 (which amounts to applying the principle of the commutativity of addition) can be considered as a useful precursor of the presentation of addition as a commutative, internal law of composition. More such precursors can be found than the error inventories reported so far would lead u s to believe, provided our goal is to search for them. By concentrating our efforts on looking for precursors of this type in the momentum study mentioned above, we were able to determine what experimental conditions and what questions may cause students to activate conceptions considered to be precursors of a concept (Lemeignan & Weil-Barais. 1988). A similar project is now under way for teaching algebra to poor students (Vergnaud et al., 1988). The activation of "precursors" is a preliminary step towards a generalization process, which of course must still be guided. Indeed, when students are told that a n invariant they employed to deal with a given situation (which thus may appear specific to them) is in fact general, the problem of the class of situations and events to which it can legitimately be applied is still not solved. All efforts to establish relationships between the conceptions of students and the conceptions developed in mathematics and physics run into the obstacle of existing gaps between the thinking of the psychological subject and the thought processes applied in mathematics and physics. Such gaps cannot be bypassed. And in

80

A.

Weil-Barais

& G.

Vergnaud

our mind, they cannot be overcome unless subjects make the effort to become aware of such gaps, and to move away from their own ways of thinking towards those developed in the discipline. These phenomena are worth further investigation in empirical studies, which have only just begun to take interest in this issue. Such studies are still too limited in number to enable u s to specify the conditions required for conceptual change in students. There is no a priori reason to believe that the conditions are the same in each of the disciplines under consideration. We are t h u s currently conducting field studies in an attempt to determine the factors necessary to such awareness and conceptual progress in each individual discipline or branch. For example, one of most serious difficulties in teaching algebra in junior high school is that negative numbers and in particular, a negative solution to an equation, have virtually no meaning to students. And since the notion of number is associated with the notion of measure (quantity, magnitude), a solution can only be positive. In an experiment on teaching eighth graders in a vocational high school, we introduced the possibility of finding a negative solution to a problem in which the unknown was a transformation. The amount of numerical data was such that the students could nok really anticipate the sign of the transformation. They thus put the problem into an equation by symbolizing the action and the transformation with a "+x". As they solved the problem, they realized they were dealing with a loss, s o they attempted to substitute "-x" for "+x". The teacher insisted they complete the calculation with "+x", and was thus able to make them aware of the fact that a negative solution to an equation can be interpreted as a decrease. Intervention by the teacher is often critical in allowing "prise de conscience" or awareness phenomena (of the type mentioned above) to take place. Indeed, when the students' activity is not sufficiently guided, they manage to make their problem solution compatible with their initial representation: this is their means of avoiding contradictory situations. In the preceding example, the teacher's guidance dealt with the application of the rules for writing algebraic expressions. In the momentum study, the guidance dealt with the various possible ways of interpreting the experiments. The choice made (Lemeignan & Weil-Barais. 1987) consisted of triggering the progression of experiment interpretation modes by asking a series of questions

Students' conceptions in p h y s i c s a n d mathematics

81

about a s e t of experimental situations. Starting from the interpretation mode available to students when responding to requests for predictions about events and experimental verification of those events (event-related interpretation), we then asked them to do causal interpretation based on known physical entities (the study of covariations and counter-variations in the mass and velocity of mobiles). These two interpretation modes were associated with graphic representations illustrating the elements of the situation (small trains represented in their spatial extension). The third comprehension mode, which involved determining the quantity of motion before and after the hitting event, was achieved by asking for predictions about the state of the system. The change in representation occurring at t h a t point served the purpose of translating the properties of this interpretation mode: division in time, consideration of the two states of the system (the "before" and the "after", to the detriment of the "during"), and establishment of relationships between the states.

Before

ml

After

+

m2

Figure 2. The symbolism used.

ml

+

m2

82

A . W e i l - B a r a i s & G . Vergnaud

The effects of such practices -- which consist of clearly distinguishing interpretation modes through the use of different types of meaningful questioning for each mode, and through the use of symbolic representations specific to each -- can only be assessed in the long run. Other than the scattered experiments like the ones we have conducted, such practices are not currently being used to teach physics. Quite the contrary, as can be seen by examining school text books, various different interpretation modes are constantly being intermixed. This may be a non-negligible source of bias, and should be a legitimate concern of psychologists. CONCLUSION

Thus, considering the conceptions of individuals both in regards t o the bias they may introduce into the thought processes of mathematics and physics, and in regards to the help they may provide in accessing those thought processes, is a complex task for the psychologist attempting to investigate this issue. Indeed, to situate the conceptions of subjects relative to these two dimensions (biases or helps), a solely psychological approach does not suffice. It is indeed necessary to refer to the concepts, models, types of reasoning, and symbolic systems developed in the disciplines taught in the schools. This implies taking an epistemological approach to the knowledge domains under consideration here, in order t o understand the difficulties encountered by students. By comparing s t u d e n t conceptions t o those developed in the concerned disciplines, the functional value of students’ thought processes can be diagnosed. In the light of the considerable gap that exists between the invariants subjects are likely to construct during their interaction with the environment, and the invariants that make up mathematics and physics knowledge, the existence of cognitive biases due to the implementation of previously defined invariants is not surprising. It is of particular interest to u s to find out more about such invariants, especially if they resist instructional intervention, since such knowledge will enable u s to more accurately determine the cognitive obstacles subjects must overcome. In order to better understand the processes of intellectual development, the knowledge of biases is not sufficient. It is necessary to study the processes by means of which subjects manage to control and overcome these biases. In the disciplines of interest

Students

' conceptions in physics and mathematics

83

to us here, overcoming the biases evoked above is very highly dependent upon the construction of concepts. This has led u s to consider these conceptions as "steps and helps" in the construction of the necessary concepts. It appears that in certain cases, students' initial conceptions may be the precursors of the to-beconstructed concepts. In other cases, the needed concepts can only be constructed if the personal conceptions of the subject are set aside. I t is no doubt in the latter case t h a t a teacher's intervention is necessary, particularly because it helps students point out their insufficiencies and develop methods for controlling their thought processes. The preliminary observations made in our study of the effects of such intervention (within the teaching series we are conducting on a n experimental basis) upon the evolution of the problem-solving modes of students support our analysis and provide evidence of the fact t h a t in the disciplines studied, conceptual change takes time. REFERENCES Brissiaud. R. (1988). De 1'8ge du capitaine. a l'age du berger. Quel contrdle de la validitt d'un enonce de probleme au CE27 pevue Francaise de PedagQgk. 82. 23-31. Brousseau. G. (1986).Fondements et mtthodes de la Didactique des Mathematiques, Fecherches en Didactique des Mathtma-tiaues. Z(2).33-177. Champagne. A.B., Klopfer, L.E.. & Anderson, J . H . (1980). Factors influencing the learning of classical mechanics. American Journal of Phvsics. 48,10741079. Clement, J. (1982). Students' preconceptions in introductory mechanics. American Journal of Phvsics. 66-7 1. Closset, J.L. (1988). Une possible methodologie pour la recherche s u r le raisonnement nature1 en physique; probltmatique et resultats dans le cas de l'tlectrocinetique. Technologies. Ideologies. Pratiaues. 2. 43-58. Di Sessa. A.A. (1983). "Phenomenology"and the evolution of intuition. In D. Gentner & AL. Stevens (Eds.).Mental models. Hillsdale. NJ: Lawrence Erlbaum. Driver, R.. & Eriksen, G. (1983). Theories-in-action: some theoretical and empirical issues in the study of students' conceptual frameworks in science. Studies in Science Education, 37-60. Gelman, R., & Gallistel. R. (1978). The child's understanding of number. Cambridge: Harvard University Press. Gilbert, J.K., & Watts (1983). Concepts, misconceptions and alternative conceptions: changing perspectives in Science Education. Studies in Science Education, 22, 61-98. Gilbert, J. K.. & Swift, D.J. (1985). Towards a Lakatosian analysis of the Piagetian and alternative conceptions research programs. Science Education, 69(5), 681-696. Kralbani. S.(1984). Registres d'intemretation des d h e s et des D rofesseurs de coll&e b n s le domaine de la mecaniaut. Doctoral dissertation, University of Paris 7. Lemeignan, G., & Wefl-Barais. A. (1987). Amrentissage de la modtlisation a DroDoS de l'enseimement de la mtcaniaue au lvcte. End of contract report, "Action

a,

a,

84

A . Weil-Barais

& G.

Vergnaud

Concertee: Recherche en education el en formation". Xeroxed LIRESPT document. Lemeignan. G.. & Well-Barais, A. (1988).Gestion d'activitts de modelisation en classe. ASTER, Special issue: ModCles et modtlisation. 2, 121-141. Martinand, J. L. (1987). Enseignement et amrentissage de la modelisation. End of contract report, "ActionConcertee: Recherche en tducation et en formation". Xeroxed LIRESPT document. Intuitive physics. Scientific American 122-130. McKloskey, M. (1983). McKloskey, M., & Kaiser, M.K. (1984). Children's intuitive physics. The Scienceg. 24, 40-25. Ogburn, J. (1985).Understanding students' understandings: a n example from dynamics. EuroDean Journal of Science Education, 2(2),141-150. Rostand. F. (1985).Proctdts de la penste mathtmatique. d 'Histolre et & PhilosoDhie des Sciences. II. Socitte Francaise d'HistoLre des Sciences et des Techniques. The gaseous state. In R Driver, E. Guesne, &A. Tiberghien (Eds.). Sere, M.G. (1985). Children's ideas in science (pp 105-123).Philadelphia: Open University. Sere, M.G.. & Well-Barais. A. (1988).Nouvelle approche de la conservation des grandeurs physiques: etude transversale de la conservation de la quantitk de gaz a u p e s dtlives de 11 a 15 ans. Enfance, 21-37. Siegler, R.S.(1983).How knowledge influences learning. American Scientist, 631-638. Tiberghien. A., Erickson. G. (1985). Heat and temperature. In R Driver, E. Guesne, & A. Tiberghien (Eds). Children's ideas in science. Philadelphia: Open University. Tiberghien. A.. & Weil- Barais, A. (1987).Les modiles spontants dans la penste commune et enfantine. In A. Giordan & J.L. Martinand (Eds.). Mod tles et simulation (pp 47-55). Conference "Neuviemes Journtes Internationales sur 1'Education Scientifique. U.E.R. de Didactique des Disciplines". University of Paris 7. Trowbridge, D.E.. & McDermott. L.C. (1981). Investigation of student understanding of the concept of acceleration in one dimension. American Journal of phvsics, 39(3).242-253. Verngaud, G. (1987).Les fonctions de l'action et de la symbolisation dans la formation des connaissances chez l'enfant. In J. Piaget. P. Mounoud. J.P. Bronckard (Eds.).EncvcloDtdie de la Pleiade: Psvcholocie. Paris: Gallimard. Vergnaud. G. (1988).Theoretical frameworks and mathematical facts in t he psvcholom of mathematics educatioq, I C M E VI: Budapest. Vergnaud. G., Cortes, A., Favre-Artlgue. P. (1988).Introduction de I'algCbre aupris des debutants faibles: problCmes tpisttmologiques et dldactiques. In G. Vergnaud. G . Broussaud, & M. Hulin (Eds.), Didactiaue et acauisition des connaissances scientifiaues, Proceedings of the Sivres Colloquium, May 1987,Grenoble: La Penste Sauvage. Viennot, L. (1979).Spontaneous reasoning in elementary dynamics. EuroDean Journal of Science Education, 1(2),205-222. West, L.H.T., & Pines, A.L. (Eds). (1985). Coanitive structure and conceDtua1chance. New York Academic Press. Well-Barais. A.. Lemeignan, G.. & Sere, M. G. (1990). Acquisition de connaissances scientifiques et developpement. In G. Nerchine (Ed.). Modeles aCntraux et modiles locaux du dheloDDernent cormitif, Paris: P.U.F.

,a.

a(1).

u,

SECTION 2 Biases in reasoning pragmatics

This Page Intentionally Left Blank

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez (Eds.) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland), 1990

CONVERSATIONALAND WORLD KNOWLEDGE CONSTRAINTS ON DEDUCTIVE REASONING Vittorio Girotto* and Guy Politzef * * * * CNRS

I n s t i t u t e of Psychology, CNR, Rome, and U n i v e r s i t y of P a r i s V I I I ,

Italy

Saint-Denis,

France

Abstract. The ef€ects of pragmatic factors on deductive reasoning are reviewed. Firstly. linguistic factors are considered. The various interpretational effects of the laws of language use on linguistic units (such a s quantifiers and connectives) and their consequences on immediate inferences, conditional reasoning, and propositional reasoning are described. Secondly, the influence of world knowledge is considered. The literature reviewed has shown that when an individual reasons in order to act in the real world, the premises actually given are constrained by attitudes and beliefs. implicit assumptions dictated by previous knowledge, and the interpretation of the context. The explanation of the latter process suggested in pragmatic reasoning schemas theory and social exchange theory are discussed.

The psychological literature on reasoning in the last two decades has been concerned with two main problems that regard both inductive and deductive reasoning: (i) biases and errors and (ii) the influence of content and context on performance (for reviews, see Evans, 1982; Nisbett & Ross, 1980). Different explanations of these phenomena have been proposed. According to Evans (1987, 1989). the theories of reasoning can be classified into four distinct approaches: (a) inference rule theories, which assume that subjects reason according to general purpose rules applied to problems across all domains (Braine, 1978; Braine, Reiser, & Rumain, 1984; Rips, 1983); (b) schema theories, which propose that people reason according to domain dependent schemas embodying procedural knowledge in the form of rules which can be applied to problems of a general type previously experienced by the subjects (Cheng & Holyoak, 1985); (c) mental models theories which assume that people envisage different states of the world described by the premises, that can accommodate a conclusion coherently (JohnsonLaird, 1983); (d) heuristic and biases theories which assume that reasoning depends on short-cut strategies which lead to correct solution in some situations and to systematic biases in others (e.g. Kahneman, Slovic, & Tversky. 1982).

88

V. Girotto

&

G. Politzer

The relative merits of these four general views on reasoning will not be discussed here. Rather, our aim is to focus on one class of influences on deductive reasoning which all theories acknowledge, although to different degrees, namely pragmatic influences. What various theorists call pragmatic is by no means clearly defined. However, a first broad distinction can be made following Braine (1988) who distinguishes between two kinds of pragmatic factors: (i) the social frame of discourse that affects comprehension through Grice's (1967, 1975) cooperative principle and (ii) knowledge of subject matter that affects comprehension and reasoning through rules that enrich the data base. Braine's viewpoint is that both types of pragmatic factors affect what he calls the primary logical skills (that is, those skills responsible for the inferences universally carried out without error and automatically in comprehension of discourse and in practical reasoning) by providing additional inferences or by making inferences more accessible. The first part of this paper will be devoted to the first kind of pragmatic influences which are linguistic in nature: then, the second kind of pragmatic influences which are related to world knowledge will be considered. 1. CONVERSATIONAL CONSTRAINTS

1.1 Basic concepts

By the very fact that the input of reasoning is formulated in natural language (no matter what transformations may occur at a later stage), reasoning is a prisoner of the rules that govern language use. Not until such rules were formulated in the fundamental work of Grice (1967) and a few others who shared the same intuitions, like Ducrot (19721, could their influence be recognized by investigators of reasoning: this explains why their concern with this matter is relatively recent. Grice's (1967, 1975) maxims of conversation are subsumed under a general cooperatiue principle to the effect that each participant in a talk exchange recognizes a common purpose. Three maxims are particularly pertinent for our present concern, namely the two maxims of quantity: (1) make your contribution as informative as is required (to the current purpose of the exchange), and (2) do not make your contribution more informative than is

C o n s t r a i n t s on d e d u c t i v e reasoning

89

required, and the m a i m of relation: be relevant. According to Grice, under normal circumstances the speaker is cooperative and he obeys the maxims. However, a speaker may be uncooperative to various degrees: overtly (opting out), e.g. when saying "I have nothing to tell you"; or covertly, e.g. with a n intention to deceive: or, more importantly, apparently: this is the case where an implicature Q is conveyed. Grice says that a speaker exploits a maxim by saying P when (i) he apparently violates a maxim he was supposed to observe, (ii) he is aware that Q is necessary to reconcile his saying P with the observance of the maxim and (iii) he estimates the hearer capable of working out (ii). That the current trend in research is to emphasize the role of language pragmatics is exemplified by Sperber and Wilson's theory (1986). They propose t h a t discourse comprehension can be explained on the basis of the concept of relevance. Starting from the new information and the context, the hearer makes inferences by use of a set of deduction rules and retains those inferences that maximise the relevance of the information being processed. Although this theory addresses more specifically the problem of spontaneous (automatic and unconscious) inferences made in the process of language comprehension, it has obvious implications for deductive reasoning in general. 1.2 Horn scales

How Gricean implicatures affect the interpretation of linguistic units (numerals, quantifiers, modalities and connectives) and therefore how they affect reasoning has been explicated by Horn (1972). A set of terms that are equally lexicalized and ordered along a common dimension is called a Horn scale: This is the case for numerals [ ..., 9, 8. 7, ..., 2, 1, 0).quantifiers [all, some), modals [necessarily P, possibly P) and connectives [and, or]. When a speaker uses a term that is on such a scale, he is supposed to observe the first maxim of quantity to the effect that he should be as informative as he can. The implicature is calculated by a subtle counterfactual argument. Consider the scale [tn, ..., ti+l, ti, ti-1, ..., t 2 , t l ) . The speaker h a s used the term ti which is not the maximum of the scale to state sentence S(ti). If S(ti+l)were the case he should have used t i + l . Since he did not use ti+l. S(ti+l)is not the case. In brief, on a Horn scale S(ti) logically entails S(ti-1)

90

V. Girotto

&

G. Politzer

and pragmatically implicates not-S(ti+1). This is why confessing to three crimes when the murderer has actually committed five is deceptive: and this is also why some children have difficulty understanding the relationships between members of nested classes (a square is a rectangle but pragmatically if you talk of a rectangle it cannot be a square due to the existence of the scale (square, rectangle, parallelogram, ...}). Similarly, Horn has proposed a n explanation of the predominant interpretation of or as exclusive. Lexically, or is inclusive. It is positioned on the lower end of the scale (and, or}because it is less informative than and. Consequently, its use in A or B conveys the implicature not (A a n d BJ, which renders it exclusive. 1.3 Interpretation of quantifiers

Granted that {all, some} constitutes a Horn scale, it follows that if the speaker utters some, he implicates that not all is the case. This latter prediction has extremely important and testable consequences for the interpretation of, and reasoning with, quantified sentences. This phenomenon had long been assumed on intuitive bases but it was tested only recently using the immediate deduction paradigm. Immediate deductions are deductions from one of the four categorical sentences (all X are Y, some X are Y, no X are Y, some X are not Y) to another one: it is called "immediate" because, unlike syllogisms, it has only one premise (at least explicitly). The pragmatic theory predicts that in deductions where a particular sentence is involved (some X are Y, some X are not Y), the quantifier some will convey a not all implicature so that (i) deductions between all and some will implicate a contradiction (all- some but not all) and similarly between no and some ... are not, (no- some are not, but some are) so that the conclusion will be evaluated as false, and (ii) deductions between some and some ... are not will give rise to a response true (whereas the logically correct response is indeterminate) because the implicature conveyed by one sentence is equivalent to the other sentence. Newstead and Griggs (1983) using abstract sentences did find 60% of the subjects who declared true a particular conclusion drawn from the other particular premise, and 70% who declared false the universal conclusion all or no drawn from the particular premise some or some ... are not, respectively (but only 30% in the

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

91

opposite direction). Politzer (1990) used sentences that referred to marbles of different sizes and colors or to people's occupations and activities. He found similar results to Newstead and Griggs' except that for the deductions all-some and no-some ... are not the percentage of responses false was around 60%; the discrepancy between this percentage and Newstead and Griggs' (30%)may be due to differences in the response format: whereas the subjects received one deduction per page, Newstead and Griggs' subjects had on each page one premise with eight conclusions to evaluate. The results of Begg and Harris (1982), obtained with another paradigm, go in the same direction. These authors studied the association between the four categorical sentences and the four Euler diagrams that are candidates for the interpretation of the sentences. Among other results they found that (i) some was not associated with inclusion, i.e. it excludes all, and (ii) some and some ... are not were associated with the same diagrams and had therefore become synonymous. Newstead and Griggs' data as well as those of Begg and Harris were obtained with English speaking subjects. Politzer (1990) obtained his data from speakers of a non-Indo-European language (Malay), thus suggesting the universality of the Gricean approach. This conclusion is borne out by the results of another experiment using similar materials where the language of reasoning (English vs Malay) was manipulated and the subjects' native language (Chinese vs Tamil) controlled (Politzer, in press). Finally, it should be emphasized t h a t the implicature generated by the particular quantifiers is such a robust phenomenon that it even resists instructions that attempt to cancel it: in both the Newstead and Griggs and B e g and Harris studies, the subjects had received a warning regarding the logical meaning of some and yet the majority performed as predicted by the pragmatic theory. This observation was confirmed by another study (Newstead, 1989). 1.4 Interpretation of conditionals

The thorny problem of the interpretation of conditional sentences constitutes another area where the laws of conversation interact with reasoning. Geis and Zwicky (1971) introduced the principle of conditional perfection which says that in the case of promises and also for some predictions, threats, and commands, a sentence in i f X then Y form suggests. or invites the inferences of iJ

92

V. Girotto

&

G. Politzer

not X , then not Y. With their famous example, If you mow the lawn, I'll give youfrve dollars, they argue that people normally draw the invited inference, Zf you don't mow the lawn, Z will not give you five dollars. which technically is the obverse of the initial sentence. Horn (1972) argues that invited inferences are a special case of implicature: given a promise i f X , then Y the addressee is entitled to infer i j not X,then not Y because if the speaker intended to perform Y even in the absence of X, his restrictive conditional utterance would not respect the maxim of quantity. Fillenbaum (1975) showed the generality of invited inferences attached to conditional statements and their variability as a function of the conceptual properties of the constituent sentences. He presented his subjects with conditional sentences and asked them whether they accepted the obverse of each sentence as a conclusion. The percentages of accepted inferences were 85% for promises and threats, 81% and 73% for temporal-causal (according to whether the conclusion was factually true or possibly true) and 64% for contingent universal sentences. Braine's (1978, 1988: Braine, Reiser, & Rumain, 1984) theory of natural logic accommodates conversational constraints, because for this author inference schemas act on semantic representations and a t this stage the implicatures have already contributed to the interpretation of the sentences. In line with this conception, Rumain, Connell, and Braine (1983) presented children and adults with two kinds of conditional syllogisms: some in their usual form (i.e, with a conditional of the type ZfP then R),some others with a n additional conditional premise, Zf Q then R, aimed a t countermanding the initial inference by suggesting Q as a n alternative antecedent to P. They found that the subjects committed the classical fallacies in the first experimental condition b u t not in the second one. Similar results were obtained by Markovits (1984, 1985). 1.5 Selection of information: biases in formal tasks

Evans (1984: Evans, Ball, & Brooks, 1987) has acknowledged the role of natural language use in the selection of information, although he did not qualify it with the term "pragmatic". According to Evans (1984) there are two types of thought processes, heuristic and analytic. The heuristic processes select information: their outcome is a judgement of relevance. Relevant

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

93

information is then subjected to analytic processes which serve to generate inferences (but these processes do not normally do so by logical reasoning: they follow the normal linguistic usage of syntactic forms). How does the heuristic process operate? A typical illustration is provided by the phenomenon of matching bias. In general terms, it consists of the subject's tendency to select responses that match the features of the problem. Matching bias has been observed in a number of tasks such as truth table evaluation of conditionals (Evans, 1972). conditional syllogism solving (Evans, 1977), categorical syllogism solving (see the well-known atmosphere and figural effects), disjunctive reasoning (cf. the 'intuitive error' on the THOG problem, Wason. 1977; Griggs & Newstead, 1983), and Wason's selection task. Evans et al. (1987) regard the matching responses manifested in the latter task as a case where subjects are induced to find a solution by a heuristic judgement of relevance determined by a linguistic presupposition attached to the natural language use of negation. In the classical version of the selection task (Wason, 1966), subjects are given a rule 'If there is a vowel on one side ofa card, then there is an euen number on the other side'. and shown four cards displaying letters and numbers such as an A, a B. a 4. and a 7. The subjects have to select just those cards that are necessary to turn over in order to decide whether the rule is true or false. (In general terms, the rule presented to the subjects is in 'up then q' form and the four cards may be designated by p , not-p, q, not-q.) In the version of the task using abstract problem content, such as letters and numbers, most subjects fail to select all the cards that are potential counter-examples (that is p and not-q: in the present case, the vowel and the odd number). Subjects typically select either p alone or p and q. However, subjects seem more able to solve the problem when the rule has a negative consequent ( ' u p then not-q', e.g. 'Ifthere is an A on one side of the card then there is not a 7 on the other side of the card'). Studies by Evans and Lynch (19731, Manktelow and Evans (1979) indicate that this result is understandable in terms of matching bias: subjects tend to choose the cards that are mentioned in the rule, irrespective of the logically critical presence of a negative. In the case where there is a negative consequent, the logically based selection, namely the negation of the consequent, coincides with the matching bias selection, namely q. hence the correct solution.

V. Girotto

94

& G.

Politzer

Evans et al. (1987) attribute the underlying mechanism of matching bias to the natural language use of negation: The subject‘s attention is focused on the presupposition which, in a negative statement, coincides with positive information. Here, the positive and negative statements correspond respectively to the given/ new information in the Clark and Haviland (1977) sense. In other words, negatives make statements about affirmatives, so that, in a statement such as ‘IfA, then not 7,attention is directed towards 7. 1.6 Selection of information: Roblem solving As mathematics and science teachers commonly observe,

students usually assume that the givens of a textbook problem are all relevant. This has acquired the status of a “rule of the game” between teacher and student. I t can be viewed as an application of what Ducrot (1972) has called the law of economy of qualfiation which states that each particular qualification introduced in an affirmative statement should be informative. This law might also explain subjects’ failure on Wason’s 2,4,6 problem (Wason, 1960) which consists in finding out the rule followed by the experimenter to construct sequences of three numbers: the initial sequence 2, 4, 6 is presented by the experimenter, following which the subjects are requested to produce their own sequences in order to test their hypothesis by getting feedback from the experimenter. The actual rule intended by the experimenter is any three increasing numbers. The salient features of the sequence 2. 4, 6 (that is, even numbers increasing by two) are assumed to be relevant by virtue of the law of economy, which explains why subjects do not question the relevance of these features, and consequently will not put them to test. This last example shows that inductive tasks also are under the influence of the laws of language use. 1.7 Propositional reasoning

The last type of determination due to language pragmatics to be considered regards propositional reasoning. Here we shall consider deductive arguments in such a way that the premise is the logical form of the speaker’s statement and the conclusion the logical form of a statement inferred either by the speaker himself or by the listener. Under these circumstances, there is a fundamental

C o n s t r a i n t s on d e d u c t i v e reasoning

95

constraint of language use on logical reasoning which is the following. Whereas from a logical point of view in any deduction the conclusion cannot be more informative than the premise, from a pragmatic point of view the conclusion cannot be less informative than the premise. This is because the aim of human communication is to convey information, be it explicitly stated by the speaker or inferred by the listener. Consequently a deduction s u c h as P F ( P V Q ) (law of or-introduction) where the disjunct P V Q is less informative than the premise P violates this basic pragmatic requirement (McCawley, 1981; Politzer, 19861. It can be verified on both intuitive and empirical bases that it is not a n acceptable inference. Intuitively, consider: ( 1 ) The meeting took place on the 14th; therefore the meeting took place on the 14th or on the 15th.

The feeling of oddity aroused by such a n argument stems from the contradiction between the decreasing informativeness imposed by the logical form and the pragmatic expectation of increasing informativeness. Empirically, Braine. Reiser, and Rumain (1984) found that only one half of their subjects accepted inferences following this schema and Rips (1983) found an availability parameter of 20% only. I t can be shown that the paradoxes of implication are explainable on similar bases (Politzer, 1986). In summary, laws of language pragmatics operate at all the stages of the reasoning process. They constrain the selection of information by inducing judgements of relevance: they control the interpretation of quantifiers and connectives by generating implicatures; they affect the acceptability of inferences through conflicting demands for informativeness. 2. WORLD KNOWLEDGE CONSTRAINTS

In this section we shall consider the role of world knowledge on purely deductive and hypothetico-deductive reasoning. 2.1 Attitude and beliefs

World knowledge may influence deductive reasoning in several manners. The first one has been more widely investigated. It deals

96

V. Girotto

&

G. Politzer

with the role of attitudes and beliefs on the evaluation of the validity of deductions. The most common paradigm consists of asking the subject to evaluate the conclusion of a syllogism: the conclusion may match or mismatch the subject's own attitudes or beliefs. Virtually all studies have reported an effect of belief on judgement of logical validity: subjects solve neutrally toned syllogisms more correctly than emotionally toned syllogisms (Lefford. 1946; Thistlethwaite. 1950) and their beliefs influence their response in the direction of their convictions (Evans, Barston, & Pollard, 1983: Feather, 1964: Kaufmann & Goldstein, 1967; Oakhill & Johnson-Laird, 1985). Nowadays the issue is not whether or not belief affects reasoning, but to what extent and how it does so. While for some authors human thought is irrational and lacks objectivity, for others such influences are due to the fact that attitudes and cognitive processes normally interact (Henle & Michael, 1956) and more specifically, the effect observed would reflect a rational strategy in order to solve a contradiction between conclusion and belief (Revlin. Leirer, Yopp, & Yopp, 1980). We would like to suggest that those results might be better understood as reflecting a process of decision under uncertainty: that is, when presented with a deduction whose validity seems uncertain, the subject's best bet may be to produce a response that matches what he believes to be the case. In brief, the results of these studies can be easily interpreted in connection with the fact that there are many schemas that are borderline in terms of the subjects' inferential capabilities: therefore, it is natural that on these schemas subjects are sensitive to non-logical influences. 2.2 Imported premises

The second sense in which world knowledge influences deductive reasoning is deeper, in that instead of influencing the judgement of truth or validity, it provides the reasoner with a stock of premises. In this sense, world knowledge determines reasoning. This occurs systematically in reasoning applied to specific subject matters as well as in daily life. For example, from the single premise it rains the use of a modus ponendo ponens argument to arrive a t the conclusion there are clouds is made possible because the additional premise fi it rains, there are clouds is available as an element of world knowledge. Reasoning in natural language is built on this process of furnishing inferences schemas with imported premises.

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

91

This phenomenon provides part of the explanation of a class of paradoxes that characterize reasoning in natural language. For example, granted that the natural reasoning negation of a conditional i f P then Q generally is obtained by negating the consequent alone (producing ifP then not Q), the striking failure of contraposition to apply in some circumstances is easily explained. Consider the following statement: (2) If Pierre goes to England, he does not stop in London.

Its contrapositive is: (3) If Pierre stops in London, he does not go to England. I t might be tempting to conclude from such examples that contraposition is not valid in natural language. This would be misleading. The paradox stems from the clash between (3) and (4): (4) If Pierre stops in London, he goes

to England.

where (3) has been obtained by inference and (4) stems from our world knowledge. Sentences (3) and (4) clash because, for natural reasoning, they are the negation of each other, whereas logically taken together they simply amount to:

(5) Pierre does not stop in London. In brief, the paradox occurs because, within natural reasoning, world knowledge contradicts the inferred sentence. 2.3. Context and content of sentences

The third sense in which world knowledge influences reasoning has been investigated by varying the context and the content of the sentences: it amounts to manipulating the interpretation of the sentences. We shall limit our discussion to the paradigms that have received the most attention, namely conditional reasoning and the four-card selection task in its "thematic" versions. 2.3.1 Conditional reasoning Several studies have shown that the interpretation of the conditional link between the antecedent and the consequent depends on what these components refer to.

98

V. Girotto

& G.

Politzer

Given a sentence ifP then Q, the experimental task consists of deciding the truth value of the sentence when P and Q are true, P is true and Q is false, etc. (truth table evaluation). The results indicate marked differences between abstract and meaningful sentences (Leahey, 1977) and between different kinds of meaningful sentences. Marcus and Rips (1979) observed a biconditional interpretation when the statement suggested a causal mechanism ('Ifthe ball rolls left, then the red light jlashes') but not when the statement referred to arbitrary associations ('Ifthe fish is red, then it is striped or 'Ifthere is a B on the l e 3 side, then there is a 1 on the right side'). Similarly, Politzer (1981) observed more conditional and biconditional and fewer conjunctive interpretations when the occurrences of the two components of the sentence were strongly correlated in the real world ( ' I f 1 wear ajacket, then I put on a tie') than when they were more loosely correlated ('If1drink coffee, then Z e a t biscuits'). In a different task, O'Brien, Costa, and Overton (1986) presented their subjects with causal statements that belonged to either mechanical or medical domains (e.g. 'If the bone chips a r e removed, then the p a i n will b e eliminated'). Given the contingencies P, Q : P, not-Q; etc. as evidence, the subjects had to decide whether the rule was true or false. They found that subjects considered the P, Q case as a proof more often with the mechanical than with the medical content, and that subjects considered the not-P, Q case as a counter-example to the rule for the mechanical, but not for the medical content: it is likely that cases of spontaneous remission are more available to the subjects in medicine than in mechanics. Given the effect of content on the interpretation of conditional statements, it should not be surprising that subjects' evaluation of the conclusion of conditional syllogisms depends on the relationship between the antecedent and the consequent of the conditional premise. This is precisely what was observed in a few studies. Roberge (1982) found fewer cases of fallacious affirmation of the consequent with a class inclusion rule ('If there is a trout, then there is afih') than with a causal-temporal rule ('lfthe alarm clock rings, then I wake up'). Marcus and Rips (1979) found fewer cases of acceptance of the two fallacious schemas that result from an implicative reading of the rule (but not under a bi-implicative reading, i.e. Denial of the Antecedent and Affirmation of the Consequent) with the arbitrary rule mentioned above [ ' Z j t h e f i h is

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

99

red, then it is striped') than with the causal rule. All these results concur to show that the conditional formulation of deterministic causal relations gives rise more often to a biconditional interpretation than arbitrary, class-inclusion or necessity relations and that people reason in accordance with their interpretation. 2.3.2 The selection task The issue of world knowledge effects on reasoning has been treated recently within a framework proposed to study inductive processes (Holland, Holyoak, Nisbett. & Thagard, 1986). These authors opt for a pragmatically oriented approach to the study of inferences, which is characterized by a specific consideration of the context and goals of the cognitive system. Along this perspective, the notion of pragmatic schemas which they develop seems to offer a promising line for exploring content effects on reasoning. Pragmatic schemas are defined as clusters of abstract rules concerning goals and actions (Cheng & Holyoak, 1985). A class of these schemas, the regulative ones, such as permission ('Ifone wants to d o action A, then one must satisfy precondition B') and obligation ('If condition A occurs, then action B must be fulfilled'), have production rules which correspond to the logic of implication. For example, a permission schema has a rule which corresponds to the contrapositive inference ( ' r f precondition B is not satisBed, then action A must not be taken'), and rules which block the classical fallacies of Denying the Antecedent and Affirming the Consequent (respectively: 'If action A is not to be taken, then it is irrelevant whether or not precondition B is satisfred', and 'Ifprecondition B is satisfied, then action A can be taken'). Given this correspondence, the activation of one of these schemas should produce correct performance in tasks that require inferences following from the material conditional, like in the selection task. Empirical results indicate that this is the case: The activation of a permission (e.g. Cheng & Holyoak, 1985; Girotto, Light, & Colbourn, 1988) or a n obligation schema (Cheng, Holyoak, Nisbett, & Oliver, 1986; Girotto, Blaye. & Farioli, 1989, Exp. 3) produces facilitation on selection t a s k performance, with adults as well as with children. Interestingly, a facilitation effect was elicited even when the rules to be tested were not directly familiar to the subjects. For example, subjects without experience of the specific content of a permission rule such as ' I f a letter is sealed, then it must carry a 20-cent stamp' or 'All the buzzing bees must stay outside', reached the same

100

V. Girotto

&

G. Politzer

performance as 'expert' subjects, if an explicit purpose for the rules was provided (cf. Cheng & Holyoak. 1985; Girotto et aL. 1988). In addition, Cheng and Holyoak (1985, Exp. 2) showed that even the purely abstract description of a permission situation (i.e. with no reference to any concrete content, e.g. 'Zfone is to take action A, then one must first satisfy precondition B')elicited a high rate of correct performance on the selection task. The empirical evidence that people are able to search for counter-examples of unfamiliar or abstract regulative rules seems difficult to explain by alternative approaches. In particular, an explanation in terms of familiarity or in terms of the availability of counter-examples (cf. Griggs, 1983; Pollard, 1982) cannot easily account for the data. Taken together, these results seem to corroborate the hypothesis that people often reason by using abstracted clusters of rules such as permission and obligation schemas. The degree of abstraction of pragmatic schemas. however, is relative. Even if these knowledge structures can be distinguished from representation of specific experience, they do not attain as high a degree of abstraction as syntactic inference schemas. Pragmatic schemas are in fact context-sensitive: consequently, the outcome of their production rules may not coincide with the result obtained by the application of a n inference schema after a formal, non-contextual reading of the sentence being tested. This point is clearly illustrated by two recent pieces of research. Politzer and Nguyen-Xuan (1988) used a promise rule of the type ' I f you perform action A, then you will receive benefit B' (e.g. 'If the purchase exceeds 10,000 francs, then the salesman must stick on a voucher gifl for a golden bracelet on the back of the receipt'). They showed that, depending on the point of view that the subject had to take, different selection combinations were obtained, while keeping the wording of the rule unchanged. Nearly all the subjects who produced selection combinations characterized by the presence of the antecedent of the rule (that is. attending to the performance of action A, card P: a purchase exceeding 10,000 francs), and by the presence of the negation of the consequent of the rule (that is, attending to the absence of the benefit B, card not9: not receiving the gift) had been instructed to consider the rule from the point of view of the recipient of the promise (i.e. the customer). Likewise, all the subjects who produced combinations characterized by the presence of the consequent of the rule (that is,

C o n s t r a i n t s on d e d u c t i v e reason i n g

101

attending to the reception of benefit B, card Q : receiving the bracelet), and by the presence of the negation of the antecedent (that is, attending to the absence of the performance A, card not-P a purchase of less than 10,000 francs) had been instructed to consider the rule from the point of view of the donor of the promise (i.e. the manager). Formally, the relations between the two sets of selection combinations is an exchange of t h e antecedent (performance of A) and the consequent (reception of B). These results suggest that inferences that people tend to draw when requested to check the validity of regulative rules depend on the pragmatically relevant context in which they are embedded. Similar conclusions are suggested by the results of another investigation. Girotto, Gilly, Blaye, and Light (1989) showed that rules that have been cast syntactically as permissions produce different performance with the variation of their inherent intelligibility as social regulations. If the content of the rules was not directly familiar to the subjects (10- and 14-year olds), performance was facilitated when the expressed relation did not appear arbitrary. The plausibility or intelligibility of the rules was supported either by the provision of an explicit rationale or by the ready availability to the child of an implicit rationale. For example, an unfamiliar permission rule s u c h as 'If one drives over 100 km/h, then one must haue a fluorescent car' elicited a high rate of correct performance (significantly superior to that elicited by the standard rule relating vowels and even numbers), regardless of the presence of an explicit rationale (such as 'The government decided this law because at high speed cars must be visible at a distance, and therefore the fluorescent bodywork colour is a safety measure'). With this version of the rule, children were able to imagine a purpose for its formulation, when not explicitly given one. On the contrary, with an implausible version of the rule ('Ifone drives under 100 Km/h, then one must have a Juorescent car'), most children were unable to think of any reason, nor were they able to correctly perform the task. Interestingly, the latter version elicited a higher frequency of selections of the not-P card, which is the card that corresponds to the case 'driving fast' (i.e. the action that is subject to control in the real world). Therefore, individuals seem to search for pragmatically meaningful potential counter-examples, rather than for the formally relevant ones.

102

V. Girotto

&

G. Politzer

2.3.3 The social contract algorithms A different explanation of content effects in the selection task has been proposed by Cosmides (1985. 1989). According to this author, people use specialized cognitive processes to reason about social exchange, which is defined as "adaptive cooperation between two or more individuals for mutual benefit" (Cosmides, 1989). In particular, she postulates that a specific cognitive procedure, the "social contract algorithm" (henceforth SC algorithm), has naturally evolved. Its aims are (i) assess the costs and benefits of various courses of action: (ii) detect individuals who cheat, that is, individuals who fail to pay the cost (or meet the requirement) of the exchange in order to receive the benefit. Thus, in a typical situation of SC like ' r f you take the benefit, then you have to pay the cost', the activation of a 'look for cheaters' procedure would draw attention to any individual who received the stated benefit and to any individual who has not paid the required cost. Now, if this type of rule is tested in the selection task setting, the activation of such a procedure would lead the subjects to select t h e c a r d s corresponding to the person who received the benefit and to the person who did not pay the cost. Therefore, their selection would not be determined by a formal analysis of the ifsentence in terms of antecedent and consequent. This prediction has been empirically supported in a series of experiments in which subjects had to solve selection tasks concerning unfamiliar SC rules. Cosmides (1989, Exp. 1 and 3) presented such a rule ('rfaman eats cassaua root, then he must have a tatoo on his face') in a scenario which depicted a Polynesian tribe where cassava root was described as a rationed benefit and having a tatoo as a cost requirement. When this type of SC rule was stated following the standard formulation ('Zfa man takes the benefit, then he p a y s the cost') subjects tended to select the formally correct cases P ('benefit accepted') and not-Q ('cost not-paid'). On the contrary, when the S C rules were switched ('rfa man pays the cost, then he takes the benejit'), the cards more frequently selected were still those representing the cases 'cost not-paid' and 'benefit accepted'. But, in this condition, they correspond to the formally incorrect cases not-P and Q . Thus, reasoning about a social rule seems to be governed by a 'look for cheaters' procedure, irrespective of the formal status of the rule components. In a second series of experiments, Cosmides (1989, Exp. 5-9) found that permission rules lacking the benefit-cost requirement

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

103

structure of a n SC produced less facilitation than rules having this structure. For example, the above 'cassava root' rule produced a significantly higher rate of correct performance (80%) than the same rule presented in a context in which the stated action (eating cassava) was not a benefit (compared to eating another food: molo nuts), and in which having a tatoo was not a cost (correct selection: 45 To). According to Cosmides. these data corroborate her hypothesis that only SC permissions facilitate the task, and that SC algorithms are not induced from everyday life experience by some general purpose processes: on the contrary, they are special purpose innate algorithm. Leaving aside the general problem of the origins of reasoning mechanisms, which is an issue that clearly differentiates Cosmides' position from all the other theoretical perspectives on reasoning, a few words on the empirical corroboration of the SC theory are in order, particularly with respect to the alternative view of pragmatic schemas. Even if it is possible that SC permission rules are normally interpreted in a conditional way, it should be mentioned that other empirical studies showed that (i) obligation rules having a weak cost-benefit s t r u c t u r e produce t h e same rate of correct performance as a n SC permission rule (Girotto, Blaye, & Farioli, 1989, Exp. 3); (ii) facilitation can be obtained with prudential rules t h a t do not involve a direct social exchange (cf. the above 'fluorescent car' rule, Girotto et al. 1989; cf. also Manktelow & Over, in press), even for prudential rules in which the required precaution is not costly (cf. Cheng & Holyoak, 1989): finally, (iii) SC rules can be insufficient to produce a high rate of facilitation: Politzer and Nguyen-Xuan (1988) obtained only 34% correct performance with the above jewellery rule, when tested from the point of view of the customer, that is, in the case that was the most favorable for the activation of a 'look for cheaters' procedure. Further empirical work will assess the relative merits of the two discussed approaches, and the possible interaction between the content of the tested rule and other apparently less relevant factors, like the type of instructions. (Note that contrary to Cosmides, Politzer and Nguyen-Xuan did not give explicit cues to those subjects who were playing the role of customers to identify the manager of the jewellery as a 'potential cheater'. Also note that in the non-SC permission conditions, Cosmides presented the people

104

V. Girotto

& G.

Politzer

subject to the regulation as "law-abiding people", and the rule itself more as a scientific hypothesis than as an established regulation.) In any case, from a theoretical point of view, Cosmides' position has a major weakness. Following her definition of SC, it could be possible to explain the facilitation obtained with rules not involving social exchange. Consider, for example, the prudential rules mentioned above. In these cases, the regulations typically express the requirement to meet in order to be entitled to do a certain action (e.g. the fluorescent color of the bodywork is a necessary requirement for the action of driving fast). As we have seen, Cosmides' definition is large enough to include situations, like these, where there are no really exchangeable entities to give in payment for some "individual (or group)". However, this broader definition of exchange (or "pseudo-exchange theory" (cf. Cheng & Holyoak, 1989) does not really differ from the definition of permission schemas: "Cosmides' "benefits" become a subset of Cheng and Holyoak's "action to be taken" in their (not necessarily social) permission schema, and her "requirements" become the "preconditions" in the permission schema" (Cheng & Holyoak, 1989). Therefore, in order to avoid t h e weakening of its explanatory power, the SC theory has been reduced to a specific case of the pragmatic schemas theory. In general, the latter seems to explain more parsimoniously a larger amount of content effect in reasoning outside the realm of SC's, in both social and non-social domains. As a provisional conclusion, the available results indicate that there is a domain of world knowledge in which people possess the ability to search for counter-examples: it consists of a subclass of social rules which produce the representation of their potential violators. CONCLUSIONS

In conclusion, the theoretical considerations and the empirical results just reviewed underline the importance of the pragmatic perspective in the elucidation of reasoning. The application of two types of considerations commonly referred to as 'pragmatic', namely, the laws of language use on the one hand, and real-world knowledge on the other hand, appear essential to explaining errors, biases and correct performance in reasoning. However, the question of the relationships between these two fields

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

105

is clearly a n open problem. Notice that although the term 'pragmatics' in its second broad sense is widely used, it has seldom received a precise definition in the psychological literature. This applies, inter alia. to the work of Holland et al. (1986) for whose theory this concept is crucial. By and large, it seems that most researchers who are interested in that kind of pragmatics are concerned with the regulation of the subject's action in order to achieve a goal in the real world. What is common between this world knowledge and language pragmatics? There are striking cases where both types of analysis coincide, for instance in the promise situation: Politzer and Nguyen-Xuan (1988) were able to make concurring predictions in terms of a n analysis based on Gricean implicatures and in terms of pragmatic schemas. Given the social foundation of the Cooperative Principle (Grice (1975) states that he assumes that such a principle is basic in all social endeavours), one might speculate that language pragmatics on the one hand and goal directed activity in social contexts on the other are derived from a common set of rules whose aim is to enable members of complex social organizations to interact.

REFERENCES Begg, I.. & Harris, G. (1982).On the interpretation of syllogisms. Journal of Verb4 Jxarninff and Verbal Behavior. 595-620. Braine. M.D.S. (1978).On the relation between the natural logic of reasoning and standard logic. Psvcholoeical Review, &, 1-21. Braine. M.D.S. (1988). The "natural logic" approach to reasoning. In W.F. Overton (Ed.). Reason ine. necessitv and logic DeveIoDmental DersDectiv a . Hillsdale. NJ: Lawrence Erlbaum. Braine. M.D.S., Reiser, B.J.. & Rumain. B. (1984). Some empirical justification for a theory of natural propositional logic. In G.H. Bower (Ed.), The Dsvcholom of leamine and motivation, (Vol. 18).New York Academic Press. Cheng. P.W., & Holyoak. K.J. (1985).Pragmatic reasoning schemas. Coenitive J'svchdo~, 17,391-416. On the natural selection of reasoning theories. Cheng. P.W., & Holyoak. K.J. (1989). 285-313. Qenition. Cheng, P.W.. Holyoak. K.J.. Nisbett, RE., & Oliver, L.M. (1986).Pragmatic versus syntactic approaches to training deductive reasoning. Cognitive Psvcholoe. 18.293-328. Clark, F H . , & Haviland. S.E. (1977). Comprehension and the given-new contract. In RO. Freedle (Ed.), Discourse Droduction and commehension. Hillsdale, NJ: Lawrence Erlbaum. Cosmides, L. (1985).Deduction or Darwinian aleorithms? An exDlanation of the Doctoral dissertation, Harvard University. University Microfilms. Cosmides. L. (1989). The logic of social exchange. Has natural selection shaped how humans reason? Studies with the Wason selection task. Coenition, 187276. Ducrot, 0. (1972). Dire et ne Das d b. Paris: Hermann.

a,

a,

'w.

a,

V. Girotto

106

&

G. Politzer

Evans, J.St.B.T. (1977).Linguistic factors in reasoning. Quarterlv Journal of Fxuerimental Psvchology, 29,297-306. m e Psvcholom of Deductive R e a s o a . London: Routledge & Evans, J.3.B.T. (1982). Kegan Paul. Evans, J.St.B.T. (1984).Heuristic and analytic processes in reasoning. British Journal of Psvcholo~y, 451-468. Evans, J.St.B.T. (1987).Reasoning In H. Beloff & A.M. Colman (Eds.), Psvcholoizv Survey. (Vol 6).Leicester: The British Psychological Society. Bias in human reasonina. C w e s and conseauenca . Hove Evans, J.St.B.T. (1989). and London: Lawrence Erlbaum. Evans, J.S.B.T.. Ball, L.J., & Brooks, P.G. (1987).Attentional bias and decision order in a reasoning task. British Journal of Psvcholoa, 28,385-394. Evans, J3.B.T.. Barston, J.. & Pollard, P. (1983). On the conflict between logic and belief in syllogistic reasoning. Jvlemorv and COPnitioq. U, 295-306. Evans, J.St.B.T.. & Lynch, J.S. (1973).Matching bias on the selection task. British &urnalo f PsvchoI O U , 391-397. Feather, N.T. (1964). Acceptance and rejection of arguments in relation to attitude strength, critical ability, and intolerance of inconsistency. Journal of Abnormal and Social Psvcholo@, 127-136. Fillenbaum, S. (1975). If: some uses. psvchoI~gicalResearch, 2, 245-260 Gels, M.L., & Zwlcky, A.M. (1971).On invited inferences. I i n g u t s t i c q . 2,561-

a,

a,

m,

566. Girotto, V.. Blaye. A.. & Farioli. F. (1989).A reason to reason: Pragmatic basis of children's search for counterexamples. Cahiers de Psvcholoaie Coenitive/ EuroDean Bulletin of Cognitive Psvcholocy!.9, 297-321. Girotto, V.. Gilly, M.. Blaye, A.. & Light, P.H. (1989). Children's performance in the selection task Plausibility and familiarity. British Journal of PsvcholQgy, 79-95. Girotto. V.. Light, P.H.. & Colbourn. C. (1988).Pragmatic schemas and conditional reasoning in children. Quarterlv Journal of ExDerimental Psvcholoe, m, 342-359. Grice, H.P. (1967).Logic and conversation. William James lectures. Harvard University. Grice. H.P. (1975).Logic and conversation. In P. Cole & J.L. Morgan (Eds.).svntax and semantics Nol. 9).New York: Academic Press. Griggs, R.A. (1983). The role of problem content in the selection task and THOG problem. In J.St.B.T. Evans (Ed.), Thinking and Reasoning: Psvchological 4DDroacheS. London: Routledge and Kegan Paul. Griggs. R.A.. & Newstead. S.E. (1983).The source of intuitive errors in Wason's THOG problem. British Journal of PsvcholQgy, 74.451-459. Henle, M.,& Michael. M. (1956). The influence of attitudes on syllogistic reasoning. Journal of Social Psvcholom. 44,115-127. Holland, J.H.. Holyoak, K.J., Nisbett. R.E.. & Thagard, P.R (1986).Induction Processes of Inference. Learnine and Discovery. Cambridge, MA: Brandford Books/MIT Press. Horn, L. (1972).0n the semantic urQuerties of loaical ouerators in English. Bloomlngton, IN: Indiana University Linguistics Club. Johnson-Laird, P.N. (1983). Mental models. Cambridge: Cambridge University Press. Kahneman, D., Slovic. P., & lkersky, A. (1982).Judgement under uncertaintv: Uuristic and biases. Cambridge: Cambridge University Press. Kaufmann, H., & Goldstein, S. (1967). The effects of emotional value of conclusions upon distortion in syllogistic reasoning. psvchonomic Science, Z. 367-368. Leahey. T.H. (1977).The effect of sentential and abstract rules on implicational reasoning and judgement. Journal of General Psvcholocy, 92. 185-191. Lefford. A. (1946). The influence of emotional subject matter on logical reasoning. Journal of General Psvcholom, 2, 127-151.

a,

C o n s t r a i n t s on d e d u c t i v e r e a s o n i n g

107

Manktelow. K.I.. & Evans, J.St.B.T. (1979).Facilitation of reasoning by realism effect or non-effect? British Journal of Psvcholo&, ZQ, 477-488. Manktelow, K.I.. & Over. D.E. (1990). Deontic thought and the selection task. In K.J. Gilhooly. M. Keane, R H . Logie. & G. Erdos (Eds.), Lines of thou2ht: Reflections gn the DSVC holom of thinking. London: Wiley. Marcus, S.L.. & Rips, L.J. (1979). Conditional reasoning. Journal of Verbal Learning and Verbal Behavior, Is.199-223. Markovits, H. (1984). Awareness of the 'possible' a s a mediator of formal thinking in conditional reasoning problems. British Jou ma1 of PsvcholoQ!. 367-376. Markovits, H. (1985).Incorrect conditional reasoning among adults Competence or performance? British Journal of Psvcholo , z,241-247. Newstead, S.E. (1989).Interpretational errors syllogistic reasoning. Journal of Memorv and Lar@ua&. =,78-91. Newstead. S.E., & Griggs. R.A. (1983).Drawing inferences from quantified statements: A study of the square of opposition. Journal of Verbal Learning m d Verbal Behavior, 535-546. Nisbett R.E.. & Ross, L. (1980).Human inference strategies and shortcomings of social iudcements. Englewood Cliffs, NJ: Prentice-Hall. Oakhill, J.V..-& Johnson-Laird. P.N. (1985). The effects of belief on the spontaneous production of syllogistic conclusions. Quarterlv Journal of Ernerimentd psvcholo@, =A, 553-569. O'Brien. D.P., Costa, G.. & Overton. W.F. (1986). Evaluation of causal and conditional hypothesis. Quarterlv Journal of ExDerimental PsvcholQgy. 493-512. Politzer, G. (1981). Differences in interpretation of implication. American Journal of PWcholo&, 94.461-477. Politzer. G. (1986).Laws of language use and formal logic. Journal of 47-92. Psvcholinauistic Research, Politzer. G. (1990).Immediate deduction between quantified sentences. In K.J. Gilhooly. M.T.G. Keane, R.H. Logie. & G. Erdos (Eds.), Lines of thought; Reflections on the Dsvcho1om of thinking. London: John Wiley. Politzer. G. (in press). A cross-linguistic comparison of deductive abilities. Journal Df Cross-cultural PsvcholoPy. Politzer, G.. & Nguyen-Xuan (1988). praamatic reasonina schemas: Promises and the four-card selection task. Unpublished manuscript. University of Parls VIII. Pollard. P. (1982).Human reasoning: Some possible effects of availability. Cowition, 12,65-96. Revlin. R.. Leirer. V., Yopp. H.. & Yopp, R. (1980).The belief-bias effect in formal reasoning: The influence of knowledge on logic. Memorv and Cognition. B. 447-458. Rips. L.J. (1983).Cognitive processes in propositional reasoning. Psvcho1 m pevie~. 38-71. Roberge, J.J. (1982)Linguistic factors in conditional reasoning. Quarterlv Journal of Emerimental Psvcholoa. 275-284. Rumain. B. Connell. J.. & Braine. M.D.S. (1983).Conversational comprehension processes are responsible for fallacies in children as well a s in adults Ifis not the biconditional. DeveloDmental Psvcholo&, 19.471-481. Sperber, D., & Wilson. D. (1986). Relevance. Communication and cognition. London: Blackwell. Thistlethwaite. D. (1950).Attitude and structure a s factors in the distorsion of reasoning. Journal of Abnormal and Social Psvcholom. 442-458 Wason. P.C. (1960).On the failure to eliminate hypotheses in a conceptual task. Quarterlv Journal of ExDerimental Psvchology, 12, 129-140. Wason. P.C. (1966). Reasoning. In B.M. Foss (Ed.),New horizons in Dsvchology (vol. I). New York: Pelican. Wason. P.C. (1978).Hypothesis testing and reasoning. Unit 25. Block 4. Cocnitive Psvcholoa. Milton Keynes: Open University.

a,

m.

z,

m.

u,

w,

m,

s,

This Page Intentionally Left Blank

Cognitive Biases J. -P. Caverni, J. -M. F a b r e a n d M. Gonzalez (Eds.) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

REMEMBERING CONCLUSIONS W E HAVE INFERRED: WHAT BIASES REVEAC Ruth M.J. Byme* and P.N. Johnson-Laird** *

U n i v e r s i t y of Wales, C o l l e g e o f C a r d i f f , Wales

**

P r i n c e t o n U n i v e r s i t y , USA

Abstract. The theory of mental models proposes that reasoners make inferences by constructing an initial model of the premises, and then searching for alternative models that refute a putative conclusion. Where there is no valid conclusion, reasoners will nevertheless have considered, only to reject, a conclusion based on their initiaI model. In order to test this theory, we asked subjects what conclusion. if any, followed from pairs of syllogistic premises. half of which did not support any valid conclusion. Subsequently, we gave them an unexpected recognition test in which they had to pick out which conclusion, if any, they had originally drawn from the premises. Our prediction was corroborated: when subjects had correctly responded "no valid conclusion" but erroneously picked out a conclusion in the recognition test, then with no exception they chose, not a foil. but a conclusion corresponding to the initial model of the premises. A subsequent experiment suggested that the phenomenon probably arises as a result of making the inference again during the recognition test: when the premises were not included in the recognition test, the subjects' selections were essentially made at random. The results cannot be explained by current theories based on formal rules of inference or on inferential heuristics such a s the atmosphere hypothesis.

THEORIES OF REASONING

Most theorists believe t h a t reasoning processes can be represented by a coherent set of principles, but they disagree about the fundamental nature of these principles. Some theorists propose that the validity of a conclusion is tested by semantic procedures that search for alternative models of the premises that falsify the conclusion: people make errors because they do not have a simple deterministic procedure for this search (e.g., Johnson-Laird, 1983; Byrne, 1989; Johnson-Laird, Byme, & Tabossi, 1989). Other theorists propose that there are formal rules of inference in the mind for deriving proofs of conclusions: people make errors because they fail to apply the correct rules. or because they apply the correct rule in the wrong way (e.g., Braine, Reiser & Rumain, 1984; Osherson, 1975; Rips, 1983). Still others suggest that people possess heuristics to produce pragmatically useful and often logically correct conclusions (Pollard & Evans, 1987). These heuristics may

R.M. J. Byrne

110

& P.N.

Johnson-Laird

take the form of memories organised into generalised rule-based schemas (e.g., Griggs & Cox, 1982: Cheng & Holyoak. 1985). Alternatively, they may be "meta-reasoning" heuristics that govern attention to certain aspects of a task (e.g. Evans, 1982:Woodworth & Sells, 1935). People make errors because the heuristics apply only in a limited range of situations: in other situations what was a heuristic becomes a bias that leads them astray. How can we decide between the three theories? Studies of the final products of reasoning, i.e. the conclusions that people make, have not weighed heavily against any of the theories. Hence, we decided to obtain evidence about the intermediate processes of reasoning. INTERMEDIATE PROCESSES OF REASONING The model theory

The three theories each propose that different information is inferred at intermediate stages during reasoning. The theory of reasoning based on mental models proposes that there are three stages to reasoning: first, the premises are understood by constructing a representation based on their meaning and general knowledge: next, a putative conclusion supported by the representation is formulated: and finally, its validity is tested by attempting to revise the initial model. If the model can be revised so that the conclusion is false in the new model, then the inference is invalid: if the model cannot be revised, the inference is valid. Thus, on this account, an intermediate stage of reasoning is the consideration of a putative conclusion, which might well be erroneous. These processes can best be illustrated by an example. Consider the following premise: All of the swimmers are horseriders The theory proposes that reasoners attempt to construct a model of it: swimmer horserider swimmer horserider o horserider o horserider

Remembering i n f e r r e d c o n c l u s i o n s

111

where each line represents a n individual and each token that corresponds to a swimmer also corresponds to a horserider. The possibility that there might be horseriders who are not swimmers is represented by some tokens tagged by a n "0"(for "optional"). If the second premise is: All of the horseriders are psychologists there is no choice about where to insert the tokens for psychologists into the model, the horseriders must all be tagged as being psychologists : swimmer horserider psychologist swimmer horserider psychologist o horserider psychologist o horserider psychologist o psychologist This model supports the conclusion:

AU of the swimmers are psychologists. There is no way to revise the model so that it still represents the premises and yet falsifies this conclusion, and so the inference is valid. If, however, the second premise is: Some of the horseriders are psychologists t h e n there is a choice about which horseriders to tag as psychologists: either the horseriders who are swimmers, or the horseriders who are not swimmers, or some mixture of the two. Whenever there is such a choice, the theory predicts that the inferential task should be harder, because more than one model of the premises has to be constructed. One such model: swimmer horserider psychologist swimmer horserider psychologist swimmer horserider o horserider o psychologist supports the conclusion: Some of the swimmers are psychologists

R.M. J. Byrne & P.N. Johnson-Laird

112

or its converse. However, there is an alternative model, equally compatible with the premises: swimmer horserider swimmer horserider o horserider psychologist o horserider psychologist o horserider o psychologist This model falsifies the initial conclusion, and the two models do not support any conclusions in common. Hence, there can be no valid inference inter-relating swimmers and psychologists. The model theory divides inferential problems into three categories: those that depend on one model and that support a valid conclusion: those that depend on more than one model and that support a valid conclusion: and those that depend on more than one model and that do not support a valid conclusion. In the case of the multiple-model problems, a n intermediate inferential step is to consider an erroneous conclusion based on the initial model, before rejecting it because it is refuted by a subsequent model.

The rule theory The theory of reasoning based on formal rules of inference proposes very different intermediate steps. A pair of premises such as: Some of the swimmers are horseriders All of the horseriders are psychologists is interpreted by recovering each premise’s logical form. For example, in the notation of the predicate calculus, the form of the first premise is: [there exists an x, such that xis a swimmer and x is a horserider] and the form of the second premise is: 1. ( 3 X ) ( s X

A

-)

=m

[for any y. if y is a horserider then y is a psychologist] After the logical form is recovered for both premises, the premises are instantiated. The rule of existential instantiation applied to the premise with the existential quantifier (1) yields: 2. (VY)(HY

Remembering i n f e r r e d c o n c l u s i o n s

113

[ais a swimmer and a is a horserider] 3. SahHa where "a" denotes a hypothetical individual introduced by the rule. The rule of universal instantiation applied to the premise with the universal quantifier (2) yields: [If a is a horserider then a is a psychologist]

4.Ha-h

where this rule allows any individual (including the previously instantiated individual, a) to be substituted as the value of the variable. Next, the elementary formal rules of inference go to work on these premises to provide a proof [conjunction elimination, from 31 [modus ponens, from 4 and 51 [conjunction elimination, from 31 [conjunction introduction, from 6 and 71

5.H a 6.Pa 7.sa 8.SahPa

This conclusion is then generalised by a rule to restore the quantifier: 9.

(3X)(sX

A

CX)

[existential generalisation of 81

and finally translated back into natural language to produce: Some of the swimmers are psychologists or its converse. Since the conclusion can be proved, the inference is valid. If the rules cannot be applied to derive a conclusion, there is no valid conclusion. Thus, the formal theory suggests that the intermediate steps in a n inference are likely to consist in a series of immediate inferences such as the ones above. If there is no proof, then reasoners will get stuck in their efforts to apply the inferential rules and they will never come up with a quantified conclusion. Heuristics and biases

Proponents of heuristics propose a different s e t of intermediate processes. The "atmosphere" heuristic has been proposed to operate for pairs of quantified premises (Woodworth & Sells, 1935; Begg & Denny, 1969). An initial conclusion is constructed solely on the basis of the "mood" of the premises, i.e.

114

R.M.J.

Byrne

&

P.N. Johnson-Laird

whether the quantifiers are negative ( n o and some-not) or affirmative (all and some), and whether they are universal (all and no) or particular (some and some-not). Given premises such as: None of the swimmers are horseriders Some of the horseriders are psychologists the particular quantifier, some, overrides the universal, no, and so the conclusion will be a particular one; and the negative quantifier, no, overrides the affirmative, some, and so the conclusion will be a negative one. The two constraints together yield a conclusion that is both negative and particular, i.e. some-not. The correct conclusion for this pair of premises is, in fact: Some of the psychologists are not swimmers and so the atmosphere hypothesis can lead to a correct response. But, for a problem without a valid conclusion, such as: Some of the swimmers are not horseriders None of the horseriders are psychologists the atmosphere hypothesis still yields a conclusion. Hence, if people use the heuristic alone, they will fail to make the correct "no valid conclusion" response. Instead, they will produce a some-not conclusion (because both quantifiers in the premises are negative and the particular overrides the universal). On this account, the intermediate steps in producing a conclusion are manipulations sensitive to the mood of the quantifiers in the premises. RE-CONSTRUCTING CONCLUSIONS A test of the intermediate processes of reasoning

We examined the intermediate steps of reasoning, not directly. but by using a reconstruction technique. We asked subjects, first, to construct conclusions to a series of pairs of premises, similar to the examples above. Their task was to make an inference connecting the two end-terms, if one was possible: otherwise they were to respond that there was no valid conclusion. We were particularly interested in pairs of premises that do not support a valid conclusion, so that we could obtain indirect evidence

Remembering i n f e r r e d c o n c l u s i o n s

115

about the intermediate steps in reasoning. However, we obviously could not use only such problems, and so half of the problems had valid conclusions and the other half did not. After they had constructed their conclusions, the main phase of the experiment began. The subjects had to re-do the problems in the guise of an unexpected "recognition" test. We presented them with each pair of premises again, this time accompanied by a selection of conclusions. We asked them to choose which conclusion, if any, was the one they had originally drawn. In the selection of conclusions from which our subjects were to choose, w e neuer included their actual "no valid conclusion" response. Instead, we included two sorts of conclusions: those that the model theory predicts reasoners consider briefly on the basis of their initial model, before rejecting them in favour of the correct response: and some foils consisting of the negation of these conclusions. There are two ways in which the recognition task can be carried out. On the one hand, subjects can try to remember how they responded to the problem originally - - in which case, their performance may be biased by the initial conclusion that they fleetingly considered only to reject subsequently because it was refuted by an alternative model. On the other hand, subjects may attempt to make the inference ab initio - - in which case, we can predict exactly the same bias towards recognising a n initial conclusion as the one they originally drew even though, in fact, they had correctly responded "no valid conclusion". Subjects may indeed use both of these strategies, which are not mutually exclusive. We can illustrate this experiment with a n example. In the first phase, the subjects received a pair of premises, such as: All of the programmers are runners Some of the runners are drivers from which they were to say what, if anything, followed. In the second phase of the experiment they were presented with the pair of premises again, and a selection of four conclusions: All of the programmers are runners Some of the runners are drivers (1) Some of the programmers are drivers (2)Some of the drivers are programmers (3)None of the programmers are drivers (4) None of the drivers are programmers

116

R.M.J. Byrne

&

P.N. Johnson-Laird

Their task was to indicate which of these conclusions, if any, they had originally drawn. Two of the conclusions (those containing the quantifier some) are supported by the initial model of the premises, and two are not. Granted that they responded "no valid conclusion" correctly in the first phase of the experiment, we predicted that if our subjects mistakenly recognised one of the conclusions, it would be one of the conclusions supported by the initial model rather than one of the foils. These predictions cannot be made by either the theory based on formal rules or the atmosphere hypothesis. The results confirmed our prediction. Four subjects always erroneously drew conclusions from the invalid problems. The remairilng twelve subjects made the correct "no valid conclusion" response on 27% of the trials with these problems. Their performance in the subsequent "recognition" test for these problems is as follows: on 20% of trials. they realised that their "no valid conclusion" responses were not among the selection: and on 6% of trials, they responded that they could not identify their conclusion. On the remaining 74% of trials, eleven subjects erroneously chose one of the conclusions: every single choice was of an initial model conclusion (binomial test, p = 0.511). The subjects were not misled by the fact that their original conclusion had been excluded from the set as their responses to other problems show. Conversely, where their correct conclusion was among the set, they correctly identified it on the majority of trials.

Consequences for the three theories When the subjects correctly inferred that there was no valid conclusion to a problem, they often mistakenly judged in the recognition test that they had drawn a conclusion. Whenever they made this error, the conclusion that they falsely recognised was without exception one supported by the initial model of the premises. This finding supports the model theory, because the theory predicts that subjects who make the correct response to these problems have nonetheless considered, at an intermediate stage, such erroneous conclusions. The phenomenon is contrary to the rule theory's account of intermediate processes: if subjects were constructing proofs, it is difficult to see how they could have reached any quantified conclusion -- precisely because there are no

Remembering i n f e r r e d c o n c l u s i o n s

117

proofs for these problems. At best, they are likely only to have reached some point in a derivation that corresponds to, say:

a is a horserider before they would be forced to abandon the proof. Hence, either they should have realised that their conclusion was not among the choice set, or they should have made a random choice. The results also fail to support the heuristic theory based on the "mood" of the premises. This atmosphere hypothesis makes the same predictions as the model theory for some of the problems, but the two theories diverge on other problems. We will consider two such problems. First, premises of the form: None of the B are A Some of the C are not B do not support a valid conclusion. According to the model theory, the initial model is of the form: C

oc

b b

a a where the lines represent negative barriers between the tokens (cf. Johnson-Laird & Bara, 1984, p. 54). It supports the initial conclusion: None of the C are A which is subsequently refuted by a n alternative model of the premises, According to the atmosphere hypothesis, people should be biased towards the erroneous conclusion: Some of the C are not A (cf. Woodworth & Sells, 1935, p. 454). Our subjects made the correct response on 47% of trials to this problem and to the related

R.M. J. Byrne

118

&

P.N. Johnson-Laird

problem containing the premises in the opposite order, which the atmosphere hypothesis cannot explain. The recognition test offered only a choice of no conclusions and all conclusions. Hence, the atmosphere hypothesis does not predict any bias, whereas the model theory predicts that the no conclusion will be falsely recognised. Where the subjects chose a conclusion in the recognition test, they invariably chose a no conclusion. Perhaps proponents of the atmosphere hypothesis might argue that reasoners will be biased to chose whatever conclusion is closest to the atmosphere of the premises, and so they will choose a no conclusion in the previous case. However, this conjecture is refuted by the data from a second problem with premises of the form: All of the B are A Some of the B are not C for which the correct conclusion is: Some of the A are not C The subjects drew this conclusion, which is also predicted by the atmosphere hypothesis, on 22% of the trials for this problem and for the related problem containing the premises in the opposite order. In the subsequent recognition test, however, these subjects recognised that the conclusion was not among the choice set. mistakenly selecting the no conclusion on only 14% of the trials. Even in the case of those subjects who drew a n erroneous initial conclusion, there was no reliable bias towards selecting a n o conclusion in the recognition test: subjects chose it on 46% of the relevant trials, whereas they chose a n all conclusion on the remaining 54% of trials. In summary, neither the rule theory nor the heuristic theory can explain the results of this experiment. When reasoners re-construct conclusions in the guise of a recognition test, they appear to go through the intermediate processes postulated by the model theory. Do the subjects actually remember the initial-model conclusions that they had constructed at an intermediate stage, or do they make a new inference in the recognition test? Because the premises were presented to the subjects again in the "recognition" phase of the experiment, they could make a new inference from them, rather than try to remember their original inference. If the

Remembering i n f e r r e d conclusions

119

biases in the "recognition" phase arise because subjects try to remember their conclusions, then we would expect such biases even when the premises are not presented again. We carried out a second experiment, in which we gave subjects only a choice of conclusions in the recognition test, i.e., we did not present the premises again. In this case, their choice of conclusions was essentially random. Thus, it seems probable that our subjects were, in fact, making new inferences. CONCLUSIONS

Our experiments support the theory of reasoning based on mental models: reasoners construct a n initial model of the premises, formulate a putative conclusion on the basis of this model, and then try to test their conclusion by revising their model to one where the premises are true but the conclusion is false. When reasoners attempt to remember a conclusion that they have previously drawn, they are systematically biased in a recognition test towards a conclusion compatible with the intermediate stages of reasoning postulated by the mental model theory. The effect could reflect their memory of the original process of inference or it could arise when they carry out the inference again. The data from our second experiment suggest that reasoners are unlikely to have retrieved much information from their initial inference. REFERENCES Begg. I.. & Denny, J. (1969). Empirical reconciliation of atmosphere and conversion interpretations of syllogistic reasoning. Journal of ExDerimental J'svcholoa!, 351-354. Braine. M.D.S.. Reiser, B.J.,& Rumain, B. (1984).Some empirical justification for a theory of natural propositional logic. In G.H. Bower (Ed.). The Psvchol o d Leamim and Motivation, 18.New York Academic Press. Byme. R M . J . (1989).Suppressing valid inferences with conditionals. Coenition. 31. 61-83. Cheng. P.N.. & Holyoak. K.J. (1985).Pragmatic reasoning schemas. Cocnitive P ~ c h o l o a17. , 391-416. Evans, J. St. B.T. (1982). The Psvcholom of Deductive Reasoning. London: Routledge and Kegan Paul. Griggs. R.A., & Cox, J.R. (1982). The elusive thematic-materials effect in Wason's 407-420. selection task. British Journal of PsvcholQgy. Johnson-Laird, P.N. (1983).Mental Models: Towards a Cognitive Science of Lancuace. Inference and Consciousness. Cambridge: Cambridge University Press. Syllogistic inference. Coenition. 1-61. Johnson-Laird, P.N.. & Bara. B. G. (1984). Johnson-Laird, P.N., Byme. R.M.,J., & Tabossi. P. (1989). Reasoning by model: the case of multiple quantification. Psvcholoaical Review, 96,658-673.

a,

a,

a.

120

R.M.J. Byrne

&

P.N. Johnson-Laird

Osherson. D. (1975). Logic and models of logical thinking. In R.J. Falmagne (Ed.). &asonin@:R e D r e s u i o n and Procea. New York Wiley. Pollard, P.. & Evans. J3t.B.T. (1987). Content and context effects in reasoning. h & € i i i i ~ UM. . 41-60. RIDS. L.J. (1983). Cognitive processes in -propositional reasoning. - Psvcholoeical . -

Review. 9Q,38%. Woodworth. RS.. & Sells. S.B. (19351. An atmosDhere effect in formal svllogistic - reasoning..Journa1 of Emerimental Psvcholc& ,J&, 45 1-460.

Cognitive Biases J.-P. C a v e r n i , J.-M. F a b r e a n d M. G o n z a l e z ( E d s . ) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

SYTLOGISTIC REASONING WITH PRORABILITIES AND CONTINUOUS TRUTH VALUES Jean Costermans and Vkronique Heuschen Experimental Psychology Department University of Louvain, Belgium

Abstract. Classical syllogisms were modified by using quantities. For example: Among the 100 A's. exactly 70 are also Bs. Among the 100 Bs. exactly 30 are also C's. Consequently, among the 100As. I guess that

.... are also C's." The subjects were required to write down their "best estimate", and to express their judgment about the truth value of the resulting proposition on a continuum. Six main strategies have been described and detected through their various predictions as to the responses. The interpretation of these strategies focuses on the way the subjects fill in the gaps in the information contained in the premises by adding one or more principles of their own. INTRODUCTION

The cognitive processes involved in syllogistic reasoning have elicited some extensive experiments since the pioneer work by Sells (19361, and Chapman and Chapman (1959). Several rather sophisticated models have now been constructed from a careful examination of error patterns in human performance (Revlis, 1975; Ceraso & Provitera, 1971; Erickson, 1974; Dickstein. 1978; Guyote & Sternberg, 1981; Johnson-Laird & Bara, 1984). Almost all of these experiments, however, deal with "classical" syllogisms, the formulation of which has not been fundamentally revised since Aristotle's time. As such, they constitute a n interesting field for psychological inquiry, because this field is very complex and well defined at the same time. Indeed, syllogisms generate many errors with precise locuses and direction. Nevertheless, some limitations are obvious as regards so-called "natural" thinking. The four types of premises, namely: A (universal affirmative) : E (universal negative) : I (particular affirmative) : 0 (particular negative) :

All As are B's No As are B's Some As are B's Some A's are not B's

122

J. Costermans

&

V. Heuschen

may be ambiguous and difficult to handle when presented in the linguistic "surface" form above (Ceraso & Provitera, 1971; Erikson, 1974; Just & Carpenter, 1971; Bucci, 1978; Revlin & Leirer, 1980; Tsal, 1977; Newstead 81 Griggs. 1983). A-propositions may have two meanings: identity of set A and set B when these sets are of equal size, or inclusion of A in B when B is larger than A (therefore, the size of the sets should be mentioned in the problem). The lexical item some, as used in I- and O-propositions, is quite different in everyday language from the logical symbol it is supposed t o represent here, which means something like "at least one, possibly all". Thus, I-propositions may refer to a n intersection between A and B. a n inclusion of A in B, a n inclusion of B in A, or even a n identity. O-propositions may refer to an intersection, an inclusion of B in A, or a disjunction of sets A and B. E-propositions only, which express a disjunction, are unambiguous (even though the surface form no may appear as a peculiar combination of all and not). Besides the fact t h a t classical logic h a s often been contaminated by the peculiarities of surface formulations, two other, perhaps more fundamental, limitations must be stressed. First, the so-called "quantifiers" [all, some, none) actually represent a rather weak quantification. Since nowadays deductive reasoning often takes a mathematical form, it could be informative to study syllogism-like structures including numbers: given a set size t, number t, when inserted in a proposition, will be equivalent to all, number 0 will be equivalent to none, and all other numbers in between will be equivalent to various, but precise values of some. Second, given the two premises, the conclusion of a classical syllogism is supposed to have a binary truth value (true or false). But, when numbers are inserted in the premises and are also allowed in the conclusion, some quantities may look more acceptable than others. This should be in accordance with the development of non-binary truth systems in modern logic, and, presumably, with their use in "natural" thinking.

METHOD In the present, quite exploratory experiment, classical syllogisms have been modified t o be presented in the following way (example translated from French):

Syllogistic r e a s o n i n g

123

Among the 100 As, exactly 70 are also B's. Among the 100 B's. exactly 30 are also C's. Consequently, among the 100 As, I guess that

..... are also C's.

The subjects were required to write down their "best estimate". This formulation (which actually is a result of various preliminary attempts) has the shape of a syllogism of the first figure: A --- B B --- C

-----------

A --- C

except that the minor is in the first position and the major in the second (an ordering also preferred by Johnson-Laird & Bara, 1984). To avoid further complexity at the present time, it was decided to ascribe a unique constant size (t = 100) to the sets A. B, and C. and to use only the following quantities in the premises: 0, 30, 50. 70. and 100. Thus, "0 A s are B's" is equivalent to an E-proposition (disjunction), and "100 As are B's" is equivalent to an A-proposition referring to a n identity. The other three numbers generate propositions which are similar (but not equivalent) to I-propositions referring to an intersection. Furthermore, when, say, "Exactly 70 As are B's", it may be inferred that "Exactly 30 A's are not B's", which is similar (but not equivalent) to an O-proposition; therefore, we can dispense with negative premises in the present experiment. If m is the quantity used in the first premise (the minor) , and n the quantity used in the second (the major), using five quantities allows one to construct man = 25 combinations. However, when these two numbers differ, it was decided to always put the largest in the first premise: this allows for only 15 syllogisms. Among these 15 problems, only the five using the quantity 100 (in the first premise) may be seen as equivalent to classical "valid" syllogisms (problems 100/100, 100/70, 100/50, 100/30, and 100/0), the right answer being n. As to the other ten problems, we clearly confronted our subjects with a n "impossible" task, because the information they received was really insufficient to impose a unique solution. But this also characterizes the so-called "invalid" classical syllogisms. The aim of this experiment was precisely to observe how the subjects could cope with such a situation of uncertainty. After having written down their "best estimate", the subjects were asked to express their judgment about its truth value. From

124

J . Costermans

& V.

Heuschen

preliminary attempts, it appeared that subjects have some trouble with probabilities when facing an invalid problem: the probability for a guess to be correct is difficult to conceive of when there actually exists no "correct" solution. Therefore, it was decided to ask the subjects for their confidence level, by estimating to what extent they felt their reponse could be "justified". or was rather provided "at random". They had to express this evaluation by placing a vertical marker somewhere on a continuous line, presented as follows: "My estimate is: random

justifiable ."

Furthermore, to facilitate the interpretation of their responses, the subjects' comments were recorded. After the whole test had been completed, the subjects were invited to explain the rationale of their responses, as f a r as possible. The subjects were 45 students in their last year of secondary school (17-18 years old), and included 24 boys and 21 girls. They were taking various teaching courses, including mathematics and sometimes probability theory, but never formal logic. They received the 15 problems in random order, a different one for each subject.

RESULTS Among the 45 subjects, three refused to participate in the experiment in the proposed way. One provided correct answers to the five valid problems, and refused to complete the other ones, arguing that information was deficient. The other two also refused to give any precise number as their "best estimate" for the invalid problems, but they proposed a margin. For one of them, who was absolutely confident in his responses, this margin indeed corresponds to the exact solution, which may be summarized as follows: x = n & (t - m) (Strategy I) The maximum number of As which may be C's is derived through the following rationale: (a) since n -< rn , the n B's which are C's may also be As, and (b) the ( t - ml As which are not B's may be C's; thus, a t most, n + (t - m) A's may be C's. Regarding the minimum value: (c) among the n B's which are C's, there are at most ( t - m) B's which are As, so, n - ( t - m) B's which are C's must also be As: (d) the (t - rn) As which are not B's must not be C's: thus at least n - (t - m) As must be C's. This rationale may be visualized

125

Syllogistic reasoning

through some "mental model" of the Johnson-Laird style (JohnsonLaird & Bara, 1984); Figure 1 shows such an illustration, applied to problem 70/50. The responses provided by Strategy I to the 15 problems are listed in Table 1 . The other subject, who was slightly less confident, followed a similar rule, b u t made three errors in his maximum estimate (he wrote 100 instead of 80. 60, and 80 for problems 70/50,70/30,and 50/30).From his comments it appears that he temporarily reversed A and C in the surface formulation of principle (a) above, which becomes: (a') the B's which are As may also be C's. Applying this rule leads to: x(rnaxl= rn + (t - ml = t = 100. Maximum value: n

+ It-m) = 80 C C

a

a a

B B B B

a a

C

b b b

b b b b b b b

C C C C

C C

Minimum value: n a a a a b a b a b a b a b a b la b b b b

-

(t-ml= 20

C C C

C C C C

C C C

Figure 1. The correct margin for problem 70/50,as illustrated by a Johnson-Laird style "mental model". Each lower case letter (a. b. c) stands for 10 items of sets A. B, and C. In order to characterize this error and the other errors below, it may be useful to distinguish the main steps involved in syllogistic reasoning, as postulated by most models, and to adapt them to the present situation. The main steps could be listed as follows:

12 6

J. Costermans

&

V. Heuschen

Step 1: Translate the linguistic form of the premises into semantic representations. Step 2: Construct a n integrated "mental model" of the whole situation. Step 3: Derive some verbal formulation of the strategy to be followed. Step 4: Translate this strategy into a set of arithmetic operations.

Once a n appropriate strategy has been derived as to the first problems encountered, it may be generalized in the form of a verbal rationale (step 3) or of an arithmetic formula (step 4). and applied as such to the subsequent syllogisms. Although quite schematic, this classification of the involved processing levels may serve as a guide for a better understanding of what was going on. Thus, the mistake described above clearly appears to have occurred at step 3 in its generalization phase. Among the 42 remaining subjects, who tried to perform the task as required, seven appeared to provide disparate responses, which they were unable to justify: these subjects expressed very low confidence in their responses. Most of the others proved to have followed some more or less coherent way of reasoning. To fill in the gaps in the information contained in the premises, they chose t o add one or more principles of their own (which presumably are inserted at step 2). We shall focus now on the nature of these principles. An large group of 22 subjects, who often were very confident in their mode of processing, devised a strategy resting upon a postulate of independence between events: "for a n A to be B" and "for a B to be C" were two events viewed as independent of each other. From there, the subjects reasoned in the following way: the chance that a B is a C (which is n/t) is the same whether that B is an A or not: therefore, the proportion n / t should be applied to the m B's which are As (or, synonymously, to the m As which are B's). The advocated computation is thus:

x = mw/t

(Strategy 11)

(See Table 1 for the responses predicted by this strategy.) I t must be mentioned that two subjects applied this strategy only for a part of the problems: another used it, but gave only approximations (step 4 seems deficient): three more subjects did not use it when n = 0,

Syll o g i s t i c r e a s o n i n g

127

favoring Strategy IV (see below) in this case. Finally, one subject used a simplified version of Strategy 11:

x = m.1/2

(Strategy 111)

in which it is clear that the information provided by the second premise was not decoded (step 1). or at least not integrated (step 21, although this subject expressed maximal confidence in his

responses. Instead of viewing the events as independent, 6 subjects seem to have considered that an A may not be a B and a C at the same time (5 subjects), or that a B may not be an A and a C (1 subject). Such intersections are thus viewed as mutually exclusive events. Among the former 5 subjects, 3 consider that "All the As which are not B's must be C's", which gives rise to Strategy Iv: x=t-m

(Strategy IVl

Another subject explained that "All the As which are not B's may be C's", and proposed the above value as a maximum: x I t - rn While Strategy I1 yields correct responses to the valid problems, Strategy IV (and also Strategies V and VI below) does not: on the contrary, as it appears in Table 1, it predicts quite aberrant estimates, which the subjects appear to defend with great confidence. Furthermore, it is clear that Strategy IV, like Strategy 111, fails to make use of the information provided by the second premise. Another subject, instead of assuming that all As which are not B's must (or may) be C's, postulates that (a) among the As which are not B's, there is a proportion which are C's, and (b) that the probability for one of these As to be a C is the same as the probability for a B to be a C: thus: x = (t - m)*n/t

(Strategy V)

Although this subject is quite confident, his strategy gives 0 as the response to all the valid problems. Finally, as mentioned above, one subject considers that a B may not be an A and a C a t the same time. His strategy may be summarized as: x=t-n

(Strategy VI)

J. Costermans

128

&

V. Heuschen

and fails to integrate the first premise. But this subject expressed very poor confidence. Out of our 45 subjects, the last 8 appear to have applied two or more of the above strategies. When rn differs from n, it happens that some of them apply the rule: x = rn - n; and when rn = n , one sometimes observes the rule: x = rn = n. One subject applied the rule: x = 150 - (rn + n) to most problems. These subjects had much difficulty justifying their responses, which seem indeed to rest on processing of the premises that hardly goes beyond step 1.

Table 1. Responses predicted by various observed strategies. Strategy I: x = n +(t- m)

Strategy 11 :x = m*n/t

m D Z Q Z 2 3 Q

uLQlOo70 50 30 zn 40-10020-80 0-60 0-100 0-80 M 0-100 a2

-0

0 0-30 0-50 0-70 0-100

Strategy I I l x = m*1/2

.a

35

50 a2 Q

35 25

5 0 5 0

35 25 15

35 25 15

O

Strategy V x = (t - m)*n/t 5Q

3

Q

70

15 25

0 9 15 21

0 0

50

o

o

21

332 Q

0

La 4

70 49

-

a 50

a2 30 21 15 9

35 25

Q

0 0 0 0 0

m L n l l l Q Z Q a 3 Q JQQ ZQ 50

0

0 30

a

0

30 50

0 30 50 70

Q

Q

0 30 50 70 100

Stratem VI: x = t - n

mLnuMZa

m

a

100

Strategym x = t - m

L u L R L Q Q Z Q ~ Q ~ Q m505050

mLn4QQZQ

mLn4QQ

ZQ Z2 3Q lQQ 0 3 0 5 0 7 0

70

0 0

a

O

Q

3

30

50 50

70

70 70

Q

100 100 100 100 100

CONCLUSION As a whole. most of our subjects seem to have coped with their "impossible" task in a rather acceptable way: they actually integrated

Syllogistic r e a s o n i n g

129

the available information (step 2). even if it meant inserting at this level some complementary links of their own to make such integration possible. In most cases, such postulates seem plausible as regards "natural" thinking (i.e. they may seem appropriate in many everyday situations). Some subjects, however, were rather insufficient at step 2 because they neglected to integrate the information provided by one of the premises (generally the second one). Some subjects did not go beyond step 1, and tried to find a way of computing on the basis of some numerical cues present in the premises. A few others, who integrated fairly well, had trouble with the formulation of their rationale (step 3). or simply made computational errors (step 4). A remarkable feature of our observations is that most subjects, through their confidence ratings and further comments, proved to be convinced that their postulates were right and that they were the only way to solve the problems. This conviction leads some of them to defend aberrant responses to the valid problems. One of the encountered strategies (Strategy 11) dominates the others, since it was observed in 50% of the subjects. This strategy can also be expressed in a different way:

x = m/t*n/t*t which is clearly equivalent but might correspond more closely to what happens psychologically (we are indebted to a n anonymous reviewer for that suggestion). Indeed, subjects might adopt a simple multiplicative rule: thus, if the first proportion is 70% and the second 30%, subjects multiply these together to arrive at the answer of 21%. Such a reformulation of the strategy is not trivial. In fact, other research on verbal quantifiers has also hinted at such a rule (Ekberg & Lopes, 1979a. 1979b. 1980; Newstead, Pollard, & Griggs. 1986). This may prove to be a fairly general strategy for combining information. It may be appropriate to compare what we have done with experimentation on classical syllogisms. I t may be that strategies like those described above are effective in the latter case, but they will be very hard to detect. Indeed, the majority of our invalid problems correspond to the following classical syllogism: Some As are B's, Some B's are C's. for which we know from previous experiments that subjects tend to accept the conclusion: Some As are C's. When evaluated in binary truth values, this conclusion is false. But such a problem is poorly sensitive to different strategies.

J . Costerrnans & V . Heuschen

130

This is not the case when numbers are used, because different strategies will then predict different response structures. Some further research could be devoted to problems where rn is smaller than n, or where the sets are of unequal size, or to problems including negative propositions, or corresponding to other syllogistic figures. Indeed, the list of strategies outlined above is assumed to be far from exhaustive, and further insight into them could shed some light on the biases affecting current reasoning processes. REFERENCES Bucci,

v.

(1978). The interpretation of universal affirmative propositions. Cognition. 6, 55-77. Ceraso, J., & Provitera, A. (1971). Sources of error in syllogistic reasoning. Cognitive p W c h o l o ~2. , 400-410. Chapman, I.J.. & Chapman, J.P. (1959). Atmosphere effects reexamined. Journal of ExDerfmental Psvcholo&, 220-266. Dickstein. L.S. (1978). Error processes in syllogistic reasoning. Memorv a n d Comition. 6, 537-543. Ekberg. P.-H.S.. & Lopes, L.L. (1979). Syllogistic reasoning: A review. EUteborg ESvchological ReDoa .9(1). Ekberg, P.-H.S.. & Lopes. L.L. (1979). Tests of a natural reasoning model for syllogistic reasoning. GUtebog PsvcholQg&d ReDorts, 9(3). Ekberg. P.-H.S., & Lopes, L.L. (1980). Fuzzy quantifiers in syllogistic reasoning. GUtebore Psvchological ReDorts. u ( 6 ) . Erickson, J. (1974). A set analysis theory of behavior in formal syllogistic reasoning tasks. In R.L. Solso (Ed.), Theories in coenitive Dsvchology. Hillsdale. N J : Erlbaum. Guyote. M.J..& Sternberg. R.J. (1981). A transitive-chain theory of syllogistic reasoning. Cognitive Psvcholoa, 13.461-525. Johnson-Laird, P.N.. & Bara, B. (1984). Syllogistic inference. Cognition. 16, 1-61. J u s t , M.A.. & Carpenter, P.A. (1971). Comprehension of negation with quantiflcation. Journa 1 of Verbal Learning and Verbal Beha244-253. Newstead. S.E.. & Griggs. R.A. (1983). Drawing inferences from quantified statements: a study of the square of oppositions. Journal of Verbal Learning gnd Verbal Behavior. 22, 535-546. Newstead. S.E., Pollard, P.. & Griggs, R.A. (1986). Response bias in relational reasoning. Bulletin of the Psvchonomic Society, 95-98. Revlin, R., Ammerman, K.. Petersen, K.. & Leirer V. (1978). Category relations and syllogistic reasoning. JQumal of Educational PSVChOlO~. 613-625. Revlin. R. & Leirer. V. (1980). Understanding quantified categorical expressions. Memorv and Cognition. 8.447-458. Revlis. R. (1975). Two models of syllogistic reasoning: feature selection and conversion. Journal of Verbal Leamine and Verbal Behavior. 14. 180-195. Sells, S.B. (1936). The atmosphere effect: an experimental study of reasoning. &chives of Psv3-72. Tsal, Y. (1977). Symmetry and transitivity assumptions about nonspecified logical relations. Quarterlv Journal of Emerimental Psvcholom. 29.677-684.

a,

.u,

a,

a,

,a.

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e a n d M . G o n z a l e z ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B.V. ( N o r t h - H o l l a n d ) ,

1990

BELIEF BIAS AND PROBLEM COMPLEXITY IN DEDUCTIVE REASONING Jonathan St. B. T. Evans* and Paul Pollard*+ *

P o l y t e c h n i c S o u t h West, P l y m o u t h , E n g l a n d

**

Lancashire P o l y t e c h n i c , England

Abstract. Two experiments are reported in which subjects are asked to assess the validity of deductive arguments whose conclusions vary in prior believability. Both experiments demonstrate a "belief bias" effect, in which subjects tend to rate conclusions a s more valid when they conform with prior beliefs. This finding is generalised over several procedural variations relative to previous research, including use of attitudinal rather than factual beliefs and between versus within subject design. The research also provides some new results of relevance to the interpretation of the belief bias effect. In both experiments the logical complexity of the arguments was manipulated. Surprisingly, while more complex arguments significantly reduced the numbers of inferences that subjects were able to generate from the premises. no increase in the extent of the belief bias effect was observed as a result. Each experiment also introduced belief neutral materials and found significant evidence only for a negative belief bias, 1.e. unbelievable conclusions were rejected more often than neutral or believable ones. These findings are discussed with reference to alternative theoretical models of belief bias.

A deductive reasoning task is one in which people are asked to judge the logical validity of a conclusion on the basis of certain premises. An argument is valid provided that the truth of the premises entails the truth of the conclusion. An argument is invalid so long as its conclusion need not necessarily be true, given the truth of its premises. Hence, a valid argument may lead to a false conclusion, if at least one premise is false. Equally, the conclusion may be true even though the argument lacks logical validity. Of the numerous claims of bias in reasoning, "belief bias" is one of the oldest, dating from the work of Wilkins (1928). The claim is that when subjects hold a priori beliefs about the truth of a conclusion, this will bias their assessment of an argument's validity. Specifically, they will tend to say that arguments with apparently true conclusions are valid and those with false conclusions invalid, regardless of their logical relation to the premises. A number of reports of belief biases are scattered through the literature, mostly employing problems based on syllogistic logic, although many of these earlier studies suffer from methodological flaws (see Evans,

132

J. E v a n s

&

P. Pollard

1982. pp. 107-111; Barston, 1986, Ch. 4). There has, however, been a recent resurgence of interest in the phenomenon leading to the report of better designed studies. A study by Revlin. Leirer, Yopp and Yopp (1980) found significant evidence of belief bias, although the authors claimed it was weak in relation to the influence of logic. Evans, Barston and Pollard (1983) argued that Revlin et al. might have underestimated the bias due to aspects of their methodology and went on to demonstrate substantial belief bias effects in three experiments of their own. Evans et a1 also observed a strong tendency for subjects to take account of the logic of the problems and for an interaction to occur between the two factors. Specifically, belief bias was substantially more marked on problems which were logically invalid as opposed to valid. Evans et al. proposed two main alternative models to explain the interaction between logic and belief in their experiments which were later named as the Selective Scrutiny Model and Misinterpreted Necessity Model by Barston (1986) and are given detailed discussion by Evans (1989). The two models are shown schematically in Table 1. The Selective Scrutiny model assumes that subjects first assess the believability of the conclusion. If it accords with prior belief then there is a strong tendency to accept it without any logical analysis of the premises. If, on the other hand, the conclusion is unbelievable, they are more likely to check the logic before making a decision. The uncritical acceptance of believable conclusions causes error when the arguments are invalid and thus results in the more marked belief bias observed on such problems. The Misinterpreted Necessity Model is derived from the work of Dickstein (1980, 1981) who presented evidence that subjects have difficulty understanding the concept of logical necessity. Specifically, they fail to realise that a conclusion is only valid if it necessarily follows from the premises -- it is not sufficient for it to be merely consistent with them. The invalid arguments used by Evans et al. did not produce conclusions which actually contradicted the premises. Specifically. this model makes the contrary assumption to the Selective Scrutiny Model that the conclusion is first evaluated with respect to its logical relation to the premises. If it can be shown to be definitely true or definitely false then the conclusion is accepted or rejected accordingly. In the event that the logical analysis does not produce a determinate outcome then

Belief bias and problem complexity

133

the subject makes a decision based on its prior believability. This model also accounts for the interaction observed by Evans et al. since invalid problems -- which in their design failed logically to determine the truth of the conclusions -- would be more liable to belief bias. Table 1. Two Models of Belief Bias (adapted from Barston. 1986). A. SELECTIVE SCRUTINY MODEL I s conclusion believable? If YES then ACCEPT CONCLUSION If NO then Is conclusion necessitated by the premises? If YES then ACCEPTCONCLUSION If NO then REJECTCONCLUSION

B.

MISINTERPRETED NECESSITY MODEL

Is conclusion falsified by the premises? If YES then REJECTCONCLUSION If NO then I s conclusion necessitated by the premises? If YES then ACCEFTCONCLUSION If NO then Is conclusion believable? If YES then ACCEPTCONCLUSION If NO then REJECTCONCLUSION

The first model assumes conclusion centered or belief-first reasoning in which the asymmetry is between the treatment of believable and unbelievable conclusions. The second assumes premise centered or logic-first reasoning in which the asymmetry arises in the treatment of determined a n d non-determined conclusions. Evans et al.favoured the former model on the basis of an analysis of think aloud protocols collected in their experiments. They found that belief bias was more marked on problems in which subjects referred only to the conclusion, or to the conclusion prior to mentioning the premises than on problems where the premises were mentioned before the conclusion. The Selective Scrutiny Model was also favoured by Barston (1986) in the light of several

134

J. E v a n s

&

P. Pollard

experiments in which some groups of subjects were given intensive instruction in the principle of logical necessity. Contrary to the predictions of t h e Misinterpreted Necessity Model, s u c h instructions did not generally reduce either the extent of the belief bias effect or its interaction with validity. The notion that people are more likely to attempt to evaluate arguments whose conclusions they do not agree with is also supported by some research on belief maintenance in social psychology. For example, Lord, Ross and Lepper (1979) asked subjects to evaluate the methodology employed in imaginary research studies whose results concorded or conflicted with their own prior beliefs. Subjects tended to criticise the study whose results they disagreed with regardless of the methodology ascribed to it, which was counterbalanced in the experiment. There was also evidence that this tendency was based on cognitive, rather than motivational factors. Hence, the Selective Scrutiny Model is supported by considerations of cognitive economy: why work hard assessing the evidence for claims that one already believes? Oakhill and Johnson-Laird (1985) have applied the theory of reasoning based upon manipulation of mental models (cf. JohnsonLaird, 1983) to the explanation of belief biases. They demonstrated belief biases on problems where no conclusion was presented for evaluation and subjects were asked to produce them spontaneously. At first sight this might seem to support the premise centered model, though the mechanism proposed by the authors is more similar to our belief-first model. In the mental models theory, people are assumed to assess validity by constructing alternative models of the world which are compatible with the premises. in order to ensure that the conclusion always holds: a method which checks for logical necessity. Oakhill and Johnson-Laird propose, however, that if a conclusion is generated which is congenial with prior beliefs subjects may be less inclined to carry out this exhaustive checking for counterexamples than if it conflicts with belief. There are also indications that the belief bias effect is weaker when no conclusion is presented for evaluation. A set of experiments by Barston (1986), using similar materials to those of Evans et al. (1983). failed to find any evidence of belief bias on a conclusion production task. In fact, Oakhill and Johnson-Laird also failed to observe the effect using problem materials of the same kind, in which general beliefs are involved -- e.g. "All police dogs are highly trained." The effect was only observed on problem materials

Belief bias and problem complexity

135

whose conclusions were true or false by definition, e.g. "Some of the millionaires are not rich." The experiments reported here introduce a new variable of relevance to the two models of belief bias shown in Table 1, namely the logical complexity of the arguments. Clearly,there is a logical component of performance in the experiments referred to above, in that subjects are frequently able correctly to distinguish valid from invalid conclusions. If we are able to manipulate the complexity of the logical structures to make them relatively easy or difficult then we would naturally expect the frequency of logical errors to change accordingly. In this event, however, the two models will make different predictions about the implications for the belief bias effect. Consider first the Selective Scrutiny Model. This model assumes that belief bias occurs due to uncritical acceptance of believable conclusions prior to any logical analysis of the premises. Hence, the size of the belief bias effect should be essentially independent of the difficulty of the logic. Of course, the subsequent logical analysis -- if it takes place -- will be influenced by logical complexity, b u t any errors are assumed to be random and not systematically related to beliefs. On the other hand the Misinterpreted Necessity Model makes the clear prediction that as logical complexity and consequent errors increase, then the belief bias effect should correspondingly become more marked. This is because this model views belief bias as a heuristic which is resorted to whenever the prior logical analysis fails to achieve a definitive result -- evidently more likely with complex problems. I t appears common sensical that complexity and belief bias would trade off against one another such that easy problems would be less susceptible to the bias. Hence, the Selective Scrutiny Model -- favoured by the authors in view of other evidence discussed above -- generates the counter-intuitive prediction that belief bias will be independent of complexity. In addition, Experiment 1 involves a useful generalisation of the belief bias effect across a number of procedural variables. First of all, Experiment 1 tests the generality of the effect across problem materials. Although the materials used by Evans et al. (1983) were weaker than definitionally true or false statements, they were nevertheless based on matters of general belief or opinion. The problems used here were based upon group specific attitudes, namely stereotyped opinions about the likely leisure pursuits of students, as a function of their chosen course of study.

J. E v a n s

136

&

P. Pollard

Prior beliefs were taken from a separate study of psychology undergraduates conducted by Pollard and Sumiga (1983). In addition, the construction of problem materials is extended from those used in the studies cited above to include believable, unbelievable and neutral materials. It is not clear from earlier studies whether believable conclusions facilitate valid judgements, or whether unbelievable conclusions inhibit them -- or both. The introduction of neutral conclusions allows both positive and negative aspects of belief bias to be investigated. The semantic relations chosen to construct the three types of problem are shown in Table 2. In contrast with other recent studies, Experiment 1 also employed a between rather than within subject manipulation of the believability of problem content. Demonstration of belief bias with this design is not then open to criticism on the grounds that subjects may perceive the purpose of the experiment by noticing the main independent variable. In addition, the experiment generalises earlier findings by employing deductive problems which deviate in their structure from the classical syllogism and invite solution by a method known as "conditional proof' [see Method below).

Table 2. Empirical semantic relations used in Experiments 1 and 2. Belief Value

Subject of study

Leisure interest

Believable

Maritime studies Sociology Architecture Mechanical engineering

Sailing Left wing political activity Artistic pursuits Car maintenance

Neutral

Environmental sciences Electrical engineering Physical science Architecture

Football Reading Sailing Pacifist campaigns

Unbelievable Social work Electrical engineering Maritime studies Sociology

Home electronics Meditation Artistic pursuits Right wing political activity

Belief bias and problem complexity

137

EXPERIMENT 1

Method

Design Subjects were divided into six experimental groups, by manipulation of two factors: Believability (Believable, Neutral, Unbelievable) and Complexity (Simple. Complex). Each subject received four reasoning problems as follows: Valid-True (VT):

A valid inference that the conclusion stated was true was permitted by the premises. Valid-False (VF): A valid inference was again permitted, but this time entailed the falsity of the conclusion. Invalid-True (IT): No valid inference was possible, but employment of a plausible fallacy would entail the truth of the conclusion. Invalid-False (IF): In this case employment of the fallacy entailed the falsity of the conclusion, though no valid inference was possible.

Materials The semantic relations used to construct the problems were those shown in Table 2, while the logical structures used are shown in Table 3. In each case, P represents a category of students defined by chosen subject areas, R the leisure activity and Q an arbitrary linking term concerned with society membership. A n example of a Complex, Unbelievable, Valid-True problem is: 1) No members of club A are social work students.

2) All students that do NOT have home electronics as one of their leisure activities are members of club A. The conclusion to be evaluated would be: "ONE OF THE LEISURE ACTMTIES OF SOCIAL WORK STUDENTS IS HOME ELECTRONICS." To understand the proofs of these problems one must first note that there are four basic inferences which can be drawn from logical implications, two of which -- Modus Ponens (MP) and Modus Tollens (MT) -- are classically considered to be valid, and two of

J. E v a n s

138

&

P. Pollard

which -- Affirmation of the Consequent (AC) and Denial of the Antecedent (DA) are considered fallacious (see Table 4). Experimental studies of reasoning with conditional sentences of the form "If p then q" -- normally regarded a s expressing a logical implication -- suggest, however, that endorsement of all four inferences by experimental subjects is quite common (see Evans, 1982, Ch. 8). Table 3. The logical structures employed in the two experiments.

SIMPLE LOGIC Valid-True IVT) P --> Q Q --> R P --> R

Invalid -True (IT) P --> Q R --> Q P --> R

Valid-False (VF) P --> Q

Q -->"R -----------P --> R Invalid-False (VF) P --> Q "R - - > Q ------------P --> R

COMPLEX LOGIC Valid-False (VF) Q - ->"P

R --> Q P --> R Invalid-True (IT) Q -->"P

Q -->"R _-----------P --> R Notes:

Invalid-False (IF) Q - ->"P Q --> R -----------P --> R

(1) P - -> Q means ' P implies 9' etc. (2) "P means 'not P' etc.

B e l i e f b i a s and problem c o m p l e x i t y

139

Table 4. Four inferences associated with material implication.

Modus Ponens (MP) P - -> Q. P ... Q

(Valid)

Denial of the Antecedent IDA) P - -> Q, "P .. Q (Invalid) Affirmation of the consequent (AC) P - -> Q, P :. Q (Invalid) Modus Tollens (MT) P - -> Q, "Q .. -P

(Valid)

The rule known as Conditional Proof (CP) states that if by assuming the truth of one proposition (say p) one can derive another (say q) then the implication p --> q may be drawn as a conclusion. In the case of the above argument, the validity of the conclusion can be demonstrated by CP together with two MT inferences as follows: Given Social Work students It follows by MT from premise 1) that they do not belong to club A. (iii) Hence by MT from premise 2) they must have home electronics as a leisure activity.

(i) (ii)

CONCLUSION: One of the leisure activities of Social Work students is home electronics.

Assume P Infer -Q Infer R

P --> R

In the corresponding Valid-False problem -R is inferred at step (iii) leading to a determinately false conclusion, i.e. the derived conclusion (P --> -R) directly contradicts the one to be evaluated (P --> R). The Complex Invalid problems are similar except that the truth value of the conclusion can only be inferred by application of an invalid DA inference at step (iii). On Simple Valid problems the conclusion can be deduced by two steps of MP, while on Simple

140

J. Evans

&

P. Pollard

Invalid problems an MP and an AC inference are required to generate deductions. The rationale for the Simple/Complex classification rests upon previous research which shows that subjects more often endorse MP than MT. and more often endorse AC than DA (See Evans, 1982, Ch. 8). By this definition simplicity refers to the likelihood that the subject will be able to generate a deduction, whether it is valid or not. The point is that we wish to distinguish inferences drawn from the premises (regardless of formal validity) from direct evaluations of conclusions. The Simple problems are ones where such inferences should be relatively easy for subjects to generate. The problems employed a response task which is unusual in modern studies of deductive reasoning and hence requires some comment and justification. Instead of being asked simply to judge whether the conclusion presented was true, false or indeterminate in the light of the evidence of the premises, subjects were given five choice categories to allow "probably" true and false as well as "definitely" true and false choices. This may seem odd since in logic a conclusion is either necessitated by the premises or not. However, the purpose of the study is to study conflict between logic and belief which we expect to cause uncertainty in the subject's mind in many cases. Hence, the provision of response categories including uncertainty may add to the sensitivity of the measurement. Subjects are, of course, under no obligation to use the intermediate responses.

Subjects Sixty-six first year undergraduate psychology students a t Plymouth Polytechnic served as subjects, 1 1 in each group. They were sampled from the succeeding year to the students who had been used for the attitude test conducted by Pollard and Sumiga (1983)when in their first year. Procedure Subjects were group tested and given a booklet with a n instruction page followed by four further pages each presenting a separate problem. The instructions were as follows: This experiment is designed to test people's reasoning ability on a number of separate problems. On each problem you will be given some information about a different polytechnic student, including their main

Belief bias and problem complexity

141

subject of study. The other information concerns the polytechnic clubs that the student may join, although these will be labelled with arbitrary letters. The clubs are connected with students' leisure activities in ways that will be indicated. You must assume for the sake of the exercise that all this information is true. The problem is then to assess the truth of a test statement which MAY OR MAY NOT BE TRUE. The truth of this statement must be judged solely on the basis of the information presented. You will be given five choices, abbreviated as follows: DT Definitely True PF Probably True I Indeterminate PF Probably False DF Definitely False A choice of "indeterminate" means that t h e information presented gives no indication about the truth value of the statement. Please work through the problems in the order presented. Do not refer forwards or backwards, or change a n answer once you have left one problem and gone on to the next.

Note that the logically correct answers, assuming the validity of component inferences as shown in Table 4, would be DT for Valid-True problems, DF for Valid-False and I for all Invalid problems. The PT and PF response categories are included to permit subjects to express degrees of uncertainty in their own judgements and do not correspond to a "correct" answer on any problem. Results

For purposes of data analysis subjects' responses were assigned a score of +2 for definitely true, + 1 for probably true, 0 for indeterminate, - 1 for probably false and -2 for definitely false. The mean ratings are shown in Table 5. The first matter of interest is to see whether a belief bias is exhibited despite the various generalisations of the procedure described in the introduction. It can be seen from Table 5 that the overall mean ratings are in the predicted direction, dropping from

J. Evans

142

&

P. Pollard

Believable to Neutral to Unbelievable, as are the sub-means for Simple and Complex problems considered separately. An analysis of the trend on the combined data for Simple and Complex problems was found to be highly significant using the Kendall S test for between group trend (z = 3.24.p < 0.001. 1 tailed). In order to test for positive and negative aspects of belief bias, the ratings of Neutral problems were compared with t tests to each of the others in turn. Unbelievable problems had significantly lower ratings (t = 2.299. p c 0.025) but the difference between Neutral and Believable conditions fell short of significance (t = 1.269). Table 5. Mean truth ratings in Experiment 1 .

BELIEVABILITY Believable

Neutral

Unbelievable

Mean

1.91 -1.18 0.82 0.64

1.54 -2.00 0.82 -1.00

1.18

1.54 -1.48 0.61 -1.00

0.23

-0.16

-0.31

-0.09 -0.55 0.46 -0.46

0.00 -0.82 0.64

-0.55

-0.18 -1.46 0.36 -0.82

-0.16

-0.18

-0.53

0.03

-0.17

-0.42

SIMPLE U X I C

VT VF

IT IF Mean

- 1.27 0.10

- 1.36

COMPLEX LOGIC

VT VF

IT IF Mean

OVERALL MEAN

-0.09 -0.94 0.49 -0.61

The next matter of interest is whether or not the extent of belief bias interacted with the complexity of the problems. First, however, it is necessary to establish that the complexity manipulation was effective. Inspecting the right hand column of

Belief bias a n d problem complexity

143

Table 5, one can see a strong general tendency for subjects to rate True conclusions more positively than False conclusions, regardless of validity, with a more marked difference for Simple than Complex problems. Since subjects showed a tendency both to rate True conclusions positively and False conclusions negatively a combined test of these trends was made to test the effectiveness of the Complexity manipulation. This was done by reversing the sign of ratings on False problems and adding scores across problems for each subject. (False problems are ones whose conclusions are falsified by the inferences suggested by their premises, so a negative rating is regarded as according with logic -- or subjective logic in the case of Invalid problems.) I t was found that the ratings in favour of the truth value suggested by the premises (validly or invalidly) was highly significantly larger on Simple than on Complex problems (t64 = 4.435, p < 0.001). Hence, subjects did indeed more readily make inferences from the problems classed as Simple. Since both Believability and Complexity produced the expected effects, we can test the hypothesis that belief bias will have a more marked effect on Complex than on Simple problems. Inspection of Table 5, however, reveals that the range between the means on Believable and Unbelievable is 0.54 for the Simple group, and 0.37 in the Complex group. which is against the direction predicted by the Misinterpreted Necessity Model. Strictly speaking no statistical comparison should therefore be made, but it was felt important to establish that there was no indication of any difference here. To do this, data from the Neutral groups were dropped, and all other subjects were coded in terms of their rating in favour of belief. Thus the sign of the ratings was reversed for the Unbelievable groups and then combined with the data of the Believable groups. Hence Simple versus Complex groups of 22 each were compared on degree of belief bias. The comparison fell well short of the significance level appropriate for a two tailed test (t42 = 0.518). There is, then, clear evidence that the extent of belief bias was not increased by the presence of more complex problems and no indication of a significant trend in the reverse direction.

Discussion Experiment 1 has successfully demonstrated that the belief bias effect generalises across a number of variations in procedure, including between rather than within subject design, attitudinal

144

J. E v a n s

&

P. Pollard

materials. non-standard syllogisms and so on. The introduction of a Neutral problem condition produced intermediate ratings between Believable and Unbelievable. Although the difference between Neutral and Believable group ratings fell short of individual significance, the findings suggests that there may be both positive and negative aspects to belief bias. Experiment 2 permits further investigation of this variable. On Simple problems, as predicted, subjects made more polarised ratings of the conclusion in the direction (true or false) suggested by the logic of the premises. Logical validity is not really relevant here since we know that the "fallacious" DA and AC inferences are frequently endorsed when presented in isolation and thus have subjective validity. The point is that the drawing of inferences from the premises, whether logically valid or not, was inhibited on problems classed as Complex. According to the Misinterpreted Necessity Model we would have expected more belief bias with such problems, because this model sees belief bias as reflecting a heuristic resorted to when deduction fails. However, there was no significant difference in belief bias between Simple and Complex groups, and the mean difference was against the predicted direction. The lack of interaction between complexity and belief bias would seem to provide further support for the Selective Scrutiny Model. However, since we need to be cautious in making inferences from negative results it was decided to conduct a further, and stronger test for the presence of this interaction in Experiment 2. EXPERIMENT 2

Although the effect of belief bias and logical complexity were both significant in Experiment 1 , it is arguable that neither manipulation was a strong as it might have been. For example, the size of the belief bias effect appeared smaller than that observed in most previous studies, and may have resulted in a n insufficiently powerful test of the hypothesis that there is both a positive and negative aspect to belief bias (only the latter was significant). There are two possible reasons for this. Firstly, the use of a between subject manipulation may have reduced the salience of the belief manipulation -- suggesting, perhaps, that the effect might result in part from the demand characteristics of the within subject design. Secondly, the attitudinal material concerning student stereotypes

Belief bias and problem complexity

145

may have had a relatively weak effect in inducing differential believability of statements. As mentioned previously, Oakhill and Johnson-Laird (1985) reported more powerful belief bias effects using materials that were definitionally rather than empirically true or false. In Experiment 2, believability was manipulated as a within subject factor and a n extra condition was introduced using definitionally as opposed to empirically true and false statements. Whilst half the subjects received materials of the form used in Experiment 1, t h e others were presented with problems concerning the classification system of a n eccentric naturalist, leading to inferences about the classification of animals. The Believable, Unbelievable and Neutral materials are shown in Table 6 . All conclusions were expressed in universal form, e.g. "All crocodiles are reptiles" or "All snakes are mannites." Note that the neutral problems were constructed by assigning invented nonsense category names. Table 6. Definitional semantic relations employed in Experiment 2.

Belief Value

Animal

Classification

Believable

Robin Crocodile Dog Herring

Bird Reptile Mammal Fish

Neutral

Horse Chaffiich Trout Snake

Traxite Priscile Scholl Mannite

Unbelievable

Cat Sparrow Lizard Shark

Bird Mammal Fish

Reptile

146

J. Evans

&

P. Pollard

The complexity manipulation was also strengthened in Experiment 2. In Experiment 1 the "complex" problems differed from the simple ones only by the use of component inferences shown by previous research to be qualitatively different. In Experiment 2 the simple problems were the same as those used in Experiment 1, whereas the complex problems required subjects to reason with two additional premises and a total of five as opposed to three terms. The manner in which these four premise problems were constructed is described below. I t was expected that belief bias and problem complexity would significantly influence the data as in Experiment 1. The principal aim of Experiment 2 was to decide whether or not there is an interaction between the extent of belief bias and the more powerful manipulation of problem complexity used in this experiment. In addition the study should provide further evidence concerning positive and negative aspects of belief bias, and the effects of the use of empirical versus definitional materials on the extent of the belief bias effect. Method Design Subjects were divided into four experimental groups by manipulation of problem complexity (two vs four premise problems) and by use of empirically or definitionally true materials. The four groups are consequently labelled 2PE, 2PD, 4PE and 4PD. Each subject received 12 reasoning problems to solve -- produced by the manipulation of two factors: Believability (believable, neutral, unbelievable) and Logic (VT, VF, IT and IF as in Experiment 1). The presentation order of problems was randomised independently for each subject. Materials The four premise problems were constructed from the Simple problems of Experiment 1 by adding the same two premises in each case, the effect of which was to expand the P --> Q first premise of each into something derivable from the following three premises: P

-->

s

T --> " S "T --> Q

Belief bias and p r o b l e m complexity

147

By conditional proof, the assumption of P leads to S (by MP), S leads to -T (by MT.) and "T leads to Q by MP. Since the fourth premise was always the same as the second used in the Simple version, the logical evaluation of the conclusion was unaffected by these additional premises. In a n attempt to stop subjects from noticing that each problem involved the same initial s e t of inferences, the order of the four premises was randomised across problems. For consistency, the order of the two premise problems -- fixed in Experiment 1 -- was also randomised. The problems presented to subjects in Group 2PE were identical to those used in the Simple Logic condition of Experiment 1 except for the randomisation of premise order. Group 4PE received similar problems with two additional premises referring to further arbitrarily labelled student clubs. An example set of premises is as follows: 1 ) All mechanical engineering students are members of club T. 2 ) Any student who is NOT a member of club I belongs to club S . 3 ) No members of club I belong to club T. 4 ) All members of club S have car maintenance as one of the leisure

activities. The conclusion to be evaluated for the above problem was "ONE OF THE LEISURE ACTIVITIES OF THE MECHANICAL ENGINEERING STUDENTS IS CAR MAINTENANCE." The materials presented to Groups 2PD and 4PD referred to arbitrary classification classes instead of arbitrary student clubs. For example, the premises corresponding logically to the above example for Group 4PD were as follows: 1 ) All robins are members of class P. 2 ) Any animal that is not a member of class Q belongs to class R. 3 ) No members of class Q belong to class P. 4 ) All member of class R are birds.

All conclusions for the Definitional groups were phrased in a conventional universal form. For the above premises subjects would be asked to evaluate the statement, "ALL ROBINS ARE BIRDS."

J. Evans

148

&

P. Pollard

Subjects Fifty-six undergraduate psychology students served as subjects, 14 in each experimental group. None had participated previously in Experiment 1 or in the Pollard & Sumiga rating study. Procedure As in Experiment 1, subjects were group tested and provided with a booklet containing written instructions followed by a separate sheet for each problem. The Empirical groups received the same instructions as used in Experiment 1. The Definitional group instructions differed only in the second paragraph which was substituted as follows:

Each problem concerns a classification system used by an enthusiastic but eccentric naturalist. In each case you will be given some information about the way in which he classifies animals, including classes labelled with arbitrary letters. Results and discussion As in Experiment 1. subjects' ratings of Definitely True to Definitely False were coded as numbers in the range +2 to -2. The mean ratings for each condition in each group are shown in Table 7. Inspection of the means reveals that, in general, ratings decreased from Believable to Neutral to Unbelievable, in line with the belief bias hypothesis. An exception, for which there is no obvious explanation, arises in group 4PE who give highest ratings to neutral conclusions. I t can also be seen that True conclusions were generally rated more highly than False ones, regardless of the validity, although this trend appears much clearer on the simpler two premise problems, as expected. (As before, True conclusions are those supported and False conclusions those contradicted by subjectively valid inferences.) In order to investigate these trends more closely, the data were recoded in terms of a Logic Index (LI) and a Belief Index (BI). The LI was computed as the mean rating in favour of the truth status (true or false) suggested by the premises and could range. in theory, from -2to +2. In fact, only 8 of the 56 subjects produced a negative Logic Index, indicating a massively significant effect of logic as assessed by a Binomial test. The real interest, however, lies in comparison between the groups on this measure (see Table 8(1)).A

Belief bias and problem complexity

149

two way analysis of variance computed on the LI scores demonstrated that there was a highly significant difference between the 2 and 4 premise groups (F1,52 = 88.02, p < 0,001) but no effect of problem materials and no interaction between the two factors. Hence, as hoped, the complexity manipulation was highly effective in this experiment. Table 7. Mean truth ratings in Experiment 7. ~~

Believable

-

DEFINITTONAL 0.79 -0.29 0.00 0.64 0.29

0.43 -0.64 0.57 0.29 0.16

DEFINITIONAL 2.00 VF -1.79 IT 1.07 IF -1.57 - 0.07

1.79 -1.93 0.43 -1.93 - 0.41

4PREMISE

VT VF IT IF Mean 2PREMISE

4PREMISE

0.64 -0.07 -0.43 0.07 0.05

~

Mean

0.62 -0.33 0.05 0.33

1.43 -1.79

0.50 -1.86 - 0.43

1.74 -1.83 0.67 -1.79

-

EMPIRICAL 0.43 VF 0.07 IT 0.36 IF -0.57 0.07

VT

Mean

~

Unbelievable

-

VT

Mean

~

Neutral

2PREMISE - EMPIRICAL VT 1.86 VF -1.64 IT 1.07 IF -1.29 0.00 Mean OVERALL MEAN

0.07

0.57 0.14 0.79 0.43 0.48

-0.21 -0.07

0.26

0.43

0.52 -0.29

-0.71 -0.14

1.36 -1.71 1.14 -1.71 -0.23

1.57 -1.64 0.71 -1.36 -0.18

0.00

-0.17

0.05

1.60 -1.67 0.98 -1.45

J. E v a n s & P. Pollard

150

Table 8. Mean logic and belief indices in Experiment 2. (1) Logic indices (possible range -2 to +2)

2 premise

Definitional Empirical

1.51 1.42

4 premise 0.17 0.27

(2) Belief indices (possible range 0 to 3)

2 premise Definitional Empirical

2.00 2.00

4 premise

1.93 1.36

The Belief Index was scored by computing the P statistic (related to Kendall's S) for each subject. Essentially, P is a count of the number of paired comparisons of the three conditions that are in the predicted order. The maximum value is 3 (all as predicted), minimum 0 (all against predicted order), with an expected value of 1.5 under the null hypothesis that there was no trend. Tied pairs were scored as 0.5. The mean BI scores for each group are shown in Table 8(2). Over the four groups combined, 30 subjects showed BI scores above 1.5 and 12 below 1.5 (with 14 equal). This provides significant evidence of a n overall belief bias in the predicted direction (p c 0.02, Binomial test). The two issues of principal interest concerning belief bias were (i) whether it has both positive and negative aspects and (ii) whether or not its extent interacts with problem complexity. In order to address the first question, two 2 way ANOVAs were conducted based upon the raw rating scores as shown in Table 7. In each ANOVA Materials (Definitional vs Empirical) was taken out as a factor since this might be expected to affect the degree of belief bias observed. The first ANOVA compared Believable with Neutral ratings as the second (within subject) variable, dropping the Unbelievable data, in order to test for a positive belief bias. Neither Believability nor Materials approached significance, although there was a near significant (p c 0.07) interaction between the two factors. This reflects a curious cross-over effect in which Neutral problems

Belief bias and problem complexity

151

receive higher ratings than Believables with Empirical materials, though the expected trend is present with Definitional materials. As the trend was not present in the Empirical materials of Experiment 1 and the interaction was short of significance, the cautious interpretation of these results is simply that there is no evidence for a positive belief bias. The second ANOVA was similar, except that Neutrals were compared with Unbelievables with the Believable data dropped. Again problem materials had no effect on the extent of the belief bias, but in this case Believability approached significance (F1.54 = 3.406). In fact, an analysis of variance with one degree of freedom is formally equivalent to a two tailed t test and it is quite legitimate to halve the probability value where a prior prediction of direction of effect has been made. On this basis, there is significant evidence of a negative belief bias (p < 0.05, one tailed). Over the two experiments combined, then, the evidence suggests that belief bias may reflect more of a negative effect in which unbelievable conclusions are rejected, rather than a positive one in which believable conclusions are favoured. The test for interaction of Complexity with Belief Bias is especially interesting in Experiment 2 in view of the very extensive effect of the problem complexity manipulation on the Logic Index described earlier (cf. Table 8(1)). The effect predicted by the Misinterpreted Necessity Model is that the degree of belief bias, as measured by the Belief Index, should be greater for the more complex (four premise) problems. Inspection of the means in Table 8(2) shows that the trend was, as in Experiment 1. in the wrong direction. A two way analysis of variance failed to find any significant effects of either Complexity or Materials or a n interaction between the two. In summary, Experiment 2 provides strong evidence that the belief bias effect is essentially independent of a very obvious and effective manipulation of problem complexity. GENERAL DISCUSSION

Both experiments have succeeded in demonstrating a belief bias in deductive reasoning problems with a useful generalisation across changes in materials and procedural details compared with most previous studies. The introduction of belief-neutral materials has also permitted investigation of the question of whether belief bias is due to a tendency to accept believable conclusions or reject

152

J. Evans

&

P. Pollard

unbelievable ones. Insofar as our data provide an answer to this question, they suggest that the latter, negative interpretation of belief bias is better supported. Whilst, on average, ratings for these were intermediate between those for believable and unbelievable conclusions, each experiment found significant evidence only for a negative belief bias effect. Hence, it appears that belief bias may reflect more a tendency to reject unbelievable conclusions rather than to accept believable ones. This conclusion should be treated with great caution, however, since the difference between the positive and negative measures was not sufficiently large to be significant in itself. If belief bias is a predominantly negative effect, then this might seem to cause problems for the Selective Scrutiny Model, as presented in Table 1, since it assumes that belief bias reflects acceptance of believable conclusions, preempting subsequent analytic reasoning. However, a simple modification to the model could accommodate equivalent treatment by subjects of neutral and believable conclusions, namely the assumption that conclusions are accepted unless they are unbelievable. Hence, the first question asked in the model should be "Isthe conclusion unbelievable?" with a NO answer leading to acceptance and a YES resulting in inspection of the premises. The primary purpose of this study was to investigate the effects of problem complexity on the belief bias phenomenon, and to test the prediction of the Selective Scrutiny Model -- in contrast with that of the Misinterpreted Necessity Model -- that they would be statistically independent. In both experiments, the logical complexity of the arguments was manipulated as a between group variable. In Experiment 1, the manipulation was purely qualitative, being based upon the inclusion of more difficult types of inference in the problems. I t nevertheless significantly reduced the number of inferences subjects were able to draw from the premises of the arguments. In Experiment 2 a more powerful complexity manipulation involved the addition of two extra premises to each argument with a consequently massive effect on the rate of premise based inferences. However, in both experiments the extent of the belief bias effect did not, as one might intuitively expect, increase in the groups given more complex problems. Indeed, a non-significant decrease was observed in each case. With the effect in the wrong direction and a total of 122 subjects tested there are grounds for confidence that the independence between complexity and belief

Belief bias and problem complexity

153

bias is a genuine result that cannot plausibly be explained away as a type 2 error. This finding is counter-intuitive in that one might well expect that subjects would do their best to follow the instructions to generate deductive inferences and only fall back on a belief heuristic when this process failed. However, the result is consistent with the Selective Scrutiny Model of belief bias preferred by Evans et al. (1983) and Barston (1986). More generally, it conforms to the general theoretical proposals of Evans (1984, 1989) concerning a distinction between heuristic and analytic stages of reasoning. I t was proposed that heuristic processes precede and often preclude analyhc reasoning processes. In this case a heuristic judgment of acceptability based upon belief would preclude any subsequent analysis of the logic of the premises. I t is worth noting that the proposals of Evans (1984) were not based upon studies of belief bias, but primarily motivated by an attempt to explain behaviour on a quite different reasoning problem known as the Wason selection task (see Evans, 1982. Ch. 9). The Selective Scrutiny Model as shown in Table 1 is schematic, in that procedures shown as definitive -- e.g. accepting a believable conclusion -- are in fact simply probable. Two approaches are possible to explaining the probabilistic nature of the data actually observed. One interpretation is that subjects first inspect the conclusion and, with some probability, make a judgement based upon its believability. Failing this, they go on to make an attempt to generate a deduction which leads with a further probability either to success or random error, the latter being more likely with more complex problems. An alternative possibility is that some subjects evaluate the conclusion -- leading to more or less constant average belief bias effects, whilst others attempt to generate deductions with more or less success depending upon their complexity. However, it should be noted that the protocol analyses of Evans et al. (1983) failed to find evidence of individual differences in the extent of belief bias and the former interpretation is preferred by the present authors. In conclusion, this study has further demonstrated the belief bias effect across a range of procedural variables and provided some suggestive evidence that the effect may be more negative than positive in nature. This finding is not conclusive and it is suggested that belief-neutral materials should be included where possible in future studies to provide further evidence on this question. The

J. Evans

154

&

P. Pollard

major finding is the strong and counter-intuitive effect in which belief bias has been shown to be independent of the logical complexity of the problems with which subjects reason. This provides further support for the Selective Scrutiny Model of belief bias in particular, and the proposal that reasoning errors may result from preemptive heuristic decision making in general. REFERENCES Barston, J.L. (1986).An investkfation into belief biases in reasonirg. Unpublished Ph.D. thesis, Polytechnic South West, England. Dickstein. L.S. (1980).The effect of figure on syllogistic reasoning. Memorv and ,6.537-543. Dickstein;, L.S. (1981).Conversion and possibility in syllogistic reasoning. . - Bulletin nf the Psvchonomic Societv,G,414-416. The P s v d w of Deductive Reasoning. London: Routledge Evans, J.St.B.T. (1982). and Kegan Paul. Evans, J 3 . B . T . (1984).Heuristic and analytic processes in reasoning. British , 451-468. Evans, J.St.B.T. (1989). Bias in Human ReasoninP: Causes and Conseouencea. Hove and London: Erlbaum. Evans, J.St.B.T.. Barston. J.L., & Pollard, P. (1983). On the conflict between logic and belief in syllogistic reasoning. Memory and Coenition, 11.295-306. Johnson-Laird, P.N. (1983). Mental Models. Cambridge: Cambridge University Press. Johnson-Laird. P.N., & Bara, B.G. (1984).Syllogistic inference. ,u,1-61. Lord, C., Ross, L. & Lepper. M.R. (1979).Biased assimilation and attitude polarisation: The effect of prior theories on subsequently considered evidence. Journal of Persona litv and Social Psv . ~hology, z,2098-2109. Oakhill, J., & Johnson-Laird, P.N. (1985). The effect of belief on the spontaneous production of syllogistic conclusions. Quarterlv Journal of Emerimental P ~ c h o l ~ . 553-570. -nts' DerceDtions of themselves and Pollard, P.. & Sumiga. L. (1983). ntber students. Unpublished manuscript. Polytechnic South West, England. Revlin, R.. Leirer, V.,Yopp, H., & Yopp. R (1980). The belief bias effect in formal reasonlng: The influence of knowledge on logic. Memorv and Cormition, 8. 584-592. WilMns, M.C. (1928).The effect of changed material on the ability to do formal syllogistic reasoning. Nchives of Psvchology New York. No. 102.

a,

m,

.

Acknowledgements. The authors would like to thank Anita Waran for her assistance in the conduct of the second experiment reported in this paper.

Cognitive Biases J . - P . C a v e r n i , J.-M. F a b r e a n d M . G o n z a l e z ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B. V . ( N o r t h - H o l l a n d ) ,

1990

BIASES IN CHILDREN'S CONDITIONAL REASONING Vittorio Girotto I n s t i t u t e of Psychology,

CNR,

Rome,

Italy

Abstract.

This paper presents a brief review of the recent literature on children's conditional reasoning. The possible sources (e.g. concrete operational competence, linguistically based invited inference, confirmation bias) of children's systematic errors on a specific class of inference (modus tollens syllogisms) and meta-inference [selection) tasks are considered. The potential effects of these sources are compared to the debiasing effects of the activation of pragmatic reasoning schemas.

Solving problems about conditional relationships (i.e., i f p then ql has been considered to play a central part in the acquisition of deductive reasoning abilities. (cf. among others, Braine, 1978; Braine & O'Brien, 1989; Kuhn, 1977: Moshman, 1979; O'Brien & Overton, 1980, 1982). The research in this area has demonstrated difficulties and biases in children's reasoning both on inference and meta-inference tasks. In particular, many of these difficulties seem to concern tasks directly involving the use of a specific argument (modus tollens), and more complex tasks whose solution requires, a t least from a formal point of view, a n understanding of this argument. In the present paper the potential sources of children's errors (logical competence in terms of concrete operations, use or ordinary mode of discourse comprehension, confirmation bias) will be briefly discussed. Their potential effects will be then compared to the debiasing effects of pragmatic knowledge activation. 1. CHILDREN'S ERRORS WITH CONDITIONAL SYLLOGISMS

Two research strategies have been used in the study of children's conditional reasoning. The first has studied the development of performance on classical conditional syllogisms: rnodus ponens, denial of the antecedent, assertion of the consequent, and rnodus tollens. In this traditional research paradigm, subjects are given the two premises and asked to evaluate the truth value of the conclusion, or they are asked to evaluate the validity of the whole argument. As usually presented, the first premise of a conditional syllogism is a sentence of the form i f p then

156

V.

Girotto

q ; the second premise is either the affirmation or denial of the antecedent p or the consequent q. Modus ponens. or asserting the antecedent (i.e. the proposition p as second premise), and modus tollens, or denying the consequent (i.e. the proposition not-q as second premise), have necessary conclusions. The proposition q is the conclusion for modus ponens, and the proposition not-q is the conclusion for modus tollens. These two syllogisms are valid forms. By contrast, from denial of antecedent (i.e. the proposition not-p as second premise) and assertion of consequent (i.e. the proposition q as second premise), defined fallacious syllogisms, no valid conclusion can be drawn, Studies of young school children's and adolescents' ability to solve problems involving these four basic forms of syllogism (e.g. Kuhn, 1977; Rumain, Connell. & Braine, 1983; Shapiro & O'Brien, 1970; Staudenmayer & Bourne, 1977; Taplin, Staudenmayer, & Taddonio. 1974; Wildman & Fletcher, 1977) indicate that modus ponens is the easiest form of conditional syllogism and can be solved by 6-year-olds, and that the two invalid forms are the most difficult at all ages: in these forms children have almost always fallen into the classical fallacies concluding not-q to denying the antecedent and p to asserting the consequent. The modus tollens form presents a developmental pattern with a deviation from the trend of improvement in performance with increasing age groups. Performance with this form seems to decline with age (Wildman & Fletcher, 1977). Various authors have explained this developmental trend as a false positive assessment of younger children's performance. The basis for this explanation is the assumption that children interpret ifin the first premise as the biconditional if and onZy if (e.g. Matalon. 1962; O'Brien & Overton, 1982; Taplin et al., 1974; Wildman & Fletcher, 1977). This misinterpretation prevents them from arriving at the correct responses in denial of the antecedent and assertion of the consequent problems, but leads them to the correct response, for the wrong reasons, in modus tollens problems (i.e. first premise: f p then q interpreted as p if and only if q; second premise: not-q; conclusion: not-p). A similar explanation of the early solution of modus tollens problems is proposed, in a Ragetian framework, by Knifong (1974). who proposes the concept of "transductive logic". This concept (see Raget, 1946/1951) appears to mean that events represented by the two clauses of a conditional are viewed by young children simply as

Conditional reasoning

157

going together -both present or both absent. It follows that the modus tollens form should be correctly answered by application of a transductive pre-operational logic. A different interpretation of these findings follows from the assumption that the child's lexical entry for if is not the biconditional (Rumain et a l , 1983; cf. also Braine & O'Brien. 1989). According to this explanation both children and adults interpret conditional problems using comprehension processes that are usual in ordinary discourse. Linguistic analysis [Geis & Zwicky. 1971) introduced the concept of invited inference to define the phenomenon of inferring from a conditional statement, for example a conditional promise ("If you mow the lawn, then I'll give you five dollars"), the obverse statement ("If you don't mow the lawn, then I will not give you five dollars"), which is an invalid inference (from Z f p then q to If not-p then not-q]. Empirical studies (e.g. Fillenbaum, 1975. 1976. 1977; Markovits & Lesange, this volume) have subsequently showed that most adults consider as correct, or spontaneously draw, logically invalid inferences from conditional statements of promises and threats. In general, these inferential errors correspond to inviting the assumption of the biconditional: "If and only if you mow the lawn, I'll give you five dollars". According to Rumain et al. (1983) both adults and children c a n d i s t i n g u i s h the necessary and the invited inferences in understanding conditional statements, but children find it more difficult to abandon the ordinary modes of comprehension: they accept the invited inferences and construe fi as the biconditional. Consequently, they make more errors than adults on the two invalid syllogisms, and produce false positive responses on modus tollens problems. In two experiments, Rumain et aL (1983) demonstrated that children (at least from around age seven onwards) respond appropriately to premises in which the invited inferences are countermanded. For example, first premise: "If the bottom has 3, then the top has R. But if the bottom doesn't have 3. then the top may have R or it may have some other letter" (iJp then q, but ifnotp then q may or may not be true); second premise: "The bottom doesn't have 3" (not-p);conclusion: "Can't tell". In the case of deniaI of the antecedent and assertion of the consequent, the expanded first premise significantly raised the (formally correct) "can't tell" responses. In the case of modus tollens problems, this type of premise significantly decreased not-p responding (formally correct)

158

V.

Girotto

in 10-year-olds. and significantly increased "can't tell" responding. According to the authors, children as well as most adults (cf. the erroneous performance produced by the majority of adult subjects in the selection task, described in the next section) do n o t spontaneously have available a line of reasoning that could lead to the correct solution of modus tollens problems: namely, the reductb ad absurdurn strategy (if p were true, then q would be, but q isn't, s o p is f a k e ; cf. Wason & Johnson-Laird, 1972). Therefore, in the absence of this sophisticated strategy. the countermanding of the invited inferences can reduce the apparently correct answers of younger children, but it cannot produce real comprehension of the problem. In conclusion, research on conditional syllogisms seems to indicate that only modus ponens problems are solved around schoolentering age. The other three forms elicit erroneous performance in most children. In particular, the ability to solve modus tollens problems seems not to be an early-developing ability, since attempts to prevent children from making the invited inferences, a potential source of errors in these problems, appear to produce only responses of intermediate sophistication.

2. ERRORS IN CONDITIONAL RULE TESTING

People's difficulties with reasoning problems that involve steps concerning the evaluation of not-q cases, and at least formally require the use of a modus tollens argument, have also been demonstrated by adults' systematic errors on tasks that involve assessing conditional statements against evidence. In its basic version, the well-known Wason four-card selection task (Wason. 1966. 1968) consists of four cards (each with a letter on one side and a number on the other side) displaying a vowel (p), a consonant (not-p). a n even number (q) and an odd number (not-q),together with the sentence: "If there is a vowel on one side of the card, there is a n even number on the other" ( g p then 4). The subject is asked to indicate the cards which must be turned over in order to find out whether the rule is true or false. The correct solution is to select the vowel (p) and the odd number (not-q). Most adults subjects (including undergraduates with formal training in logic, cf. Cheng, Holyoak, Nisbett. & Oliver, 1986) perform very poorly on selection tasks of this type. The commonest pattern of response for adults is the choice of the vowel (p) and the

Conditional reasoning

159

even number (4).A similar pattern of results has been observed in analogous tasks by Kuhn (1977),Moshman (1979) and O'Brien and Overton (1980) using children as subjects. Despite theoretically remarkable differences, t h e various approaches to the problem of the development of reasoning abilities have produced the same general conclusion: assessing conditional statements against data is a developmentally advanced skill, beyond young children's competence. According to the authors who partially accept the Piagetian distinction between the concrete and formal operation stages (e.g. Kuhn, 1977: Moshman 1979; O'Brien & Overton 1980, 1982). this ability belongs to the more sophisticated formal stage. According to Braine and Rumain (1983). who interpret the development of reasoning in terms of the gradual acquisition of 'natural' logical abilities other than the Piagetian operations (cf. Braine, 1978) there is a general confurnation bias at the basis of people's failure to solve selection tasks: most subjects, including adults, perform inadequately since they "have a strong bias to evaluate by trying to verify rather than falsify". Even if the source of this bias is not clear, "the poor quality of the performance on these tasks confirms that subjects' logical understanding is reflected more directly in simple deductions than in evaluations and truth judgments" (Braine & Rumain. 1983, p. 311). This pessimistic conclusion will be evaluated in the next section. 2.1 Incorrect performance: confirmation bias and debiasing effects

of knowledge Braine and Rumain's position seems plausible for two reasons. First, there is strong evidence that people fail on meta-inference tasks such as the selection task, which requires subjects to make inferences about the condition that would allow a valid inference: second, as indicated, people also fail to solve simple deduction tasks, s u c h as syllogisms in modus tollens form. However, a considerable corpus of evidence exists that shows how subjects' performance is not always interpretable in terms of a confirmation bias, and above all, how adults and children are in some cases able to solve conditional problems requiring a search for potential counterexamples First of all, an analysis of erroneous performance on the selection task simply in terms of confirmation bias (or the

160

V.

Girotto

verification principle, cf. Wason, 1966) cannot explain the correct performance obtained with versions of the task in which the tested rule presents a negative consequent (e.g. "if there is a n A on one side of the card, then there is not a 2 on the other side of the card"). This and related phenomena (cf. Evans, 1989, for a review: cf. also Girotto & Politzer, this volume) have been explained in terms of a matching bias, that is, the subject's tendency to match his/her response to the stimuli named in the test sentence. Adult performance seems therefore to be determined more by linguistic factors, which direct attention to only a selected number of logical cases, than by their general inability to falsify hypotheses. Moreover, there seem to be robust debiasing effects produced by real-world knowledge. Adults, in fact, are able to solve selection problems even with test sentences presented in the affirmative form if they refer to 'thematic' content, in particular when they express regulative rules such as permission or obligation. For example, Johnson-Laird, Legrenzi, and Sonino-Legrenzi (1972) asked their subjects to imagine they were post office workers examining sealed and unsealed letters with stamps of different value (40 and 50 liras), and that they had to determine whether the rule "If a letter is sealed, then it has a 50 lira stamp on it" was violated. Most subjects solved the task, turning over the sealed letter and the letter with the 40 lira stamp. Thus, in the case of certain thematic contexts (see Griggs, 1983; Pollard. 1982 for reviews) subjects appear to recognise the force of the potential falsifying cases. Cheng and Holyoak (1985) interpreted these results without assuming the use of formal rules of inference, such as modus tollens, in solving selection tasks, since this position does not in itself explain the variability in performance across contexts. According to these authors, people often reason using neither formal rules of inference, nor memories of specific experiences, but rather pragmatic reasoning schemas, which are clusters of contextsensitive rules, defined in terms of actions and goals. Under certain circumstances, some of these schemas lead to the correct solution of reasoning problems, in a way that appears consistent with standard logic. For example, the permission schema can be defined in terms of four production rules which, although not coinciding, correspond to those of the material implication (e.g. from a rule such as "If action A is to be taken, then precondition C must be satisfied", it is possible to derive the contrapositive rule "If precondition C is not satisfied, then action A cannot be taken".

Conditional reasoning

161

which implies the impossibility of having, at the same time, "action A" ( p ) and "precondition C not satisfied" (not-q).without using a n indirect strategy such as reductio ad absurdum). This means that the evocation of a permission schema should help subjects solve a conditional problem such as the selection task. This prediction, supported by the results of previous studies in which the versions shown to be successful in producing facilitation concerned permission rules, was further corroborated by Cheng and Holyoak's (1985) own experiments. In one case, they showed that subjects without experience on a specific permission rule but with a rationale enabling them to understand it, perform like expert subjects. Furthermore, they showed that abstract wording of a permission rule produces significant facilitation relative to the standard version concerning concrete b u t arbitrary relationships between letters and numbers. In a series of studies (Girotto, 1987; Girotto, Gilly, Blaye, & Light, 1989; Girotto. Light, & Colbourn, 1988; Light, Blaye, Gilly, & Girotto, 1989). we applied this approach to the study of children's deductive reasoning. Since previous results indicated that adults were able to solve selection tasks if presented in terms of social regulations such as permission and obligation, and since research on the development of social cognition has demonstrated that children develop early sophisticated knowledge of social rules (cf. Turiel, 1983, for a review), we hypothesized that the conditions which facilitate adult reasoning would produce the same effects on child reasoning. This general hypothesis was corroborated by the results of our experiments. In one of these studies (Girotto et al.. 1988). it was demonstrated that 9-10 year-olds were able to solve a reduced version of the selection task (the FWST, cf. Johnson-Laird & Wason, 1970; Wason & Green, 1984) when phrased in terms of permission. When an unfamiliar regulation ("All the buzzing bees must stay inside") was presented as a permission rule made by an authority (the queen bee) in order to reach a specific goal (to increase the safety of the bees), about 70% of the children solved the task. In another study (Light et al., 1989) it was shown that even for young children (6 to 8 year-olds) permission rules can dramatically improve performance'. Both these studies used the simplified 1. These r e s u l t s a r e c o n s i s t e n t w i t h t h e f i n d i n g of a n e a r l i e r s t u d y ( L e g r e n z i & Murino, 1 9 7 4 ) i n w h i c h it was shown t h a t 6-7 y e a r - o l d c h i l d r e n a r e a b l e t o s o l v e a

162

V.

Girotto

version of the task. But even with the full selection task, Girotto e t al. (1989) showed that both 9-10 and 14-15 year-olds can do quite well in situations which are well contextualised as permissions, including those in which a permission rule was not familiar and was plausible only on the basis of an available implicit rationale (e.g. "If one drives over 100 km/h. then one must have a fluorescent car"). In the next section the proposed interpretation of these results in terms of pragmatic schema activation will be compared with other possible explanations of the same data. 3. CORRECT PERFORMANCE: PRAGMATIC SCHEMA ACTIVATION, INVITED INFERENCES AND CONDITIONAL BIASES

I t might be argued that, given the indicated difficulties in solving modus tollens problems and given the fact that some of the production rules of permission and obligation schemas correspond to the expanded premises2 used by Rumain et a1 (1983),children's correct performance on selection tasks concerning permission or obligation rules is not due to schema activation per se, but to a sort of implicit countermanding of invited inferences. However, it should be noted that the explicit countermanding of such inferences in the Rumain et al. study resulted in an increase in incorrect "can't tell" responses on modus tollens problems. By contrast, in our experiments the correct performance elicited on the permission problems were correct. For example, in the RAST studies (e.g. Girotto et al., 19881, the successful children not only correctly selected the not-q elements as potential counterexamples, b u t actually tested them in a n exclusive manner. These results also show that children's correct solutions cannot be explained in terms of false positives arising from biconditional reasoning.

complete s e l e c t i o n t a s k a f t e r t r a i n i n g on a p e r m i s s i o n r u l e . The i n t e r p r e t a t i o n proposed by t h e a u t h o r s a t t h a t t i m e was based on t h e ' r e a l i s m ' of t h e s i t u a t i o n ( a s c h o o l r u l e concerning p u p i l s ' o u t f i t s ) . 2 . Consider, f o r example, r u l e 2 of a p e r m i s s i o n schema " I f t h e a c t i o n i s not t o be t a k e n , t h e n t h e p r e c o n d i t i o n need I t corresponds t o t h e second p a r t of not be s a t i s f i e d " . t h e premise "if p t h e n q, b u t if not-p t h e n q m a y o r m a y not be t r u e " .

Condi t i ona 1 r e a s o n i n g

163

Another alternative explanation, similar to the first, refers to the fact that since both conditional permission and obligation statements tend to not invite the biconditional [ p ifand only ifq) or converse (if q then p ) assumptions, their conditional bias is sumient to explain facilitation. More generally, it might be argued that children's (and adults') correct performance can be explained in terms of a simpler notion than pragmatic schema activation. To provide evidence that a class of conditional reasoning performance cannot be explained in terms of the conditional bias of certain regulative rules, we devised a n experiment in which children's judgments on syntactic transformations of statements with the same conditional bias were compared (cf. Girotto, Blaye, & Farioli, 1989/Exp. 3). In fact, permission and obligation rules, which elicit similar selection performance explicable in terms of conditional biases, differ in regard to the possibility of inferring logically equivalent statements (cf. Cheng et aL, 1986). An if-then permission statement can be rephrased into the logically equivalent only-if form, e.g. the rule "If action A is to be taken, t h e n precondition B must be satisfied" is equivalent to "Action A is to be taken only if precondition B is satisfied". In contrast, an if-then obligation statement cannot be rephrased in a n only-Y form. Since this form stresses the temporal priority and the necessity of the consequent for the antecedent (cf. Evans, 1977; Evans & Newstead, 1977), its application to an obligation rule (by definition relating the occurrence of a certain event to a subsequent action) will produce a pragmatically anomalous statement3. For instance, the rule "ZJ a child is sick, then he/she must stay inside" should imply "A child is sick only i j he/she stays inside". However, such a difference in pragmatic appropriateness will not occur for the application of the only-if form to the contrapositive ( r f not-q then not-p) of either permission or obligation if-then statements. In both cases, the only-if form of the contrapositive will carry the pragmatically valid connotation that the consequent is necessary for the antecedent. For instance, the permission "lf a child wants to play, then he/she 3 . T h i s d i f f e r e n c e between p e r m i s s i o n and o b l i g a t i o n r u l e s has a l s o been used ( G i r o t t o e t a l . , 1989/Exp. 3 ) t o compare t h e d i f f e r e n t p r e d i c t i o n s about s e l e c t i o n performance d e r i v i n g from t h e pragmatic r e a s o n i n g schema approach and a competing approach r e c e n t l y proposed: t h e ' s o c i a l c o n t r a c t t h e o r y ' (Cosmides, 1985, 1989; c f . t h e subsequent d e b a t e i n Cheng & Holyoak, 1989; G i r o t t o & P o l i t z e r , t h i s volume).

164

V.

Girotto

must stay outside" is pragmatically equivalent to "A child can stay inside only if he/she is not playing". Similarly, the obligation "lf a child is sick then he/she must stay inside" is equivalent to "A child can stay outside only ifhe/she is not sick'. To test the prediction that statements with the same conditional bias produce different response patterns in a task concerning these linguistic transformations, we asked 7-year-old children to judge the appropriateness of the only-if rephrasing for permission and obligation rules. The results of this experiment were consistent with the pragmatic schema predictions. While in the case of the contrapositive rule the transformation was accepted by a large majority of the children in both the permission (95%) and obligation (92%) conditions, in the case of the direct rule, the only-if form was accepted by only 42% of the children in the obligation condition, a percentage significantly lower than that of the permission condition (86%). Results of further experiments confirm the conclusion that the notion of conditional bias is not sufficient to explain the debiasing effect of pragmatic knowledge activation in children's (and. a fortiori, adults') reasoning. Given the same syntactic structure of the tested rule, and given the same potential invited inferences, variables at the level of regulative rule content and context, such as the plausibility of a permission rule (Girotto et al., 1989). or knowledge of the goals and motivations of a promisor whose behaviour has to be tested (Light, Girotto & Legrenzi, 1990; Politzer & Nguyen-Xuan, 1988; cf. Girotto & Politzer, this volume), influence reasoning performance in a manner that the conditional bias notion cannot account for. CONCLUSIONS

The arguments in this chapter suggest that the pragmatic schema approach can be usefully applied to the study of children's conditional reasoning. In general, our studies support the prediction that the ability to understand the power of potential disconfirming data in assessing universal statements, which in Piagetian terms requires formal reasoning (Beth & Piaget, 1966). is available before adolescence. Moreover, these results cannot be considered as empirical evidence of early formal competence in children's reasoning, since anticipation of the appearance of complex logical abilities cannot

Conditional reasoning

165

explain the systematic errors made by adults in the arbitrary versions of the selection task. On the other hand, the version of the Ragetian theory in which both formal logical competence (acquired during adolescence) and world knowledge are considered as necessary features of adequate reasoning performance (Overton, Ward, Noveck. Black, & O'Brien. 1987) still cannot explain adult errors. It could be argued that even for adults. formal competence can be elicited only in the presence of familiar contents. However, such a claim would obviously reduce the explanatory power of the theory. In conclusion, even if the pragmatic schema approach needs to be tested on a larger range of problems, including simple deduction tasks (cf. Markovits & Lesange, this volume), and even if the notion of pragmatic reasoning schemas is probably not in itself sufficient to explain differences in performance which seem to depend on the f o r m of the reasoning problems, in a contentindependent way (cf. Braine & O'Brien. 1989; Evans, 19891, such a notion nevertheless has the merit of explaining in a n economic manner the debiasing effects of pragmatic knowledge on complex meta-inference tasks REFERENCES Beth. E.. & Piaget, J. (1966).-1set molom and P s v c u. Dordrecht: Reidel (French original work published in 1961). Braine. M.D.S. (1978).On the relation between the natural logic of reasoning and the Review. ~, 1-21. standard logic. -ical Braine, M.D.S.. & O'Brien. D.P. (1989).49 theory of If: A lexlcalentrv. reasoning r m. an r a m a tic ~ r l n c i ~ l @ Manuscript . submitted for publication. Brahe!%%., &?urnah. B. (1983).Logical reasoning. In J.H. Flavell & E.M. Markman (Eds.). Handbook of Child Psvcholom. Volume 3: Coanitive DevelODment (4th ed.). New York Wfley. Cheng. P.W.. & Holyoak, K.J. (1985).Pragmatic reasoning schemas. Cognitive Psvcholm, 391-416. Cheng. P.W.. & Holyoak. K.J. (1989).On the natural selection of reasoning theories. 285-313. Cheng, P.W.. Holyoak. K.J., Nisbett. R.E.. & Oliver, L.M. (1986). Pragmatic versus syntactic approaches to training deductive reasoning. CoPnitive Psvchologv, J&, 293-328. Cosmides. L. (1985).peduction or Darwinian An e a t i o n o f the "elusive" content effect on the Wason selection task . Doctoral dissertation, Harvard University. The logic of social exchange: has natural selection shaped how Cosmides. L. (1989). humans reason? Coenition, 3113).187-296. Evans, J.St.B.T. (1977).Linguistic factors in reasoning. Quarterlv Journal of JCmerlmental Psvchology!.B, 297-306. Evans.J.St.B.T. (1989). Biases in human reasoning . Hove, UK.: Lawrence Erlbaum.

u, m.a,

m?:

V.

166

Girotto

Evans, J.St.B.T., & Newstead. S.E. (1977). Language and reasoning: a study of temporal factors. B. 265-283. Ffflenbaum, S. (1975). IF: some uses.1 Resea&. 245-260. Fillenbaum. S. (1976). Inducements: on the phrasing and logic of conditional promises, threats, and warnings.Psychological Research. B ,231-250. Fillenbaum. S. (1977).A condition on plausible inducements. Language and SDeech,

-.

x,

29. 136-141.

Gels, M.L., & Zwlcky, A.M.(1971). On invited inferences., -

2.561-

566. Girotto, V. (1987). -the o -e r

nei bambinl. Doctoral dissertation, University of Bologna. Girotto. V.. Blaye. A , & Farloll. F. (1989). A reason to reason. Pragmatic basis of children's search for counterexamples. Eurooean Bu1let.D of Coanit ive

ErJrrhnlney. 9.297-32 1. Girotto. V., Gffly. M.. Blaye, A., & Light, P.H. (1989). Children's performance in the . cholom, selection task: Plausibility and experience. British Journa1 of Psv 89.79-95.

Girotto, V.. Light, P.H.. & Colbourn, C. (1988). Pragmatic schemas and conditional PsvcholQgy m. reasoning in children. Quarterlv Journal of1-E

.

469-482. Grlggs. R.A (1983). The role of problem content in the selection task and THOG

reas-: Psychol& problem. In J.St.B.T. Evans (Ed.), 9DDrOaChes. London: Routledge and Kegan Paul. Henle, M. (1962). On the relation between logic and thinking. Psvcholoelcal Revlew. 69.366-378.

Holland, J.H.. Holyoak. K.J.. Nisbett, R.E.. & Thagard, P.R. (1986). h d u c t ion: ses of inferlearning & discoverv. Cambridge, MA. Brandford Books/MIT Press. Johnson-Laird, P.N., Legrenzi. P.. & Sonino-Legrenzi. M. (1972). Reasoning and a sense of reality, British Journal of Psvcholom, 395-400. Johnson-Laird, P.N.. & Wason, P.C. (1970). Insight into a logical relation. Quarterly Journalof Ernerlrna,&l Psvchology, 22.49-61. Knifong, J.D. (1974). Logical abilities of young children -two styles of approach.

a,

G . ~ ~ J U 42578-83. Q ,

Kuhn, D. (1977). Conditional reasoning in children. DeveloDmental Psvcholom,

u,

342-353. Legremi. P.. & Murlno. M. (1974). Falsification at the pre-operational level. ,1.361-368. Light. P.H.. Blaye. A. Gffly, M.. & Girotto. V. (1989). Pragmatic schemas and logical reasoning in six to eight year olds. w t l v e Devel,a. 49-64 Light. P.H.. Girotto. V.,& Legremi, P. (in press). Children's reasoning on conditional

promises and permissions. Cognitive DeveloDment. Moshman. D. (1979). Development of formal hypothesis-testing ability. 104-112. pevelo~mentalPsvcholocy, OBrlen. D.P.. & Overton. W.F. (1980). Conditional reasoning following contradictory evidence: A developmental analysis. Journal of ExDerimental C hild PWcholo~. 44-60. O'Brlen. D.P.. & Overton, W.F. (1982). Conditional reasoning and the competenceperformance issue: a developmental analysis of a training task. Journa 1 of &Derimg&al Child -P , 274-290. Overton, W.F.. Ward, S.L.. Noveck. I.A. & OBrlen. D.P. (1987). Form and content in the development of deductive reasoning. PevelpEIDental Psychology, 22-

s,

a,

a,

a,

30.

Politzer, G. & Nguyen-Xuan. A. (1988). Pragmatic reasonine schemas: Promises and UIG four-card s e l e m. Unpublished manuscript, Centre National de la Recherche Scientiflque, Paris.

Conditional reasoning

167

Pollard. P. (1982).Human reasoning: some possible effects of availability. , 12.65-69. Rumain B.. Connell. J.. & Braine M.D.S. (1983).Conversational comprehension processes are responsible for reasoning fallacies in children a s well a s adults: If is not the biconditional. ,471-481. Shapiro. B.J., & O'Brlen. T.C. (1970). Logical thinking in children ages 6 through 13. , 823-829. Staudenmayer. H.. & Bourne. L.E.Jr. (1977).Learning to interpret conditional sentences: A developmental study. peveloDmenta1P s v c u , 616-623. Taplin, J.E., Staudenmayer, H.. & Taddonio. J.L. (1974).Developmental changes in conditional reasoning: Linguistic or logical? Journal of ExDerlmental Child Psvcholo&, 17.360-373. Turlel. E. (1983).The develoDment of social knowl-. Cambridge: Cambridge University Press. h l Wason. P.C. (1966).Reasoning. In B. Foss (Ed.), Harmondsworth: Penguin Books. Wason, P.C. (1968).Reasoning about a rule. Quarterlv Journal of ExDerlmental Psvcholocy!. 273-2811 Wason. P.C.. & Green, D.W. (1984). Reasoning and mental representation. Quarterly 597-610. Journal of ExDerimental Psvcholo-, Wason. P.C., & Johnson-Laird. P.N. (1972). psvch&@v of Reasonina: Structure and Content. Cambridge, MA: Haward University Press. Wildman, T.M.. & Fletcher, H.J. (1977).Developmental increases and decreases in solutions of conditional syllogistic problems. DeveloDmental Psvcholo& .l.& 630-636.

s,

a.

u,

.

a,

a,

Acknowledgements. Preparation of this paper was supported by a grant from CNR. The research described here was supported by grants from Fondation F'yssen (Paris). I gratefully acknowledge the hospitality of the CREPCO (CNRS and Universite de Provence, Aixen-Provence), during the 1986-87and 1987-88academic years. I thank in particular Agnes Blaye, Fernand Farioli and Michel Gilly for their collaboration. Correspondence and requests for reprints should be sent to Vittorio Girotto. Istituto di Psicologia, CNR. Viale K. Marx 15. 00137, Roma, Italy (Electronic mail: [email protected]).

This Page Intentionally Left Blank

Cognitive Biases J.-P. Caverni, J . - M . F a b r e and M . Gonzalez (Eds.) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B.V. (North-Holland),

1990

ARE THERE BIASES IN ANALQGICAL REASONING? Mark T.G. Keane School of Psychology, U n i v e r s i t y of Wales C o l l e g e of C a r d i f f , C a r d i f f , Wales

Abstract. I will attempt in this paper to determine whether there is any evidence of biases in analogical reasoning. First, I will consider the special problems surrounding the definition of biases which exist in the area of analogy research. Then I will survey some data that is suggestive of biases. I conclude that there is very little evidence of systematic biases in people's analogical abilities. However, a caveat to this conclusion is necessary since there is a paucity of research examining when people's analogical heuristics might lead them astray. INTRODUCTION

Analogical reasoning can be viewed as an umbrella term for a variety of reasoning heuristics for using knowledge about one domain to understand another domain. Typically, an analogy allows one to reason about an unfamiliar domain by referring to one's knowledge of a familiar but semantically-disparate domain. So, for example, if you know nothing about the atom and I say to you "The atom is like a miniature solar system: the electrons correspond to the planets and the nucleus to the sun", it is possible for you to infer analogically that the electrons revolve around the nucleus and that the nucleus attracts the electrons (see Figure 1). Analogical phenomena have been divided traditionally into two distinct issues: analogical mapping and analogue retrieval. We will concentrate on analogical mapping because it is the central intellectual process in analogical reasoning. The question of biases in analogical reasoning has not been considered before. In deciding whether or not there are biases, the first problem one faces is that of determining what constitutes a bias in this research area. This turns out to be a non-trivial problem. Many biases are determined with respect to normative theories about what is correct or reasonable behaviour in a certain task or problem situation [see e.g., the use of Bayesian probability theory in determining what probability judgements are and are not reasonable: Kahneman & Tversky. 1973). However, it is a moot

170

M. T. G Keane

point whether a normative theory of analogy exists. Two immediate questions therefore beset us: (i) is there an analogy theory which can be taken to be a normative theory, (ii) if no such theory exists how is a bias to be defined in this area. In the next section I consider possible candidates for a normative theory of analogy. After choosing Gentner's structuremapping theory as a likely candidate I go on to argue that the theory is not a n appropriate normative theory. Then I attempt to define biases in analogy without the aid of a normative theory. Later, the core issue of whether there are biases in analogical reasoning is discussed. DETERMINING BIASES IN ANALOGY

Finding a candidate normative theory Normative theories are, to use Palmer & Kimchi's (1986) terminology, fnJormation mapping theories. Such theories may not even have been proposed initially as models of psychological processes but are often adopted wholesale by psychologists as competence-type theories [see e.g., Henle, 1978; Inhelder & Piaget, 1958; Fitts & Posner. 1967; Garner, 1962). They provide specific, usually formalised, statements of the mapping between a specified input and an output. Usually, the statement of the input and the output can be characterised easily in the formalism of the theory, via some operational definition. So, for example, in the Kahneman & Tversky experiments on judgements of probability, the problems they gave subjects could be easily formulated in terms of Bayesian probability theory and the probability value output. Similarly, in deductive reasoning research the materials given to subjects can be formalised in their abstract logical skeleton (e.g., t f p then q, p ) , as can subjects' responses (e.g., therefore q). We can specify formal rules from the predicate calculus which characterise the move from a set of premises to a conclusion (in the above example, modus ponens). This provides a precise statement of the mapping from the input to the output. I t is, however, debatable whether it tells you anything about the underlying psychological processes; although it places constraints on the nature of these processes. So, what theory is a likely candidate as a normative t h e o m In one sense, it can be argued that a syntactic theory, like Gentner's (1983) structure-mapping theory, is normative in that it lays down what is considered to be the "correct" way to map analogically from

B i a s e s and a n a l o g y

171

one domain to another when one is either reasoning, learning or problem solving by analogy. Indeed, Gentner (1989) has proposed that structure-mapping is a competence-type theory. The alternative pragmatic theories (Burstein, 1986: Carbonell, 1986: Holyoak, 1985: Gick & Holyoak. 1980, 1983: Holland, Holyoak. Nisbett. & Thagard. 1986: Keane. 1985. 1987, 1988a. 1988b) have emphasised the role of plans, goals and purpose in analogy. The essential difference being that the mapping process is critically under the control of the problem solver's current goals (see Holyoak, 1985: Keane, 1985, 1988a). However, these theories have made few competence-type claims as Gentner has done. So, of all the various theories of analogy this makes hers the best candidate for a normative theory. Let us therefore examine the central tenets of her theory. Gentner's Syntactic Theory as a Normative Theory of Analogy Gentner uses predicate calculus-type representations to make a number of representational distinctions which underlie h e r theory. The main distinctions are as follows: Objects represent individual entities, attributes are one-place predicates [e.g.. RED(x)] and relations are multi-placed predicates [HIT(x, y) or GIVE(x, y, z)]. A further distinction is made between types of relation: higherorder and lower-order relations. A lower-order relation is one which take objects as its arguments [like HIT(x. y)] while a higherorder relation is one which takes lower-order relations as its arguments. Examples of typical higher-order relations are causal relations [e.g., CAUSE(BOUNCEpCROSS(bal1. street), HIT(bal1. wall))]. Others include relations like IMPLIES and ENABLE. Using these distinctions, Gentner maintains that an analogy between a base domain (B) and a target domain (T) involves the following mapping rules: (1)Attributes of objects are dropped: e.g.. RED (bi) -/-> RED (ti) (2) Relations between objects in the base tend to be mapped across: e.g., COLLIDE(bi, bj) --> COLLIDE(ti, tj)

(3)The particular relations mapped are determined by systematicity, as defined by the existence of higher-order constraining relations that can themselves be mapped: e.g., CAUSE[PUSH(bi, bj), COLLIDE(bj, bk)] --> CAUSE[PUSH(ti, tj). COLLIDE(tj, tk)]

172

M.T.G

electron

nuc Ie u s

Figure 1. The solar system/atom analogy.

Keane

Biases and analogy

173

So. as in Figure 1, the relation HO'ITER-THAN is not mapped into the target atom-domain because it is not systematically interconnected to the other relations in the solar system domain, all of which are constrained by higher-order relations to do with gravitation. I t is important to notice that structure-mapping theory operates in a predominantly syntactic manner, in two central senses. First, in determining what is to be matched/mapped it works off the "-arity" of the predicates involved (i.e., whether they are one- or multi-placed predicates). Second, higher-order relations are determined purely on syntactic grounds. That is, if a fairly meaningless relation, such as a predicate called IS-BLUE-ONTUESDAYS took two propositions as its arguments then, as a higher-order relation. it would influence what is to be mapped into a target domain. Granted, structure-mapping theory proposes a limited type of semantic processing in that it matches relations with the same label. But, it does not critically direct processing on the basis of semantic or pragmatic decisions. Rather decisions and evaluations of the goodness of the mapping are made mainly on the basis of the syntax of the representations. This is of course reminiscent of a variety of theories of deductive reasoning [Rips. 1983; Braine, 1978). So, how does this theory match up to the criteria we laid down earlier regarding normative theories? These sorts of criteria cannot be met by structure-mapping theory because the translation of natural language statements into predicate calculus representations is oftentimes an arbitrary process [Keane, 1988b). While this is true of any attempt to use the predicate calculus, the problem is especially acute in structure-mapping theory for three reasons: The grain of the analysis is very fine. We do not simply have to determine propositions but specific predicates (i.e., relations, attributes, higher-order relations) while paying attention to the "-arity" of these predicates [i.e., whether they are single- or multiplace predicates). It is not possible to be unequivocal about such fine-grained distinctions. Higher-order relations often have to be inferred from natural language statements rather than being explicitly stated. When one should and should not infer such relations is a serious problem in itself deserving research.

114

M . T . G Keane

The mapping from input to output depends crucially on these distinctions. This means that structure-mapping theory is not an adequate information mapping theory. If it were to fulfil this role it needs to be modified in a number of radically different ways. As such it is not of use as a normative theory. It is therefore necessary to consider how one can define a bias in the absence of a normative theory of analogy.

If one were to imagine an exemplar of a bias in analogical reasoning it would have to have a number of distinct characteristics. First, most people would have to agree that there was an obvious analogy to be drawn between the two domains. So, in the solar system/atom case most people would agree that when the analogy was specified it was a good analogy and could be easily understood. Second, despite this agreement, people would make systematic errors when they were given the analogy to draw. So, for instance, in the solar system/atom analogy we might find that people consistently mis-map the object-to-object correspondences and become confused about whether the sun corresponds to a n electron or the nucleus. The conclusion to be drawn from this line of thought is that, in analogy research, a bias is best determined by some consensual, empirically-supported notion of adaptive behaviour. The norm, in this sense, thus derives from what is generally agreed to be adaptive behaviour in the world and not from a normative theory. We can apply this characterisation of a bias to experiments in the literature. Consider an example. In a typical analogy experiment, subjects are presented with a base analogue (see e.g., Gick & Holyoak's (1980) stories about "A general attacking the fortress from multiple directions"). This analogue has been written to suggest a specific analogous solution to a target problem (in the above case this was Duncker's, 1945, Radiation Problem about a doctor attacking a tumour from different directions). On the basis of his/her intuitions, the experimenter supposes that the analogy is, in fact, a n analogy and that the analogous solution will solve t h e target problem. The experimenter's intuitions can then be substantiated with rating tasks which determine whether the subject population also agrees that the analogy is a good one (as Gick & Holyoak, 1980. do in their

B i a s e s and analogy

175

experiments). Given this empirical consensus, if a sufficient number of subjects in a particular condition of a n analogy experiment fail to draw the analogy, then one can say that they are in error and that a bias has occurred. However. this is too simplistic a view because it assumes that all correct answers are the result of the application of adaptive heuristics and all incorrect answers are the result of mal-adaptive heuristics. The picture is actually a shade more complicated (see Table 1). We also need to consider the actual or conceived representativeness of the conditions under which the correct (successful) or incorrect (failed) responses occur. If we assume that various heuristics underlie analogical processing (as theorists have proposed: see Keane, 1985, 1988: Gentner. 1983: Holyoak, 1985: and later sections) then, as in other research areas, biases occur when in certain situations these heuristics turn out to be inappropriate. There are three distinct categories of bias. Table 1. The different categories of responses Responses

Conditions Representative

Non-repre sentative

Correct

globally adaptive

accidentallyadaptive bias

Incorrect

global bias

local bias

First, one can have a global bias. This is where most subjects produce an incorrect response under representative conditions. By representative, I mean conditions which widely hoId in many realworld situations. This definition also assumes that these conditions are conceived of by the subject in the manner in which they were intended by the experimenter. These are clearly the sorts of biases which should be our main concern in determining whether people are capable of producing optimal responses. These can be readily opposed to the second category of biases: local biases. These are biases which only occur under specific, local conditions which are

176

M.T.G

Keane

not found generally in the world. As such, they are conditions which are non-representatiue of those in the world. This definition again assumes that the conditions are conceived of in the manner in which they were intended. Such local conditions may even be particular to the experimental situation in which the bias is found (traditionally many of such biases are simply termed "artifacts"). These are clearly a class of biases which should not retain our attention for long. Finally, there is a class of biases which I have called accldentally-adaptive biases. These are responses which appear to be correct responses but are in fact only accidentally so because otherwise representative conditions have been conceived of in a non-representative manner (e.g., Evans' matching bias sometimes leads to the correct response on abstract versions of the selection task for the wrong reasons: see Evans, 1982). So, the heuristics which the subject are applying are non-adaptive ones but the conditions the subject selectively attends to result in the accidental production of a correct answer. It should be noted that this last category of biases cannot be determined using the consensual characterisation of a bias outlined earlier. This possibly points to the limitations of this method of defining biases. 1

BIASES IN ANALOGICAL MAPPING? Effects of analogical relatedness There is some evidence that people get worse at analogical problem solving when the analogical relatedness of the two domains is changed. However, these responses cannot be considered to be biases, because they are adaptive and merely reflect decreasing degrees of analogy between the domains presented (see Gick & Holyoak. 1980; Holyoak & Koh, 1987; Keane. 1985, 1988a). Many experiments have been designed to test the effects of analogical relatedness. For example, in a recent experiment (Keane, 1988a. Experiment 6) I varied the relatedness of the goal of a storyanalogue relative to that of the presented problem and examined the effects of such a change. Specifically, subjects were given a story analogue to the Radiation Problem about an engineer trying to deal with a blockage in a machine component, using a chemical. The 1. Thanks t o Ruth Byrne f o r s u g g e s t i n g most of t h e s e points.

B i a s e s a n d analogy

111

story came in one of two forms. Both versions of the story had identical solution statements b u t they differed in terms of the goal adopted by the engineer. In one version the engineer had the goal of "destroying the blockage using a chemical" while in the other he/she had the goal of "removing the blockage by lubricating it with a chemical". While this change was relatively minor, it was found to have significant effects on the types of errors subjects made when drawing the analogy. However, these responses simply reflect subjects' sensitivity to the degree of disanalogy between the base and target domains. For, in the destroy-story. the goal is more congruent with the destroy-goal of the Radiation Problem while the blockage-story lacks this goal congruence. So, subjects' analogical processes were operating effectively.

Effects of attributes: bias or adaptation? I t may be recalled that one of the central proposals made by Gentner's theory was that analogies were about mapping relations and ignoring attributes. However, Keane (1985, 1988a) has argued and shown that if certain attributes are present in both domains then subjects' ability to analogise improves. These attributes were called functionally-relevant attributes (FRAs) and were ones which become salient when an object is predicated by certain classes of manipulative relation. So, for example, it has been shown by Anderson & Ortony (1975) that in the sentences "He tuned the piano" and "He lifted the piano" people represent the former "piano" as having musical attributes and the latter "piano" as having attributes to do with weight. In Keane (1985). subjects in two conditions received one of two analogous solutions to the Radiation Problem. The only difference between the two solutions was that the critical object/concepts of one had the same FRAS as those in the target domain, while those of the other did not. As predicted, significantly more subjects receiving the former analogous solution solved the problem than those receiving the latter. This improvement in performance on analogising is, setting aside Gentner's theoretical proposals, best classed as a further effect of analogical relatedness. This is because what it basically reflects is t h e appropriate a n d adaptive exploitation of points of correspondence between the two domains in the analogy. To put it another way, it would be mal-adaptive for people to ignore these attribute correspondences in their analogising.

178

M. T. G Keane

However, it can be argued from other evidence in a n experiment by Gentner & Toupin (1986) that in certain situations similar exploitation of attributes proves to be non-adaptive. Gentner & Toupin gave two age-groups of children (5-7 year olds and 8-10 year olds) a task in which they first memorised a simple story, through acting it out, and then re-enacted the story with other characters. As such, the re-enactment demanded that the subjects analogically map the first story onto the characters of the second story. Two variables were manipulated in these story materials: (1) The systematicity or explicit causal structure of the originally-learned story was modified. The systematic version of the story has an added moral which lent a more coherent causal structure to it, by providing a rationale for the main actor's actions. This moral was absent from the non-systematic versions of the story. ( 2 ) The transparency of the object correspondences was made progressively harder by introducing various different characters into the to-be-enacted story. In the simplest case, the actors who had parallel roles in the target were from the same general category as their counterparts in the base story. So, for example, if the walrus was the main character in the base story, then a seal was the main actor in the to-be-enacted target. In the most difficult or nontransparent case, the actors in both domains were from the same general categories but the roles they took differed. So, if the walrus was the main actor in the base then the seagull might fill this same role in the target, while the seal played the role of some secondary actor. The general finding of interest for present purposes, was that in the latter case where category information encouraged the mismapping of objects to one another, subjects experienced a greater difficulty in performing accurate mappings. This effect was, however, lessened in the older-age groups where it was found that these subjects were exploiting the systematicity of the story in order to perform in a successful fashion. The fact, that this effect of object attributes was lessened as subjects got older and the unusual nature of the experimental situation all point to this effect being a local bias. That is, that in the normal course of events, the exploitation of attribute correspondences between the objects in both domains is a n effective and adaptive heuristic: however, in this experimental situation it is specifically non-adaptive to employ such a heuristic. So, it is a form of bias but merely a local one. There is an alternative

Biases and analogy

179

argument to this one which says that this is the result of a global bias in children. This view is supported by the known tendency of young children to deal with comparisons in an attributional rather than a relational manner. Exactly which of these interpretations is true remains to be determined by further research. The role of knowledge: bias or adaptation? Finally, in some recent experiments I have shown that making apparently minor changes to base analogues have profound effects on the success of the mapping process. This looks like responding which may be appropriately classified as a bias. However, as we shall see, this particular way of responding, which is not predicted by Gentner's syntactic theory, does seem to be highly adaptive. Typically, in these experiments, subjects were first presented with simple sentences, to read, of the form: "The football bounced across the street and hit the wall". They were told to read this sentence carefully and to imagine the state of affairs it described. Then, this sentence was removed and they were presented with three objects. Their task was to "tell the same story about these objects using the previous sentence". So, on being given "roof, basketball, tree" they would have to produce "The basketball bounced across the roof and hit the tree". Typically, subjects were given one of two sets of objects: they either received the basketball set or else "green, golf-ball flag-pole'' to produce "The golf-ball bounced across the green and hit the flag-pole". The difference between these two manipulations may seem slight, and structuremapping theory would predict that there should be no difference, but it has been reliably found that in the basketball case, where the events to be described are not commonplace, more errors are made in mapping. Furthermore, when one examines the times in the two conditions, one finds that the mapping is executed more slowly in the basketball-case than in the golf-ball case. The explanation proposed for these results is that the relations in the sentences are being mapped one-by-one in a n incremental fashion from the presented sentence to the objects. When an appropriate predication is found for the first relation (e.g., it is discovered that BOUNCE can be used to predicate the objects GOLF-BALLand GREEN, a long-term memory look-up occurs. This memory look-up mechanism searches long-term memory for causal schemata which have the same antecedent condition

180

M. T.G Keane

BOUNCE(G0LF-BALL, GREEN). If a match is found it then applies the consequent part of the schema to the remainder of the base sentence. So, if this long-term memory look-up succeeds the process switches from mapping from the base to the target to mapping from the schema to the base sentence. We would argue that this schema-fitting process is faster than mapping from the base so that in cases (like the basketball one) where there are no much-used schemata to be found this look-up fails and the time to respond takes longer. Furthermore. the error differences found are a direct consequence of this slowing up of processing because as more time elapses the base sentence, stored in working memory, is lost bit by bit. So, at a certain point when processes return to working memory to map the sentence, the information has been lost and errors are made (see Keane & Brayshaw. 1988, for a computer simulation of some aspects of this explanation). Again, when one considers this behaviour it is highly adaptive because processing is made more efficient by relying on a vast store of previously-encountered situations. The general overload on the system is reduced and fewer errors are made. In short, behaviour is not biased in any way. CONCLUSIONS

In conclusion, it is clear that at present there is little evidence for the presence of biases in analogical reasoning. However, this may simply be a function of the paucity of research in the field. I t is clear that even given what is known about analogising there may be certain conditions in which biases occur. For example, the above mentioned influence of long-term memory on analogising could conceivably give rise to distortions when the retrieved schema is inappropriate. What is clearly needed is more research on analogy so that these aspects of analogising are brought to light. REFERENCES Anderson, RC.. & Ortony, A. (1975).On putting apples in bottles: A problem of polysemy. c=ognitive Psvchology,2.167-180. Braine, M. (1978).On the relation between the natural logic of reasoning and standard logic. psvchQlnrrica1 Review.&, 1-21. Burstein. M.H. (1986).Concept formation by incremental analogical reasoning and debugging. In RS. Michalski, J.C. Carbonell. & J.M. Mitchell (Eds.), Machine learnine 11: An artificial intellieence a mroach. Los Altos, CA: Kaufmann. Carbonell. J.G. (19861.Perivational ana1cgv machine leaminc 11. Las Altos, CA: Kaufmann.

Biases and a n a l o g y

181

.

Evans, J 3 . B . T . (1982). n e DSVCholom of deductive reasoning London: RKP. Fitts. P.M.. & Posner, M.I. (1967). Human Derf0. Belmont. CA: Brooks Cole. Gamer, W. R (1962). LJ h 1 i n 8. NewYork Wiley. Gentner, D. (1983).Structure-mapping: A theoretical framework for analogy. Cognitive Science. z. 155-170. Gentner. D. (1989). Mechanisms of analogical learning. In S. Vosniadou & A. Ortony (Eds.), n Cambridge: CUP. Gentner, D.. & Toupin, C. (1986).Systematicity and surface similarity in the development of analogy. Cognitive Science, J&, 227-300. Gick. M.L.. & Holyoak, K.J. (1980).Analogical problem solving. Cognitive PSVC~O~O@, 2,306-355. Gick. M.L., & Holyoak. K.J. (1983).Schema induction in analogical transfer. Cognitive PsvcholQgy. G,1-38. Henle, M. (1978).Foreword to R. Revlin & R E . Mayer (Eds.). Human reasoning. Washington, DC: Winston. Holland, J.H., Holyoak, K.J.. Nisbett. RE.. & Thagard, P. (1986). Induction: Processes in inference. learning and discoverv. Cambridge, MA: MIT Press. 1 f Holyoak. K.J. (1985).The pragmatics of analogical transfer. Leamine and Mot ivatloq, 19,59-87. Holyoak. K.J.. & Koh, K. (1987).Surface and structural similarity in analogical transfer. Memow and Cognition. 15, 332-340. Inhelder. B.. & Piaget. J. (1958).m r o w t h of logical thinking from childhood to adolescence. London: Routledge & Kegan Paul. Kahneman. D.. & Tversky,,A. (1973). On the psychology of prediction. Psvcholoaicd Review. Sp, 237-251. Keane. M.T.G. (1985). On drawing analogies when solving problems. British J oumal Df P S V C ~ O I26. O ~ 449-458, , Keane. M.T.G. (1987).On retrieving analogues when solving problems. Quarterly Journal of Emerimental Psvcholom. =A. 29-41. Keane. M.T.G. (1988a). . g - h Chichester: Ellis Horwood. Keane. M.T.G. (1988b).Where’s the beef?: The absence of pragmatic factors in pragmatic theories of analogy. proceediqp n mificial Intelligence ‘88.London: Pitman. Keane. M.T.G., & Brayshaw, M. (1988).The Incremental Analogical Machine: A Computational Model of Analogy. In D. Sleeman (Ed.). EuroDean Working S p London: Pitman. Palmer, S.E.. & Ktmchi. R. (1986).The information processing approach to cognition. In T. Knapp & L.C. Robertson (Eds.). ADDroaches to cognition. Hillsdale. N J : LEA. Rips, L. (1983).Cognitive processes in propositional reasoning. Psvchological Review. 90.38-71.

: .

m

m

Acknowledgements . This work was funded by an Open University, Post-Doctoral Fellowship granted to the author. I would like to thank all those who have commented on the paper: Marc Eisenstadt, Mike Brayshaw, and Arthur Stutt. I would like to extend an especial gratitude to Ruth Byrne who h a s formed the paper through her criticisms.

This Page Intentionally Left Blank

Cognitive Biases J.-P. C a v e r n i , J . - M . F a b r e and M. Gonzalez ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B . V . (North-Holland),

1990

PRAGMATIC REASONING SCHEMAS FOR CONDITIONAL PROMISES:CONTEXT AND REPRESENTATION Henry Markovits and Christiane Lesage Department of Psychology, U n i v e r s i t y of Quebec, M o n t r e a l , Canada

Abstract. Two studies are presented that attempted to examine the influence of pragmatic factors (Cheng & Holyoak, 1985: Fillenbaum, 1975) on logical reasoning in adults. The first study hypothesized that reasoning with conditional promises of the form "if you do P then I'll do Q" should promote a biconditional mode of responding. Results of a paper and pencil test presenting both neutral conditionals and conditional promises supported this hypothesis. The second study examined the influence of context on the effect of conditional promises. Conditional promises were embedded in short paragraphs that described differently the agents involved in the conditional relations. In the authoritarian context, the promise was described as coming from a strong person, while in the non-authoritarian context, the person was described a s weak. As hypothesized. there were more biconditional responses in the former context. However, it was also found that there is an interaction between the effect of context and subjects' ability to reason correctly on neutral problems.

STUDY 1

Conditional (i.e. 'if-then') reasoning is one of the basic elements of deductive logic. Despite its importance, it is not clear what are the processes that adult subjects use to generate responses to conditional inferences. While different models have been advanced, one of the more intriguing involves the notion of pragmatic reasoning schema introduced by Cheng & Holyoak (1985). Their results specifically suggest that adults possess a general schema for permissions or obligations that is expressed in terms of 'if-then' reasoning, and which generates a formal structure that is identical to that of the logical conditional. Such schemas are the product of experience with conditional relations of the kind: "if you want to have a driver's licence, you must be more than 18 years of age". Repeated exposure to similar relations generates expectations as to the permissible combinations of antecedent and consequent terms (e.g. you cannot have a licence and be less than 18 years old), which may then be transferred to logical reasoning problems, A somewhat similar argumentation was developed on linguistic grounds by Fillenbaum (1975, 1976), who suggested that in certain

184

H. Markovits

&

C.

Lesage

instances it would make good pragmatic sense for an "if-then'' utterance to be interpreted as "if, and only if, then". His argument used the notion of invited inference (Geis & Zwicky, 1971). According to this notion, ordinary discourse depends on a n unspoken contract which allows speakers to make inferences based on convention and experience that go beyond the overt meaning of verbal exchanges. Among other things, Fillenbaum claimed that the assumption of contingency which is basic to the sense of a conditional promise ("If you do P then I will do 9"). makes it reasonable that the addressee should infer that if he [she] does not do P then the inducement (9) offered will not come. This implies t h a t repeated exposure to s u c h promises would generate expectations as to the permissible combinations of antecedent and consequent terms by the same process used to account for permission schemas. Fillenbaum's linguistic argument thus suggests that adults may also possess a pragmatic schema for conditional ("ifthen") promises, which (contrary to permissions) should generate a formal structure that is identical to that of the logical biconditional when used in reasoning problems. This study attempted to examine this hypothesis. Subjects were presented with a series of five conditional reasoning problems. The initial two involved if-then relations without human agents, the following three involved conditional promises. It was hypothesized that there would be a greater rate of biconditional responding to the promises than to the 'neutral' problems. Methodology Subjects A total of 130 university students (average age: 25 years: 6 months) of various concentrations were given the reasoning problems. Materials A paper-and-pencil test was used. This consisted of five conditional reasoning problems, all of which were in the same format. Each of the problems presented the statement "Suppose it is true that" at the top of the page followed by a n "if P then 9'' statement. Below this, were four multiple choice questions which took the following form:

Reasoning schernas for c o n d i t i o n a l promises

185

If P is true, you can say a) that it is certain that Q is true. b) that it is certain that Q is false. c) that you do not know whether Q is true or not.

The three other questions started with the statements "If Q is true", "If P is false" and "If Q is false". The order of the four questions varied between problems. The five conditional reasoning problems used the following ifthen relations: If there is an electrical failure, school will be closed. (SCHOOL) If a fish is red, it has wings. (FISH) If Lise cleans up her room, she will go to the movies. (LISE) If Claude gives his money back to Pierre, Pierre will lend him his car. (PIERRE) 5) If Josee is punctual, she will get a raise in salary. (JOSEE)

1) 2) 3) 4)

The first two problems (SCHOOL, FISH) did not involve promises nor human agents and were always presented in the given order. Subsequent to the SCHOOL and FISH problems, the three relations involving promises (LISE. PIERRE, JOSEE) were presented. The order of presentation of the latter three problems was systematically varied. Procedure Questionnaires were administered to complete classes. No specific instructions were set and subjects were allowed to continue until they had answered all questions to their personal satisfaction. Results

For each problem, four reasoning principles were examined. These corresponded to "P is true" (modus ponens), "Q is true" (converse), "P is false" (inverse) and "Q is false" (modus tollens). For both modus ponens and modus tollens, the conditional and biconditional interpretations give the same response ("Qis true" and "P is false", respectively). The major difference in reasoning patterns between a conditional and a biconditional interpretation occurs on the converse and inverse. For these questions, the conditional interpretation leads to an uncertainty response, while

186

H. Markovits

&

C. L e s a g e

the biconditional interpretation leads to responses of "P is true" and "Q is false". The observed results indicate that, as would be expected by the hypothesis, there are more biconditional responses for the inverse and the converse on the three conditional promises (LISE, PIERRE, JOSEE) than on the two neutral problems (SCHOOL. FISH), while there is not much difference on modus ponens and modus tollens. In order to examine the specific influence of the two problem types on the converse and inverse, a combined score for the number of biconditional responses on these two questions was calculated for each problem. In addition, a cumulative score for number of biconditional responses for the converse and inverse on the neutral and conditional promises combined was also calculated. This indicated that there was a n average of .65 biconditional responses on the converse and inverse (maximum score of 2) for the neutral problems and a significantly higher average of 1.38 biconditional responses on the conditional promises (t = 12.32, p < ,001). One question that might be raised with the global measure is that the conditional promises were presented after the neutral problems. It is thus possible that the increase in biconditional responding is due to this effect. To examine this, the number of biconditional responses on the inverse and converse was compared for each of the five problems. Table 1 gives the mean differences in biconditional responses over all five problems. The pattern of differences is on the whole consistent with the hypothesis. There is no difference between the SCHOOL and FISH problems, despite the fact that the second problem always preceded the first. However, the expected differences between each of the two neutral problems and the conditional promises are clearly present. Among the three promises, only the difference between the problems PIERRE and JOSEE achieved significance, although the absolute difference was much less than that observed between the neutral problems and the promises. These results thus confirm the basic hypothesis of this study and indicate that subjects do appear to possess a 'promise' schema that leads them to give biconditional interpretations on reasoning problems. Another question that can be raised concerns the relation between the level of biconditional responses on the neutral problems and that on the conditional promises. Although there is a higher level of biconditional responses on the conditional promises, the Pearson correlation between the two is quite high, .665

Reasoning schemas f o r conditional promises

187

(p < .OOl). This indicates that the tendency for subjects to respond biconditionally to the neutral problems affects the extent to which the conditional promises provoke biconditional interpretations, Table 1. Difference in the proportion of biconditional responses to the converse and inverse combined over the five reasoning problems. FISH SCHOOL FISH LISE PIERRE

.05 -

-

LISE

-. 6 8 * * -.72**

-

PIERRE

JOSEE

-.80** 85**

-. 6 1 * * -. 6 5 * *

-.12

-.07

-.

-

.19**

* * Indicates a significant difference. STUDY 2

These results are consistent with the notion that adult subjects possess a pragmatic schema for conditional promises, the structure of which is identical to that of the logical biconditional. They thus reinforce the notions of Cheng and Holyoak (19851,in that they indicate that there may be a variety of such schemas and that subjects' responses to logical reasoning problems depend to a certain extent on which of these schemas might be activated. The second study attempted to extend these findings by examining the effect of context. The pragmatic schemas approach (and others) assumes that context should affect reasoning by affecting the representation of the premisses and thus the strength of the inferences generated by the schema. This implies that it should be possible to influence the use of the conditional promise schema by incorporating the basic premisses within a context that alters the information provided not about the promise itself, but about the nature of the agents involved. Specifically, two kinds of contexts were generated. The first (authoritarian) kind, involved promises made by agents who were portrayed as being strong, bad-tempered, etc. The second (non-authoritarian) kind, involved promises made by agents who were portrayed as being weak, kindly-tempered. etc. In both cases, the promises were identical, both in substance and in

188

H. Markovits

&

C. L e s a g e

linguistic form. The only difference in the presentation of the conditional inferences that were required of subjects was in the preliminary context. In the authoritarian condition, promises were presented subsequent to a paragraph that attempted to implicitly strengthen the possible conclusion that the given "if-then'' relation could well be taken as an "if, and only if, then" relation. This was done by portraying the person who was said to have made the promise as severe, angry, etc. In the non-authoritarian condition, subjects were presented with a preliminary paragraph that was designed to discourage the implicit conclusion that the "if- then" relation was in reality an "if, and only if, then" relation. This was done by portraying the person who was said to have made the promise as friendly, weak, etc. It was specifically hypothesized that a greater proportion of biconditional responses would be generated with the threatening context than with the non-threatening context. In order to provide an explicit control for subjects' responses to other types of conditional problems, the same two neutral problems as in the initial study were presented before the conditional promises. Methodology Subjects A total of 218 university students (average age: 24 years: 1 1 months) of various concentrations were given the reasoning test. Of these, 107 did the threatening condition and 1 1 1 did the nonthreatening condition.

Materials Two paper-and-pencil tests were used. These consisted of the same five conditional reasoning problems used in Study 1, in basically the same format. The first two problems (SCHOOL, FISH) were considered not to involve promises and were presented without context for both tests. Subsequent to the SCHOOL and FISH problems, the three relations considered to involve promises (LISE. PIERRE, JOSEE) were presented in one of two conditions. In the t h r e a t e n i n g condition, they were presented subsequent to a paragraph that attempted to implicitly strengthen the possible conclusion that the given "if-then'' relation could well be taken as an "if, and only if, then" relation. A n example of one of the three threatening contexts is the following:

Reasoning schemas f o r conditional promises

189

Lise's mother is authoritarian and often angry, and attaches a great deal of importance to cleanliness. She demands that everything be carefully p u t away. Today, s h e is particularly intransigent on the subject since she is expecting the visit of her inlaws. While she is getting everything spotless, Lise asks permission to go to the movies. Exasperated by the messy state of Lise's bedroom, her mother replies to her in a very sharp way: "If you clean u p your room, you will go to the movies." Now, suppose it is true that If Lise cleans UD her room, she will ao to the movies. and respond to the following four multiple choice questions. In the non-authoritarian condition, subjects were presented with a preliminary paragraph that was designed to discourage the implicit conclusion that the "if-then'' relation was in reality a n "if, and only if, then" relation. This was done by portraying the person who was said to have made the promise as friendly, weak, etc. An example of one of the three non-threatening contexts is the following: Lise's mother is friendly and weak with her children and attaches very little importance t.o cleanliness. Today, s h e is expecting friends to dinner and is making an effort to put the house in order. In the mean time, Lise asks permission to go to the movies. Her mother replies to her: "If you clean up your room, you will go to the movies." Now, suppose it is true that If Lise cleans UD her room. she will ao to the movies. and respond to the following four multiple choice questions. Procedure The tests were administered to complete classes. The two forms of questionnaire were alternated in a large pile and distributed randomly to students. No specific instructions were set and subjects were allowed to continue until they had answered all questions to their personal satisfaction. Results For each subject, the average number of biconditional responses to the converse and inverse combined was calculated for the both the neutral problems and for the conditional promises. An analysis of variance with the context (threatening, non-threatening) as independent variable and the proportion of biconditional

190

H . Markovits

&

C. Lesage

responses on the conditional promises as dependent variable was performed. In order to control for the level of biconditional responses on the neutral problem, this was included in the model as a covariate. The resulting analysis of variance indicated that the context did have a n effect on the proportion of biconditional responses generated on the conditional promises (F(217.1)= 4.42. p < .05). As hypothesized, the average number of biconditional responses for the combined converse and inverse on the conditional promises was higher (4.59)with the threatening context than with the non-threatening context (4.00). The preceding analysis examined the overall effect of the conditional promises schema on the extent to which subjects produce biconditional responses to specific logical forms. However, it has been shown previously (Markovits, 1985. 1988) that understanding variations of the kind examined here can also incorporate the notion of correct reasoning. Thus, the final analysis examined the extent to which the effect of context interacted with the ability of subjects to reason correctly. In order to do this, subjects' responses to the two neutral problems were scored as logically correct if they responded correctly to all four logical forms on a given problem. Thus, scores varying from 0 to 2 were generated. Responses on the three conditional reasoning problems were graded along a three-point scale following Markovits (1985. 1988) in order to examine in more detail the effects of context variation. Subjects giving a fully correct response to all four logical forms on a given problem received 2 points. Subjects giving one of three types of intermediate responses (these are responses that differ from the logically correct response on one of the logical forms) received 1 points. All other responses were given 0 points. The scale thus examines the degree to which subjects are able to give logically correct responses irrespective of the effect of the conditional promises schema. A Mann-Whitney procedure was used to examine differences between threatening and non-threatening contexts for subjects giving 0, 1 or 2 correct responses to the neutral problems. This indicated that there was no significant difference between the two contexts for subjects giving 0 (z = 1.35. n.s.1 and 1 (z = 1.41,n.s.) correct responses. The total score was however higher for the non-threatening context for subjects giving two 2 responses to the neutral problems (z = 2.21.p < .01).This indicates that the effect of context was concentrated among subjects

R e a s o n i n g s c h e m a s for c o n d i t i o n a l promises

191

able to respond correctly to the two neutral problems. Generally, the results indicate that the overall proportion of correct (conditional) responses on the conditional promises increases with the level of correct responding on the neutral problems. It is thus clear that subjects that can respond correctly to the neutral problems appear generally more resistant to the effect of the conditional promises. More interesting in this respect is the fact that the effect of context is stronger among subjects responding correctly to both neutral problems than for the others. In the case of the latter, the threatening condition produces fewer conditional and intermediate responses and more biconditional responses than the non-threatening condition. DISCUSSION

These results indicate that the force of the conditional promises schema depends to a certain extent on the context in which it is embedded. They are thus consistent with the notion that when subjects reason, they may do s o by incorporating information that is not directly related to the premisses but which may influence the deductions that they make. However, they also indicate that there is a relation between the ability of subjects to reason correctly on more neutral content and the way that information provided by context is processed. In this case, the difference between the threatening and non-threatening context was more clearly apparent with subjects able to correctly respond to both neutral problems than with the others. This indicates that subjects who are better able to reason are more sensitive to the different kinds of information contained in two contexts. These results are thus consistent with previous ones (Markovits, 1988) that indicate that the way that subjects use information in reasoning is subject to internal constraints related to their level of cognitive competence. REFERENCES Cheng, P.W.. & Holyoak, K.J. (19851. Pragmatic reasoning schemas. Coenftfve Psvcholo&, l7.391-416. Fillenbaum, S. (1975).If: Some uses. Psvcholoeical Research, z,245-260. Fillenbaum. S. (1976).Inducements: On the phrasing and logic of conditional promises, threats and warnings. Psvcholocical Research, 23 1-250.

a,

192

H. Markovits

&

C. L e s a g e

wIna

Geis. M.L.. & Zwicky. A M . (1971).On invited inferences. 2(4). 561566. Markovits. H.(1985). Incorrect conditional reasoning among adults: Competence or performance? Journal of Psy&&gy. 26. 241-247. Markovits. H. (1988).Conditional reasoning, representation and empirical evidence on a concrete task. Quarterlv Journal of Emerlmental Psvcholom, in press.

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e a n d M. Gonzalez (Eds.) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

NON-LOGICAL SOLVING OF CATEGORICAL SYLLOGISMS Guy Politzer CNRS and U n i v e r s i t y of P a r i s V I I I ,

Saint-Denis,

France

Abstract.

I t is shown that a model based on linguistic (pragmatic and grammatical) principles can account for responses to formal categorical syllogisms. The model gives an explanation of the well-known biases in syllogistic reasoning (in particular the atmosphere effect and the figural effect). They are attributed mainly to the requirements of the testing format, which discourages subjects to make use of their reasoning abilities.

INTRODUCTION

The study of how subjects solve formal categorical syllogisms h a s uncovered the existence of a number of response biases attached to that task, one of which, namely the atmosphere effect, is often regarded as prototypical of the biases that affect human reasoning. I t will be argued that, as far as syllogism solving is concerned, these biases are essentially a consequence of the testing format and of the artificiality of the task which trigger a strategy based on a linguistic, rather than a logical, treatment of the syllogisms. DESCRIPTION OF SYLLOGISMS

Categorical syllogisms are deductive arguments made of categorical propositions. A categorical proposition is a proposition that affirms or denies that a class S (subject) is included in a class P (predicate) either in whole (universally) or in part (particularly), hence the following four propositions with their traditional abbreviation: A E

I 0

universal affirmative: universal negative: particular affirmative: particular negative:

All S are P No S are P Some S are P Some S are not P

The quantity refers to the universal/particular opposition and the quality to the affirmative/negative opposition.

G. Politzer

194

Categorical syllogisms are made of three categorical propositions. The first two propositions are assumed to be true: they are called premises (the first one is called the major and the second one, the minor). The third proposition is the conclusion whose truth value has to be evaluated. When the conclusion is necessarily true on the basis of the truth of the premises alone, the syllogism is said to be valid. In syllogisms, the propositions refer to exactly three classes: one of them is common to the premises and does not appear in the conclusion: it is called the middle term(M). The other classes that appear in the minor and in the major are noted S and P. respectively. Since M can occupy the place of the subject or of the predicate, there are four types of pattern, calledfiure. as follows: 2

1

4

3

M

P

P

M

M

P

P

M

(major)

S

M

S

M

M

S

M

S

( m i n o r)

Allocating each of the four proposition types (A, E, I , 0) to each of the two lines produces 42 = 16 pairs of premises for each figure, hence a total number of 16 (pairs) x 4 (figures) = 64 problems which are called "syllogisms" in psychological context, although the conclusion is not expressed. This is because the subjects of the experiments generate the conclusion or evaluate it in a multiple choice response format: in the latter case there are five options, namely, A, E,I, 0, (all S are P. etc.) and a fifth option of the type "no valid conclusion is possible" (henceforth NVC). Similarly, the mood refers to the 16 types of pairs of premises, disregarding again the conclusion. Notice that in logic the conclusion is such that S and P occupy the place of the subject and of the predicate, respectively ( S P conclusion). However, if the individual is free to produce his own response, he may state it in the reverse direction as well (PS conclusion). THE PHENOMENA TO BE EXPLAINED

Most investigators of categorical syllogisms have used the fiveoption multiple choice format with the presentation of the 64 problems on booklets. Most phenomena that characterize subjects' behavior have been observed in this standard paradigm (with formal

Syllogisms

195

material, i.e., material such that the individual cannot use his knowledge of the world to evaluate the conclusion). The present paper aims at explaining this behaviour. The main phenomena to be accounted for are the following. First of all, the subjects' categories of response: For each problem, the frequency distribution of the responses (A, E, I , 0, N V C ) is fairly stable across studies. Second, the well-known atmosphere effect which says that when at least one of the premises is negative, then the most frequent conclusion is negative: and when at least one of the premises is particular, then the most frequent conclusion is particular (Begg & Denny, 1969). Third, the figural effect (observed when the individual is free to generate his response): it consists in the individual's giving more SP, than PS, conclusions on the first figure whereas it is the reverse on the fourth figure (Johnson-Laird & Bara, 1984). DESCRIPTION OF THE MODEL

The present model is based on six principles. Its main metatheoretical assumptions are (i) the existence of betweensubject differences in strategy: consequently, the model claims to explain the behaviour of the majority of the subjects only; (ii) the non-homogeneity of syllogisms, i.e.. there exist a few syllogisms which have very salient formal characteristics that invite subjects to use a heuristic that is both available and adequate, that is, a transitivity schema applied to syllogisms such as AA-1 (all M are P; all S are M),AI-1 (all M are P: some S are M).EI-1 (no M are P; some S are M), etc. The first four principles of the model determine the category of the response (A, E, I, 0, W C ) ; they are based on pragmatic considerations. The last two principles determine the direction of the response ( S P or PSI: one is based on grammar, while the other is a procedure for combining the premises.

pragmatics The use of letters to designate the subject class and the predicate class of the premises in most studies goes counter to what individuals are accustomed to in their daily reasoning, that is, welldefined universes. (Besides, most syllogisms have little ecological validity, in spite of the claim to the contrary made in the past). It is

196

G. Politzer

assumed that, because the classes have no reference, individuals interpret the two premises as the assertion of rather unspecified laws. What differentiates the sentences is basically the quantifier, that is, the degree of generality of the sentence. I t is also assumed that the individuals regard sentences which are both affirmative and universal, that is, A sentences, as the standard form and also the best form to convey a law. Further, the other sentences (I, E, 0) which assert that some X are..., some X are not..., a n d no X are ..., respectively, are restrictive assertions as compared to A sentences. This is especially clear and well documented for Isentences to which the majority of the subjects add the Gricean implicature, but not all (Begg & Harris, 1982; Newstead, 1989; Newstead & Griggs, 1983; Politzer, 1990). Negative sentences 0 and E convey a restriction even more radically since 0 sentences negate A sentences (both logically and pragmatically) and E s e n t e n c e s pragmatically negate A sentences (even more strongly than 0 sentences do) (Politzer, 1990). Consequently, the negative quality is a kind of mark of restrictiveness, so that when a n E or a n 0 sentence is presented, the gist of the message being conveyed is a restriction. In brief, categorical sentences follow a hierarchy from the most general to the most restrictivem: A before I, and I in turn before E and 0. Next, research on the interpretation of quantifiers with formal materials has shown that I and 0 sentences are virtually synonymous (Begg & Harris, 1982; Newstead, 1989; Newstead & Griggs, 1983; Politzer, 1990) so that spontaneous inferences between them are likely to occur. In particular, inference from 0 to I is likely to be performed as a kind of normalization to obtain an affirmative, more acceptable, statement. I t is further assumed that subjects are aware that the structure of syllogisms is conjunctive, i.e., the conclusion is determined by both premises. In particular, the degree of generality of the conclusion depends on that of each conjunct and cannot be higher than that of any one of the premises. In other words, although subjects are inclined to assert as general a conclusion as possible, there is a n upper bound which is determined by the premise that is the most restrictive. Consequently, although the conclusion should be as general as possible (ideally a n A sentence) the premises limit the generality of the conclusion in that (i) if both premises are of the same category, the conclusion cannot be more general than that of the common category, and (ii) if the premises are of different

Syllogisms

197

categories, the conclusion cannot be more general than the less general of the two. The last assumption is that subjects who are presented with two premises formulated in terms of P and M, and S and M, respectively, will formulate their conclusion in terms of S and P. that is, without M. The reason is that the middle term should be perceived only as a means to relate S and P since its presence in the conclusion together with one of the end terms would render the premise that contains the other end term irrelevant, and the premise that contains the same end term either logically or pragmatically contradictory to, or redundant with, the conclusion. This point was made by Johnson-Laird & Bara (1984)based on slightly different considerations, namely, informativeness. The foregoing considerations on pragmatics justify the following four rules which constitute the first set of principles of the model.

Rule 1. I f there is an A (or I) premise, then the response category is that of the other premise (with A taking precedence over I). Rule 2. I f there is an E or an 0 premise, then the response is E or 0. Rule 3. An 0 response may be transformed into I. Rule 4. The conclusion is formulated in terms of S and P only (the middle term does not appear).

We shall now consider the subject and the predicate of categorical sentences. I t is generally agreed that the subject and the predicate of categorical sentences share differently the information conveyed by the sentence (Clark & Clark, 1977). It will be assumed that the subject carries the information related to the quantity, and the predicate carries the information related to the quality. This assumption is in line with grammar and logic from Aristotle to the modem times. Hence:

Rule 5. In categorical sentences, the subject encodes the quantity and the predicate encodes the quality.

198

G. Politzer

Premise combination

The last principle makes use of the notion of afeature. Each categorical sentence is defined by two features that represent its quantity (+ universal: - particular) and its quality (+ affirmative; - negative), respectively, so that: A = (+ +): I = (- +); E = (+ -1: 0 = (- -) The pair of signs that defines a categorical sentence will be called a frame. Introducing the subject X and the predicate Y into the frame, a proposition such as, e.g.. some X are Y will be represented by (-X +Y) where the attachment of the quantity (-1 to the subject and the quality (+) to the predicate is motivated by, and makes sense only'through, rule 5. Now, rule 6 spells out which terms will be selected to produce a conclusion (if any) when the premises are combined. When the premises are being combined, rules 1 and 2 determine the category of a tentative response, so that the features of that response can be identified: they constitute a frame that needs to be filled u p with terms: rules 4 and 5 come into force to determine the selection of these terms. According to rule 5, the components just abstracted from a premise tend to attract into the frame the terms that are attached to them in the premises. But generally the result cannot be a final response, because when an M term is attached to a component, rule 4 prevents it from entering the frame. Consequently, the individual will reject this way of filling the frame with an M term. He now has two possibilities. Either he decides that there is no valid conclusion Such a decision is made with a probability that is a parameter of the population. Or, he alters the M term. This can be done by substituting one of the end terms for M, which is possible because either S or Pis always available. For, if the frame attracts M and S , P is available and can be substituted for M; and similarly, if the frame attracts M and P. S can be substituted for M. A categorical response follows as determined by the frame and expressed in terms of S and P. (In a multiple choice response format, this process of substitution is to some extent compelled by the presence of the option in terms of S and P.) Notice that by assuming that S or P is introduced into the frame to replace M , while the other term has already been positioned, the theory predicts and explains the directionality of the response ( S P versus PS) as will be seen in detail below.

199

Syllogisms

Rule 6. A selected component (quantity or quality) attracts the term that is attached to it in the premises in order to constitute the conclusion. I f the term attracted is S [or P), then it appears in the conclusion. I f the term attracted is M , then either an NVC response is given, or the P (or S ) term is substituted for M. DERIVATION OF THE RESPONSE AND MAIN PREDICTIONS

The derivation of the response will be treated in detail for one pair of premises. I t will be easy to generalize the production of the response to the other pairs. We shall consider the IA pair and we shall treat the four figures in parallel. Lines 1 and 2 below represent the two premises. Line 3 shows the frame that is abstracted from the lower premise in the hierarchy, viz. I. according to rule 1. As shown by lines 4 and 6, there are two ways of filling up the frame, since there is a (+) quality feature in both premises. The filling of the frame (which is more likely to follow line 4 than line 6 because of the within-premise association of the terms and of the contiguity in the premises of the terms that constitute the former) obeys rule 6. When an M term is attached to a feature that serves to constitute the frame, it is replaced with a triangle (A) to indicate that M cannot enter the frame, according to rule 4; lines 5 and 7 show the term substitution as described by rule 6. Line 8 and below indicate the predicted responses obtained by summation of lines 5 and 7. 1

2

3

4

-M +P +S tM

-P +M +S +M

-M +P

-P +M

:I 1

+M +S (+)

+M +S

:A

(-

(-A -s

+P) +P

(-P +S) -P +s

4 5

(-A +S) +s

(-P tS) -P +s

7

-s

tP -P

-P

+S

(-

+)

(-

(-A

tP) tP

(-P +A)

(-A tA) -P +s -S tP

(-P tA) -P +s

-s +P

-P ts

-s

or 1) 2)

-P

-P

+)

+s

+s

NVC

NVC

-P

1) 2)

NVC

2

3

+)

6

+s

NVC (weak)

8

200

G. Politzer

The result is an I conclusion on all four figures. On figure 1, there will be more conclusions in the S P than in the PS direction; on figure 3 also, but relatively fewer SP responses than on figure 1 (because the second filling does not contribute to any S P conclusion). For figures 2 and 4, there should be (ideally) no S P responses (but one should expect a background "noise") and the absolute number of PS responses will be higher on the fourth than on the second figure; this is due to the fact that there will be fewer NVC responses on the fourth figure because there is no M term involved in one of the two fillings. (The process of substitution for M is the source of the NVC response, according to rule 6.) We can summarize these comparative predictions by using for each pair of premises, for each figure and for each response category the following ratio: frequency of responses in the SP direction = frequency of responses in the PS direction Then, for the ZA case under consideration, we have the following prediction: r l > r3 > r2 > r4. This kind of relation will be called a figural relation. A similar derivation can be made for the opposite pair of premises, AZ. It yields the figural relation r i > r2 > r 3 > r4 (notice the different order of the indices than in the IA case). The smallest rate of W C responses is predicted to occur on the first figure. Similarly, for all pairs of premises, it is possible to derive one, two or three categorical responses together with their figural relation attached to these responses, and predictions regarding the relative rate of NVC responses across figure. These predictions are summarized in Table 1. EVALUATION OF THE MODEL

Two sources will be considered: one, the data available in published reports known to the present author that are comprehensive enough, namely Chapman and Chapman (1959); Dickstein (1978); Guyote and Sternberg (1981); Johnson-Laird and Bara (1984); Roberge (1970); and, two, results of a new experiment. Categories of response All the categories presently predicted occurred in all the reports just mentioned; in other words, there are no false positives.

201

Syllogisms

However, there are a few cases of false negatives (unpredicted responses that appeared), namely, on AA-3 and AA-4, response I; on EA-3 and EA-4, response 0 and on the EI and IE problems, response 0. All these cases regard valid syllogisms to which the subjects gave correct conclusions. They clearly belong to those cases where some subjects reason correctly: such problems have been identified as solvable by specific heuristics. In brief, over the 64 problems, the predictions are well satisfied. Table 1. Predicted conclusion, figural relation, and relative rate of NVC responses for each pair of premises. Predictions Premise pair

Conclusion

F i g u r a l relation

NVC responses

A0 OA EI IE

rl

AE EA A1 IA EO OE EO OE I0 01

AA EE I1 00

F i g u r e w i t h fewer

=

r 2 > r3

r l = r3 > ‘2

r3 > r2 r2 > r3

=

r

=

4 4

r4 r4

r1

=

rl

=

rl r l r l r l r l r l r l

> r 2 > r3 > r 4

4

> r > r > r 4 3 2 > r 2 > r > r4 3 > r > r > r 4 3 2 > r > r 2 > 4r 3 > r > r > r 4 2 3 > r > r > r 2 3 4 > r3 > r2 > r 4 > r 2 > r > r 4 3 > r3 > r2 > r 4 > r2 = r3 > r 4 > r2 = r3 > r 4 > r2 = r3 > r 4

1

rl r l

r1 r1 rl

= =

rl r1 > r 2 = r3 > r

4

1 4 1 & 4 1 & 4 1 & 4 1 & 4 4 1

1 & 4 1 & 4 1 & 4 1 & 4

202

G. Politzer

Rate of "NOValid Conclusion"responses Predictions were made regarding t h e figure that would produce fewer NVC responses, for most pairs of premises (that is, wherever applicable). Counting the number of differences that are in the predicted direction, the results of all the studies confirmed the predictions. Two studies show only a tendency: the other three give levels of significance of 3 ~ 1 0 -3~~, 1 0 -and ~ . 1 ~ 1 0(sign - ~ test).

Figural relations They could be tested only with the data of Johnson-Laird and Bara (1984) since the other studies did not allow free responding. Results show excellent agreement on all pairs of premises but two (OA and EO where the agreement is fair only). These three series of predictions were tested in a new experiment where subjects were presented with a nine-option multiple choice test, in which four options were in the SP direction (identically to the standard material) and the four matched options in the PS direction (the last option being N V C ) . The results gave categories of conclusion as predicted (except that there were again 15% to 30% of unpredicted 0 responses on four of the EZ/ZE syllogisms). The analysis by individual problem gave a fit of ,84. For the N V C responses, 20 out of the 22 comparisons were in the expected direction and there were two ties ( p c 1 ~ 1 0 - ~The ). figural relations were tested by a Treatment (Figure) x Subjects analysis of variance which confirmed that all the eleven relations obtained. The 33 equations and inequalities were tested by t-tests with appropriate contrasts or by a non-parametric test: all the results were significant a t levels that ranged between 1 ~ 1 0 and -~ MO-~. The derivation of the responses described above applies to the standard five-option paradigm as well as to the free response paradigm. It was predicted that, with the former, the putative P S conclusions would be converted into S P conclusions, hence the predictions that (i) the rate of categorical responses would be higher with the five-option than with the nine-option format on those problems that offer a conclusion in terms of P S only, and (ii) there would be no difference between the two response formats on those problems that offer an SP-only conclusion. These predictions

Syllogisms

203

were confirmed by comparing the experimental group with a control group that received the standard five-option material. The "effects":Atmosphere and Figure

The atmosphere effect can easily be accounted for in the present framework: It is just a n approximation to the operation of rules 1 and 2. Notice that the notorious failure of the atmosphere theory to predict responses accurately on the EO and OE pairs of premises is not repeated by the present model. As far as the figural effect is concerned, it can be considered that its description by Johnson-Laird and Bara (1984) as a phenomenon that concerns the first and fourth figures only is a rough approximation to a phenomenon that is both more subtle and more general: In the present framework it was predicted, and it was empirically verified, that all four figures affect the direction of the response. The present model is powerful enough to make predictions case by case, on each pair of premises, as exhibited by the figural relations (Table 1). CONCLUSION

The classical study of syllogism solving does not tell much about how subjects reason. Rather, it tells what kind of strategy subjects resort to when they are presented with problems (i) a majority of which are beyond their capabilities: (ii) that are to be solved with time constraints (even implicit): (iii) whose degree of difficulty varies greatly but that have high superficial similarity to each other and so can give rise to uncontrolled effects such as "einstellung" or invalid generalization. I t is the experimenter who induces the observed biases in presenting such a task: the biases become the object of studies in a vicious circle. One way to escape it is to abandon this experimental paradigm. Much could be learnt from the use of the few syllogisms that lie within the subjects' capabilities (in particular, the EZ/ZE pairs and some of the pairs with an A premise) provided the problems were presented one at a time in a well controlled manner. For a study of reasoning to take place, the subject should be given an opportunity to reason.

G. Politzer

204

REFERENCES Begg, I.. & Denny, J.P. (1969). Empirical reconciliation of atmosphere and conversion interpretations of syllogistic reasoning errors. ExDerimental Psvcholoa. 81,351-354. B e g . I.. & Harris, G. (1982). On the interpretation of syllogisms. Journal of Verbal Behaw. 595-620. 1 f Chapman. I.J.. & Chapman, J.P. (1959). Atmosphere effect reexamined.

a,

. m E

58.220-226.

Guyote. M.J.. & Sternberg. R.J. (1981). A transitive-chain theoIy of syllogistic reasoning. Cornitive Psychology, 461-525. Johnson-Laird, P.N., & Bara. B.G. (1984). Syllogistic inference. Cognition. 16, 1-61. Newstead. S.E. (1989). Interpretational errors in syllogistic reasoning. Journal Q€ ,28.78-9 1. Newstead. S.E.. & Griggs, R.A. (1983). Drawing inferences from quantified statements: A study of the square of opposition. Journal of Verbal Learning and Verbal Behavior. 22.535-546. Politzer. G. (1990). Immediate deduction between quantified sentences. In K.J. Gilhooly. M.T.G. Keane. RH. Logie. & G. Erdos (Eds.). m e s of thin kina; ReflecUm on the DSVof .-t Vol. 1. London: John Wiley. Roberge. J.J. (1970). A reexamination of the interpretation of errors in formal S w . 331-333. syllogistic reasoning. ci-

u,

u,

SECTION 3 Response biases and context effects

This Page Intentionally Left Blank

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e a n d M. G o n z a l e z ( E d s . ) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

RESPONSE BIAS AND CONTEXTUALEFFECTS: WHEN BIASED? Allen Parducci U n i v e r s i t y of C a l i f o r n i a ,

Los Angeles, USA

Abstract. In a very particular but crucial sense of the term, "response bias" may be defined a s systematic distortion in the reporting of experience. This chapter challenges the common belief that there is response bias when the judgment of a stimulus can be shown to depend upon the context for judgment. The emphasis is upon my range-frequency theory of how category ratings are determined by the context. The challenge to claims for response bias is also applied to the theories of signal detection and information integration and to the pragmatics of everyday discourse.

The term "response bias" can refer to systematic distortion in the reporting of experiences. Although this meaning seems to push "response bias" beyond the pale of prescriptive behaviourism (Watson, 19 13). the pejorative connotation of "distortion" often creeps into behavioristic interpretations. This seems obvious when there is a motive to deceive, as when test-takers distort their selfreports to gain social acceptance (Edwards, 1957). A similar emphasis upon distortion seems intrinsic to interpretations of the effects of changing signal probabilities in experiments on signal detection (Green & Swets, 1966) where it is not clear that experience remains constant in spite of changes in the criteria for detection. My concern here is to challenge the widely-held assumption that contextual effects in judgment imply distorted reporting of unchanging experiences. The assumption that contextual effects imply response bias is itself based on a network of more basic assumptions. Different assumptions yield different interpretations of the same data. I propose to illustrate these differences with a consideration of how the distribution of recently presented stimuli determines the category rating of any particular stimulus. I will be comparing different interpretations, with special emphasis on my rangefrequency theory of judgment.

208

A . Parducci

RANGE-FREQUENCYTHEORY This theory of contextual effects (Parducci, 1965, 1983) is based on Gestalt relationalism, developed and tested with simple psychophysical experiments, and applied primarily to category ratings. Gestalt Relationalism

In spite of Gestalt psychology's historical preoccupation with "wholes" (e.g., Wertheimer, 1938),its most enduring contribution may prove to be its emphasis upon stimulus relationships as the determinants of phenomenal experience. A particular stimulus may be experienced quite differently in different contexts. For example, Helson's (1938)and Wallach's (1948)experimental analyses of color induction specify quantitatively the dependence of hue and brightness upon the relationship between the focal stimulus and its immediate surround. Even if one were to assume (with Helmholtz) that the changes in reported color are the product of "unconscious inferences," direct personal experience precludes interpreting t h e s e c h a n g e s as merely intellectual - - r a t h e r t h a n phenomenological. The Gestalters' rejection of the assumption of a one-to-one relationship between stimulus elements and experience makes application of the term "response bias" much more problematical. Category Ratings

The relational effects of context seem particularly obvious when experiences are reported as category ratings. A soft drink sweetened with a particular concentration of sucrose is rated much sweeter when lower concentrations predominate in the set of drinks presented for judgment (Riskey, Parducci. 81 Beauchamp, 1979). Category ratings, like "deliciously sweet." "good," or "slightly disappointing," are the way we ordinarily express the value components of experience. Sometimes there may be distortion as when we tell a musician that we enjoyed his performance even though we were not actually attentive to it. This overt inconsistency might support a behavioristic conception of response bias (Ryle, 1949). But we also use category ratings when we are making silent evaluations to ourselves and in conditions where we are trying to

Response bias and c o n t e x t u a l e f f e c t s

209

characterize without distortion exactly what we have been experiencing. Combining the Gestalt assumption that experiences depend upon stimulus relationships with this assumption that category ratings are natural expressions of value experiences suggests that changes in stimulus relationships should produce predictable changes in reports of experience. Changes in category ratings of unchanging stimuli may thus be veridical, i.e., made without response bias. Indeed, failure of the rating to change with changing stimulus relationships may itself be evidence for bias. The Stimulus Context for Category Ratings In simple psychophysical research, the experimenter tries to control the important stimulus relationships affecting the judgments, viz., their context. For example, in the research on category ratings of sweetness, the experimenters varied the skewing of the frequency distribution of concentrations of sucrose presented during the experimental session. Because the ratings proved so predictable when range-frequency theory was applied to each distribution, it seems reasonable to assume that the effective context for judgment was the set actually presented by the experimenter. I t is something of a mystery how the entire set gets represented with such accuracy. Because the different concentrations were presented in random order (sampling without replacement from short, representative blocks), the effective context might have been based upon only the most recent presentations, perhaps as few as the last dozen (Wedell, 1984). However, there seems no reason to assume that we are conscious of these contextual elements when making a category rating. The rating seems to describe only the focal stimulus, the stimulus that is being judged - even though experiments demonstrate that the rating depends upon the relationship between this stimulus and the other stimuli that have been presented. Suppose that the representations of contextual stimuli were to shift systematically in memory. This has been assumed when interpreting classical time-order errors, as when a 1OO-gram comparison weight is regularly judged "heavier" than the 100-gram standard that preceded it. In this case, we might say that the memory for the standard has grown "lighter." I s this systematic

210

A . Parducci

tendency to report the comparison stimulus as "heavier" a response bias? The ratings distort the physical relationships between the stimuli: but if the standard has lightened in memory, the ratings may not be distorting the subject's experience of comparison.

The Contextual Range Range-frequency theory places particular emphasis upon the endpoints of the context, the lowest and highest of the stimulus values affecting the judgment of any particular stimulus. The respective rating-categories are assigned to equal contextual subranges. In the simple number-judging experiments for which range-frequency models were first developed (Parducci. Calfee, Marshall, & Davidson, 1960). these subranges correspond to equal stimulus differences. For example, when numbers in a set from 100 to 1000 are rated using nine categories, each category is assumed to correspond to a contextual subrange of 100 (100 to 200, 200 to 300, etc.). But this simple division rarely works with physical dimensions: category ratings of the heaviness of lifted weights tend to mark off subranges defined by equal physical ratios rather than equal physical differences. Following Fechner, Helson (1964) assumed this logarithmic scaling for his applications of adaptationlevel theory. S.S. Stevens (1971) objected, interpreting any tendency of category ratings to follow Fechner as evidence for response bias: Stevens assumed, on the basis of subjects' "magnitude estimations" (i.e., their numerical estimates of the ratios of sensations), that the magnitudes of experienced sensations must be a power function of stimulus intensities. Neither the logarithmic nor the power assumption is incorporated into range-frequency theory. Indeed, the Gestalt assumption that there is no one-to-one relationship between stimulus element and experienced sensation, that experience depends upon stimulus relationships. precludes the possibility of a simple "psychophysical law," whether Stevens' or Fechner's (Parducci, 1982). The subjective location of the stimulus in the range of contextual stimuli assumes nothing about what is actually experienced: there is no claim that the hypothetically-equal subranges are psychologically equal, only that this range values are useful for predicting how stimuli will be experienced in various distributions.

Response b i a s and contextual e f f e c t s

211

Birnbaum (1974; 1982) has argued that constancy of these inferred scale values provides a necessary restriction for testing range-frequency theory or any other algebraic model in which stimulus values are represented. His assumption of "scale convergence," i.e., constancy or inferred stimulus values for different tasks or conditions, is thus a methodological convenience that does not address the question of what is consciously experienced. Deviation of the overt category ratings from these inferred range values is not prima facie evidence for response bias in the sense of ratings distorting subjective experience. If the range values, conceived as the judgments that the stimuli would have elicited if determined only by position in the contextual range, are not themselves conscious experiences, what are they? I think of them as inferred parameters whose values can usually be assumed to be independent of differences in the frequencies or spacing of the stimuli. Each stimulus is represented theoretically by its position in the contextual range, a representation that is useful in predicting how the stimulus will be experienced. Any lawful relationship between the range values and the physical values of the stimuli would depend upon various features of the context, including anchors (Sarris, 1967; Sarris & Parducci, 1978). For example, the inferred upper-endpoint of the contextual range of square sizes is larger than the largest square in the set actually presented. The magnitude of this upward extension of the contextual range varies directly with background size (Parducci & Perrett. 1971). Since this effect is inferred from changes in category rating, i.e., from changes in the reports of how the size of any particular stimulus is experienced, some might label it "a response bias." But that would be a rejection of the Gestalt assumption that changes in the immediate background change how the stimulus is experienced. The range values in range-frequency theory are usually inferred from the category ratings in conjunction with a specific range-frequency model. This inference might be made from ratings of equally-spaced stimuli and then used to predict ratings of unequally-spaced stimuli (Parducci & Perrett, 1971). Alternatively, a best-fit function relating range values to physical values might be inferred for many different sets of stimuli, again assuming that range values do not change when the endpoints and background are held constant (Parducci & Wedell, 1986).

212

A. Parducci

The Contextual Frequencies Category ratings are extremely sensitive to the relative frequencies of the stimuli presented for judgment. The plot of the ratings against stimuli has the general form of the cumulativefrequency function, steeper where stimuli are more frequent. My favored interpretation of this frequency principle is that subjects tend to put the same number of contextual stimuli in each of the available categories. If the context for judgment is simply the set of presented stimuli, then frequency values. i.e.. what the ratings would be if subjects simply followed the frequency principle, can be calculated a priorf from the stimulus frequencies. When there are the same number of categories as stimuli and the different stimuli are presented with equal frequency, a different category should be used for each stimulus; but when the same stimuli are presented with unequal frequency, the ratings of the most frequent stimuli must be divided between two or more categories. For example, if the smallest of five sizes is presented on 40% of the trials, it must evoke the lowest two of the five categories, each for half of its presentations. This assumes that there is perfect discrimination in the inferred process of determining frequency values. Although an early range-frequency model allowed Thurstone-like discriminal dispersions (Parducci. 1965). the added complication did not seem justified by the differences in the respective fits to the data (Parducci & Perrett. 1971). Another statement of the frequency principle, one favored by those who think of it as a response bias (e.g., Anderson, 1982; Upshaw & Ostrum, 1984; Poulton, 1979). is that subjects tend to use the available categories with equal frequency. Does this statement differ in meaning from my assumption that subjects tend to put the same number of stimuli in each category? I t seems to me that the two statements have different implications, and I interpret some experimental evidence as favoring my own version (Parducci, Knobel, & Thomas, 1976). If one thinks of category ratings as a way of identifying a particular stimulus from a known set, equalizing frequencies may be the most generally useful heuristic. The frequency principle (either way it is stated) tends to insure maximum transmission of information in Shannon's sense (Garner, 1962). Although there are specific situations in which it does not optimize correct identifications (Fabre, 1987). it works fairly well across a broad

Response bias and c o n t e x t u a l e f f e c t s

213

variety of stimulus sets without having to be modified for different distributions. However, when the frequency principle requires applying different categories to the same stimulus, there may be a counteracting consistency rule that modifies the judgments (Fabre, 1987; Parducci 81 Wedell, 1986). Since the equalizing of frequencies seems to be applied unconsciously, in what sense can the rule be taken as a response bias? I t is not as though the subject said to himself, "this size looks average to me, but there have been so many larger sizes that I had better call it small." Rather the subject might say, "compared with all those larger sizes, this one is really small." Its smallness is a n objective feature of its relationship to the other sizes in the context for judgment. I t is this relationship that determines the experience of smallness. A response bias would be manifest if the subject said, "this size looks small but I better call it medium so as not to seem influenced by all the larger sizes."

Range-FrequencyCompromise There seems n o reason to believe that this kind of hypothetical articulation by the subjects occurs even for the simultaneously-presented sets of numbers. Persistent postexperimental prodding can elicit some form of either the range or frequency principle from an occasional subject, but even in such exceptional cases the actual ratings do not simply follow the articulated principle. In many conditions, ratings by individual subjects exhibit a roughly equal compromise between range and frequency values: it is not that some subjects follow the range principle, others the frequency principle, with the group average consequently falling between the two; rather, the rating by each subject is intermediate between what it would have been following either the range or the frequency principle. Consider a concrete example from everyday life. A man of 195 cm would ordinarily be described as "very tall" (though not in the context of professional basketball). The same category might also be applied to a man who was 215 cm. "Very tall" can thus cover a subrange of at last 20 cm. However, "medium" or "average" height covers a subrange of less than 10 cm: 180 cm is already "tall" while 170 cm is "short." This might appear inconsistent with the range principle that successive categories correspond to equal subranges; however, there are many more men with heights near the middle of

214

A. Parducci

the range than those out toward either extreme. Even for people who have never heard of the Gaussian distribution, the contextual distribution of heights has been quasi-normal in their own experience and also in the experience of anyone to whom they are likely to be communicating. This sharing of context allows the compromise between range and frequency principles to communicate heights almost as accurately as if the estimates were made in cm. A formal statement of the range-frequency compromise is given by the following equation: [1 Jic= WRic + (1 - W]Fic where J k is the judgment of Stimulus i in the Context c, R k and F i C are its range and frequency values in this context, i.e., what its judgment would have been following either the range or frequency principle, and w is their relative weighting in determining the actual judgment. This judgment is expressed as a category rating. CR. perhaps as a verbal category (in which case the experimenter conventionally tabulates its rank in the set of categories): CRk = bJk + a i2 The category rating is thus assumed to be simply a linear transformation of the judgment, with the multiplicative constant, b. being the range of category ranks (e.g., 8 for a set of nine categories), and the additive constant, a, being the value assigned to the lowest category (usually 1). The value of w is typically close to .5. However, it can appear to vary inversely with the number of categories when stimuli are presented with unequal frequency: a n alternative interpretation holds w constant at .5, with the effective contextual frequencies of the more frequent stimuli progressively limited by increasing numbers of categories (Parducci & Wedell, 1986).

I t should be reiterated that neither the range nor frequency values are experienced consciously, and the subject or judge may not even be aware of the context in which these are established. What is conscious is the judgment, the sense of tallness, goodness, or whatever psychological value is being experienced. I t is this subjective value that is communicated directly by the overt category rating. Insofar as range-frequency theory is applicable, response bias is exhibited when the overt rating does not correspond to the subjective judgment in accordance with Equations 1 and 2.

Response bias and contextual effects

215

MODELS OF RESPONSE BIAS

Although there has been little attention to response bias in applications of range-frequency theory, the concept is often employed in applications of other theories of judgment. Signal Detection Theory

In the late 1920's. L.L. Thurstone presented a theory that proved useful for separating the discriminability of stimuli from the criteria for judgment (Thurstone, 1929; adapted by Torgerson, 1958, Ch. 10). This approach assumes that each stimulus and each category limen can be represented as a normal distribution of momentary values on the dimension of judgment. Conditions that systematically change the locations of the limens do not ordinarily affect the locations of stimuli on the dimension of judgment and hence their discriminability or differences in scale value (e.g., Parducci & Perrett, 1971; Thurstone. 1931; Upshaw. Ostrum & Ward, 1970). For example, although particular political statements about war are rated more militaristic by those people that hold more pacifistic positions, the statements have the same Thurstone values on the scale of "dovishness-hawkishness"for both groups. The issue of response bias here then is whether, as suggested by the category ratings, particular statements really seem more militaristic to "doves" than to "hawks." The emphasis upon response bias has been most central to the theory of signal detection which used many of the same assumptions employed by Thurstone to analyze absolute thresholds for psychophysical stimuli (see Green 8t Swets. 1966, for a comprehensive review, and Swets, 1988, for more recent applications). There is a large body of empirical evidence supporting the generalization that the measure of discriminability, d. remains constant in spite of dramatic shifts in the probabilities of reporting that the signal has been presented, viz., the probabilities of "hits" and "false alarms." These shifts can be produced by changes in instructions, emphasizing either boldness or caution in reporting the signal, by changes in the matrix of rewards and penalties for "hits" and "false alarms," or by changes in the probability of the signal. Applying the Neyman-Pearson analysis from statistics (cf,, Gigerenzer & Murray, 19871, these shifts can be derived from shifts in the criterion. The term "response bias" then

216

A . Parducci

refers to the locating of the criterion. The same logic is applied when there is more than one criterion or limen (as in ratings of confidence) and also to ratings of suprathreshold stimuli, as in Thurstone. The location of the criterion in signal detection does not seem to reflect response bias in the sense of distorting the observer's experience. For example, the observer experiences a given level of sound and reports whether that level is best considered as evidence for the signal or for noise alone. In a difficult discrimination that often requires guessing, the observer makes decisions about uncertain information - just as the theory of signal detection assumes. If the observer knows that the signal is presented on most trials, lowering his criterion increases the proportion of correct responses. The direction of this shift is actually contrary to the shifts in category ratings where suprathreshold tones are rated "softer" when the louder tones are presented more frequently (Parducci & Sandusky, 1970). The degree of experienced loudness might thus be expected to decrease when the probability of the signal is greater. My objection is not to the well-specified use of the term "response bias" in signal detection theory but rather to the blurring of the distinction between criterion shifts and deceptive reporting. This encourages a generalized tendency to label any shift in standards as "response bias," including shifts in the ratings of suprathreshold stimuli. There may sometimes be response bias in the sense that I am using the term. Reporting "no signal" when one clearly hears the signal would be a legitimate case of response bias. But under what conditions would such a bias occur?

Information Integration Theorg Anderson's theory of information integration (Anderson, 1981) articulates the distinction between input and output functions, with response bias manifested by nonlinearity of the output function. In this approach, the input of physical stimuli is transformed into subjective values (like Thurstone scale values or the range values of range-frequency theory) which are combined algebraically to form a psychological impression which is transformed into output as a category rating or other overt response. Because it is the integrated impression that is consciously experienced (Anderson. 198I), there

Response b i a s a n d contextual e f f e c t s

217

is response bias insofar as this impression is not reported directly, i.e.. linearly. Birnbaum (1982) has emphasized "scale convergence," i.e., constancy of the input function, as a restriction necessary for testing between alternative cognitive models. For example, differences between instructions to judge differences or ratios are assumed not to affect the input function: however, the output function depends upon whether subjects were instructed to make category ratings or magnitude estimations. Mellers and Birnbaum (1982) demonstrated how one could determine within this framework whether particular contextual effects are located in the input function, as they found to be the case with cross-modality comparisons. When contextual effects can be located in the output function, one has a clear case of response bias. Thus, magnitude estimations were interpreted as distorting subjective impressions by an antilog transformation (Birnbaum. 1982). Anderson ( 1982) prescribes various procedures for stabilizing the output function, including endpoint-anchoring at stimulus values much more extreme than any presented for judgment. Although this may be useful in testing between alternative cognitive models of information integration, the Gestalt assumption is that any such procedure at best standardizes the contextual effects upon the integrated impression or judgment - without eliminating them.

Pragmatic Response Bias

My main concern is that the effects of context should not be indiscriminately interpreted as response bias, in the sense of distorting the subjective experiences that are being reported. When these experiences do change with context, it would be a biased response that would report them as unchanging. This would be a classical case of the "stimulus error," viz., the sin of reporting the stimulus rather than the experience (Titchener, 1909). We deliberately report the stimulus in many of our communications with others. For example, a disappointed tradesman might complain: "I did all that work, but I was only paid $lo!" Here, the report is the actual payment, $10 - though it might have seemed like only $5; however, the "all that work' establishes the context for evaluating the $10, and the modifier "only" tells us that this payment was experienced as inadequate.

A . Parducci

218

In this case, the total statement conveys the feeling of having been underpaid. In other cases, response bias is used to change the experiences and behavior of others. Moral and aesthetic judgments are often expressed using categories more extreme than the speaker's actual feelings. For example, a n old-fashioned mother might claim that her daughter's skirt was "scandalously short." knowing full well that it was not really out of line with current fashion. Terms like "obscene" and "antediluvian" are applied to political targets, not as accurate reports of the speaker's judgments but rather to evoke the listener's disapproval of the targets. So too with advertising where inflated testimonials are used to sell products. However, the response bias inherent in all such pragmatics may be much less than would appear: we have learned to discount superlatives. Our knowledge of a speaker's tendency to use superlatives becomes part of the context for evaluating their meaning. Thus, the degree of response bias is regularly moderated. We resort to response bias for the purpose of persuading others, but in the long run our efforts are often self-defeating. REFERENCES Anderson, N.H. (1981).Foundations of Information w ro n Theory . New York: Academic Press. Anderson, N.H. (1982).Methods of Information Intearation Theory. New York: Academic Press. Bimbaum. M.H. (1974).Using contextual effects to derive psychophysical scales. 89-96. perceDtion and PsvchoDhvsicS, Bimbaum, M.H. (1982). Controversies in psychological measurement. In B. Wegener Psvcmhvsical Measurement. Hillsdale. N J : (Eds.). Social AttitErlbaum. m m . New Edwards, A.L. (1957). York: Dryden. Fabre. J.-M. (1987).J&J relativisation des jgaemen&. Unpublished dissertation, Universite de Provence, Aix-en-Provence. Gamer, W.R. (1962).Uncertafntv and Structure as Psvcholoeical Concepts. New York: Wiley. Gigerenzer. G.. & Murray, D.J. (1987). CorCnition a s Intuitive Statistics. Hfllsdale. N J : Erlb aum . Green, D.M., & Swets. J.A.(1966). Signal Detection Theorv and PsvchoDhvsicS. New York: Wilev. Helson. H. (1938):Fundamental problems in color vision. I. The principle governing changes in hue, saturation, and lightness of non-selective samples in chromatic illumination. p a , 439-476. Helson. H.(1964). &&&J tio on-level Theorv. New York: Harper & Row. Mellers. B.A.. & Bimbaum. M.H. (1982).Loci of contextual effects in judgment. l a : Human performance and judgment, 8, n 1P . 582-601. Parducci, A. (1965).Category judgment: A range-frequency model. Psvcholoeical M e w , 72,407-418.

a,

a,

Response b i a s and contextual effects

219

Parduccl. A. (1982).Scale values and phenomenal experience: There is no psychophysical law! In H.G. Geissler. P. PetzoId. H.F.J.M. Buffart, & Y.M. Zabrodin (Eds.), Psvch n. Berlin: VEB Deutscher Verlag der Wissenschaften. Parducci, A. (1983).Category ratings and the relational character of judgment. In H.G. Geissler, H.F.J.M. B~~ffart, E.L.J. Leeuwenberg. & V. Sarris (Eds.),Modem Issues in PerceDtion. B e r m VEB Deutscher Verlag der Wissenschaften. Parducci. A., Calfee, RC.. Marshall, L.M.. & Davidson. L.P. (1960).Context effects in judgment: adaptation level a s a function of the mean, midpoint. and median of the stimuli. Journal of a1 PsvcholQgy, m. 65-77. Parducci. A.. Knobel, S., & Thomas, C. (1976).Independent contexts for category ratings: A range-frequency analysis. PerceDtion and PsvchoDhvsics. 20. 360366. Parducci. A.. & Perrett, L.P. (1971).Rating scales: Effects of relative spacing and frequency of stimulus values. Journal of ExDerimental PsvcholQgy 427-452. Monoma Parducci;,A. & Sandusky, A.J. (1970). Limits on the appllcabillty of signal detection theory. PerceDtion and PsvchoDhvsicS, 5,434436. Parducci. A., & Wedell. D. (1986). The category effect with rating scales: Number of categories, number of stimuli. and method of presentation. Journal of ExDerlmental Psv: Human P e r c a o n and Performancc. 496-516. Poulton, E.C. (1979).Models for the biases in judging sensory magnitude. ,86, 777-803. FUles, G. (1949).The ConceDt of Mind. New York Barnes and Noble. Torgerson, W.S. (1958). Theorv and Methods of Scaling. New York Wfley. Riskey, D.R. Parducci. A, & Beauchamp. G.K. (1979). Effects of context in judgments of sweetness and pleasantness. m D t l o n and PsvChODhvsica. 26,171-176. Sarris. V. (1967).Adaptation-level theory: T w o critical experiments on Helson's weighted-average model. American Journal of Psvch010&, &J, 331-334. Sarrls. V.. & Parducci. A. (1978).Multiple anchoring of category rating scales. PerceDtion and PsvchoDhvsics. 24. 35-39. Stevens, S.S. (1971). Issues in psychophysical measurement. psvcholoeical Review, 426-450. Swets, J.A. (1988).Measuring the accuracy of diagnostic systems. Science, 12851293. Thurstone. L.L. (1927).A law of comparative judgment. Psvchological Review. 2. 273-286. Thurstone. L.L. (1931).The measurement of social attitudes. Journal of Abnormal and Social Psvchology, 26.249-269. Titchener. E.B. (1909). A Textbook of PsvcholQey. New York: Macmtllan. Upshaw, H.S.. Ostrom. T.M.. & Ward, C.D. (1970).Content versus self-rating in attitude research. Journal of Emerimental Social Psvchology, S , 272-279. Upshaw. H.S.. & Ostrom, T.M. (1984). Psychological perspective in attitude research. In J.R Eiser (Ed.).Attitudinal J u d m e n t . New York Springer. Watson, J.B. (1913). Psychology as a behaviorist views it. psvcholouical Review, 158-177. Wallach. H. (1948).Brightness constancy and the nature of achromatic colors. Journal of Exaerlmental Psvchology, 22, 339-368. Wedell. D.H. (1984). A Process Model for PsvchoDhvsical Judgment. Unpublished dissertation. University of California. Los Angeles. Werthelrner. M. (1938).Gestalt theory. In W.D. Ellis (Ed.), A Sourcebook of Gestalt psvcholoa. London: Routledge and Kegan Paul.

,a.

u,

a.

m,

a,

This Page Intentionally Left Blank

Coqni t i v e B i a s e s J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez (Eds.) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

PSYCHOPHYSICAL APPROACHES, CONTEXTUAL EFFECTS AND RESPONSE BIAS Claude Bonnet

* L a b o r a t o r y of E x p e r i m e n t a l Psychology U n i v e r s i t y of P a r i s V, EPHE, EHESS, CNRS ( U R A 3 1 6 ) Abstract. Modem psychophysics stresses the assumption according to which no response directly reflects the sensory functioning of the organism. The discussion proposed here is based on a n analysis of psychophysical tasks in terms of information processing levels. Basically, perceptual processes are separable from judgment processes. The processing context refers to the former while the experimental context leading to response biases refers to the latter. Detailed analysis of well-known biases is presented to support such a view.

The understanding of contextual effects and of response bias in psychophysics is frequently obscured by a lack of explicitness in specifying t h e aim of the research. In the framework of psychophysics, two main approaches can be distinguished. Some people use psychophysical methods, almost exclusively scaling, in order to understand the processes of judgment as such. Others use psychophysical methods in order to understand the information processing going on in the organism as a result of the presentation of a stimulus. The term "sensory scientists" (Gescheider, 1988) is too restricted to refer to the latter since they may be interested in different problems like perception, memory, learning, psycholinguistics etc. Between the tenants of these two approaches, there is frequently more than a difference of emphasis on two complementary aspects of information processing. Psychophysics is the quantitative approach to the relationships between stimuli and responses. I t is generally restricted to detection, discrimination and scaling methods (Gescheider, 1976: Baird & Noma, 1978). For the present author (Bonnet, 1986). psychophysics should include every method allowing for quantitative studies relating behavioral responses to some aspect of the environment. Most of the research concerning the study of the perceptual aspects of information processing in humans uses some kind of psychophysical method. The information processing approach emphasizes the study of mental operations at work in given tasks. Psychophysics is more than a set of methods like technology for instance: psychophysics

222

C. Bonnet

constitutes a scientific paradigm (Baird & Noma, 1978). As such, psychophysical theory has always been considered, implicitly or explicitly. to make assumptions about how the organism processes information, whether these processes can or cannot be observed directly . I t should be mentioned here that psychophysical models are frequently only descriptive. This tendency may have resulted from the dominance of the neo-positivist position of behaviorism which avoids any inference of internal processes. However, even before the information processing approach became popular, psychophysics was really never able to stand up to a strong behaviorist position. For instance, classical psychophysics. mainly in the field of scaling, has attempted to establish how the "magnitude of sensation" varies with that of the stimulus. The magnitude of sensation is at best an "intervening variable" (Gescheider, 1988).i.e. some mental event inferred from the results. The paradigm shared by psychophysicists (those who use psychophysical methods and models) is that "perceptual and cognitive processes are open to quantitative treatment in a framework provided by the experimental method" (Baird & Noma, 1978, p. 260). The present paper will attempt to further specify the paradigm by proposing that the aim of any psychophysical approach should be to disentangle the perceptual, or more generally, the specific components of the response from those components directly related to the processes of response production. 1. MODELS OF THE PSYCHOPHYSICALAPPROACH

Any psychophysical approach to a problem consists of establishing a relationship between a set of stimulus values ( S ) and a set of response values (R),R = f(S). The psychometric function describing the relationship between the probability of correct detection and the "intensity" of the stimulus is one example of such a relationship. Scaling functions are other popular examples (Torgerson, 1958; Baird & Noma, 1978; Bonnet, 1986; Gescheider, 1988). Similar relationships are also described for reaction times (Luce, 1986). In fact, such an observed relationship was assumed to be established for two transformation functions (Bonnet, 1969; Shepard, 1981). A stimulus transformation function describes the relationship between the dimension(s) of the stimulus which is (or

Psychophysical approaches

223

are) really processed by the organism and its sensory effect (an intervening variable). A response transformation function relates the latter to the observed response. Whether or not these two transformation functions can be estimated will not be a central issue here. Even better, we will stress the idea that the psychophysical paradigm necessitates careful analysis of the two intermediate transformation functions through the use of three orders of models (Bonnet, 1986): stimulus models, sensory models, and response or judgmental models. In other words, it is presumed that no psychophysical response directly reflects the sensory functioning of the organism. An information processing analysis of every psychophysical experiment is necessary before the data can be interpreted theoretically. Before showing some examples of such an analysis, a few questions related to the assumption that three levels of modeling exist have to be clarified. 1.1 Response models

Psychophysicists, like other psychologists, use the overt responses of subjects to infer some knowledge about internal processes. Think about perceptual problems. Researchers like the gestaltists have relied heavily on the phenomenological method, i.e. on the reported experience of observers. In fact, from a n information processing point of view, such reports a t best reflect the cognitive representations of t h e subjects, i.e. some interpretation of sensory information. This is far from sufficient if we want to understand the mechanisms and processes which generate s u c h experiences, in which case we must use a n experimental approach, and essentially psychophysical methods. Then the responses we are dealing with are simple and conventional, and we examine their distributions. Psychophysical responses are random variables. This means first that they are "a numerical quantity associated with a system that is to be observed" (Luce, 1986, p. 7).and second, that they vary. Such variation is assumed to reflect some properties of the system under study. Moreover, we assume that part of that variation is systematic and part is random. In other words, we assume that a response can be broken down into components. Most of the efforts of the psychophysical approach are to build models which allow for analyzing response components. As we shall see below, two of them

224

C.

Bonnet

are of interest: those concerning sensory processing and those concerning the response process itself. In order to produce a response according to the rules defined in the instructions, the subject has to choose the information that is relevant to his or her task, and must disregard the information that is not relevant. He or she has then to establish a correspondence between the selected sensory information and the response categories which have been defined. However, there are many aspects of the experiment which can affect the response process and which are not relevant to sensory processing itself. These We shall see variables form what we call the .txenoc- the effect of variables such as the range of stimulus values, their frequency of appearance, their sequential order of appearance, etc. Responses are modified, i.e. biased, as a result of the effect of these variables on expectancies, subjective probabilities, or stated another way, on the decision process. Response biases concern the response transformation function (see review in Gescheider, 1988) and not the stimulus transformation function. Response models should then be developed in order to separate these two components from the observed response. 1.2 The stimulus question

By definition, the term Stimulus refers to the dimension(s) of the physical world (external or internal] that activate the organism, i.e. that really cause the phenomenal experience under investigation. However, the current use of the word stimulus leads to frequent confusions. The term is used to refer to the physical variables (the factors) which are intentionally manipulated by the experimenter, as if there were no uncertainty about the very nature of the stimulus and about how it is processed by the organism. Consequently, different types of "stimulus-error" are found, Some of them will now be reviewed. In any psychophysical task, the experimenter chooses a "stimulus" on the basis of a theoretical model of processing related to his or her knowledge of physical reality. Such knowledge is in fact a cognitive representation of the world. One example of this is the case of the so-called optico-geometric illusions. The figures used in such experiments are defined in abstract geometric terms. A Muller-Lyer (ML) figure is described as being made up of two segments of "equal length". one terminated by inside shafts, the other, by outside shafts. The psychophysical task is defined by a

Psychophysical approaches

225

matching of a linear segment to one of the two ML figures. For the sensory system, the stimulus is not a linear segment with shafts (two separate components), b u t rather a shafted-segment as a whole. In fact, a functional approach suggests that there is no illusion at the sensory level. The information which is processed is veridical, given the filter properties of the sensory system. The "illusion" is then essentially a misfit between the (sensory) stimulus and its cognitive and abstract definition. What is separable in terms of geometry is not necessarily separable for the sensory system. It should be mentioned that the naive conception about the nature of the stimulus has been challenged by gestaltists as well as by authors such as Gibson. However these authors, who are not interested in processing per se (the former talk about "isomorphism" and the latter about "direct" perception), approximately define the stimulus as the s t r u c t u r e of this world which is similar to our phenomenological description of it. A related aspect of the stimulus problem is to make sure that in psychophysical tasks, the subject is really judging what the instructions request him or her to judge. For instance when subjects are requested to scale heaviness, their results generally give rise to a power function with a n exponent of 1.45. Warren and Warren ( 1956) have argued that this apparently non-veridical exponent most likely results from the size-weight "illusion". In other words, our sensation of weight may be more determined by density than by weight itself. In this respect, it can be said that most of the efforts of sensory psychophysicists are to seek for a relevant definition of the stimulus. Such definitions cannot be firmly established without precise knowledge of the way the organism processes information (see 1.2). Another aspect of the stimulus problem is the status of contextual variables. We have mentioned above some of the variables that form the experimental context and are assumed to influence the response process. Some other variables determine the sensory intensity of a given dimension. This is the case for the duration of a stimulus, for instance. These factors can be shown to modulate the sensory response to the dimension tested in the experiment. As frequently stressed (cf. Marks, 1986; Bonnet, 19861, a given sensory response results from a constellation of factors which fashion the processing- context to the way the system responds to the stimulus under study. For instance, the visual detection threshold of a light is measured by the luminance of the source, others factors being

226

C. Bonnet

constant. These factors are the angular size of the light, its duration of exposure, the wavelength distribution of the light, the precise retinal locus stimulated, in particular its eccentricity, the ambient light of the field surrounding the stimulus during the trial, the ambient light received before the trial etc. All these factors are known to modify the conditions under which the visual system responds to the stimulus. The same factors can influence matching responses, discrimination responses, as well as scaling responses and the corresponding response times. However, their effect may not be related in a monotonic way to the change in their values. Consider the effect of exposure time on the apparent brightness of a light. Starting from very short exposures, the brightness first increases with duration, reaches a maximum called the Broca-Sulzer effect and then decreases a little. In the long range of exposure times, new phenomena will appear, such as an adaptation effect, which will make brightness decline in time. In fact, no stimulus can be considered as uni-dimensional from the point of view of its neuro-sensory processing. There is no such a thing as a contrast threshold, but a contrast threshold for a given spatial frequency of a grating, shown a t a given mean luminance level, with a given field size, and with some temporal characteristics, etc. All these factors are known to influence the processing of contrast, and their effects reveal something about neurosensory processing (De Valois & De Valois, 1988). 1.3 Perceptual models

Assuming that we know which dimensions of the stimulus are relevant for sensory processing, we now have to have assumptions about the fact that the information provided a t that level is or is not distorted by experimental contextual effects. This point represents a real break between sensory and judgmental psychophysicists. The first kind generally assume that sensory information is immune to judgmental bias, and that the latter are cognitive in nature. They hence assume t h a t some invariant sensory response can be extracted from variations of the observed responses due to the experimental context. They call such a response bias-free. The second type of psychophysicist seems to assume that the two levels of functioning cannot be distinguished. They stress the effects of experimental context and their generality. Any information processing analysis of the responses obtained in a psychophysical task implies at least two main stages. At the

Psychophysical approaches

227

first stage, the organism extracts and codes the information contained in the stimulus on the basis of mechanisms whose functioning is very dependent on the processing context. At the second stage, the subject builds a response on the basis of that information. The response is conventional in that it has been restricted by the experimental constraints defined in the instructions. For instance a guessing response should be identical (Yes or No) to a most certain one. Such a response implies judgments and choice. Now the very question is to find out whether these two stages are separable or not. A large part of the efforts of contemporary psychophysicists is precisely to attempt to demonstrate such separability, which is the necessary condition for scientific knowledge of any psychological processing. In effect, what knowledge of a perceptual process, for instance, can be taken for granted, if responses are essentially determined by the experimental context in which they are observed? I a m not claiming that response bias does not exist. On the contrary, I contend that we should always be aware of it in order to correctly interpret psychophysical results. To my knowledge, there is no convincing experimental evidence that cognitive factors can influence the very primitive levels of sensory processing (see 3.2). As a matter of fact, pre-attentive levels of processing are defined as being insensitive to cognitive factors. I believe this is a necessary postulate, and no psychophysical knowledge of the early level of perceptual processing would be possible without it. 2. SEPARATING PERCEPTUAL PROCESSES FROM JUDGMENTAL PROCESSES

Following these considerations, any psychophysical study of perceptual mechanisms should presuppose that overt responses are determined by two different orders of constraints: in the first, the perceptual process, and in the second, the judgmental process. The psychophysical models which are of interest in an information processing approach are those which allow u s to dissociate these two kinds of processes. In fact, as already said above, anyone experimenting in fields like perception, memory, learning, psycholinguistics, etc. is faced with similar questions. Psychophysical knowledge of information processing should then assume the separability of (at least) two components in the observed responses.

228

C. Bonnet

2.1 Signal Detection Model as a canonical model

Originating in the stimulating ideas put forward by Thurstone (1927), signal detection theorists (Green & Swets. 1966) proposed a formal model of the processes giving rise to detection responses which separates sensory effects from decisional effects. Their model applies essentially to detection and discrimination tasks. Any response (a judgment) is considered as the end product of two orders of mechanisms. One is sensory (or perceptual) and related only to the processing of sensory information, for instance the sensitivity of the system. Sensory aspects of the response are said to be unbiased in the sense that they are assumed to be independent of the subject's expectations. But a response implies a judgment, i.e. some cognitive operations which act independently of the sensory capacities of the subject. By definition, response biases refer to those aspects of the response which relate to judgmental mechanisms. They are therefore cognitive. The signal detection model operationalizes sensitivity in terms of discrimination between stimuli, i.e. a distance, and proposes a sensitivity measure known as d'. On the other hand, the strategy used by the subject is modeled as the choice of a value of a likelihood ratio (termed B) which is a function of the payoffs and frequencies of appearance of stimuli. However, the independence of these two indices (d' and B) is only proved when the assumption about the equality of the variance of two hypothetical distributions is proved. Under these conditions, the separability of sensory and decisional mechanisms can be assumed (Green 81 Swets, 1966). 2.2 'hvo complementary assumptions

To be valid, such a model imposes experimental constraints which reduce the generalizability of its application. However, in our point of view, any psychophysical model aiming to clarify the stages of information processing should stress the separability of sensory and decisional factors. When, and this seems to be the general case, a model does not propose two separate parameters for estimating the effects of these two orders of factors, another research strategy is to be used. Instead of within-experiment comparisons it consists of between-experiment comparisons. Then, two general assumptions allow us to conceptually, and one can suppose experimentally, separate these two main components in psychophysical responses.

Psychophysi cal approaches

229

Assumption 1. It is possible to infer some identical knowledge about sensory or perceptual processes from different types of experimental procedures: detection, discrimination, identification, reaction time, scaling, etc. (Luce. 1986). Such a n assumption is related to Luce's choice axiom (Luce, 1959). AssumDtion 2. Complementarily, it is possible to infer some identical knowledge about the judgmental processes at work in the same experimental procedure by comparing judgment responses obtained with different kinds of stimuli. The same experimental context variables should have the same effect on the responses in different psychophysical procedures. These two assumptions rely on the view according to which an observed psychophysical relationship results from two transformation functions. Assumption 1 implies that there is no such thing as direct measurement of a sensory process. The stimulus transformation function relates the sensation magnitude and the stimulus. It cannot be observed directly. In principle, it may be estimated through the use of different psychophysical methods using the same stimuli. When the aim of the researchers is to demonstrate the existence and specific characteristics of some sensory subsystems, this is exactly the strategy they follow. One of the most documented examples is the psychophysical demonstration of the transient and sustained channels in the visual system (review in Breitmeyer, 1984). What is specific to a given experimental procedure will affect the way in which the subject chooses decision strategies. For instance, regardless of the sensory dimension of the stimulus, if two stimuli to be identified are presented with unequal frequencies, response biases are expected. In general, the frequencies of the two kinds of responses are less different than they should be. In the first case the subject seems to be expecting equal frequencies of appearance. Such a stimulus-frequency bias (Poulton, 1979) appears not only in detection or discrimination methods but is well documented in category scaling (Parducci, 1982) or in magnitude scaling (Bonnet, 1969). It also acts very strongly in reaction time experiments (Luce, 1986) in which, at least for correct responses, reaction time decreases as frequency of appearance increases. 3. EXPERIMENTAL CONTEXT AND RESPONSE BIAS

According to assumption 2, any effect of experimental context which shows up with different psychophysical methods should be

230

C . Bonnet

attributed to judgmental processes, and hence be considered as a response bias. Such a statement should obviously be qualified since different experimental procedures may introduce constraints that change the processing context. Three biases will be reviewed now: the stimulus-range bias, the stimulus-frequency bias and sequential effects. Many other biases have been mentioned in the literature (cf. Poulton. 1979). In some cases, the separability of sensory and judgmental effects is not easy to establish. 3.1 The stimulus-range bias

Every psychophysical experiment uses only a limited range of the values of the stimulus under study. For instance in scaling tasks, it would 'be impossible to use the entire range of intensities in a given stimulus continuum between the absolute threshold and the upper threshold. It is then found (cf. Poulton, 1968; Bonnet, 1969) that in magnitude and in ratio estimation methods the exponent of the psychophysical power function decreases as the range of the stimulus values increases. Similar effects are obtained in category ratings (Parducci, 1982). Related to that effect is the fact that the responses of subjects tend to be centered on the range of the values used in the experiment. Combining with the effect of the spacing of the stimuli, these biases account for phenomena such as Helson's (1964) level of adaptation. They are included in Parducci's rangefrequency model (1963; Parducci & Perrett, 1971; see also this volume). In measuring a threshold with a constant method, it is likely that the range of the stimuli and the position of the expected threshold within the range will have an effect on the estimated threshold. The range effect and its related stimulus-spacing effect may have at least two explanations. In the first, it is clear that the more spaced the values of a stimulus are, the more discriminable they are. Hence, spacing may affect the discriminability of the stimuli. Measurements of Weber fractions and of channel capacity with absolute judgments are inversely related (Baird, 1970) and estimate the discriminative capacities of the sensory system under study. Different sensory systems show different discriminative capacities. Baird and Noma (1978) argue that the exponent of psychophysical scales also reflects these discriminative capacities. Baird ( 1970b) has shown that it is generally related to channel capacity. Increasing the range of the stimulus values should increase their discriminability. However from the point of view of information

Psychophysi cal approaches

231

transmission, such an effect should level off. In fact, Braida and Durlach (1972). a n d Luce. Green, and Weber (1976) have demonstrated this. Such a limitation cannot be attributed to a sensory factor. In fact it was proposed that it depends either upon a memory factor (Durlach & Braida, 1969; Gravetter & Lockhead, 1973) or upon the limitation of attention to a restricted band of the stimulus continuum (Luce et d ,1976). That fact that discriminability has an effect on psychophysical responses has also been seen in reaction time experiments. Swensson (1972) concluded that reaction time for errors, RT(e), tends to be shorter than reaction time for correct responses, RT(c), when both stimuli are easy to discriminate. Now, RT(e) tends to be longer than RT(c) when discrimination is difficult and especially if the constraint for accuracy is strong. As in most experiments where the time constraint and accuracy rely on the subjects' obedience to instructions ("Respond as fast as possible without error"), it appears necessary to try to determine the kind of compromise made by subjects during accomplishment of the task. This is the aim of the study of speed-accuracy trade-off functions. Basically, as suggested by Luce (1986),one should distinguish two types of functions of this sort. A Conditional Accuracy Function (CAF) is the study of such a trade-off within experimental conditions. All the CRT obtained are ranked and divided into classes. For each class a n accuracy index (d' or -In q for instance) is then computed. A response bias index such as -6 or In D can also be computed. Generally, it is observed that, within limits, accuracy increases linearly with the increase in mean CRT (Lappin & Disch, 1972; Lappin, 1978). Correlatively, if there is no response bias in the CRT, the decision index should remain constant (Lappin, 1978). A Speed-Accuracy Trade-off function (SATF, Luce, 1986) is the study of the relationship between accuracy and CRT across experimental conditions. This provides a way to attempt to separate the effects of sensory and decisional factors in RT. The speedaccuracy trade-off function is different, whether the stimuli are difficult to discriminate or highly discriminable. Response speed increases as the stimuli become more discriminable (Link & Tindall, 1971; Vickers. 1980). However, the change in the two aspects of the response depending on distance between stimuli is different. While accuracy shows a ceiling effect for a given stimulus separation, reaction times continue to decline beyond t h a t separation. In fact, the latter are likely to reflect the confidence of

232

C. Bonnet

the subject in judging the difference between the stimuli (see Luce, 1986).

Now, if the stimuli are highly discriminable, the observed relationship between speed and accuracy is generally reversed: the higher the accuracy, the longer the reaction time. With different procedures, it is possible to influence the response strategy of the subject through the relative weight given to the speed and accuracy of the response. One of these procedures is to impose a temporal response window so that the response be given before a deadline. As anyone can guess, the closer the deadline to the stimulus appearance, the shorter the reaction time but the lower the accuracy. The few results reported here suggest that the range effect cannot univocally be explained as a response bias, i.e. as due to judgmental factors. Changing the range of the stimuli also affects their relative discriminability. More systematic investigations using a research strategy founded on the two assumptions stated above are necessary to clarify this matter. 3.2 The stimulus-frequencybias

In any psychophysical experiment, a difference in the frequency of appearance of the stimuli has an effect on the observed performance. In a signal detection experiment, increasing the number of presentations of the signal, and correlatively reducing the number of presentations of the noise, increases the conditional frequency of correct detection. Beta, a likelihood ratio, happens to have a value below unity when the number of signal trials is greater than the number of noise trials, and above unity in the reverse case. In spite of these changes, d' remains invariant in most examples (see Green & Swets, 1966; Bonnet, 1986). This is at least true when the slope of the ROC curve in normal deviate coordinates is equal to unity. Actually, if d' changes systematically with the proportion of signals, the slope of the ROC curve will depart from unity. Such a departure is generally interpreted as meaning that the variances of the two underlying distributions are not equal. If the variance of the signal distribution is greater than the variance of the noise distribution the slope of the ROC curve will be smaller than unity, and vice versa. If the frequency of appearance makes d' vary systematically, two alternative conclusions are possible: One may conclude that the frequency of appearance has a n effect on sensitivity, which in other

Psychophysica1 approaches

233

words would mean that the signal detection model is invalid! Another conclusion is that the frequency of appearance changes the variance of the distribution and for that reason makes d' an invalid descriptor of sensitivity. A practical solution to this dilemma is to describe sensitivity with a non-parametric statistic such as A' (Craig, 1979) which makes no presuppositions about the assumption of equal variance. The feasibility of such a solution will be shown with some reaction time d a t a using a n identification paradigm. Parenthetically, these considerations stress the fact that some conclusions a r e closely dependent upon the choice of t h e descriptive indices of the response and related assumptions (cf. Swets, 1986a & b). Luce (1986) reported several sets of experiments using a twoalternative choice reaction time task in which the relative frequency of appearance of the two stimuli was varied (Carterette, Friedman, & Cosmides, 1965; Laming, 1968; Link, 1975; Green, Smith, & von Gierke, 1981). According to our computations, except in the Green et al. experiment, all the others exhibit some effect of stimulus frequency on d' (see Figure 1).

Figure 1. Examples of data showing a change in d' with the frequency of S1 and S2. Black circles: data from Link (1975). open circles: d a t a from Laming (1968). Dashed lines are the linear regressions fitted to the data. However, if sensitivity is recomputed as a n A statistic, the response bias does not show u p on that statistic. A similar

234

C. Bonnet

conclusion was reached by Proctor and Weeks (1989) who reanalyzed Ratcliff and Hacker’s (1981) data in a sameldifferent matching paradigm. In conclusion, as proposed by signal detection theory, the stimulus-frequency bias should not affect sensitivity. This can be shown as long as the right sensitivity statistic is chosen. In each of the above experiments, the judgmental statistic, of course, is strongly affected by stimulus frequency. In scaling experiments, a stimulus-frequency bias has been repeatedly demonstrated (cf. Poulton, 1968, 1979; Bonnet, 1969; Parducci in the present book). In short, the response to less frequent stimuli is more contrasting than the response to equally frequent stimuli. Consequently, a stronger and less frequent stimulus will be judged as much stronger than a frequently presented one, and a weaker and less frequent stimulus will be judged as much weaker. The very question is now to determine whether such changes in the scale values result from sensory processing or from the subsequent judgment process. What about reaction times? As we have computed on the data from the experiments by Carterette et al. (1965). Laming (1968). Link (1975). and Green et al. (1981), mean reaction time (RTm) does not vary systematically with the relative frequency of the stimuli. However, in each of the experiments, the RT for the correct responses declines as stimulus frequency increases. The interpretation of such a result is ambiguous. In effect, increasing the stimulus frequency also increases the conditional probabilities of correct responses. The relationship between the RT of the errors and stimulus-frequency varies from experiment to experiment. According to Swensson (1972), reaction times for errors, W(e). are shorter than reaction times for correct responses, TrT(c), when discrimination of both stimuli is easy and the experiment is run with a strong constraint on speed. This is clearly the case in the experiments by Laming (1968). Link (1975), and Green et al. (1981). Intuitively, it can be supposed that the errors are essentially anticipatory errors. Different versions of a fast-guess model have been designed to explain such results (cf. Luce, 1986). Their suitability is not completely satisfactory because fast-guessing applies only to part of the responses. The data reported here suggest that the stimulus-frequency effect is basically a response bias, i.e. dependent upon the judgmental process.

Psychophysi c a l approaches

235

3.3 Sequential effects

Ideally, in every psychophysical task, a response to a given stimulus on a given trial is assumed to be independent of the other values of the stimuli used in the experiment and the responses given to the stimuli. Such an assumption is implied by the statistical model of the response as a random variable. We all know that sequential effects are likely to appear in every psychological experiment. For this reason, great care is taken to counterbalance the order of stimuli, hoping that the average sequential effects will be negligible. However, in the background of a study of experimental context effects, sequential effects have to be taken into consideration since they reflect some basic properties of the judgmental process. Many studies have demonstrated that in magnitude estimation as well as in category judgment and in absolute judgment, the response on a given trial tends to be assimilated with or contrasted to the response on the previous trial (cf. Gescheider, 1988). In magnitude estimation tasks, sequential effects tend to reduce the value of the exponent of the psychophysical function, a result which suggests the prevalence of assimilation effects. As remarked by Cross (1973), a high-intensity stimulus is more likely to be preceded by a stimulus of a lower intensity. The response to the high-intensity stimulus tends to be assimilated with the response to the previous lowerintensity stimulus and therefore is reduced. Complementarily, a low-intensity stimulus is more likely to be preceded by a higherintensity one and the response to the lower intensity then tends to increase by assimilation with the previous response. Ward (1972, 1973) demonstrated this effect both in category judgment and in magnitude estimation by asking subjects to consider the stimulus shown at the previous trial as the standard for the judgment of the actual stimulus value. Jesteadt, Luce, and Green (1977) have suggested that these sequential effects of assimilation do not extend back over more than one trial. They are of lesser importance when the difference between the values of successive stimuli increases. However, in a n absolute judgment task, Staddon. King, and Lockhead (1980) found sequential effects extending back over 7 to 8 trials. The estimated extent of the assimilation effects depends upon the statistical technique used to measure it. Sequential effects have also been investigated in reaction time experiments (cf. Kirby, 1980; Luce, 1986). Since these

236

C.

Bonnet

experiments most frequently use only two stimuli (and two responses), the effects differ in kind. Laming (1968) and Remington (1969) show that, in a two alternative experiment, RT decreases with the number of repetitions of the same stimulus, while it increases when the stimulus is preceded by repetitions of another stimulus. These effects have been investigated over 5 back trials. When enough errors are produced, their probability decreases with repetitions. One possible explanation of sequential effects is based on the interactions between the present stimulus and sensory traces of the previous ones. Using masking techniques, Sperling (1960) has estimated the duration of that sensory trace to about 750 ms. In most experiments, the inter-trial interval is generally larger than this limit and consequently such a sensory interpretation is unlikely. Among the judgment process explanations, it has been suggested (cf. Kirby, 1980) that local regularities in the succession of the stimuli generate expectancies (or subjective probabilities). In fact, when some local regularity is present in a sequence of trials (alternation or repetition), the results suggest that the subject is expecting the pattern to continue (Laming, 1968). In that respect, sequential effects present some similarity with stimulus-frequency effects. 4. A challenging question: practice effects in psychophysics

Practically every psychophysical model presumes the stability of the subject's behavior. Empirical experience provides evidence against such an assumption. In fact, in order to deal with the disadvantage of performance t h a t changes with practice, researchers choose one of two strategies. Most frequently they use groups of naive subjects and sample the random variation between subjects. On some occasions, however, they prefer to sample within one subject. For this to be possible they use trained subjects, i.e. subjects who have practiced until their mean performance no longer changes with further practice. In one sense, practice effects are long term sequential effects. A prior&practice effects are made up of several components. These components can be related to separate stages of processing. Following Donder's idea (1868). four such stages are conveniently distinguished. - The first stage is represented by the "absolute" sensory processing of the signal.

Psychophysical approaches

237

The second stage is the one at which sensory discrimination is processed. It implies a comparison between different sensory channels and a decision about which of the channels is more highly activated. - The third stage is the one at which the organism chooses a response as a function of the sensory output with respect to the instructional rules. - The fourth stage concerns the production of the response. In most of the experiments it implies a motor component: pressing a key, producing a vocal sound, etc. Most of the psychophysical experiments are designed in order to minimize the complexity of the (motor) response so that practice only has very limited effects on that component. -

One of the basic questions in that field is whether the first sensory stage can be affected by practice effects. Using experimental procedures designed to eliminate response bias (forced choice) several authors have demonstrated an improvement in sensory discrimination with practice. This is the case in vision with retinal disparity discrimination (Ramachandran & Braddick. 1973). spatial frequency discrimination (Fiorentini & Berardi. 1981). direction of motion discrimination (Ball & Sekuler, 1987). audition with frequency discrimination (Demany, 1985, 1987). and auditory localization (Hartman. 1983). But what exactly has improved with practice? I s it the intensity of the sensory response (the signal/noise ratio) or the capacity to use that information and to select it from among the noise? J u s t imagine that practice reduces the variance of the signal distribution and leaves unchanged the variance of the noise in a signal detection model: the result would be that d' values will be mechanically greater. This argument shows that more data are necessary before improvement in sensitivitv with practice can be considered to be proven. Many scientists believe that most practice effects result from the second and third components. In that respect, they are assumed to be essentially cognitive. At stage two, the task of the organism depends upon the instructions (discrimination, identification, or matching of the same/different type). While in discrimination or identification paradigms, the task is of an OR-type (A OR B). in matching, the task is more of an IF-NOT-THEN-type (IF NOT Same THEN Different). Consequently, while reaction times are expected to be identical for the two alternatives in a discrimination

238

C. Bonnet

or identlfication task, reaction times for the Different responses are expected to be longer than reaction times for the Same responses in matching tasks. At stage three, the perceptual representations constructed during stage two are transcoded into classes of responses defined by the instructions. Depending upon the existing compatibility between the stimulus system and the response system (cf. Welford, 1980; Luce, 1986), such transcoding will require more or less training to become automated. Several kinds of compatibility can be distinguished. I t is certainly easier for a naive subject to respond with his or her left hand when the stimulus is on the left side and vice versa if the stimulus is on the right side. This is a case of spatial mmpatibility. A second kind of compatibility concerns the relationship between the modality of the signal and the nature of the response. One instance of this compatibility is comparing a motor response and a vocal response to a vocal stimulus, or the famous question of the difference in time required for naming pictures of common objects and reading their names (cf. Theios & Amrhein, 1989). Another case of compatibility concerns the number of alternatives of the stimulus and of the responses. Consequently, the response bias mentioned above should diminish or even disappear after sufficient training. It is classically reported that RT increases with the number of alternatives (Hick, 1952). The effect is attributed to the transmitted information. However, in a situation in which stimulus and response were highly compatible (press the key with the finger that is stimulated by a vibration), Leonard (1959) did not find a change in RT with the number of stimuli. Moreover, Teichner and Krebs (1974) analyzing a large set of data published on that question showed that the effect of the number of alternatives on RT diminishes with practice. They interpolated that with about a million trials, the effect would disappear. Such a large number of trials has never been used with humans, but has been employed in experiments with monkeys, where no effect of the number of alternatives is observed. CONCLUSION

In the view stressed here, whether the main interest of the psychophysicist focuses on sensory or judgemental processes, contextual effects and response biases will be a problem he or she will have to face. I t is true that sensory psychophysicists try to

Psychophysi ca 1 approaches

239

develop methods for obtaining unbiased responses. In measuring thresholds, forced choice methods are claimed to lead to unbiased estimates of the threshold. This claim means that the measured threshold corresponds to a known probability of correct responses and cannot in principle be biased by the expectancies of the subject. The absolute judgment method allows for the study of the relationship between transmitted information and the number of stimulus alternatives. Within limits, this relationship is linear (cf. Laming, 1984). This means that within such limits, rating judgments (identification) can be produced without bias. Zwislocki and Goodman (1980) and Ward (1987) have proposed an absolute magnitude estimation method. Basically the originality of the method lies in the instructions given to the subjects. Subjects are discouraged to judge stimuli relative to one another. They are asked instead to assign numbers to stimuli in such a way that the numbers match their impression of how intense the stimuli are. The results show that experimental context effects are relatively small under these conditions. The method is said to be "absolute", not because its results are free from context effects, but because the resulting scale has the property of a ratio scale: it remains invariant only for a linear transformation. Such absolute scales enable the study of sensory processes such as adaptation (Scharf, 1983). The two assumptions mentioned in the beginning of this paper need to be worked out further. In effect, different psychophysical methods can be employed to tackle processes at different levels since they request the subject to make use of different aspects of the available sensory information. Using a given method implies some constraints on the stimulus, on the sensory process, and also on the response process. For instance, sensitivity estimated by threshold measurement in vision as well as in audition is not a good predictor of what happens at supra-threshold levels. The threshold curve as a function of the frequency of the stimulus is an iso-intensity curve: namely the zero of the sensory scale. The higher the physical intensity of the stimulus, the flatter the isointensity curves for supra-threshold levels. Different methods have to be used at threshold and above threshold. The most currently accepted explanation of these differences is not based on the difference in methods but on the difference in the efficiency of the sensory process as a function of the intensity of the stimulus: the

C. Bonnet

240

curves get flatter with increasing intensity. In fact, an efficient psychophysical approach cannot be developed without consideration of the three classes of models mentioned in the beginning. Model makers interested in developing response models have used either very simple stimuli such as length of lines, or dimensions which have no physical reality such as beauty or happiness, in order to avoid the complexity of the entire approach. Psychologists interested in other aspects of information processing such as perception, memory, or language frequently undervalue their considerations for response bias. On some occasions they do not analyze parts of the responses they have requested from the subject. The fact that in some fields such a practice is systematically used does not riiake it justifiable! While it is, by definition, not possible to overpass the sensory capacities of the organism, it is very likely that experimental context effects producing response bias can be corrected with appropriate training. More research should be developed in this area. Finally, one may speculatively remark that the higher the order of the process under investigation, the more difficult it becomes to separate its specific components from the decisional aspects of the response.

REFERENCES Baird, J.C. (1970). A cognitive theory of psychophysics. I: Information transmission, partitioning, and Weber's law. i 1 3546.(a) Baird. J.C. (1970). A cognitive theory of psychophysics. 11: Fechner's law and Stevens' law. Scandinavian Journal of Psvcholom, 11, 89-102.(b) Baird. J.C.. & Noma. E. (1978). Fundamentals of scaline and DsvchoDhvsics. New York John Wiley and Sons. Ball, K., & Sekuler, R. (1987).Direction-specific improvement in motion discrimination. 953-965. Bonnet, C. (1969).Les Cchelles subjectives directes en Psychophysique. L'Ann&e Psvcholoeiaue. 69.247-264. Bonnet, C. (1986). Manuel oratiaue de PsvchoDhvsiaue. Paris: Armand Colin. Visual perception in context. Cahiers de Psvcholock Coe_nitive. 6. Bonnet, C. (1986). 137-155. Braida. L.D.. & Durlach. N.I. (1972).Intensity perception 11. Resolution in oneical Societv of America. U .483interval paradigms. Journal oft-t 502. Breitmeyer. B. (1984).Visual masking: An inteerative amroach. New York Oxford University Press. Carterette, E.C.. Friedman, M.P.. & Cosmides. R. (1965).Reaction-time distributions in the detection of weak signals in noise. Journal of the Acoust ical Societv of America. 531-542. Cross, D.V. (1973).Sequential dependencies and regression in psychophysical judgments. Percention and PsvchoDhvsicS. 14,547-552.

,u,

.=.

a,

Psych o p h y s i c a 1 a p p r o a ch e s

241

Demany. L. (1985).Perceptual learning in frequency discrimination. &uma 1 of the Acoustical Societv of Amerim 1118-1120. De Valois. RL.. & De Valois, K.K. (1988). 1visioq. New York: Oxford Science Publications. Donders, F.C. (1968).Die Schnelligkeit Psychischer Prozesse. Archlv fur Anatomie und Phvsioloeie und wissenschaftliche Medezin (English translation in W.G. 1969. 412Koster (Ed.). Attention and Performance 11. ActaPsvcho 431). Fiorentino, A.. & Berardi, N. (1981).Learning in grating waveform discrimination: Specificity for orientation and spatial frequency. V M R e s a , 11491158. Gescheider, G A . (1976).PsvchoDhvsics. theorv and method. Hillsdale, N J : Lawrence Erlbaum. Gescheider, GA. (1988).Psychophysical Scaling. m u a l Review of Psvchology, 169-200. Gravetter, F.. & Lockhead, G. R (1973).Criteria1 range as a frame of reference for stimulus judgment. Psvcholoeical Review. 80.203-216. Green, D.M.. Smith A.F.. & von Gierke. S.M. (1983).Choice reaction time with random foreperiod. PerceDtion and PsvchoDhvsi ~2, , 195-208. Green, D.M.. & Swets. J.A (1966). Siena1 Detection Theorv a nd PsvchoDhvslcs. New York John Wiley & Sons. Hartmann. W.M. (1983). Localisation of sound in rooms. Journal of the Acoust ical Societv of America, 74.1380-1385. Hick, W.E. (1952).On the rate of gain of information. Quarterlv Journal of ,21,11-26. Jesteadt, W.,Luce. RD.. & Green, D.M. (1977).Sequential effects in judgments of loudness. Journal of Emerimental Psvcholom: Human PerceDtion and performance. 9, 92-104. Kirby, N. (1980).Sequential effects in two-choice reaction time: Automatic facilitation or subjective expectancy? In A.T. Wetford (Ed.). Reaction Times. London: Academic Press. Laming, D.R.J. (1968).hformation Theorv of Choice-Reaction Tim- . London: Academic Press. Lappin. J.S. (1978).The relativity of choice behavior and the effect of prior knowledge in the speed and accuracy of recognition. In N.J. Castellan & F. Restle (Eds.). Comltive Theoq. Vol 111. Hillsdale, NJ: Lawrence Erlbaum. Lappin. J.S., & Ditch, K. (1972). The latency operating characteristic. I: Effects of the stimulus probability on choice reaction time. Journal of ExDerimentd Psvcholo-, 92.419-427.(a) Lappin. J.S., & Ditch, K. (1972). The latency operating characteristic. 11: Effects of stimulus intensity on choice reaction time. Journal of ExDerimental P ~ C h O l O93.367-372. ~, (b) Leonard, J.A. (1959).Tactual reactions I. Quarterlv J o u rnal of Exnerimental psvchol 1-38. Link. S d The relative judgment theory of two-choice response time. Journal of Mathematical Psycholomy.12.1 14-135. Link. S.W., & Tindall. AD. (1971).Speed and accuracy in comparative judgments of line length. PerceDtion and PsvchoDhvsics. 9,284-288. Luce. RD. (1959). Individual Choice Behavior. New York John Wiley & Sons. Luce. R.D. (1986).ResDonse times: Their role in inferrlne elementarv mental m a n b a t i o n . New York: Oxford University Press. Luce. R.D.. Green, D.M.. & Weber. D.L. (1976).Attention bands in absolute identification. Percention and PsvchoDhvsics. 49-54. M a r k s , L.E. (1986).Context and sensory processes: in serach of basic laws. Cahiers de Psvcholoeie Coanitivc. jj, 121-136.

.a.

a,

a.

a,

,a,

a,

242

C. Bonnet

Parducci, A. (1965). Category judgment: a range frequency model. & y C b o l oi~ d =, 22.407-418. Parducci. A. (1982). Category ratings: still more contextual effects! In B. Wegener (Ed.), Social atWudes and W h v s i c a l measurement. Hillsdale, NJ: Lawrence Erlbaum. Parducci, A.. & Perrett, L.F. (1971). Category rating scales: effect of relative spacing and frequency of stimulus values. Journal of EXDerimental Psvcholofi 89.427-452. Poulton. E.C. (1968).The new psychophysics: six models for magnitude estimation. ical B W .m, 1-19. P o u l t o n m 9 7 9 ) . Models of biases in judging sensory magnitude. Psvcholoeical Bulletin. 69. 1-19. Proctor, R.W., & Weeks, D.J. (1989). Instructional and probability manipulations of bias in multiletter matching. PerceDtion and PsychODh m . 45.55-65. Ratcliff. R.. & Hacker. M.J. (1981). Speed and accuracy of same and different responses in perceptual matching. m e D t i o n and PsvchoDhvsks, 3Q.303307:. Remington, RJ. (1969). Analysis of sequential effects in choice reaction times. 1 of E m ) t a .l-P 82. 250-257. Shepard. R.N. (1981). Psychological relations and psychophysical scales: on the status of "direct" psychophysical measurement. Journal of Mathematical 2p. 2 1-57. Sperling, G. (1960). The information available in brief visual presentations. Psvcholorrical MonouraDhs. 74. Staddon, J.E.R. King. M.C., & Lockhead, G.R (1980).On sequential effects in absolute judgment experiments. J 1 nal of Ex 1 . umn PerceDtion and Performan= , 6. 290-301. Swensson. R.G. (1972). The elusive tradeoff Speed versus accuracy in visual discrimination tasks. PerceDtion and PsvchoDhvsicS. 12. 16-32. Swets. J.A. (1986a). Indices of discrimination or diagnostic accuracy: Their ROCs and implied models. ESycholouical B u m 100-117. Swets. J.A. (1986b).Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theow and measurement performance. Psvcholog ical -~ miletin, 99.181-198. Teichner. W.A.. & Krebs. M.J. f1974). Laws of visual choice reaction time. Psvcholoeical Review, fi,75-98. Theios. J.. & Amrheim. P.C. (1989).Theoretical analysis of the cognitive processing of lexical and pictorial stimuli: Reading, Naming and Visual and Conceptual Comparisons. psvcholoeical RevleW. %,*5-24. Thurstone, L.L. (1927).A law of comparative judgment. psvch&gkal Review. 2%. 273-286. Torgerson, W.S. (1958). Theov and methods of scaling. New York John Wiley &

.--

m.

,=.

~

Sons. Vickers. D. (1980). Discrimination. In A.T. Welford (Ed.), &&ion Tim=, London: Academic Press. Ward, L.M. (1972).Categoryjudgments of loudness in the absence of an experimenterinduced identification function: sequential effects and power function fit. Journal of ExDerimental Psvcholoa, 179-184. Ward, L.M. (1973). Repeated magnitude estimations with a variable standard: sequential effects and other properties. =Dt ion and PsvchoDhvsics. 193-200. Ward, L.M. (1987). Remembrance of sounds past: Memory and psychophysical scaling. Journal of ExDerimental Psvcholoev: Human PerceDtion and Performance, 13.216-227. Welford. A.T. (Ed.) 11980). i . London: Academic Press.

a,

u.

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez (Eds. ) E l s e v i e r S c i e n c e P u b l i s h e r s B. V . (North-Holland),

1990

CONTEXT EFFECTS IN FACE RECOGNITION BELOW RESPONSE BIAS. THE CONTRIBUTION OF A SIMULATION Anne-Caroline Schreiber, Stephane Rousset and Guy Tiberghien L a b o r a t o r y o f E x p e r i m e n t a l Psychology,

CNRS, Grenoble, F r a n c e

Abstract. Context effects in face recognition have been interpreted either in terms of response bias. or a s evidence in favour of the specificity encoding principle. This second interpretation was studied through a connectionist simulation. From a cognitive model of face recognition. a layered network was specified and implemented to investigate the identiflcation processes. This network was tested in a n experiment to study the structuring of identity representations in various learning conditions defined by the specificity and variability of the encoding context. The simulation results showed a significant interaction in identification performance between the variability and specificity factors. Identification of faces learned in variable contexts is never affected by a contextual change during recognition, whereas non-variable faces are affected. and all the more so when their encoding contexts are non-specific. These results are discussed in terms of the "semantization" process (generalization) and the fan-effect phenomenon (categorization). The organization of memory is tackled in reference to context dynamics. Interest in such a new approach in cognitive sciences is discussed in relation to experimentation and modelling in cognitive psychology.

INTRODUCTION

Interest in the perception and recognition of faces has been very active over the past ten or so years in cognitive psychology, considering the exponential increase in publications in this field (Ellis, 1986). In many experimental as well as clinical studies (for reviews, see Davies, 1986: Tiberghien, 1986; Davies & Thomson, 1988). forensic or everyday life errors have highlighted dramatic context effects in face recognition. These context effects have been interpreted either in terms of response bias (Baddeley & Woodhead, 1982). or as evidence of the existence in memory of contextualized episodic traces (Tulving & Thomson, 1973: Peris, 1986). The best way to resolve the debate is to specify the mechanisms involved in face recognition by building theoretical models which could tell us at which level context can take effect. The face recognition model proposed by Bruce and Young in 1986 draws together and extends the main functions put forward successively by various models (for a

244

A. C. Schreiber,

S . Rousset, G . T i b e r g h i e n

historical review: Bruyer, 1987). We shall thus first present the main characteristics of this model, now used as a theoretical framework for interpreting experimental and clinical data. Then, we shall discuss how the model can account for the context effects classically encountered in face recognition. Finally, we shall conclude by presenting a connectionist simulation which alone represents an original and useful "bias" for a thorough study of context effects, and which enables us to go below the possible response bias that might be induced by the decision mechanism of a real subject. THEORETICAL BACKGROUND

This paper falls within the general framework of connectionist theory as a new paradigm for the simulation of cognitive phenomena. The advanced formalization of neural network dynamics as well as the recent discovery of learning algorithms for complex networks [Rumelhart & McClelland, 1986; McClelland & Rumelhart, 1986; Grossberg, 1988) now make it possible to consider real simulations of cognitive processes. We shall focus here on the processes underlying person identification from faces. The ability to recognize a familiar face in only a few milliseconds is puzzling as well as universal. Moreover, the recognition of this complex stimulus constitutes the most distinctive and widely used key to a person's identity. Identifying a person from his or her face thus requires not only the recognition of the tenuous characteristics which allow us to make a distinction within the face class, but also implies that identity representations can be accessed from facial information. I t is for the latter reason that the face can be considered as a universal identifier. In order to integrate this specific characteristic of faces, Bruce and Young's model not only describes s t i m u l u s recognition, b u t also models person identification from faces. Before presenting the simulation, we shall outline the principal characteristics of the cognitive model which has served as the theoretical background for the design of the FACENET system. FACENET is a connectionist system which simulates two functions put forward by all current cognitive models of face recognition: familiarity estimation (stimulus recognition) and access to the identity of the person from his/her face (identification). Firstly we shall describe how a cognitive model accounts for these two functions, and then we shall demonstrate how a connectionist

245

Context effects in face recognition

microstructure can account for the same functions in a different way. The recent cognitive model proposed by Bruce and Young (Figure 1) constitutes a general framework for analyzing the mechanisms underlying face recognition, and is used to draw together data from both patients with brain damage who fail to achieve efficient face processing (prosopagnosia) and normal subjects (everyday errors and laboratory experiments). It is a model of the functional components involved in face recognition which is couched in information-processing terms. I t hypothesizes a set of sub-processes, some of which precede others (structural encoding module -> face recognition units -> person identity nodes -> name generation module). The boxes and arrows have no status other than being a convenient schematic representation of the ways in which the different functional components interact with each other.

EXPRESSION ANALYSIS

VIEW-CEN'I'KED DESCRIPTION

& EXPKILSSIONINDEPENDENT DESCRIPTIONS

SQRWCTUlPtAIL IENCaDDrnG

Figure 1. The functional model for face recognition (Bruce & Young, 1986).

This face recognition model shares the same structure with the functional models used in the areas of word and object recognition (Warren & Morton, 1982). A box represents any processing module, or store, which plays a distinct functional role, and whose operation can be eliminated, isolated or independently manipulated either through experiments or as a consequence of

246

A.C.

Schreiber, S . Rousset, G. Tiberghien

brain damage. Arrows between boxes denote the access, conversion or recording of information, and the activation of one component by another. Three main components work towards the identification of a person from his/her face. The structural encoding component provides a perceptual description of the presented face. The perceptual processing module might operate according to the processing levels defined by Marr (1982). This perceptual description can match a previously stored representation, held in what Hay and Young (1982) defined as the Face Recognition Unit (FRU). This matching gives rise to the face familiarity feeling. Each FRU contains all the stored representations related to a particular familiar face. They are described as localized abstract units responding by overall activation. Once a FRU is activated, it can then access person identity representations held in Person Identity Nodes. These nodes are entry points to associative memory and contain the identity-specific semantic information (what a person does, where he lives, ...I that allow us to feel we have successfully identified the person. The sequential structure of module activation is based on the latency hierarchy classically found in experiments. The latencies of familiarity decisions are indeed shorter than the ones recorded in semantic judgments of occupational categorization (Young, McWeeny, Hay, & Ellis, 1986). In the same way, since naming responses are always the slowest (Young, McWeeny, Ellis, & Hay, 19861, the name generation module is the last component accessed in the hierarchy. The broadest component, the Cognitive System, is responsible for all the aspects of processing not reflected in other modules. More precisely, it can assume three distinct functions. First, it contains all of the associative memory, that is to say, all the episodic and semantic information to which the person identity nodes form an entry point. Secondly, it estimates the activation level in the various components of the model, and functions as a decision mechanism. Finally, it can produce a feedback influence on the specific processing of other modules. The feedback of the Cognitive System on Person Identity Nodes in fact denotes the role of contextual information (and the subject's expectancies) in identification judgments, and even in familiarity decisions, since in this model, Identity Nodes influence Face Recognition Units. Besides its decision-making function, the Cognitive System is responsible for top-down phenomena. According to this functional model, it is through the Cognitive System that the representational

Context effects in face recognition

241

and situational recognition context interacts with the specific processing of the face stimulus. This kind of interaction between the specific processing of a focal stimulus (face) and the context processing must be postulated in face recognition models insofar as face recognition is very sensitive to contextual change. More generally, both in verbal and visual memory studies, it is accepted t h a t encoding and retrieval processes are not merely determined by the characteristics of the stimulus itself, but also by the characteristics of the occurrence context. The encoding specificity principle (Tulving & Thomson, 1973) fits well with context effects, by postulating that retrieval probability depends on the degree of compatibility between encoding and retrieval context. This principle assumes t h a t traces in memory are always contextualized because the memorization of a fact results from the encoding of the complete episode in which that fact was perceived. Tulving (1983) makes the distinction between episodic memory (particular events), very sensitive to context effects, and semantic memory (concepts). much more independent of contextual fluctuations. While the encoding specificity principle interprets context effects in terms of the discriminability of memory traces, Baddeley and Woodhead (1982) stress that context effects could reflect a tendency for a familiar context to bias the subject towards claiming that the face is also familiar. Indeed, almost invariably, reinstating the initial context not only increases the detection rate (hits) but also the rate of false alarms. The context effect would t h u s be nothing but a response bias. In order to dissociate the influence of context on the discriminability of the memory trace from its effect on response bias, Baddeley and Woodhead set u p a crossover recognition condition by presenting "old" faces in "old" contexts which were re-paired and thus did not correspond to the original learned association. Their results show a clear effect of context on performance. In three cases out of four, this effect appears to be entirely due to the influence of context on the response criterion. However in one case, the results suggest that context may have been influencing discriminability in a way that might have been predicted on the basis of the encoding specificity principle. The authors conclude that "it would be unwise to argue that context effects in face recognition are entirely due to bias" (p. 162). Some experiments have used a forced choice technique which eliminates response bias b u t still does not eliminate the context effect!

248

A.C.

S c h r e i b e r , S . Rousset, G . T i b e r g h i e n

Moreover, PCris (1986) reviewed many experiments using a crossover recognition condition, and concluded that context plays a considerable part in the discriminability of memory traces. Likewise, in a recent review of the literature, Davies and Thomson (1988)dismiss response bias as the sole explanation of context effects. If we accept today that context effects occur in the accessibility of memory traces more than in decision making, one question can be addressed: at which level of Bruce and Young's model can context be taken into account? Bruce and Young indicate that contextual effects found in laboratory studies and everyday life are not surprising when we consider the relationship between Person Identity Nodes and context. Indeed, as Thomson, Robertson and Vogt (1982, p. 148) underline, "the context defines the person", or in other words, identity representations result from the integration of contextual information during the repeated presentation of a face (or a person) in context. Yet, Bruce and Young's model hardly accounts for the context effects involved in face recognition, except if one invokes the Cognitive System, a nebulous and omnipotent unspecified module. The authors propose in fact that contextual effects may be assimilated to semantic priming effects. But where does contextual priming exert its influence? Bruce and Young suggest that priming may occur at the level of Face Recognition Units via the Person Identity Nodes, through spreading activation within the semantic system, thus aiding in the perception of the face. Ellis, Young and Hay (1986)propose that contextual priming may also exert an influence on post-perceptual processes such as the activation of relevant person information. J u s t as sentence context effects facilitate post-lexical activation and integration of word meaning, the circumstances of the encounter may facilitate access to face meaning, which is identity. Bruce and Young emphasize that there are some problems in attributing semantic priming effects to recognition units. The most important of these relates to the time course of semantic priming effects. Semantic priming effects dissipate rapidly (5 seconds) whereas repetition priming effects are relatively long lasting (60 seconds). I t is thus implausible to attribute both effects to a common mechanism in the form of increased activation of FRUs. They suggest that it would be possible to continue to locate repetition priming effects in the activation of FRUs. and to argue

Context effects in face recognition

249

that semantic priming affects the decision process. So, six years after Baddeley and Woodhead (1982). the question of response bias is again raised. Therefore, we can state that the dynamics of context effects and their role in the general functioning of face memory remain unclear. Furthermore, according to these authors, the context's role constitutes an unresolved issue in their model. As Davies (1988) stresses, there is an urgent need to take context explicitly into account in the core of face recognition models: this need is justified by the extent of context effects reported both in experimentation (Memon & Bruce, 1985: Tiberghien, 1986) and in everyday life (Young, Hay, & Ellis, 1985). Laboratory studies (Davies & Milne. 1982; Thomson et aZ., 1982) have shown that the recognition of unfamiliar faces is strongly affected by a contextual change occurring between study and test, whereas recognition of familiar faces never seems to be affected. Bruce and Young interpret this discrepancy in reference to the existence of previously stored representations in long term memory for familiar faces (Face Recognition Units and Person Identity Nodes of course do not exist for unfamiliar faces). This interpretation should be qualified since faces that have been experimentally prefamiliarized remain very sensitive to contextual change (Thomson et al., 1982 exp. 6; Seamon, 19821, and in everyday life (Young, Hay, & Ellis, 1985). some faces, though familiar, are difficult to recognize when encountered in an inadequate context (for instance, a very familiar face - your baker - is hard to identify at a soccer match!). One must thus study the nature of these previously stored representations in a more precise way. If the content of identity representations results from context integration, then the contextual learning conditions for a face must be a determining factor in explaining recognition errors. To specify their model and to study the interactions between the various processing levels, Bruce and Young (1986)underline the fruitful role of a computational approach and simulation. Concerning the form of this simulation, according to McClelland (1988). the connectionist framework has led to the discovery of new principles that have helped to build specific models. It is in this perspective that we have designed a connectionist network - FACENET - in order to simulate face identification while integrating the dynamic role of context in the construction of identity representations.

250

A . C . Schreiber, S . R o u s s e t ,

G. Tiberghien

Recently, several researchers (Bruce & Young, 1986; Tiberghien, 1988; Young & Ellis, 1988) have proposed that a connectionist simulation could constitute a useful approach to advancing our understanding and modelling of the processes involved in face recognition. Concerning the modelling of Face Recognition Units, repetition priming experiments (Ellis, Young, Flude, & Hay, 1987) indicate that Bruce and Young's conception cannot account for the results obtained, since the same picture produces more priming than a different picture of the same face. This finding is incompatible with modelling FRUs as localized abstract units that respond to any view of a particular face. Ellis et aL(1987) stress that it is necessary to transform the notion of FRU into a more instance-based conception, and t h u s argue that a distributed and connectionist model of FRUs would be more adequate. As for context effects, we have already pointed out that it would be relevant to study the nature of Person Identity Nodes. since they are progressively constructed by the integration of contextual information. In Bruce and Young's model, we do not know how these identity nodes are constructed. These authors consider that the context enters the model through the Cognitive System at recognition time, b u t the problem is that to understand the dynamic role of context in access to identity, we must consider its role during the progressive creation of identity representations, Bruce and Young's functional model is a recognition model that does not really take into account the way identity nodes are constructed. In other words, it does not model the learning stage. Accordingly, we have chosen a connectionist model because it enables u s to explicitly model a truly interactive kind of processing between several different sources of information, and it allows for the simulation of a dynamic learning stage. This made it possible for u s to study how identity representations are constructed. The first thing of interest in such a simulation is its heuristic value. Indeed, by simulating the functions of a cognitive model on a system, we hope in return to be able to more precisely define the cognitive model of reference. To create a computer system, a large number of details which are not specified in a box and arrow model must be made. Moreover, such specification choices can be tested. This testing has also motivated the creation of the FACENET system. It can be considered as a virtual subject, on which virtual experiments can be made. The advantage is that these experiments are very precise since the subject's facial history can be controlled,

Context effects in face recognition

251

i.e. the content of its long term memory, its earlier experience, etc. Experimental factors can be manipulated and easily isolated. Of course, this kind of result has no ecological validity. but it can offer a new theoretical framework for further experiments on real subjects. The use of the connectionist framework as a simulation tool is especially relevant to characterizing interactive processing (McClelland & Rumelhart, 1986). More precisely, the connectionist model seems particularly suited to studying face recognition: the applications of its principles to pattern recognition highlights the networks' capability to efficiently process complex visual stimuli (Kohonen, 1984; O'Toole, Millward, & Anderson, 1988). Moreover, face recognition constitutes one of the cognitive skills that most likely emerges more from the repeated presentation of exemplars than from the learning of explicit rules. Thus a connectionist approach seems to fit well with face recognition. since learning through examples is a fundamental characteristic of connectionist models. Finally, and according to Hofstadter (19851, only a bottomup approach will be able to account for the 100 milliseconds you need to recognize your mother's face! FACENET ARCHITECTURE

FACENET is a multilayer network functioning according to the Gradient Back Propagation algorithm. This particular connectionist model seems especially suited to the simulation of a cognitive model. I t provides the means to model, through its architecture, several processing levels or functions, integrated into the same system. One of the specific features of layered networks is their hidden units, which are very important for this simulation since they enable t h e network to generate i t s own internal representations. And our aim is indeed to study the creation of identity representations during the learning stage. In addition, the multilayer model is not necessarily a static recognition system. I t can be equipped with a mechanism to send feedback from its output to its input, thus making it a truly dynamic system. With such feedback, a layered network can simulate a dynamic searching process in memory. FACENET models only that part of memory processing that leads to familiarity estimation and identification of a person from information stemming from the structural encoding of the face (perceptual processing) and from contextual information. At the moment, these two types of input are coded by a vector with 25

252

A . C.

Schreiber, S . Rousset, G . Tiberghien

components, and this vector is composed of 5 blocks. For the sake of simplicity, we propose that each block codes a part of the face, or a macro-feature. For further simplification, we have decided to code the contexts with a vector structure similar to the one used for the faces. Both vectors are at present arbitrary, because we have so far been more interested in the study of the interaction between the two pieces of information - facial and contextual - that create identity representation, than in the system's perceptual likelihood. FACE ECHO

R E I N J E

C T

I C

h

IDENTITY

CONTEXT ECHO

R

E I

v J

E C T I 0 N

Figure 2. FACENET architecture: 4 layers, 230 cells, and 3950 connections. The continuous lines define beams of complete connections and the dotted lines define beams of specific connections. The two lateral arrows indicate that face and context output can be reinserted into the input. FACENET explicitly takes into account contextual information, because the creation of and access to identity depend on contextual

Context effects in face recognition

253

input. In fact, as discussed in the theoretical section above, identity is nothing but the integration of contextual information. For familiar persons, part of their meaning is "where they live, what they do", and so on. For unfamiliar persons, their identity is even more bound up in the contexts with which they are associated. To enable FACENET to function directly from digitized faces, its forthcoming extension will demand a considerable increase in the number of connections (and in computation time) compared to this first prototype. The network's output can be assimilated to ecphoric information, which according to Tulving (19831, may result from the synergistic process of interaction between the structure of memory and the retrieval cues. The two lateral types of output are Face-Echo and Context-Echo. During the recognition stage, these echoes in fact represent the reminiscence of the search, which means that they contain "the mental image" of the face (and the contextual information relative to the identity of the person) emerging during the search in memory. During the learning stage, the simple self-association of the output with its respective input is performed. By this self-association, the system has to learn the face and the context. This self-association constraint may be linked to the theory proposed by Sayre (19861, according to which the human system's adaptation to its environment requires the system to constantly adjust its internal representations to those resulting from the perceptual processing. Thus, learning a face stimulus should, as Searle (1985) said, be as easy as making "footprints in sand". At the recognition level, it is the similarity between the reminiscence emerging on the Face-Echo and the information present in the output of the perceptual system that enables an immediate familiarity estimation of the face. J u s t as the learning stage (and thus familiarity acquisition) consisted in a n adjustment constrained by the input/output similarity, the percept/echo discrepancy is intrinsically a familiarity indicator during recognition, and therefore performs the same functions as the Face Recognition Units in Bruce and Young's model. The central part of the identification is a hetero-association. During the learning stage, the cell corresponding to the identity of the face presented is desired at 1 and the others at -1. This constraint is the explicit means to ensure the constancy of the identity, in spite of the contextual variations in which the face is learned. Each cell thus performs the same function as a Specific

254

A.C.

Schreiber, S . R o u s s e t , G . T i b e r g h i e n

Identity Node, because each node represents the place where semantic and episodic information (revealed by the ecphoric process) converge, in order to specify the particular identity of the face presented in the input. The value of a given identity node is the strength of the activation of this identity representation. In regards to the structure of the connections and the layers enabling the various levels of representation, the two lateral parts of layer 2 are used for the specific processing of facial information and contextual information. These lateral parts, by means of a system of specific connections, are used to process the input by instituting a hierarchy between the 5 blocks of the face (and between the 5 blocks of the context). Each block can be completely connected to 1, 2. 3. 4. or 5 blocks of the previous layer. These beams of specific connections are a rough attempt to introduce a hierarchy into the memory processing. Experimental results have shown indeed that some characteristics of the face are more important than others (Shepherd, Davies, & Ellis, 1981). The central part, which is called Face-Context association, is precisely used to create the interaction between a given face and its context. As we have already said, this association part is a determining factor for the Construction of identity representations. I t is completely connected to the input layer. This central part should allow the construction of episodic representations resulting from the association of faces and contexts: it t h u s models the dynamic c o n s t r u c t i o n of t h e contextualized episodic representations. The connections between this part and the echoes model the semantic priming effects upon the familiarity decision found in experiments (Young & Ellis, 1988). and enables the network to recall a face from context input (and vice versa). The third layer is completely connected to the Face-Context association and to the Face-Specific part in layer 2. This asymmetry in connectivity between layer 3 and 2 models the focal status of facial information for person identification. Since the FACENET system simulates person identification from a face stimulus, it is therefore obvious that the face has to be the privileged key to accessing identity. This third layer plays a fundamental role because, given its small size, it would have to generalize and find common points in its input. Moreover, since it is the only one connected to the identity output, it h a s to abstract identity representations from the contextualized episodic representations of layer 2 in order to satisfy the requirement imposed in layer 4. This

Context effects in face recognition

255

third layer, together with the second one, should model semantic and episodic memory concerning person identity. I t is in fact the connections between layers 1 and 2. 2 and 3. and 3 and 4 that actually encode t h e s t r u c t u r e a n d content of identity representations in memory. FACENET was designed to explicitly take into account what happens during this learning stage in order to explain the context effects found a t the recognition stage. It is for this reason that identity is considered as the result of the interaction between face and context input, a n interaction occurring during both the encoding stage and the recognition stage. Thanks to its reinsertion feature, the FACENET system provides dynamic functioning during recognition. During the identification stage, FACENET can reinsert Face-Echo and ContextEcho into its input. The reinsertions (feedback) are very important since they create a dynamic system. This strategy can make FACENET converge progressively to a stable state and retrieve learned information from distorted or partial input. These reinsertions also enable the dynamic simulation (or emulation) of the search process in memory which occurs during the person's identification from his/her face. The introduction of feedback in the recognition phase results from the psychological aspect as well as from the consideration of the dynamic behavior of layered networks. It essentially concerns the contraction effect of the function performed by the network. This contraction effect induces various dynamic attractors, especially when the input and output of the network are coded in the same way. Theoretical analysis of the feedback dynamics of a layered network is extremely difficult due to the nonlinear property of the function performed by the network. It is out of the scope of this article to thoroughly address this theoretical issue (see Wang & Besson, 1989). The contraction effect is a n important faculty of a neural network that enables it to generalize the answers to input that is not in the training set. The generation of the attractors around the learned patterns is derived from the contraction effect and these attractors play a key role in modelling memories. The construction of our network reflects the intention to simultaneously exploit several capabilities of layered networks in a n organic way. The results reported here serve as a theoretical basis for introducing feedback into the network. Of course feedback can also be introduced into the learning stage, as in Jordan's work (Jordan, 1986). This enables the

256

A . C . S c h r e i b e r , S . Rousset, G . Tiberghien

network to efficiently learn time-dependent sequences. I t is expected to be useful in dealing with temporal variations (successive poses or expressions), which are important in the creation of invariant face representation in memory, This feedback in the learning stage will be used to study specific face representation in addition to identification processes. SIMULATION AND RESULTS

The FACENET system underwent an experimental protocol in order to study the interaction between facial and contextual information in the structuring of identity representations. This kind of Face-Context relationship was manipulated through two experimental factors, the variability and the specificity of the encoding context. These two experimental factors are very interesting from a theoretical standpoint. Even if one admits today that a stimulus is encoded in memory with all of its contextual elements, that is to say, that contextualized traces exist in memory, it is nevertheless necessary to consider the nature of these traces in order to understand their construction, their transformation and their organization.

Factors Concerning the variability factor, a face can be learned several times in the same context (non-variable encoding condition) or in many different contexts (variable condition). According to Tulving (1983), presentation of episodes in different contexts progressively generates a semantic representation via a process of abstraction. Semantic representations are more independent of context effects than episodic ones. On the contrary, presentation of episodes in a n identical context retains the contextual specificity of this episode, which remains very dependent on context effects. Therefore, the encoding variability factor could induce the transformation of episodic representations into semantic ones (hypothetical process of episode "semantization"), Experimental evidence of this hypothetical process is reported by Tiberghien ( 1986): recognition of faces learned in four different contexts is less affected by a contextual change between study and test than recognition of faces learned four times in the same context. These types of results have been also found in word recognition (Hintzman & Stern, 1978; Smith, 1982). Tiberghien proposed to draw a distinction between

Context effects in face recognition

257

episodic and semantic familiarity, in order to take contextual factors into account during the acquisition of familiarity (i.e. during the learning stage). Concerning the specificity factor, a context can always be associated with the same face (specific encoding). On the contrary, a context can be shared by several faces (non-specific encoding). Specificity is a symmetrical factor of variability, because a variable face is encoded in several contexts, just as a non-specific context is encoded with several faces. The specificity factor is interesting because it links several faces together. Thus, the learning list is structured like a semantic network, with a face linked to a context, which in turn is linked to another face, etc. According to Anderson (1983). long-term memory is structured like a semantic network and fan-effect phenomena (spreading of activation within a semantic network) are generally observed in experimental tasks. FACENET is not a semantic network because its cells and connections have no meaning. But, it was interesting to study how a connectionist system could become structured (in a distributed way) in order to learn a list of stimuli structured like a semantic network. This enabled us to study how the similarity structure contained in the base of examples could influence the structuring of the information in the FACENET system during the learning stage. Our idea was to see if a connectionist network was able to simulate these categorization and fan-effect phenomena - classically found during retrieval - and to do so, of course, without supposing any semantic network organization, b u t simply owing to the parallel and distributed processing of a base of examples with non-specific contexts.

Method The FACENET system thus constitutes a single virtual subject. Once its architecture is defined, it has to acquire a facial history, that is to say, it must learn a list of patterns. The database used in this simulation contains 30 faces and 30 contexts. This input is in fact a series of vectors of 25 components, constituted by 5 blocks. These vectors can resemble each other on 0, 1, or 2 blocks. The learning list is made up of 139 patterns, a given pattern being a n association between one face and one context. The two experimental factors are crossed, making four encoding conditions, as represented in Figure 3. In each condition, there are four faces, two learned twice and two learned four times during one learning

258

A.C.

Schreiber, S . Rousset, G . Tiberghien

iteration. Such a difference in the number of presentations was introduced in order to study the effect of the degree of learning, but we will not develop this point here. The fourteen other faces in the database were meant precisely to give non-specific contexts their characteristic of non-specificity. VARIABLE

NON-VARIABLE

S

P

E C I

F-

C

F I C

N 0

N

-

S

P E C

I

F I C

Figure 3. The four encoding conditions obtained by crossing the two experimental factors. Before the learning started, weights were initialized at random between 0.5 and -0.5, and the parameter alpha which characterizes the sigmoidal function of each cell was also initialized at random between 0.5 and 1. Momentum was set at 0.01 and the learning rate at 0.3. The network was subjected to 1000 learning iterations, one iteration being constituted by one presentation of the 139 patterns. At the end of the learning stage, all the patterns had been learned very well. We did not want to study the effect of both experimental factors on learning speed because it depends on several random factors, such as the initial connection weights, the order of the list, etc. So, we studied the effect of the factors on recognition rate.

259

Context effects in face recognition

The recognition tests should reveal how the network structured and integrated its knowledge during the learning stage. Recognition tests concerning the two experimental factors First, faces were presented in their encoding context. This test was casual, but we wanted to ensure that all the faces to be compared later were equivalent on their identity value (our dependent variable), whatever their encoding condition was. This test shows that all the patterns were equivalent in identity strength. Thus, the learning degree would not interfere with the two manipulated factors. Therefore, we tested their effect by means of an old re-paired contexts condition. For this second condition, 15 old contexts were used, ones which had never been associated with the four testing faces (one for each encoding condition). An analysis of variance was performed on the identity strength data recorded after the first iteration of recognition.

Table 1. Table of the analysis of variance performed in the old repaired context condition, for the two experimental factors. SOURCE VARIABILITY SPECIFICITY VARIA * SPEC1 ERROR

SS 1.386 .793 .616 7.205

DF

MS

1 1 1 56

1.386 . 7 93 .616 .129

F 10.605 6.147 4.775

P .002 .015 .031

The main effect of variability is significant, which means that the identification of faces encoded in variable contexts was less affected by a contextual change at recognition than the identification of non-variable faces. In the same way, the identification of faces encoded with nonspecific contexts is significantly lower than the identification of faces learned in specific contexts. But the latter conclusion must be qualified because of the interaction effect (Figure 4). We can see indeed that for non-variable faces, there is a strong specificity effect: when a face was encoded in a specific context, its

2 60

A . C.

Schreiber, S. Rousset, G . T i b e r g h i e n

identification was better than if it was encoded in a non-specific one. In contrast, this specificity effect is no longer perceptible for faces learned in variable contexts: the identification of such faces was very good, regardless of the specificity (or the non-specificity) of their context. 1.0

-

0.5

-

5hil

8

b

v)

"c, non-specific

h

.c,

.d +-)

8 a

.I+

variable

non-variable

Figure 4. Average identity strength recorded after the first iteration in the old re-paired context condition. Discussion concerning the two experimental factors

Contextual variability during encoding seems to play a n important role in the integration of the information by the network. This variability enables the network to abstract a n identity representation from the various contexts presented. Thus, at recognition time, identity can emerge even if contextual information is irrelevant. The interaction effect corroborates this interpretation: if variability really enables progressive abstraction of identity, one can understand why at such a level of learning, there is no longer any difference between specific and non-specific encoding contexts. Contextual non-variability induces a n identity representation that is merged with the Face-Context pattern learned. In order to output a strong identity, this episodic Face-Context association must be preserved (or retrieved). Since the network was not able to

Context effects in face recognition

261

create a n identity representation that is independent of that particular association, any contextual change will induce a weakening in performance. This explanation can be applied to all of the non-variable faces. But how to explain the specificity effect concerning nonvariable faces? When a face is encoded in a specific context, the network h a s learned to give a n identity to that particular association, but the two components have never been learned with other identities. So, the episodic trace is very distinctive for the network. Faces encoded with a non-specific context are even more highly affected by a contextual change at recognition time. Nonspecific contexts indeed have been learned many times with several faces. They have therefore become real attractive nodes, around which faces are organized. Non-specific contexts have thus created categories, each category of course containing all the faces sharing the same context. Therefore, since non-specific contexts are at the root of a categorization process that organizes information in the network, they play an important part in the creation of identity. And thus, identification of non-specific and non-variable faces is very sensitive to contextual change during recognition. In order to find some clues corroborating these interpretations, we studied the dynamic evolution of the network in a third recognition condition. Faces were presented in new contexts that had never been learned, and we let the network make many reinsertions. Comparison of Figures 5 and 6 shows that the variability effect found in the statistical analysis is once again perceptible, since the identification of the faces encoded in the variable condition was not affected by a new context at recognition time. We can see that a great deal of identity constancy exists during reinsertion, and the Face-Echo is almost perfect. On the other hand, non-variable faces are affected by a new context. This is shown by the evolution of identity as well as similarity. However, this disruption differs in Figures 5 and 6. For a face encoded in a specific context (Figure 5). identity drops at the second iteration because the reinserted facial information is not as pure (no longer composed only of 1 and - 1 ) . Then, the appropriate context is progressively retrieved through the reinsertions, causing the identity strength to progress. We can see that the Face-Echo only improves from one reinsertion to the next. In fact, the Face-Echo is never affected by the retrieval of the

A . C. Schreiber, S.

2 62

Rousset, G. T i b e r g h i e n

encoding context, and thus we can say that this face-context association is very distinctive in memory and is not likely to activate other associations.

0.9

!

I

1

I

2

I

I

3

4

I

I

5

6

I

I

7

8

I

I

9

10

ITERATIONS OF THE NETWORK -*- SPE VAR(ID)

'"*"' SPE VAR(SIM)

-A- SPE NONVAR(ID) "'A' SPE NONVAR(SIM)

Figure 6. Study of the variability factor in a new recognition condition where encoding contexts were specific. Continuous lines represent identity values. Dotted lines represent the similarity between Face-input and Face-echo. These similarities (Euclidean distances) and identities (activation values) are superimposed on the same coordinate axis only for the visual comparison of the evolution patterns. For a face encoded in a non-specific context (Figure 6).the second iteration is very interesting because there is a n identity progression that is not attributable to the face. Indeed at the same time, face similarity diminished. Only the reinserted Context-Echo is responsible for this progression (access to its category). But during iterations 3 and 4, identity and face similarity fell simultaneously. This drop symbolizes a fan-effect phenomenon. The face activated its encoding context, but since this context was associated to other faces, the Face-Echo became more and more distorted because the network activated several faces at the same time (and other identities emerged). By the fifth iteration, the

2 63

Context effects in face recognition

network was finally able to choose, among all the activated associations, the one that corresponded to the face presented as input. Non-specific contexts have thus created categories. Once the category is retrieved, a fan-effect phenomenon propagates activation on other identities. To make a successful identification, the network has to make a differentiation within the activated category, otherwise a false identification can occur.

1

2

3

4

5

6

7

8

9

101112131415

ITERATIONS OF THE NETWORK -A- NONSPE NONVAR(ID) NONSPE VAR( S IM) "'A"' NONSPE NONVAR( S IM)

-*- NONSPE VAR(ID) '1'*'''

Figure 6. Study of the variability factor in a new recognition condition where encoding contexts were non-specific. Continuous lines represent identity values. Dotted lines represent the similarity between Face-input and Face-echo. Similarity and identity are superimposed on the same coordinate axis only for the visual comparison of the evolution patterns. The results of this simulation stress the fundamental role of the nature of the relationship between face and context, during the learning stage, for the structuring of identity representations. Encoding variability seems to induce a n identity representation t h a t is progressively abstracted from various contexts. Identification can thus emerge even if the recognition context is irrelevant. For instance, the identity of your mother is striking wherever you meet her, and fortunately, it is not necessary to begin a contextual search in memory each time we meet a friend!

2 64

A.C. Schreiber,

S. Rousset,

G. Tiberghien

A question can however be raised: Did the network really abstract an

identity? In other words: Did it actually "semantize" the episodes? In order to verify this point, we analyzed the internal representations in layer 3, which should allow for the investigation of the most generalized identity representations. For a variable face, the internal representations were always the same, regardless of encoding context. Figure 7 illustrates this invariance of the activation patterns in layer 3 for variable faces presented in their various encoding contexts (with only some micro-variations related to the different contexts presented). Therefore, the network was able to construct (or abstract) a general shape from contextual variations. Moreover, this general shape is not determined only by the face. We tested this by cutting (at recognition time) the connections between layers 3 and the F*C association in layer 2. The results show that the activation pattern induced only by the face part of layer 2 (Figure 8) is quite different - on cells 7, 9, 10, 13, 14, 15, 20 - from the invariant shape presented in Figure 7. Further analysis of the connection weights between layer 3 and 4 indicates that the weights of the cells on which the discrepancy is observed are comparable to the others (Figure 9). Thus, we can conclude that context has not been considered as noise. The network has really taken contextual variability into account, and has abstracted a semantic (or generalized) representation from this variability. ACTIVATION VALUE 1 0.5

0 -0.5 -1 1 2 3 4 5 6 7 8 9 1011121314151617181920

CELLS OF LAYER 3

Figure 7 . Internal representations in layer 3. Example of the activation patterns produced by a variable face presented successively at recognition time with its four encoding contexts (the four tests being illustrated by the four different motifs).

Context effects in face recognition

2 65

ACTIVATION VALUE 1-

0.5

-

0-

-0.5

-

-1

1 2 3 4 5 6 7 8 9 1011121314151617181920

CELLS OF LAYER 3

Figure 8. Internal representations in layer 3. Example of the activation pattern produced when the connections between the F*C association in layer 2 and layer 3 were cut during recognition. These data were obtained from the same variable face as the one used in Figure 7.

1 2 3 4 5 6 7 8 9 1011121314151617181920

CELLS OF LAYER 3

Figure 9. Connection weights linking the cells of layer 3 to the cell of layer 4 corresponding to the identity node of the variable face used in Figures 7 and 8.

The identity of non-variable faces is completely linked to contextual information, and what is worth remembering is that FACENET is sensitive and can simulate fan-effect phenomena. In order to test the validity of the system architecture, we devised

266

A.C.

S c h r e i b e r , S . Rousset, G . Tiberghien

another network without layer 3. In this case, the constraint of identity constancy imposed in the output layer acts directly on layer 2. In order to satisfy this constraint, the latter network was given less levels of processing and representation than FACENET. Though it was subjected to exactly the same experimental design with the same learning database, this network failed to show any categorization and fan-effect phenomena during recognition. This result highlights the importance of the system architecture in the simulation of some of the well-known effects in cognitive psychology. Tests of FACENET dynamics

The last point we want to underline concerns the system's dynamic capabilities introduced by the reinsertion feature. FACENET was given the dynamic ability to evolve in the identity search by allowing the output of one processing stage to serve as a basis for the subsequent processing level. The identity search is thus guided by the networks o w n context, that is to say, by its own functioning across iterations, the aim of the search in memory being to result in a stable state that is a function of the original input and of the memory structure. The Face-Context association part of the system, which is the core of the identification process, guides this evolution owing to its influence on the reinserted lateral echoes. From a psychological point of view, this dynamic property is particularly interesting in accounting for the robustness of recognition memory. Thus, FACENET can progressively retrieve all the information from distorted, noise-ridden, or partial input. For example, it can make cued recall and retrieve a face from a single context presented as input. Figure 10 illustrates the network's ability to retrieve the identity and evoke the features of a face when only the contextual cues are presented. This test could be a match for the usual question: "Who's the person I usually encounter in these specific circumstances?" In order to study this kind of situation, we presented a specific context and initialized the face input at zero. In this case, during reinsertion, the network is able to gradually recall all the features of the stored face while eliminating the irrelevant features that emerged at the beginning. We might note that the identity strength of the evoked face increases only when the face is almost complete. This result is related to the asymmetry introduced between layers 2 and 3 in order to model the focal status of faces in

Context effects in face recognition

261

the person identification process. It is not surprising, in this case, that the context in itself cannot directly induce an identity. It is nevertheless very interesting to observe t h a t during the reinsertions, the network can progressively recall a face that had been previously encoded with the available contextual cues. ACTIVATION IN LAYER 4

1

0.5

0

-0.5

-1 3 4 5 6 7 8 9 10 ITERATIONS OF THE NETWORK : F e a t u r e s o f t h e f a c e encoded with t h e p r e s e n t e d c o n t e x t ==A== : I d e n t i t y o f t h e f a c e encoded w i t h t h e p r e s e n t e d c o n t e x t ............ : O t h e r f e a t u r e s a c t i v a t e d on F a c e E c h o 1

2

Figure 10. Evolution of identity strength and activation on FaceEcho when a specific context is presented alone, without facial cues. By means of the reinsertion feature, FACENET can thus successfully perform a cued recall task, from a contextual cue as well as from a facial cue. However, in the latter case there is no ecological equivalence because faces cannot be perceived out of context. Thus, the recognition processes never takes effect with a context initialized at zero. The most pertinent characteristics of the network will thus be its capacity to use available contextual information to produce a n identification while modifying this information. The dynamic process of the inter-contextual search is assumed to mediate recognition in many recognition models (Mandler, 1980; Tiberghien, Cauzinille, & Mathieu, 1979). These models hypothesize that among the processes underlying recognition, one mechanism is the search for the encoding context

2 68

A.C. S c h r e i b e r , S . R o u s s e t , G. Tiberghien

on the basis of the recognition context. Moreover, the degree of compatibility between the encoding and recognition contexts is thought to play an important role in the efficiency of the memory search. We tested this dynamic process by presenting learned faces in irrelevant contexts, with inter-contextual compatibility being ensured by a resemblance on a few blocks. Figure 11 illustrates the progressive and automatic shift from the context input to the learned context, recorded on the Context-Echo during the search in memory. Further analysis showed that the higher the intercontextual compatibility, the less the network needs reinsertions to achieve a successful search. ACTIVATION IN LAYER 4 1

0.5

0

-0.5

-1 1 2

3 4

5 6

7

8 9 101112131415161718192021

ITERATIONS OF THE NETWORK

- A - Identity of the presented face The concurrent context features in a same block, on Context-Echo, are represented by the same type of line. Figure 11. Dynamics of the inter-contextual search: Evolution of identity strength and activation on Context-Echo, when the compatibility between encoding and recognition contexts is ensured by two of their blocks.

Last but not least, one interesting behavior of the network is its ability to make errors. Errors, which characterize all human activities, also constitute striking evidence of the processes underlying performance. Concerning face recognition, a characteristic of the human system is to produce false alarms, which

Context effects in face recognition

269

consist of wrongly identifying an unknown person. Two factors are usually put forward to explain the determination of false recognitions: target-distractor similarity and the role of the recognition context (expectancies induced by the context, which skew the identification). FACENET was presented with learned contexts and new faces resembling the learned ones. It was not surprising to find that FACENET succeeded in making such misidentifications, thanks to its dynamic evolution capabilities and its distributed representations. However, a less trivial result arose: the role of the encoding context in the determination of false identifications. Indeed, as already discussed in the study of the non-specificity factor, the recognition of faces encoded in non-variable and non-specific contexts needs within-category differentiation because of the faneffect phenomenon. If the presented face has not been sufficiently learned, the network can diverge towards the identity of a better learned face belonging to the same category. This kind of recognition error, induced by the very structuring of memory representations during the encoding stage, quickly pleads in favour of experimental investigations on real subjects in the field of face recognition. GENERAL CONCLUSION

The existence of context effects in memory recognition in general, and in face recognition in particular, is now wellestablished experimentally. These effects can be interpreted in three different ways: as a response bias, as a priming effect, and as evidence in favour of the encoding specificity principle. The interpretation in terms of response bias has little to say about the specific functioning of memory since it is confined to the decision mechanism. Besides, this interpretation has been dismissed today as the sole explanation of context effects. Contextual effects could be also seen as resulting from the priming of FRUs via Person Identity Nodes (Bruce & Young, 1986). But these authors themselves note that this explanation is not satisfactory for the effects recorded in episodic memory for unfamiliar faces. They conclude that the most common explanation offered in the literature is Tulving's encoding specificity theory. The third solution thus consists of accepting the encoding specificity principle, i.e. accepting that the context prevailing at presentation

270

A.C.

Schreiber, S . Rousset, G . Tiberghien

time affects the way in which a stimulus is encoded or represented internally, and that the retrieval conditions must reactivate the old encoding in order for recognition memory to be effective. Thus, in this theoretical framework, the aim of research should now be to give an account of the dynamics of these context effects and of their role in the general functioning of human memory. Starting from some of the functions of Bruce and Young's model, the aim of our simulation was thus to specify the functioning of the "boxes and arrows" by applying the encoding specificity principle. Moreover, the simulation of the functioning of memory in a connectionist network puts us necessarily below the response bias level, making the recorded effects merely memory effects. As a whole, our results show that in old re-paired context conditions (crossover recognition test), identification performance is impaired. The classical context effect is thus achieved by the system. But it should be noted that this effect varies according to the encoding conditions of the faces: only the recognition of faces encoded in non-variable contexts seems affected. These results better our understanding of why experimentally pre-familiarized faces remain very sensitive to contextual change: not just learning frequency but also learning conditions may determine the range of context effects. Through the control of contextual history during the learning stage, our simulation enabled us to study the intrinsic characteristics of contextual memory. The quantitative and qualitative analyses of the results of this simulation, as well as the cues given by the study of the internal representations, show that the variability of encoding context enables the generalization of an abstract representation from various Face-Context examples. From a psychological point of view, we can assert that an identity representation has been "semantized" from contextualized episodes, and that this "semantization" is possible owing to the variability of encoding context. This result is theoretically important since it suggests that the contextual history of memory in itself might account for the transformation of episodic representations into semantic representations, without presupposing several different memory systems (Tulving, 1983). So, as McClelland (1988) and O'Toole et al. (1988) have already emphasized, the connectionist model provides a very simple framework for explaining apparently complex, disparate or incompatible phenomena. In the present modelling, the crucial issue of the distinction between episodic and semantic memory can

Context effects in face recognition

271

be approached, thanks to the inherent characteristics of the connectionist system, which are: distributed representations, the possibility of dynamic Face-Context interaction (for the creation of contextualized episodic representations), and the existence of several processing levels (which allows for generalization). But the most important point which enables us to study the hypothetical "semantization" process is that connectionism attempts to model learning procedures. In fact, since learning is actually carried out in a network, we are able to study how identity representations are constructed in different encoding conditions. The simulation of a recognition model must indeed take into account the way information is learned. The second important result of this simulation is related to the specificity factor. The learning of faces in a non-specific context induces the creation of categories founded on contextual information. These categories are revealed at recognition time by a fan-effect phenomenon, which has already been put forward to explain the organization of information in human memory (Anderson, 1983). Thus, it is worth remembering that by simulating the encoding specificity principle on a connectionist network, the contextual history of memory in itself can account for the transformation of memory traces ("semantization") and their organization (categorization). As we have pointed out, the fan-effect is no longer perceptible in a network without layer 3. This finding shows how important the system architecture is, since the structurating of information in the network depends on the available processing levels. The FACENET system, whose architecture is derived from a cognitive model drawing together experimental and clinical data, can simulate two psychological behaviors: "semantization" and fan-effects. The study of the network's dynamics has shown that the reinsertion procedure during recognition enables the network to simulate an efficient search in memory. FACENET can perform inter-contextual searching, and if the number of iterations can be related to the latency of identification, this kind of gradual retrieval of the encoding context might correspond to the slow process of conditional search hypothesized in recognition theories (Mandler, 1980; Tiberghien et al., 1979). Moreover, the connectionist framework has enabled us to model, in an interactive, parallel and unified way, several types of processing which were assumed to be sequential in the cognitive model. For instance, FACENET never

212

A.C.

S c h r e i b e r , S . Rousset, G . Tiberghien

retrieved the identity before the correct Face-Echo. This result is compatible with the latencies usually reported in experimental studies in which familiarity responses are always given faster than identity responses (Young, McWeeny. Ellis, & Hay, 1986). This simulation thus shows that a hierarchy of latencies does not necessarily imply a sequential hierarchy of processing modules. The connectionist simulation of cognitive processes provides many opportunities for cognitive research in that it enables the creation of a virtual subject for virtual experiments. Since this virtual subject is completely verifiable, one can investigate the effect of very precise experimental factors. Moreover, with such an easily modifiable subject, one can eliminate some of its components, either to refine interpretation or compare its performance with brain damaged patients. Of course, this damaged network will only correspond to a loss of psychological functions and not directly to a specific brain lesion. The use of the connectionist model to devise the FACENECT system can stimulate research at several levels in cognitive psychology. First, it is a call for new experimental research on real subjects. Second, some extensions of the system are in order for integrating other functions such as perceptual encoding (study of face memory representations constructed from digitized pictures). access to name information, and the processing of expressions. Finally, the simulation is a useful and necessary addition to the standard empirical investigation for modelling the cognitive processes underlying face recognition. What should a simulation be if not a model in action? REFERENCES Anderson, J.R.(1983).The architecture of cognition. Cambridge, MA: Harvard University Press, Baddeley, A.D.. & Woodhead, M. (1982).Depth of processing, contexts and face recognition. p g y, 148-164. Bruce, V.. & Young, A. (1986).Understanding face recognition,folanruoJPsvcholo~!, 305-327. Davies. G.M. (1988).Faces and Places: Laboratory research on context and face recognition. In G. Davies & D.M. Thomson (Eds.), p n * n t jn Memorv. London: Wiley. Davies. G.M.. & Milne. A. (1982).Recognizing faces in and out of context. Current h c Pr ,2. 235-246. Davies. G.M.. & Thomson. D.M. (1988).M q q . London: Wiley Ellis, H.D. (1986).Processes underlying face recognition. In R Bruyer (Ed.). The neuromvcho1om of face DerceDtion and facial exmession. Hillsdale. N J : Lawrence Erlbaum Associates.

s,

n.

Context effects in face recognition

213

Ellis, A.W.. Young, A.W.. Flude. B.M.. & Hay D.C. (1987). Repetition priming on face recognition. Quarterlv Journal of Expedmental Psvcholo&, 193-203. Grossberg. S. (1988).Neural networks and natural intellieencc. S . Grossberg (Ed.). Cambridge, MA: Bradford Books. Hay, D. C.. &Young, A.W.(1982).The human face. In A.W. Ellis (Ed.). Normalitv and patholo- in cognitive functions. London: Academic Press. Hintzman. D.L.. & Stern, L.D. (1978). Contextual variability and memory for frequency. Journal of Emerimental Psvcho loev: Human Learnina and Memom, 4. 539-549. Hofstadter. D. (1985). Meta-1 i Themas: Questim for the Essence of Mind and Pattern. New York Basic Books. Jordan, M.I. (1986). Serial Order: A Parallel Distributed Processing ADDroach. ICS Report 8604. Kohonen. T. (1984). Self-oreanisation and associative memory. Berlin: SpringerVerlag. Maar. D. (1982).Vision. San Francisco: Freeman. Mandler, G. (1980) Recognizing: The judgement of previous occurrence. Psvchological Rewfw. 87.252-271. McClelland. J.L. (1988). Connectionist Models and Psychological Evidence. Journal gf Memorv and Lamuaeg, 22, 107-123. McClelland. J.L.. Rumelhart. D.E., and the PDP Research Group (1986). Parallel Distributed Processin g Explor ations in the microstruct U re of cagnition (Vol 11). Cambridge. MA: Bradford Books. Memon. A.. & Bruce, V. (1985). Context Effects in Episodic Studies of Verbal and Facial Memory. Current Psvchologlcal Research and Reviews. Winter, 349369. PCris, J.L. (1986). Reconnaissance et me.connaissance. Doctoral dissertation, University of Grenoble. Rumelhart. D.E.. McClelland, J.L., and the PDP Research Group (1986). Parallel Distributed Processing: ExDlorations in the microstructure of cognition (Vol I). Cambridge. MA: Bradford Books. O'Toole. A . Mfflward. R.B.. & Anderson. J.A.(1988).A physical system approach to recognition memory for spatially transformed faces. Neural Networks. 1. 179-199. Sayre, K.M. (1986). Intentionality and Information processing: An alternative model for cognitive science. Behavioral and Brain Sciences, 9. 121-166. Seamon, J.G. (1982). Dynamic facial recognition: Examination of a natural phenomenon. J ournal of Psvch, 363-381. Searle. J.R (1985).pu cerveau au savok. Hermann. Paris. Shepherd, J.W., Davies. G.M. & Ellis, H.D. (1981). Studies of cue saliency. In G. Davies. H.D. Ellis & J. Shepherd (Eds.), Perceiving and Remembering Faces. London: Academic Press. Smith, S.M. (1982). Enhancement of recall using multiple environmental contexts during learning. Memorv & Coanition. J&. 405-412. Thomson, D.M., Robertson, S.L.. & Vogt, R. (1982). Person recognition: the effect of context. Human Learning, 1,137-154. Tiberghien. G. (1986). Contextual effects in face recognition: some theoretical problems. In H.D. Ellis, M.A. Jeeves. F. Newcombe & A.W. Young (Eds.). )\sDects of face Drocessing. Dordrecht: Martinus Nijhoff. Tiberghien, G. (1988). Comment: What is face semantic, What is face processing? In W. Young & H.D. Ellis (Eds.). J-Iandbook of Resea rch in Face Processing. Amsterdam: North-Holland Press. Tiberghien. G., Cauzinille. E.. & Mathieu, J. (1979). Pre-decision and conditionnal search in long-term recognition memory. Acta Psycholoeica, 43.329-343. Tulving. E.. & Thomson, D.M. (1973).Encoding specificity and retrieval processes in episodic memory. psvcholoeical Review, 352-373.

a

s,

a.

274

A.C.

S c h r e i b e r , S. R o u s s e t , G . T i b e r g h i e n

Tulving. E. (1983).Elements of a o d ic memory . New York Oxford University Press. Wang, S.. & Besson. T. (1989). Attraction Effects of b e r e d Ncual Networks trained .RR-IMAG/INPG. Grenoble. The effects of priming on picture recognition. British Warren, C., & Morton, J. (1982). Journal of Psvchology, B.117-129. Young, A.W.. Hay, D.C., & Ellis. A.W. (1985). The face that launched a thousand slips: Everyday difficulties and errors in recognizing people. British J o u r nal of ,495-523. Young, A.W., McWeeny. K.H.,Hay, D.C. & Ellis, A.W. (1986). Access to identity-specific semantic codes from familiar faces. Quarterlv Journal of ExDerimental PsVChOlO~.38A,271-295. Young, A.W., McWeeny. K.H., Ellis, A.W, & Hay, D.C. (1986).Naming and categorisation latencies for faces and written names. Quarterlv Journa 1 of 297-318. -tal Psvc-, Young, A.W.. & Ellis. A.W. (1988).Semantic Processing. In A.W. Young & H.D. Ellis (Eds.). Handbook of Research on Face P r o c e s h g . Amsterdam: NorthHolland.

a,

a,

Acknowledgments. This simulation was conducted in collaboration with Shengrui Wang, Laboratoire TIM3 - IMAG/INPG. Grenoble. We are grateful to Tim Valentine of the MRC Cambridge (U.K.) for his helpful suggestions. Thanks are also due to Cecilia Carriere and Megan Gaiser for their efficient translating relief.

Cognitive B i a s e s J . -P. C a v e r n i , J . -M. Fabre and M . Gonzalez ( E d s . ) 8 Elsevier Science P u b l i s h e r s B.V. (North-Holland), 1 9 9 0

THE REIATIVE IMPORTANCE OF FACIAL EXPRESSION AND CONTEXT INFORMATION IN EMOTION ATTRIBUTIONS -- BIASES, INFLUENCE FACTORS, AND

PARADIGMS Harald G. Wallbott U n i v e r s i t y of G i e s s e n , West Germany

Abstract.

One important aspect of human cognitive processing is the understanding of other persons' emotional states. It is well known that facial expression is a central cue to such inferences about emotion. But the relative importance of facial expression (person information) and situational cues (context information) in determining emotion attributions is still a matter of debate. While there is some evidence that facial expression is the predominant source of information in inferring emotions, some studies suggest that context information may be at least a s important. Within a framework of the theoretical classification of different types of "context". a number of studies which have addressed this issue will be reported, specifically: a study employing the "person-scenario'' paradigm (photographs of facial expression combined with verbal descriptions of emotioneliciting situations). a study using the "candid picture" paradigm (photographs of actual events, conveying both information on the facial expression of the central person and situational information). a study employing the "collage" paradigm (photographs with persons exhibiting facial expressions are edited into different photographs depicting a situation), and finally a study using the "filmclip" paradigm (film clips taken from movies, depicting a situation in the first take and an actor with a facial expression reacting to that situation in the second take). Results indicate that in general, facial expression does indeed dominate context information in determining emotion attributions, but furthermore that the relative importance of both sources of information also depends on the following factors: the medium of presentation (verbal, photograph, video), t h e relative consonance/discrepancy of stimulus combinations, and the gender of the person depicted. These results indicate a number of fundamental biases in the processing of information related to emotion attributions.

1. THEORETICAL CONSIDERATIONS

One important aspect of human cognitive processing is the understanding of other persons' emotional states. Here we will attempt to study some biases observers/judges are subjected to when inferring emotional states. Let us start with some anecdotal evidence: A number of movie directors have claimed that viewers' recognition of actors' emotions is guided more by contextual information in a movie than by the actual facial expression of the actors. This phenomenon is referred to in the cinematographic

276

H. G. W a l l b o t t

literature as the "Kuleschov" effect: At the beginning of this century, Kuleschov produced three short films with two takes each. While the first take of each film was different (either a dish of soup, a child playing, or a dead woman), the second take was identical (a male actor with a neutral facial expression in close-up). Observers instructed to infer the actor's emotion judged it to be either thoughtfulness, happiness/joy, or grief/sadness, depending on the content of the first take, that is, on context information. Of course within this paradigm, the intensity and clarity of the different sources of information (context and facial expression) are by no means equal (Ekman, Friesen, & Ellsworth, 1982), so that it does not seem too surprising that the emotional information conveyed via context is "projected" into the neutral expression. Consequently, a number of studies have demonstrated that context information is by no means the most important source of information. In the only review so far concerning this topic, Ekman, Friesen, and Ellsworth (1982) concluded that facial expression usually is the predominant source of information in inferring emotion, while context information only becomes important when the facial expression is less clear, less ambiguous, or less intense as to the emotions presented. On the other hand, this review failed to systematically compare different possibilities of presenting both person information and context information (see Wallbott, 1986; 1988a). Facial expression may be presented via still photographs combined with the verbal description of an emotioneliciting situation, a paradigm introduced by Goodenough and Tinker (1931). which we shall call the "person-scenario" paradigm. Both facial expression and situation information may be presented via still photographs, a technique introduced by Munn (1940) and called the "candid picture" paradigm. Finally, both sources of information may be presented dynamically using film or video, with takes presenting situation information followed by or following takes presenting facial expression (a technique similar to that used in the Kuleschov study mentioned above, or in studies, for instance, like Goldberg's, 1951). a paradigm that will be called the "film clip" paradigm. Thus, the presentation of information from both sources may vary according to the medium of presentation (verbal, still photographs, or film/video), and can be done either simultaneously or consecutively.

Facial expression

&

context in emotion attribution

277

Here an attempt is made to compare these different paradigms as to the relative importance of facial expression and context information in determining emotion attributions, while taking into account some of the methodological criticism to which earlier studies in this area have been subjected (Ekman, Friesen, & Ellsworth, 1982). We shall attempt to show that the relative importance of both sources of information cannot be determined without accounting for the medium of presentation used (and also not without accounting for some other observer biases). 2. SOME EMPIRICAL RESULTS

One central aim in all of the studies to be briefly presented here was to employ a s many stimulus combinations (in the person and the context domain) as possible, t o use as many different encoders as possible (not j u s t one or two as in some previous studies), to use a variety of emotions encoded in the situation and facial expression, and finally, to include both consonant and discrepant combinations of person and context information. After the pre-selection of stimuli within the person and context domains, the strategy in all the studies was to gather separate emotion judgments for person information presented alone and for context information presented alone. As a second step, another group of judges was confronted with person/context combinations, which were constructed or selected as being either consonant, ambiguous, or discrepant (determined by t h e correlations between judgments of person information alone and context information alone). The judgments (person alone, context alone, and person/context combined) then were analyzed with respect to three major questions: At first, we tried to determine the relative impact of the two components of information on combination judgments by computing correlations between person alone and combination judgments as well as between context alone and combination judgments, by computing the relative beta-weights of both components in determining combination judgments, and by computing the relative shift measure (Frijda, 1969). which indicates the relative dominance of one source of information over the other. Secondly, we studied the efficiency of different models of information integration as known from social psychology, like an averaging model, a summation model, or a regression model in order to determine how the partial components of information are integrated into the combination judgments. Thirdly, a t least in

218

H.G. W a l l b o t t

some of the studies conducted we tried to study judges' strategies of information integration, especially to find out how judges coped with discrepant combinations and how they attempted to integrate the discrepant pieces of information. Here, we will focus mainly on the relative importance of person information and context information in determining emotion judgments. In the first study (Wallbott, 1986: 1988a). using the personscenario approach, we employed slides from Ekman and Friesen's (1976) slide series with facial expressions as person information and verbal descriptions of emotion-eliciting situations collected in a large-scale cross-cultural study (see Scherer, Wallbott. & Summerfield, 1986) as context information. Thirty-three combinations were presented to judges, one third of which were consonant, one third of which were ambiguous, and one third of which were discrepant. The results indicated indeed that facial expression information dominated combination judgments. This dominance was especially pronounced for discrepant combinations: When confronted with descriptions of emotion-eliciting situations that did not fit the facial expression of an actor, judges tended to rely much more on facial expression information than on context information. In consonant combinations. one the other hand, context information was at least as important as facial expression information in determining combination judgments. These results were replicated to a considerable degree in a cross-cultural study (Fernandez-Dols, Mallo & Wallbott, in press). Thus, the relative importance of both sources of information depends to a large degree on the relative consonance/discrepancy of cue combinations. In the second study (Wallbott, 1986; 1988a) based on the candid picture paradigm, we used 24 photographs taken from newspapers and magazines depicting a person showing some emotional facial expression in a n emotion-eliciting situation. The results of this study confirm the results of the previous one to a large degree. In general, and once again, facial expression dominated context information, although in this study context information was nearly as important as facial expression information. Again, facial expression was most important in discrepant combinations. When confronted with a facial expression that did not fit the situation, judges were biased, relying more on facial expression than on context information.

Facial expression

&

context in emotion attribution

279

In the third study (Wallbott, 1988c) using the film clip paradigm, we selected 60 clips from movies, each consisting of two takes. The first take depicted a situation that allowed for inferences on the emotional state of an actor or actress who was either watching the situation or involved in it. The second take always depicted this actor or actress in close-up, exhibiting some emotional facial expression. Thus, contrary to the other two studies mentioned, person and context information here were not Furthermore, presented simultaneously, but successively. information was not presented statically via photographs or verbal descriptions, but dynamically. The results obtained were indeed different from those in the first two studies. Here, context information generally determined combination judgments to a larger degree than person information did, and this was especially true for discrepant combinations. In contrast to the other two studies mentioned, context information therefore becomes especially important when a situation does not fit the facial expression of an actor. This finding may be due to a logical primacy effect (Truax, 1984) because in reality also, a situation precedes a n emotional reaction, a fact which might bias observers to pay more attention to the situation as the cause of the expression. When the facial expression does not fit the emotion inferred from the situation, especially in discrepant take combinations, observers might consider that display rules interfere with the expression (and in fact observers do think so, as we know from the analysis of information integration strategies not discussed here; see Wallbott, 1986; 1988a). Thus, they might consider facial expressions in such combinations to be unauthentic and thus the preceding context information to be a more important or valid source of information. This display-rule explanation (which is corroborated by the results of additional studies with actors and actresses conducted by Wallbott, 1988b; 1988d) may also hold for another unexpected effect found: When observers are confronted with male actors, they consider context information to be more important and neglect the facial expression of these actors, whereas when they are confronted with female actors, they pay more attention to facial expression, thereby neglecting context information (see Table 1). Again, observers might be biased by the hypothesis that female actors use display rules less, thus expressing their emotions more freely and rendering facial expression a more valid cue to emotion, while male

280

H.G. Wallbott

actors u s e display rules more, invalidating facial expression information as a cue to emotions in a given situation. Table 1. Differences due to gender of actor (from Wallbott. 1988~). Mean female male actors actors V a r ia b l e

N

=

29

31

t

df

P .051

Correlation person/entire c l i p

-77

.60

1.99

58

Correlation context/entire c l i p

.64

.72

.91

58

Relative s h i f t measure

.84

1.50

2.71

58

.009

Beta weight person

.63

.24

2.89

58

.006

Beta weight context

.30

.69

2.85

58

.006

n.s.

3. CONCLUSIONS

Comparing the three studies mentioned, we find that the relative dominance of person information and context information is different within the three paradigms. In the person/scenario approach, person information clearly dominates context information. In the candid picture paradigm, both sources of information are of about equal importance in determining emotion attributions. And in the film clip paradigm context information generally is more important than person information (see Table 2). Thus, we have identified at least three factors that bias the processing of both sources of information and their relative predominance in judging other persons' emotional states: - the medium of presentation used, which also implies differences between successive and simultaneous presentation: - the fact that cue combinations are consonant or discrepant:

Facial expression

&

281

context in emotion attribution

- the gender of the encoder or actor (in the first two studies similar gender differences were found, although they were not statistically significant). Table 2. Comparison of the results reported in Wallbott (1988a; 1988~). Personscenario study (Wallbott, 1988a)

Candid picture study

Film clip study

(Wallbott, 1 9 8 8 ~ )

Correlation person/combination

.84

.70

.68

Correlation context/combination

.45

.52

.68

.73

.94

1.18

61

.42

.38

.36

.51

33

24

60

Relative shift measure Beta weight person Beta weight context

N stimulus combinations

Person information (facial expression) is a more important cue in emotion attribution especially in the person/scenario paradigm, especially in discrepant combinations, and especially for female actors. Context information, on the other hand, is especially important t o judges in the film clip paradigm, in discrepant combinations, and for male actors. Thus, we may conclude that neither facial expression nor context information dominates emotion attributions on a general basis, as some studies in the past seem to imply. On the contrary, this relative dominance depends on a number of observer biases, some of which have been identified in the studies presented. Furthermore, we also found that judges almost never totally neglect one source of information, but only

282

H.G. Wallbott

place different emphasis on the two components depending on the factors mentioned. We shall give a final illustrative example as to how context information may change emotion attributions, although s u c h changes sometimes do not involve total changes in meaning, but slight shifts in judgments attributable to context information. In a further study we used collages of persons with emotional facial expressions inserted into two different photographs, each presenting context information (Wallbott, in press) to find out whether the emotional meaning of identical facial expressions would shift, depending on different visual context information. A n example of such a pair of collages might be the following: In the first picture a"photograph of a woman with a facial expression judged as being slightly happy is combined with the photograph of a little child. In the second picture, the same photograph of the female is combined with the photograph of another female. Judgments of these two collages indicate that the first picture results in much more positive emotion attributions, while the second picture yields judgments of at least some degree of negative emotions in the facial expression of the female. This brings us back to our initial remarks on Kuleschov and other movie directors: Although context information does not totally dominate facial expression information, it may slightly modify the meaning attributed to the facial expressions. This in turn implies that by providing different types of context information, judgments on facial expression may be "manipulated". Observers of emotional expressions and emotion-eliciting situations may be biased by a number of factors, such as whether they are confronted with verbal descriptions, photographs or film/video presentations, with female or male actors, or with consonant or discrepant information components. But observers/judges do not seem to focus on just one source of information: instead they nearly always attempt to use all information available, even if it is difficult t o integrate these components, and no matter how biased they may be in giving different weights to the different components of the available information.

REFERENCES Ekman. P.. & Friesen. W.V. (1976).Pictures of facial affect. Palo Alto, CA: Consulting Psychologists' Press, (Slide series).

F a c i a l expression

&

283

context in emotion attribution

E h a n . P., Friesen. W.V.. & Ellsworth, P. (1982).What are the relative contributions of facial behavior and contextual information to the judgment of emotion? in: P. E h a n (Ed.), Emotion in the human fa=. 2nd ed. pp. 111-127. New York Cambridge University Press. Femandez-Dols. J.M.. Mallo. M.J.. & Wallbott, H.G. (in press). Judgment of emotion from expression and context: a replication. Revista de Psicoloeia Social. Frijda. N.H. (1969). Recognition of emotion. In: L. Berkowitz (Ed.), Advances in g!x~erimental social Dsvcholof?y. v01.4. pp. 167-224. New York: Academic Press. Goldberg, H.D. (1951). The role of 'cutting' in the perception of motion picture. 35,70-71. Journal of A ~ ~ l l Psvcholo-, ed Goodenough. F.L., & Tinker, M.A. (1931). The relative potency of facial expression anaverbal description of stimulus in the judgment of emotion. ComDarativG PsvcholoPy. 12.365-370. Munn. N.L. (1940).The effect of knowledge of the situation upon of emotion -judgment from facial expression. Journal Abnormal and Social PsvcholQgy. s,324338. Scherer, K.R.. Wallbott. H.G., & Summerfield, A.B. (Eds.) (1986). ExDeriencing emotion: A cross-cultural study. Cambridge: Cambridge University Press. Truax. S.R. f19841. Determinants of emotion attributions: A unifvine view. 'Motivation and Emotion, 8, 33-54. Wallbott, H.G. (1986).Person und Kontext: Zur relativen Bedeutung von mimischem Verhalten und Situationsinformationen im Erkennen von Emotionen. Archiv ffir Psvcholoeie. 138.2 11-231. Wallbott. H.G. (1988a). Faces in context: The relative importance of facial expression and context information in determining emotion attributions. In: K.R. Scherer (Ed.),Facets of emotion. Hillsdale: Lawrence Erlbaum. Wallbott, H.G. (1988b). "Aus dem Zusammenhang gerissen" -Schauspielermimik ohne Kontextinformation. Zeitschrift ffir ExDerimentelle und Angewandte Psvchologie, z,496-510. Wallbott, H.G. (1988~). In and out of context - Influences of facial expression and context information on emotion attributions. riti i l Psvcholo@, 27.357-369. Wallbott. H.G. (1988d). Big girls don't frown. big boys don't cry - Gender differences of professional actors in communicating emotion via facial expression. Journal of Nonverbal Behavior. 12. 98-106. Wallbott, H.G. (in press). Mimlk im Kontext: Die Bedeutung verschiedener InformationskomDonenten be im Erkennen von Emotionen. Gbttingen: Hogrefe.

a

"

Y

This Page Intentionally Left Blank

Coqnitive B i a s e s J . - P . C a v e r n i , J . -M. F a b r e a n d M . G o n z a l e z ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B . V. ( N o r t h - H o l l a n d ) ,

1990

METHODS FOR DETERMINING THE LOCUS OF CONTEXT EFFECTS IN JUDGMENT Douglas H. Wedell University of Illinois, Champaign, Illinois, USA

Abstract.

Contextual effects are proposed to occur at any of five stages of processing: sensory encoding, lexical encoding, propositional/categorical encoding, dimensional analysis, and response translation. Methods for locating the effects of contextual stimuli on judgments within this five stage model are described. Response-oriented methods, which center on the type of responses made to contextual and target stimuli, appear best suited to determine whether these effects are located at the response translation stage. Process-oriented approaches, whkh either alter the sequencing of processes or introduce new judgmental operations, provide a finer grain analysis. Although more research is needed, the reviewed evidence indicates that contextual contrast for psychophysical judgment occurs during dimensional analysis and effects of priming on social judgments can occur during propositional/categorical encoding.

Context effects in judgment come in only two varieties, assimilation and contrast. This is because judgments are rendered on bidirectional scales so that the value of a target can only be displaced either towards values of contextual stimuli (an assimilation effect) or away from the values of contextual stimuli (a contrast effect). Although it would be convenient to assume that each of these two effects results from a single process, this assumption seems unreasonable on a priori grounds and h a s proven to be untenable on the basis of evidence from empirical studies. Thus, when a context effect is uncovered, the researcher is faced with the difficult problem of determining its underlying cause. The problem of locating the context effect within an information processing system is an important one because effects based on different processes have different implications concerning their generality and impact on other related judgments or choices, e.g., preference judgments. This article presents a review of different methods used for locating context effects in judgment. I begin by presenting a general process model in which context effects may be located a t one of five stages. I then review traditional response-oriented approaches and afterwards review more modern process-oriented

286

D.H. W e d e l l

approaches for locating context effects. Although no single method can provide unequivocal evidence for the location of context effects, converging evidence from several different methods may do so. A PROCESS MODEL FOR CONTEXT EFFECTS IN JUDGMENT

Processing of stimulus information is assumed to proceed sequentially through five stages: SENSORY ENCODING

+

LEXICAL -+ ENCODING

PROPOSITIONAL/ CATEGORICAL

+

DIMENSIONAL ANALYSIS

+

RESPONSE TRANS LATI ON

ENCODING

At each stage relevant contextual information is input from memory, with the output of the process becoming part of the stimulus representation upon which subsequent processes operate. The three initial stages encode the stimulus into its core representation. These encoding processes occur automatically and are not directly tied to the judgment process. The latter two stages represent the process of forming a judgment and translating it into a n appropriate response. These processes may be automatic or controlled, and their output does not directly alter the encoded core stimulus representation. A full exposition of this model is beyond the scope of this article (for a more detailed account, see Wedell, 1989a). Instead, I briefly overview each of the five stages and provide illustrative examples. I t is important to keep in mind that processing occurs sequentially so t h a t any context effect occurring early in the sequence will influence processing a t later stages. Methods for locating context effects attempt to draw a correspondence between the experimental procedures and a particular stage in the sequence of processing. Thus, obtaining a context effect with a given procedure implies that it occurs at the corresponding stage or earlier: whereas, elimination of a context effect implies that it occurs a t a later stage. 1. Sensory Encoding

These processes provide the initial sensory encoding of stimulus information. Context effects occurring at this stage may be thought of a s enduring perceptual illusions. A good example of

Determining the l o c u s of context effects

201

contrast occurring a t this level is given by Locke's water basin problem: Room-temperature water in one basin will feel cool to a hand that has recently been immersed in a basin of hot water, but it will feel warm to a hand that has recently been immersed in a basin of cold water. This particular effect occurs in the peripheral nervous system where sensory cells respond in a relative rather than absolute fashion. Sensory-based context effects are not limited to the peripheral nervous system, but they must occur early enough in the central nervous system so that they directly alter the sensory configuration. 2. Lexical Encoding

These processes are responsible for defining the denotative meaning of a sensory configuration. The lexical code for a given sensory representation may be thought of as a location, or node, within a n associative network, the meaning of the stimulus being represented by its connections with other nodes. If retrieval of the lexical code follows a process of spreading activation (Collins & Loftus, 1975),contextual stimuli can then influence what meaning is selected by increasing the activation of particular locations within the network. For example, the contextual set (inch, meter, yard] primes nodes related to measurement and hence increases the probability that the sensory representation "F 0 0 T'is encoded as a unit of measurement rather than a s a body part. This type of associative process may be considered to give rise to a n assimilation effect whereby the meaning of the target stimulus is displaced towards the meaning of contextual stimuli. There are no theories of contrast effects occurring at this level. 3. Propositional/Categorical Encoding

For any given denotative meaning structure that results from lexical encoding, there may be many possible associated connotative meanings. The connotative meaning is represented in the model in terms of a n elaborated set of propositions or categories that are coactivated, and hence associated, with the lexical node. For example, the same lexical node corresponding to "foot" as a body part will be retrieved regardless of whether the set of contextual stimuli consists of (massage, dance, tickle] or [locker room, fungus, itch]: however, the elaborative meaning structure will differ. When

288

D.H. Wedell

asked to judge the pleasantness of the concept "foot," subjects exposed to the first contextual set will be more likely to retrieve positively-valenced associates through connections with associated propositions than those exposed to the second set. and hence an assimilation effect should be observed. This type of associativebased assimilation effect has been studied most extensively in the social priming literature (e.g., Higgins. Rholes, & Jones, 1977). Although associative mechanisms imply assimilation, some researchers who describe "perceptual contrast" for judgments of nonsensory aspects of the stimulus appear to be arguing for a propositional/categorical based contrast effect [e.g., Manis, 1967). Martin (1986) has recently described a process model consistent with this.jew. 4. Dimensional Analysis

The judgment process may be conceived as locating the value of a stimulus along a specified dimension and communicating that location using a particular response language. Dimensional analysis refers to the first of these processes and uses as input the stimulus representation resulting from sensory, lexical, a n d propositional/categorical encoding. The output from this stage is conceived as the phenomenal impression of the stimulus along the specified dimension. Context effects occur at this level when the same encoded stimulus representation gives rise to different phenomenal impressions, depending on the set of contextual stimuli. A number of distributional theories of contextual contrast have been formulated at this stage (Helson, 1964; Parducci. 1983). These theories imply that values of contextual stimuli are retrieved at the time of judgment and that the subjective impression of the target stimulus is made relative to these values. Hence, $100 may seem like a lot of money when the contextual set consists of mostly smaller amounts [$lo, $15, $45, $65, $llO]. but it may seem like a small amount when the contextual set consists of mostly larger amounts ($65. $110, $185, $190, $2101, even though the encoded representation of the stimulus may not differ under these two conditions. Because the output of the dimensional process becomes part of the stimulus representation, albeit a peripheral element, dimensional analysis may be thought of as a specialized form of propositional encoding, in which the proposition/category consists of the impression of the stimulus along the designated dimension.

Determining the locus of context effects

289

Assimilation effects a t this stage have sometimes been represented in terms of an averaging process (Anderson, 1981. pp. 230-248). 5. Response Translation The second phase of judgment may be conceived as translating a subjective impression into the appropriate response language. Context effects occur at this stage when the same dimensional value gives rise to different responses, depending upon the set of contextual stimuli. One justification for the use of different responses for the same subjective impression under different contextual sets is that such a strategy may enhance communication. For example, it may be that one's subjective impression of the height of a 5'10" basketball player is the same as one's impression of the height of a 5'10" jockey: however, one may refer to the basketball player as short and the jockey as Such a strategy may be helpful in communicating norms about the two different groups to the listener. Principles of communication such as maximizing discrimination among stimuli (Upshaw, 1969) and avoiding redundancy (Bradburn, 1982) are among the explanations given for response-based context effects.

a.

RESPONSE-ORIENTEDAPPROACHES A traditional approach for determining the nature of context effects is to gauge whether they depend on the how responses are elicited. Context effects that generalize across different response methods are thought to occur early in the processing sequence, a t sensory, lexical, or propositional encoding stages. Context effects that are specific to a particular method of responding are thought to occur later in the processing sequence at dimensional analysis or response translation stages. Two basic types of response manipulations are possible: manipulations of (1) type of response made to contextual stimuli, and (2) type of response made to the target stimuli.

Responses to Contextual Stimuli

Some formulations of contextual processing have been stated explicitly in terms of category use. For example, contrast effects can result from the tendency to use categories equally often and

290

D.H. Wedell

assimilation effects can result from a tendency to match category use to stimulus frequencies, More generally, the overt responses made to the contextual stimuli may be responsible for the context effects on the target stimuli. This hypothesis is easily tested by varying the nature of the responses made to contextual stimuli. A number of studies have investigated the effects of various methods of incidental exposure (e.g., no judgment, judgment on another dimension, etc.). Many studies have demonstrated that contrast effects do not depend on judging contextual stimuli on the same scale (e.g., Kenrick & Gutierres, 1980: Parducci, 1956: Parducci & Wedell. 1986; Smith, Diener, & Wedell, 1989 -- although see Pepitone & DiNubile, 1976 for a n exception). Similarly, assimilation effects have been repeatedly found when contextual stimuli are presented incidentally as in the priming literature (e.g., Bargh & Pietromonaco, 1982: Higgins, et al., 1977; Srull & Wyer, 1979, 1980; Wyer & Srull. 1981) or when contextual stimuli are n o t overtly judged as in the halo effect literature (Anderson. 1981, pp. 165-178). The fact that context effects do not generally depend upon overt judgments of contextual stimuli does not mean that these effects are not response based, b u t it simply eliminates certain classes of response-based theories.

Responses to Target Stimuli Response control. Responses may vary in terms of the degree of conscious control a subject can exert on the response. Most overt verbal responses are potentially under the subject's full control. On the other hand, some physiological responses may be relatively immune to conscious intervention. Thus, context effects that have no impact on phenomenal experience should not be manifest for low-control response measures. Krupat (1974) conducted a n experiment that measured apprehension using both verbal reports and physiological concomitants of apprehension. Subjects in a driver-simulation task viewed films that were either highly threatening or non-threatening before viewing a n intermediate level target film. Both verbal and physiological measures demonstrated strong contextual contrast, suggesting that the locus of the contextual effect was a t a phenomenal level, i.e., below the response-translation stage. Because physiological correlates are not well-defined for most judgment dimensions, the generality of this technique appears limited.

Determining the l o c u s of context effects

2 91

Scale definition. Krantz and Campbell (1961) argued that one could separate out response- and sensory-based components of context effects by manipulating the degree of scale lability. Their subjects made judgments of the lengths of lines using either a welldefined scale of inches or a loosely-defined 100-point rating scale. Contrast was observed for responses made on both scales, but was greater (as measured in standardized units) for the rating scale. Krantz and Campbell argued that these results suggested both a perceptual and a linguistic component to the effect. However, Anderson (1982, p. 11) h a s argued that the inference of a perceptual component to the context effect is unwarranted from this experiment. First, the magnitude of the context effect on the scale of inches was too great to be sensory-based (a difference of 4.5 inches between conditions). But more importantly, there is no force of logic to the argument that finding a significant context effect on a well-defined scale means that the effect is encodingbased. The method appears only useful in falsifjnng the hypothesis that an effect is sensory-based, that is, when no significant effect of context is found on one of the scales. Naturally, because this conclusion rests upon accepting the null hypothesis, its validity is directly tied to the power of the test to detect a significant difference. Development of a well-defined scale need not be limited to the A general objective of realm of psychophysical judgments. behavioral assessment is to anchor categories to specific descriptions or stimulus examples so that different judges use the same standards. Although anchoring techniques have been generally successful in increasing reliability (see Bernardin & Beatty, 19841, there h a s been little evidence that they reduce the magnitude of contextual manipulations. For example, context effects on judgments of psychopathology have been reported to be of equal magnitude for anchored and unanchored scales (Amhoff. 1954: Perrett, 1971), although a recent study h a s demonstrated conditions under which verbal anchors can partially reduce such context effects (Wedell, Parducci, & Lane, in press). Scale constraints. When a subject is asked to express a judgment, the experimenter typically places constraints on how responses are to be made. For example, the experimenter may prescribe a set of ordered categories for the subject to use. Some response-based theories imply that context effects result directly from these response-scale constraints (e.g.. Upshaw. 1969:

2 92

D.H. W e d e l l

Volkmann, 1951). To test this hypothesis, one may observe whether these effects occur when constraints are taken away. One way to lift constraints is to use an open-ended response format. In general, contrast effects have been demonstrated across a wide variety of response formats such as category ratings (Helson, 1964), magnitude estimations (Mellers & Birnbaum, 1982). open-ended written descriptions (Simpson & Ostrom, 1976), and crossmodality matching procedures (Manis, 1967). Some exceptions to this finding do exist. Upshaw (1978) found that when judgments were made on two related scales, contextual effects were manifest only on the scale that was phrased in the language of the contextual manipulation. This finding suggests that the context effects were response based. The magnitude of context effects has also been found to depend on how many rating categories are prescribed by the experimenter for both psychophysical and social judgments (Parducci & Wedell, 1986; Wedell & Parducci. 1988). Although dependence of the context effect upon the number of categories suggests a locus at the response-translation stage, the results are also consistent with a model proposed by the researchers that locates the context effect at the level of dimensional analysis. PROCESS-ORIENTED APPROACHES

Process-oriented approaches attempt to vary different features of the judgment task in order to determine whether a context effect occurs before or after a particular process. Traditional responseoriented approaches have been concerned with two broad classifications of effects, sensory- versus response-based. The more sophisticated, process-oriented approaches are capable of making finer distinctions among the different types of processes responsible for context effects. I distinguish two basic types of manipulations: Those that alter the sequencing of encoding and judgment processes, and those that alter the operations involved in the judgment process.

Sequencing Manipulations Srull and Wyer (1980) investigated whether the assimilation effect found in social priming studies occurs during encoding or a t a later stage of processing by manipulating the sequencing of

Determining the l o c u s of context effects

2 93

contextual and target stimuli. In their study, contextual stimuli were presented incidentally in a sentence unscrambling task. The unscrambling task occurred either before or after the presentation of the target stimulus, with the judgment of the target stimulus occurring last in both cases. Assimilation was found only when the contextual stimuli preceded the target and not when the order was reversed. This pattern of results supports an encoding interpretation of the observed assimilation effect: When the contextual stimuli preceded t h e target, they altered t h e propositional encoding of the stimulus information which was reflected in the later judgment: b u t when the target stimulus preceded the contextual stimuli, encoding could not be altered and hence no effect of context occurred. If the locus of the assimilation effect had been a t dimensional or response translation stages, it should have occurred regardless of the order of presentation. Because the force of logic behind this test appears rather strong, it is surprising that this methodology h a s not been employed in testing context effects more generally.

Modeling Different Operations Thus far, the various methods that I have discussed for testing the locus of context effects have not altered the basic form of the judgment task. For example, regardless of whether judgments of hostility are expressed on a 10-point rating scale or as an openended verbal description, the subject is still making a direct evaluation of the hostility of the target. I t is possible, however, to vary the operations involved in the judgment task so that responses are not made directly to the dimension of interest but rather to the result of some operation that may build upon the dimensional judgment. If context effects occur early in the processing sequence, then they should affect judgments based on the entire set of operations. However, if context effects occur late in the processing sequence, the additional processes may operate on a contextindependent stimulus representation and hence context effects may be absent for these elaborated judgments. I discuss three basic types of elaborated judgments, those resulting from comparative, composite, and chaining operations. ComDarative oDerations. Rather than judge a single stimulus along a dimension, one may be asked to compare impressions of two different stimuli along the same dimension. The comparative

2 94

D.H. Wedell

response may be merely directional (i.e., greater than or less than), or an indication of something akin to psychological distance (i.e,, not verv different or very different), or a combination of the two (i.e.. not very much greater than, etc.). The object of the test is to determine whether the comparisons are made using contextindependent or context-dependent values of the stimuli. If contextindependent responses are obtained, i.e., if comparisons of the same stimulus values remain constant under different contextual conditions, then there is a context-independent representation of the stimulus available and hence the context effect cannot be located at one of the encoding stages. On the other hand, finding contextual dependencies in the comparative judgments supports the notion that context effects alter the subjective impression of the stimulus and hence are not mere response translation effects. The use of comparative judgment, of course, has a long history in psychology. A number of studies have demonstrated significant context effects for pairwise comparisons of psychophysical stimuli (Erlebacher & Sekuler, 1971, Helson, Michels. & Sturgeon, 1954, Massin, 1987); however, the stimuli employed in these studies were of low discriminability and the effects were small. To investigate whether the large contrast effects generally observed for highly discriminable psychophysical stimuli would be manifest in a pairwise judgment task, Mellers and Birnbaum (1982) had subjects rate the lightness of dot patterns presented singly or the difference in the lightness of dot patterns presented in pairs. They found that despite large contextual effects on ratings of single stimuli, pairwise ratings were independent of the contextual manipulations. This finding has been replicated and extended to pairwise dissimilarity ratings of multidimensional stimuli (Corter. 1988; Jones & Wedell, 1987). These results for highly discriminable stimuli imply that the large contrast effects typically observed in psychophysical experiments do not reflect a change in the underlying sensory encoding of the stimuli. However, these results by themselves do not distinguish between effects located at dimensional analysis versus response translation stages. For example, it is plausible that the subjective impression of the lightness of the dot patterns was context dependent when judged singly; however, because the underlying sensory impressions were not altered by the contextual manipulations, judgments of the differences for pairs were based on differences in the sensory representations of the stimuli rather than differences in their values resulting from dimensional analyses.

Determining the locus of context effects

2 95

A general difficulty facing researchers who study context effects in judgment is that there are usually strong carry-over effects so that within-subject designs are often not feasible. However, manipulating the context within-subjects would be ideal in a comparative judgment task because one could directly assess the effects of different contexts on equivalent targets. A within-subject manipulation of context requires that the contexts can be isolated from one another and linked to a specific target. Such a design was illustrated earlier in the Locke water basin problem: The cold and hot contexts were isolated by being in different basins and were linked to separate but equivalent targets, the left and right hands of the subject. Thus, the demonstration of the same water temperature feeling warmer to one hand than to the other represents a strong phenomenological impression of a contextually induced difference. Manis, Nelson, and Shedler (1988) have developed a procedure that attempts to manipulate context within subjects for social judgments. In their task, verbal statements varying in their degree of psychopathology were attributed to patients from one of two hospitals. Thus, two different contexts were established, one for each of the hospitals. Subjects were then asked to make comparative judgments of the psychopathology of verbal statements, each attributed to patients from different hospitals. Their finding of strong contextual effects on the painvise judgments implies that the context affected the subjective impressions of the stimuli. Composite oDerations. In many instances we are asked to make judgments based on a set of stimuli rather than a single stimulus event. For example, a judgment of hostility may be based on a series of observed behaviors. Most relevant to the current discussion is whether context effects that occur for judgments of single stimuli will influence judgments of the composite. For example, judgments of line length show strong contextual dependencies, the same length is judged to be shorter when most lines are of longer length. Now consider the task of judging the average length of a set of several lines. One possibility is that this judgment will be based on the average of the context independent values of the lines. However, Birnbaum. Parducci, and Guifford (1971) demonstrated that the judgment of the composite set of lines was based on an integration of context-dependent stimulus values. Similar results have been demonstrated for judgments of loudness (Parducci, Thaler, & Anderson, 1968) and for judgments

296

D .H . Wedel 1

of performance based upon composite test scores (Mellers & Birnbaum, 1983). These results suggest that the within-set context affected the subjective impression of the individual line lengths. Thus, when the results from composite and comparative judgments of psychophysical stimuli are taken together, they imply that the generally observed contextual contrast effects on these stimuli are located at the dimensional analysis stage: Comparative judgments demonstrating a context-independent component to the stimulus representation, but composite judgments suggesting a context effect on the subjective impression. This model of the judgment process can be represented by locating the difference operation after encoding but prior to the context-dependent dimensional analysis, with the additive operation occurring after dimensional analysis but prior to response translation. Chaining o p e r a t i o n s . A defining feature of sequential processing models is that later operations use as their input the output from earlier operations. Thus, there is established a chain of operations, each building on the other. Comparative and composite judgments represent types of chaining operations that operate directly on the relevant dimension of judgment. However, other types of operations (e.g., those that guide behavioral tendencies, preferences, and choices) may use as input information from the contextually manipulated dimension even though they do not directly express values on that dimension. Studying the effect of contextual manipulations on these further removed operations provides insight into the locus of the context effect. In a study measuring the effect of contextual manipulations on attitudinal responses, Sherman, Ahlm, Berman, and Lynn (1978) asked subjects to rate the importance of various social issues. The rated importance of the target issue, recycling, was higher when other issues were of generally lower importance, a contrast effect. The subject was then asked (in an ostensibly unrelated setting) to designate the number of hours she was willing to volunteer for a local recycling project. This behavioral measure may be conceived as reflecting a n operation that chains onto the importance dimension, the greater the importance, t h e more h o u r s volunteered. Only when subjects were reminded of their ratings of the recycling issue did the contrasted ratings of recycling correspond to the hours of volunteering. When no such reminder was given, volunteering showed the opposite trend. These results imply that the contrast effect did not occur at the encoding stage,

Determining the l o c u s of context effects

2 91

that is, the encoded attitude was not directly changed by the contextual manipulation. Instead, Sherman et al. argue that the results support a response-based interpretation of the contrast effect, although location at the dimensional analysis stage seems equally consistent. Operations underlying preferences may also be viewed as chaining onto specific dimensions of judgment. For example, preferences for drinks that vary in their sugar concentration may be guided by a process t h a t compares each drink's perceived sweetness to a n ideal point on the sweetness dimension (Coombs, 1950). Riskey, Parducci, and Beauchamp (1979) investigated whether contextual manipulations on the sweetness dimension would affect judgments of pleasantness of the different drinks. Judgments of the sweetness of different drinks showed the usual contrast effects, the same concentration of sugar being judged as more sweet when the set of contextual stimuli were of lesser sugar concentrations. If pleasantness judgments are guided by a n operation that compares the sweetness of the target to an ideal sweetness level, the relevant question is whether the ideal point is located on a context dependent or independent scale of sweetness. Judgments of pleasantness were strongly affected by the contextual manipulation, regardless of whether subjects made sweetness ratings. Following a n ideal-point model, the pleasantness ratings were single-peaked functions, with their peaks corresponding roughly to the middle category on the 9-point sweetness scale. These results argue strongly against a response-based interpretation of the contextual effect on sweetness. Furthermore, Riskey et al. cite previous work on sweetness t h a t implies the sensory representation of the target does not change under the different contextual conditions. Thus, converging evidence implicates the dimensional analysis stage a s the locus of the contextual effect in this instance. In a recent experiment, Wedell (1989b) h a s attempted to replicate the methodology of Riskey et al. for judgments on a social dimension. Subjects either rated the neatness of hypothetical stimulus persons or rated how much they would like to have that person as a roommate. Strong contextual contrast was observed on the neatness ratings, however, preference ratings were influenced by the contextual manipulation only when directly preceded by the neatness rating. This finding, in a sense, conceptually replicates

2 98

D. H. Wedell

that of Sherman et al. (1978) and once again implies that contrast effects on social judgments are not encoding-based. Degree of preference can be expressed for single stimuli, or in a comparative judgment situation. Huber. Payne, and Pluto (1983) investigated whether choice preferences were affected by contextual manipulations on underlying dimensions. In their experiments, stimuli varied on two dimensions, e.g., beers that varied in rated quality and price. Subjects chose the most preferred beer from a set of three beers that defined the contextual set. Each set consisted of two beers whose values on each dimension differed b u t when combined resulted in roughly equivalent attractiveness, e.g., a moderate quality, low priced beer equated with a high quality, high priced beer. The third alternative was of lower attractiveness than the other two, but was constructed in such a way as to increase one of the alternative's values on a particular dimension via a contrast effect. Subjects choices were significantly affected by the contextual manipulation. Because the context was manipulated within subjects, the context effect resulted in a preference reversal: the same subject chose one beer over the other in one contextual condition, but reversed this preference order when the context was changed. These results, combined with the fact that no overt judgments of the dimensional values were made, imply that the effect was not response-based. Furthermore, it is unlikely that this effect resulted from a n encoding process because the stimuli were numerical and hence unlikely to be encoded differently. The process of cross-modality matching may be viewed as directly chaining operations on one dimension to operations on another. Manis (1967) has argued that context effects found for cross-modality procedures are indicative of "perceptual" effects because the response-language component has been eliminated. Although it is true that elimination of context effects when using a cross-modality method implies a response-based effect, observation of contextual effects in cross-modality scales does not mean that the effect is necessarily a n encoding one. For example, after establishing that difference ratings of the lightness corresponding to dots patterns were context independent, and hence not encoding-based, Mellers and Birnbaum (1982) demonstrated contextual dependence for difference ratings made across modalities (e.g., judgments of whether a circle was larger than a dot pattern was dark). This result suggests that although the difference operation for within modality comparisons occurs before

Determining the locus of context effects

299

dimensional analysis, it occurs after dimensional analysis for crossmodality judgments for which the sensory representations are not commensurable. These results are consistent with locating contextual contrast for psychophysical judgments at the dimensional analysis stage. A final example of chaining operations that I will discuss concerns the sequencing of opposing contextual processes. Wedell, Parducci, and Geiselman (1987) had subjects rate the attractiveness of faces presented in pairs. Between-pair context produced a contrast effect while within-pair context produced a n assimilation effect. Because both assimilation and contrast were observed on ratings of the same target, it is possible to determine the priority of these processes. There are three possibilities: assimilation occurred first and its output was used as input for the contrastive process, contrast occurred first and its output was used as input for the assimilative process, or both processes occurred independently and used context independent values as input. Using an averaging model of assimilation and a range-frequency model of contrast, the only sequencing that provided a n adequate fit to the data was to assume that contrast occurred first. This result implies that the observed within-pair assimilation was not an encoding effect but occurred either a t dimensional analysis of response translation. GENERAC IMPLICATIONS

Context effects are ubiquitous in psychophysical and social judgment. However, it has long been acknowledged that the same context effect can arise from different processes. Identifylng what processes determine the context effect is especially important to understanding the implications of the effect for different modes of judgment and for related tasks. Traditionally, two levels of contextual processes have been identified, perceptual and linguistic (Krantz & Campbell, 1961). The model presented here describes five identifiable levels. An impressive array of methods have been developed for locating contextual effects within such an information processing system. Although any particular method typically does not uniquely identify the type of process involved, converging evidence from different methods can do so. From the review presented here, there appears to be converging evidence t h a t contextual contrast observed in psychophysical experiments occurs primarily a t the level of

300

D.H. W e d e l l

dimensional analysis. Assimilative effects found in priming studies apparently result from encoding processes, but there have been no studies to determine the locus of the contrast effects that are sometimes reported in priming studies (Herr, 1986; Martin, 1986). There is no evidence as yet to uniquely indicate an encoding-based contrast effect in social judgment, although some perceptual illusions appear to occur at this level (for exceptions see Girgus & Coren. 1982). What is needed now is not so much the development of more methods, but more coordination of the use of methods to uncover principles defining when different types of contextual processes will occur and how they interface with other processes.

REFERENCES Anderson, N.H. (1981).Foundations of informat ion inteeration theory. New York: Academic Press. Anderson, N.H. (1982).Methods of information integration. New York Academic Press. Arnhoff. F.N. (1954).Some factors influencing the unreliability of clinical judgments. Journal of Clinical Psvchology!.JQ,272-275. Bargh, J.A.. & Pietromonaco. P., (1982).Automatic information processing and social perception: The influence of trait information presented outside of conscious awareness on impression formation. Journal of Personalitv and Social Psvcholocy, 437-449. Bernardin, H.J., & Beatty, R.W. (1984).performa nce a mraisal: Assessin@human behavior at work. Boston: Kent. Birnbaum, M.H., Parducci. A.. & Cuifford, R.K. (1971).Contextual effects in information integration. J o u m a l n t a l Psvchol .88.149-157. Bradburn. N. (1982).Question-wording effects in surveys. In% Hogarth (Ed.), Question framing and resDonse consistencv (pp. 65-76).San Francisco: Jossey-Bass. Coombs. C.H. (1950).Psychological scaling without a unit of measurement. Psvcholoeical Review, 145-158. Collins, A.. & Loftus. E. (1975).A spreading-activation theory of semantic processing. Psvcholoeical Review. 407-428. Corter, J.E. (1987).Similarity. confusability. and the density hypothesis. Journal of FxDerimental Psv . cholow: General y6, 238-249. Erlebacher, A.. & Sekuler, R (1971). Response frequency equalization: A bias model for psychophysics. PerceDtion and PsvchoDhvsics 315-320. Girgus, J.S., & Coren. S. (1982). Assimilation and contrast illusions: Differences in plasticity. PerceDtion & PsvchoDhvsics. 32,555-561. Helson. H. (1964). AdaDtation-leveltheorv. New York Harper & Row. The use of comparative rating scales Helson, H.. Michels, W.C.. & Sturgeon, A. (1954). for the evaluation of psychophysical data. American Journa 1 of Psvchology. 321-326. Herr, P.M. (1986).Consequences of priming: Judgment and behavior. Journal of Personalitv and Social Psvcholo~w. 1 106-1 1 15. Higgins. E.T., Rholes. W.S., & Jones, C.R. (1977).Category accessibility and 141impression formation. Journal of ExDerlmental Social [email protected], 154.

a,

a,

a,

.

,a,

a,

a,

Determining the locus of context effects

301

Huber. J.. Payne, J.W.. & Pluto, C. (1983).Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal pf Consumer Research. 9.90-98. Jones, L.E., & Wedell, D.H. (1987).Contextual effects in multidimensional scaling; Variations in stimulus densities. Paper presented a t the fifth annual European meeting of the psychometric society, Enchede. Netherlands. Kenrlck. D.T.. & Gutierres, S.E. (1980).Contrast effects and judgments of physical attractiveness: When beauty becomes a social problem. Journal of personalitv and Social Psvcholorry. 131-140. Krantz. D.L., & Campbell, D.T. (1961). Separating perceptual and linguistic effects of context shifts upon absolute judgments. Journal of ExDerimental PsvcholocFv, 62.35-42. KrupatFE. (1974).Context a s a determinant of perceived threat. Journal of Personalitv and Social Psvcholom, 29. 731-736. Manis, M. (1967).Context effects in communication. Journal of Personalitv and Social Psvchology. 5, 326-334. Manis. M., Nelson, T.E., & Shedler. J. (1988).Stereotypes and social judgment: Extremity, assimilation, and contrast. Journal of Personalitv and Social pSvcholoa, 28-36. Martin, L.L. (1986).Set/reset: Use and disuse of concepts in impression formation. Journal of Persona litv and Social Psvcholorry, 493-504. Massin. S.C. (1987).Different biases in the methods of constant and single stimuli. Bulletin of the Pychonomic Society, 25,379-382. Mellers, B.A.. & Birnbaum. M.H. (1982).Loci of contextual effects in judgment. Journal of ExDerimental Psvcholom: Human PerceDtion and Performance, 8, 582-601. Mellers, B.A.. & Birnbaum, M.H. (1983).Contextual effects in social judgment. Journal of Exoerimental Social Psvchology. B,157-171. Parducci. A. (1956).Incidental learning of stimulus frequencies in the judgment of 169-178. single stimuli. Journal of ExDerimental Psvcholom, Parducci. A. (1983).Category ratings and the relational character of judgment. In H.G. Geissler. H.F.J.M. Buffort. E.L.J. Leeuwenberg, - & V. Sarris (Eds.), Modem issues in DerceDtion (pp. 89-105). Parducci. A.. & Wedell, D.H. (1986). The categorv effect with rating scales: Number of categories, number of stimuli, and-method of presenration. Journal of ExDerimental Psvcholom: Human PerceDtion and Performance, 12.496-516. Parducci, A.. Thaler. H.. &Anderson, N.H. (1968). Stimulus averaging and the context for judgment. PerceDtion and PsvchoDhvsicS. 3. 145-150. Pepitone. A.. & DiNubile. M. (1976). Contrast elfects in judgments of crime severity and the punishment of criminal violators. Journal of Personality and Social PSvcholom, 33,448-459. Perrett. L.F. (1971).Immediate and background contextual effects in clinical judement. Unpublished dissertation, University of California. Los Angeles. Riskey, D.R. Parducci. A. & Beauchamp. G.K. (1979).Effects of context in judgments of sweetness and pleasantness. PerceDtion and Psvchoahvsics, 26, 171-176. Sherman, S.J., Ahlm. K., Berman, L., & Lynn, S . (1978). Contrast effects and their relationship to subsequent behavior. Journal of ExDerimental Social Psvcholoi8. 14.340-350. Simpson. D.D.. & Ostrom, T.M. (1976).Contrast effects in impression formation. Journal of Personalitv and Social Psvchology, 625-629. Smith, R.H., Diener, E.. & Wedell, D.H. (1989). Intrapersonal and social comparison determinants of happiness: A range-frequency analysis. Journal of Personalitv and Social Psvcholoa, 56. 317-325. Sn.111. T.K., & Wyer, R.S.. J r . (1979).The role of category accessibility in the interpretation of information about persons: some determinants and consequences. Journal of Personalitv and Social Psvcholom. 1660-1672.

a,

s,

a,

a,

a,

z,

302

D.H. W e d e l l

Srull. T.K.. & Wyer. R.S..J r . (1980).Category accessibility and social perception: Some implications for the study of person memory and interpersonal 841-856. judgments. Journal of Personalitv and S wial Psvchology, Upshaw, H.S. (1969). The personal reference scale: An approach to social judgment. In L. Berkowitz [Ed.). A h & (Vol. 4. pp. 315-371).New York: Academic Press. Upshaw. H.S. (1978). Social influence on attitudes and on anchoring of congeneric attitude scales. Journal of Emerimental Social P s v c w . 14.327-333. Volkmann. J. (1951). Scales of judgment and their implications for social psychology. In J.H. Rohrer & M. Sherif (Eds.), Socia1 Dsvcholoav at t hc prossroads [pp. 279-2941. New York: Harper & Row. Wyer. R.S.. Jr.. & Srull, T.K. (1981).Category accessibility: Some theoretical and empirical issues concerning the processing of social stimulus information. In E.T. Higgins. C.P. Herman, & M.P. Zanna (Eds.). Social cormition: The OntariQ SWQosium on Persona litv and Social Psvchology (pp. 161-197). Hillsdale. N J : Erlbaum. Wedell, D.H. (1989a). Contrasting theories of assimilation and contrast. Paper presented at the meeting of the Midwest Psychological Association, Chicago, IL. Wedell, D.H. (1989b). Context effects on Dreference iudcments of social stimuli. Manuscript in preparation. Wedell, D.H.. & Parducci, A. (1988). The category effect in social judgment: Experimental ratings of happiness. Journal of Personalitv and Social P.Wcholo~,55.341-356. Wedell. D.H.. Parducci, A.. & Geiselman, RE. (1987).A formal analysis of ratings of physical attractiveness: Successive contrast and simultaneous assimilation. 230-249. Journal of ExDerimental Social Psvcholoa. Wedell. D.H.. Parducci, A., & Lane, M. (in press). Reducing the dependence of clinical judgment on the immediate context: Effects of number of categories and type of anchors. Journal of Personalitv and Social Psvcholo-.

a.

a,

SECTION 4 Biases relative to the categorization activity

This Page Intentionally Left Blank

Cognitive Biases J . - P . C a v e r n i , J.-M. F a b r e a n d M . G o n z a l e z ( E d s . ) Q Elsevier S c i e n c e P u b l i s h e r s B.V. ( N o r t h - H o l l a n d ) , 1 9 9 0

SOCIAL BIASES IN CATEGORIZATION PROCESSES Willem Doise Faculty of Psychology, U n i v e r s i t y of Geneva, S w i t z e r l a n d

Abstract. Social psychological studies on categorization are reviewed.

~n

important lineage of research stems from Henri Tajfel's model on the accentuation of contrasts in quantitative judgements. The relevance of the model for the study of intergroup relations is illustrated, and it is shown that the structure of these relations determines the relative strength of between-group differentiation and within-group homogenization. Deschamps' covariation model is also described: it assumes that an increase in intragroup differentiation can covary with an increase in intergroup differentiation. Finally, research based on the prototypicality model is discussed. This model can account for differences in degrees of category membership and for the intertwining of within-category singuIarization and between-category differentiation.

More than forty years ago, Bruner and Goodman (1947, p. 43) concluded their famous article "Value and Need as Organizing Factors in Perception" with the following sentence: "For too long now, perception has been virtually the exclusive domain of the Experimental psychologists with a capital E. If we are to reach an understanding of the way in which perception works in everyday life, we social psychologists and students of personality will have to join with the experimental psychologists and reexplore much of this ancient field of perception whose laws for too long have been taken for granted." In this paper I will briefly illustrate how experimental social psychologists have studied categorization processes and, more specifically, how they have studied the intervention of social regulations in these processes. At a more general level my aim is to show that Bruner and Goodman's conclusion remains rnutatis rnutandis true for contemporary cognitive psychology. Just a s social regulations intervene in perceptual processes, they also intervene in cognitive processes and a n important task for experimental social psychologists is to study the interrelationships (see Doise. 1986) between social regulations and cognitive functioning. In the realm of categorization processes such research already abounds: social biases have not only been shown to exist but cumulative research has manifested their systematic nature in many instances.

306

W. Doise

FROM CONTRASTS IN QUANTITATIVE JUDGEMENTS TO CATEGORIZATION PROCESSES

Ten years after the publication of Bruner and Goodman's article, Tajfel (1957) published a review paper on about twenty experiments investigating the overestimation phenomenon in quantitative judgements on valued stimuli. A few of these experiments have replicated the overestimation effect: others, for instance Lysak and Gilchrist (19551, studying judgements of the size of dollar bills, did not replicate the same effect. Why? Tajfel's attention had been aroused by a regularity in the results of several experiments: every time a series of stimuli of different values was to be judged (such as coins of different size and value, jugs with different quantities of cane sugar), the difference between the judgements of the extreme stimuli of the series was much greater than that between judgements on similar stimuli without value (chips, jugs containing sand). This result seemed more consistent than the overestimation effect itself. Bruner and Rodrigues (1953). for instance, did not replicate overestimation of the size of coins in comparison with judgements on metallic chips without value, but they did observe a much greater discrepancy between the size estimations of bigger and smaller coins than between the estimations of bigger and smaller chips. It should also be noticed that in the experiments with coins their value varied with their size whereas this was not the case for dollar bills since they are all the same size. The attribution of different values to a number of stimuli varying on a physical dimension (such as size and weight) would lead to a n intra-serial effect: the differences between the stimuli are overestimated for these physical characteristics which are systematically related to their value. These results highlight the importance of accentuating physical differences between stimuli of different value. Enhancement of such differences in perception would be a systematic distortion aiming a t avoiding errors in the identification of objects with different values. A new experiment with coins was carried out by Tajfel and Cawasjee (1959). It was aimed at checking that judgements on size of coins of unequal values enhance size differences. This time the experiment took place in Great Britain and the stimuli to be judged in the experimental condition were coins worth two shillings and two shillings and a half. These coins measure 1.125 inches and

Social biases in categorization processes

307

1.250 inches, respectively. In the control condition, the stimuli were two coins unknown to the subjects: these coins were the same size and had almost the same appearance as those used in the experimental condition. With regard to the estimation of the smaller size coin, the difference between the estimations of the two familiar coins was 24.2 percent compared to 17.4 percent for the two unknown coins. Although this difference was statistically significant, no differences were observed between the estimations of the sizes of the small coins in the two conditions. In his 1957 article Tajfel had already suggested (p. 202s) that relationships exist between these accentuations of contrasts on physical dimensions between valued objects and more general accentuations of contrasts between social categories: "When judgments concerning some quantifiable or ratable aspects of stimuli which fall into distinct categories are called for, differences in value or relevance cannot fail to influence the quantitative judgments in the direction of sharpening the objectively existing differences between the stimuli. (...) These judgmental effects of categorization are probably fairly general: it is likely, however, that they are particularly pronounced when judgments are made in dimensions in which scaling in magnitude is simultaneously a scaling in value. Thus, it may well be that a n accentuation of differences in size will hardly occur between two paintings, one liked and one indifferent or disliked. But when skin color, or height, or some facial traits of social "value" are concerned, there will be a marked sharpening of differences in the degree of these characteristics perceived as belonging to individuals who are assigned to different categories." Before reporting some recent research showing such accentuations of contrast in person perception I shall first reproduce the six main proposals formulated by Tajfel (1959, p. 20s) on the effects of links between, on the one hand, accentuation of contrasts in quantitative judgments, and on the other hand, variations in value and/or differences in category membership. The statements enounced as experimental predictions were as follows: "(1) When a variation in value is correlated in a series of stimuli with a variation in a physical dimension, the judged differences in this physical dimension between the elements of the series will be larger than in a series which is identical with regard

308

W. Doise

to the physical magnitudes of the stimuli, and in which the stimuli do not possess the attribute of value. (2)When differences in value exist in a series of stimuli, but are in no way correlated with the variation in a physical dimension, these differences in value will have no effect on the judged relationships between the physical magnitudes of the stimuli of the series. (3)When a classification in terms of a n attribute other than the physical dimension which is being judged is superimposed on a series of stimuli in such a way that one part of the physical series tends to fall consistently into one class, and the other into the other class, judgements of physical magnitudes of the stimuli falling into the distinct classes will show a shift in the directions determined by the class membership of the stimuli, when compared with judgements of a series identical with respect to this physical dimension, on which such a classification is not superimposed. (4) When a classification in terms of a n attribute other than the physical dimension which is being judged is superimposed on a series of stimuli, and the changes in the physical magnitudes of the stimuli bear no consistent relationship to the assignment of the stimuli to the distinct classes, this classification will have no effect on the judged relationships in the physical dimension between the stimuli of the series. (5) When a classification in terms of a n attribute other than the physical dimension which is being judged is superimposed on a series of stimuli in such a way that one part of the physical series tends to fall consistently into one class, and the other into the other class, and this classification is of inherent value or of emotional relevance to the subject, judgements of physical magnitudes of the stimuli falling into the distinct classes will show a shift in directions determined by the class membership of the stimuli when compared with judgements of a series identical with respect to this physical dimension, on which such a classification is not superimposed: this shift will be more pronounced than the shift referred to in (3) above. (6)When a classification in terms of an attribute other than the physical dimension which is being judged is superimposed on a series of stimuli, this classification being of inherent value or of emotional relevance to the subject, and the changes in the physical dimension bear no consistent relationship to the assignment of the stimuli to the distinct classes, this classification will have no effect

Social biases in categorization processes

309

on the judged relationships in the physical dimension between the stimuli falling into the distinct classes." Elsewhere, I have related in detail how each of these predictions has been experimentally validated (Doise, 1978). I should also add that in a subsequent paper Tajfel and Wilkes (1963) completed the above predictions in a n important way by adding that predicted differentiation between classes goes together with accentuation of similarity within classes. Hence the model is often no longer referred to as a model on accentuation of contrasts (see also Campbell, 1956) but as a categorization model. WITHIN AND BETWEEN GROUP DIFFERENTIATIONS

Tajfel (1959b; Tajfel, Sheikh, & Gardner, 1964) directly applied the theory of contrasts in judgements to accentuation of differences and of resemblances in the area of social stereotypes. Does this mean that explanation of social stereotypes can be reduced to that which accounts for individual organisation of judgements of physical stimuli? I t is true that the research I have reported elsewhere (Doise, 1978, 1979a). allows the conclusion that social psychologists have been usefully inspired by theories developed in a different area in their explanations of some characteristics of social stereotypes. However, it should not be forgotten that Tajfel carried out his research into perceptual judgements because he was interested in problems of a social nature. The legitimacy of using a psychophysical model in social psychology is, in our opinion, not a real problem if put in general terms. Every model is a special case, and the usefulness of its transfer into another domain should each time be proved. In order to facilitate adaptation of the psychophysical categorization model to be used in the area of intergroup relations, I have proposed a n extension of the definition of the categorization process (Doise, 1978). Three aspects of relations between groups should be distinguished: behaviour, value judgements, representations. These distinctions, which are classically used in psychology, do not imply that we are dealing with different realities. There is no intergroup behaviour which is not also accompanied by value judgements or more objective judgements. On the other hand, a judgement is already a behaviour; it is a stance taken in relation to another group and often a justification or anticipation of some act in relation to it. The hypothesis is that differentiations at the levels of cognitive representations, evaluative discriminations and behavioural

310

W. Doise

discriminations are all linked. In regard to Tajfel's theory (1959a) which links evaluative and representative discriminations, extension of the theory is related more precisely to explanation of the behavioural aspect of intergroup relations. This is a fundamental dimension which governs the entire setting of the category differentiation dynamic. If there are discriminations and contrasts in representations and evaluations, this is because they facilitate differentiated behaviour in terms of category membership between members of different groups. Discriminations between groups operate in a vicious spiral: behavioural discrimination as a function of category membership leads to evaluative and representational discrimination, which in t u r n facilitates further behavioural discrimination. The process of categorization, which allows individuals to organize their subjective experience of the social environment, thus also takes account of the structuring of a system of social interactions. Behaviour, as a function of shared or opposed memberships, structures and transforms social reality. The categorization process therefore describes, not j u s t a fundamental way of understanding reality, b u t also a form of interrelating collective behaviours. which transform reality as a function of social cleavages of highly diverse origins. Percepts, feelings and individual actions are thus oriented along collective lines of force. Elsewhere (Doise, 1978) I have reported in detail several experiments carried out in the general framework of this extension of the theory of category differentiation. Here I will summarise a few of them. One experiment (Doise, Csepeli, Dann, Gouge, Larsen, & Ostell, 1972) was designed to test the effect of behavioural factors on value judgements and representational discriminations. Subjects were divided into two groups, apparently according to their previous choices in an aesthetic judgement task but in fact at random, and anticipated either a cooperative or a competitive interaction with members of the other group. In a control situation, no interaction was expected. All subjects rated members of their own group and of the other on evaluative scales, e.g. friendly-hostile, generous-mean, and on more objective scales, namely fair-dark, tallshort, fat-thin, active-quiet. In fact, since they did not know who was in their group or the other, they had to guess their characteristics in a situation where they did or did not anticipate future interaction. As predicted, anticipated competition with

Social biases i n Categorization processes

311

something important at s t a k e led to more differentiated descriptions than in the control condition, both on the evaluative and on the more objective scales. This experiment therefore succeeded in reproducing experimentally the accentuation of differences of a physical nature observed in racist cultures (Pettigrew, Allport, & Barnett, 1958; Secord, Bevan, & Katz, 1956). Another experiment (Doise & Weinberger, 1972-73) also studied the effect of convergence or divergence at a behavioural level on discriminations at the representational level. The representations were those which boys have of girls in our society, and the aim of the experiment was to show how variables relating t o the level of interactional analysis modified these representations. Male students anticipated a cooperative or competitive interaction with female students. When their anticipation was competitive, they displayed greater evaluative discrimination in relation to female students than when their anticipation was cooperative. In the competitive condition, they also attributed more feminine characteristics to the others than in the cooperative condition. This stronger differentiation was observed when competition between two males and two females was anticipated, and not when competition was to involve only one male and one female. This difference between collective and inter-personal interactions is related to the functioning of the categorization process itself in a group interaction, convergence in respect to another member of one's own category is possible, which also leads to the complementary aspect of the process, divergence in relation to the other group. During a n interaction between two individuals, each of a different group, one aspect a t least of the process, convergence in relation to a member of one's own group, is impeded, which weakens the effect. The same difference between personal encounter and group encounter was demonstrated in an experiment studying the evaluative representations which members of two groups of different social position (apprentices and high school boys) held about themselves and about the others (Doise & Sinclair, 1973). Confrontation of one group with another can in certain conditions lead to tightening of links within the group and a "sacred union" effect. Two other experiments showed that evocation of another group or of another category is sufficient to lead to accentuation of within-category resemblances i n social representations. In one experiment (Doise. Deschamps, & Meyer,

312

W. Doise

1978) all subjects described six photographs, three girls and three boys, according to a list of traits relating to feminine and masculine stereotypes, the only difference between the two experimental conditions being that in one, subjects first described the three photographs of one sex without knowing that they would have three more of the opposite sex to describe, while in the second all six photographs were presented together. The categorization process should operate in a stronger fashion in the second condition where both categories were available from the start. This is what the results showed: in the second condition, descriptions revealed a greater differentiation between categories a n d a greater resemblance between photographs of the same category. Another experiment (Doise, Deschamps, & Meyer, 1978) looked at the same process of accentuation of resemblances and differences, b u t this time using stereotypes that Geneveses have about different regional groups in Switzerland (German-speaking Swiss, French-speaking Swiss, Italian-speaking Swiss). When two of these groups had to be described a t the same time as a third non-Swiss group, resemblances between the two Swiss groups became more salient than when they were described with a third Swiss group and without any opposition from a foreign group. Such research illustrates well how collective representations are actualized in different situations according to the categorization process. Though in myths and ideologies the social world is usually divided into two opposing camps, this dualist view does not necessarily reflect the complexity of social systems. Nor is dualism always the end product of the categorization process. The social arena is often made up of a network of affiliations which cross each other. Let me distinguish two types of situations: a situation of "simple categories" in which opposition between one membership category and another is made very salient, and a situation of "crossed categories" in which, for individuals, some members of their membership category and some members of a different category according to one criterion find themselves linked as members of a different group according to a second criterion. Let me also suppose that the two category systems are equally salient: how will the categorization process work in the crossed category situation? There should be a n increase in the differences between the two categories in the first categorization as well as an increase in the differences between the two categories in the second. At the same time, there should be accentuation of differences within the

Social biases in categorization processes

313

same category since it is, by definition, made up of members of two different groups. For the same reasons, there should be both accentuation of resemblances between members of the same and of different categories according to the first criterion since they belong to the same category according to the second. Therefore one might suppose that opposed effects would weaken the operation of category differentiation. To illustrate these dynamics, Deschamps and Doise (1978) experimentally created a situation of simple categories and a situation of crossed categories. In each situation, six girls and six boys had to evaluate their own and the others' performance, but in one situation, only gender divided them, whereas in the other three girls and three boys also belonged to a "blue" group as distinct from the other three girls and three boys in a "red" group. Results showed that crossover of membership had the predicted effect: in simple categories, subjects made different evaluations of the performance of members of their own groups from members of the other, b u t this difference disappeared in the crossed condition where neither gender nor colour category showed any effect. Deschamps and Doise (1978) found effects in the same direction by using two natural categories (men-women, young-adult) crossed or not. Results reported by Arcuri (1982) and Vanbeselaere (1987) confirm that crossed membership weakens discrimination between groups. The results of Brown and Turner (1979) are similar when the four groups resulting from crossover of two different categories are present in the experimental situation: when only two groups are present, they once again found the usual effects of inter-group differentiation. Deschamps (1977a) also studied the effect of crossed membership in judgements of physical stimuli: he again found disappearance of the effect of category when systematic crossed membership was introduced. We have thus gone full circle: a theory applied first to quantitative judgements has enriched analysis of social phenomena and has returned to its original field to test properties of the theory revealed in social psychology.

THE COVARIATON HYPOTHESIS Almost two decades ago, I (Doise, 1969) had already reported results showing that in a n experimental situation using the Wilson, Chun and Kayatani (1965) procedure, intragroup competition was

314

W. Doise

significantly correlated with intergroup competition, in two of the three main experimental conditions. When homogeneous dyads (two French or two German subjects) were competing with other such dyads, the intracell correlation between the number of intradyadic competitive choices on a mixed motive game was significantly related to the number of interdyadic competitive choices. However, this was not the case when the dyads themselves were heterogeneous, i.e., composed of both a French and a German youngster. I t was years later that I was reminded of these results when the criss-crossing experiment of boys and girls (Deschamps & Doise, 1978) exposed that differentiation between oneself and other members of the same category occurred for boys in a simple categorization situation. However, this did not occur when two categorization systems were criss-crossed. Since then, Deschamps (1977b. 1984; Deschamps & Lorenzi-Cioldi, 1981; Deschamps & Volpato, 1984; see also Lorenzi-Cioldi, 1988; Nigro & Serino. 1985) has shown several times that greater differentiation between oneself and other members of the ingroup can be linked to greater differentiation between the ingroup and the outgroup. Are these results to be considered as exceptions to the categorization model which a s s u m e s that i n t r a g r o u p homogeneisation increases with intergroup differentiation? Apparently, Deschamps' results are not so exceptional if one looks a t causability from another direction taking into account effects of within group differentiation on between group differentiation. Ng and Cram (1987) for instance report on the differences of personal status within a group as factors in strengthening or weakening intergroup discrimination. Some ingroup differences, s u c h as s t a t u s , lead to more or less ingroup/outgroup bias (see also Kennedy & Stephan, 1977). Rabbie (1982) also thinks that it is difficult to conceive of simple links between ingroup processes and intergroup relations. "One of the main themes of Sherif, (1966). and Blake and Mouton, (19611, is that intergroup competition produces an increase in ingroup cohesion or solidarity (...) There is much less support for this popular hypothesis in 'realistic conflict theory' (Levine & Campbell, 1972) than one would expect. Indeed, our own studies show that groups that expect to compete do not exhibit a greater ingroup cohesion than groups that do not expect to compete or than groups that expect to cooperate" (Rabbie, 1982, p. 132).

Social biases in categorization processes

315

Manipulating intragroup relations and enhancing cooperation or competition can, under specific conditions, both give rise to significant differentiation within the ingroup and between the ingroup and the outgroup. Wagner and Schdnbach (1984) reported data which show that it is enough to ask lower s t a t u s German subjects to describe themselves and Germans in general before they describe immigrant workers, in order to produce a more negative image of the latter. Such differentiation does not occur when the selves have not been described before the outgroup. However, the same is not true for higher status subjects, for whom the order of describing themselves before or after the immigrant worker does not play a role. I t is therefore worthwhile to carefully inspect data that show that intragroup differentiation is inversely related to intergroup differentiation. The data presented by Doise, Deschamps, and Meyer (1978) on the accentuation of intracategory similarities did not involve descriptions of selves and it is perhaps oneself that is to be considered the main source of the co-variation of intergroup and intragroup differentiation. To conclude this section, let me briefly summarize Deschamps' (1984, p. 456) covariation hypothesis. According to him, social dynamics at the interindividual level and a t the intergroup level are, to some extent, homologous and interrelated. Tensions between intergroup cooperation and competition are similar to the tensions between fusion and individuation a t the personal level. Experimentally reinforcing a pole a t one level will increase the salience of the homologous pole a t the other level. Therefore, in response to the categorization model opposes interindividual differentiation and intergroup differentiation, another point of view is postulated from which both differentiations can be seen as compatible. In another paper Deschamps (1982) had already developed the idea that the search for interpersonal distinctiveness depends on the individuals' positions in a given network of intergroup relations. Especially members of dominating groups tend to consider themselves individually as the point of reference in relation to which other people are defined: they conceive of themselves as unique individuals and do not search so much for their self-definition in terms of group membership. On the other hand, members of dominated groups define themselves more, and are also defined by others in terms of social

316

W. Doise

categorizations imposed on them. The search for differentiation from other individuals would therefore be stronger for members of dominant groups and would be enhanced when s u c h a group affiliation is made salient: but it should be understood that members of dominated groups also can occupy dominant positions in specific situations (see Lorenzi-Cioldi, 1988). PROTOTYPICAL CATEGORIZATIONS

The covariation hypothesis articulates categorization schemes which intervene at the intergroup and interpersonal level. This idea converges with current trends analyzing social differentiation processes in models derived from prototypicality theory. A prototype (see Rosch, 1975, 1978) is defined in terms of differences between other elements of its membership category fully represented by the prototype, as well as in terms of differences with elements of another category. The main novelty is that the prototypical categorization model explicitly states that degrees of category membership are possible, where prototypes are those members of a category who are more typical of the category than other members. In this sense, prototypical categorization can deal with processes which simultaneously account for both within category singularization and between category differentiation. Several aspects of prototypical categorization which are important for interrelating social and cognitive dynamics have already been studied more or less extensively. In the first instance the research of Rosch (1973, 1975) on the internal structure of "natural" categories should be mentioned. This research is based on the assumption that many categories are not "Aristotelian" in the sense that they do not imply " ... logical, clearly bounded entities, whose membership is defined by an item's possession of a simple set of criterial features, in which all instances possessing the criterial attributes have a full and equal degree of membership." (Rosch, 1975, p. 193). On the contrary the concept of internal structure refers to that "general class of conceptions of categories in which categories are not presented only as criterial features with clear-cut boundaries and in which items within categories may be considered differentially representative of the meaning of the category term." (ibidem, pp. 193-4). Rosch's research dealt mainly with color categories and with ten other categories systematically chosen to represent common concrete objects: furniture, fruit,

Social biases in categorization processes

311

vehicles, weapons, vegetables, carpentry tools, birds, sports, toys and clothing. Results obtained with identity judgements clearly show that members rated as good examples of a category are cognitively processed in a different way than members considered generally as bad examples. Subsequent research has proved the usefulness of this concept of internal structure for understanding the cognitive processing of social categories such as professions (Cohen, 1983; Dahlgren, 1985; Salmaso, & Pombeni, 1986) social situations (Cantor, Mischel, & Schwartz, 1982), personality types (Semin & Rosch, 1980; Forgas, 1985). Members who are more prototypical of a category can be considered as more important reference points in similarity judgements than less prototypical members. Therefore also in the social domain such judgements should manifest asymmetries as those obtained by Tversky (1977) with non-social objects and by Codol (1984, 1987) in comparisons involving oneself and others. Trentin a n d Arcuri (1987) investigate s u c h asymmetries in evaluative judgements on food and in the perception of regional differences in Italy (Arcuri, personal communication, see also Serino, 1988). Another much investigated topic in prototypical categorization research concerns the differences in levels of categorization according to a hierarchy ranging from the abstract to the concrete. Three levels of inclusiveness are often distinguished on this continuum: a superordinate category, for instance, the furniture category, is located a t the most inclusive and abstract level: a subordinate category, for instance, the kitchen table category, is the less inclusive and most concrete. But for the study of prototypes the most useful categories are located in between; they are basic categories such as the table category. Basic categories are at the same time rich in attributes and well differentiated from other basic categories. In that sense table, lamp and chair are three basic categories, they each evoke many features but have few in common, and they are inclusive of many different objects which can be described in great detail and which elicit the same patterns of behavior within a category but not between categories. At the more subordinate level, definitions (for instance of kitchen chair, dining room chair, living room chair) are also rich but they do not longer allow for clear-cut distinctions a t the same level of inclusiveness. Cantor and Mischel (1979, p. 25) have confirmed the usefulness of this distinction for studying the structure of categorization systems

318

W. Doise

bearing on human beings: "Our pursuit of the analogy between the categorizations of common everyday objects -- pants and chairs and cars -- and that of people -- extraverts and madmen and activists -so far has yielded considerable convergence. The consensually rated attribute lists associated with person types a t the middle level in these taxonomies were richer, in our data, than those judged to be true of members of the superordinate categories and there was less overlap between the attribute lists for neighboring categories at this level of abstraction than at the more finely tuned and detailed level of subordinate categorizations. The subordinate categories maximized richness and imageability, but at the cost of overlap (...I, such that categories at this level would be less than ideal for highlighting the differences between the types of persons in neighboring categories." Andersen and Klatzky (1987) have applied the levels of categorization idea to the distinction between on the one side personality traits, such as extravert and introvert, considered as more abstract, and on the other side social stereotypes such as comedian, politician, brain or guru, considered as more concrete. They summarize their results as follows: " ..., when subjects were presented with both trait and social stereotype labels and were asked to list their features, the social stereotypes yielded far more non-redundant associations than did the trait-defined categories: in this sense, they were the richer and better articulated of the two categories (Study 2). Social stereotypes also conveyed more physical features, behavioral characteristics, and demographic facts (see also Cantor & Mischel, 1979: Fiske & Cox, 1979). Hence, the two types of representations are substantively different, with social stereotypes being more vivid and easier to visualize (Study 2; see also Nisbett & Ross, 1980). Finally, when subjects were presented with both types of category labels and were asked to rate their association with a series of featural attributes, the social stereotypes were found to have many more idiosyncratic, distinctive associations than did the trait categories, which had virtually none (Study 3)." (Andersen & Klatzky, 1987, p. 243-4). Individual differences in categorization have been studied by Roney and Sorrentino (1987) who find that "certainty-oriented'' subjects list fewer person categories under superordinate categories and demonstrate less overlap across these categories than did "uncertainty-oriented" subjects. It is now the task for social psychologists to investigate systematically the social conditions

Social biases in categorization processes

319

which favor one or another style or level of categorization. For analyzing categorization processes within the category of French executives, the sociologist Boltanski (1982) has already based his interpretation upon Rosch's prototypical categorization model. Executives are a very heterogeneous group, which is shown in the representation they construct of themselves. "This all happens as if the structure of mental categories, that serve to define the group, is homologous to the structure of the group itself a hypothesis that is based upon the conception that mental categories have a structure. This hypothesis opposes the functioning of practical logic (which considers categories organized around a central point, but which lacks clear limits that would allow for a n absolute definition of membership or non-membership) to the functioning of learned logic (which explicitly considers categories to be homogeneous and discreet entities)" (Boltanski, 1982. p. 464). Boltanski's analysis surpasses a mere description by indicating the functions assumed by these non-precisely defined categories: - to facilitate fierce competitions in situations which lack precise evaluation due to the variety of status indices and symbols: - to mask important cleavages through the multitude and the crisscrossing or differentiated memberships: - to proceed, according to specific circumstances and strategies, toward various inclusions and exclusions. As such, one can paradoxically speak of "the cohesion of fuzzy sets" or "the strength of weak aggregates". CONCLUSION

My purpose in writing these pages has not been to offer an exhaustive report on more than forty years of research about biases in categorization processes. I have mainly commented upon one lineage of research in which especially the contribution of Tajfel has produced a coherent body of knowledge showing that so-called biases are in fact manifestations of pragmatic or social regulations. For lack of space other important contributions of the lineage have not been commented upon here. I have recently dealt with social identity theory (Tajfel, 1981; Tajfel & Turner, 1986: Turner, 1987) which of course bears on important dynamics elicited by and giving rise to biases in categorization processes (see Doise, 1988). Another important branch of the lineage started with Eiser's (1971; Eiser & Van der Pligt, 1984) reanalysis of Hovland and Sherifs

W. Doise

320

(1952) assimilation and contrast effects. His reinterpretation, already preceded by that of Zavalloni (1964), allows for the understanding of important mechanisms through which social categorizations regulate individual judgements (see Doise. 1979b). Elsewhere (Doise, 1984) I have also reported on the asymmetry between intragroup and outgroup homogeneisation. the latter occurring much more easily than the former (see for instance Park & Rothbart, 1982; Quattrone. 1986). This asymmetry can be considered as another effect of the special position of the self which is not taken into account by some traditional perceptual or cognitive interpretations of the categorization model. With the studies on prototypical categorization new models are now available. I t is a challenge for social psychologists to describe the social conditions which make individuals function according to more traditional dichotomous categorization models or to some aspects of the prototypicality model which can account for the covariation of within and between group variation.

REFERENCES Andersen. S.M., & Klatzky, R L . (1987).Traits and social stereotypes: levels of categorization in person perception. Journal of Personalitv and SOC id Psvchology, B, 235-246. Arcuri. L. (1982).Three patterns of social categorization in attribution memory. 271-282. ean J oumal of- S Blake, RR. & Mouton, J.S.(1961).Reactions to intergroup competition under winlose conditions. m a e ement Science, Z,420-435. Boltanski. J.L. (1982). 1 . Paris: Minuit. Brown, RJ., & Turner, J.C. (1979). The criss-cross categorization effect in intergroup discrimination. British Journal of Social and Clinical Psvcholoav, 18. 371383. Bruner, J.S.. & Goodman, C.C. (1947).Value and need a s organizing factors in perception. Journal of Abnormal and Socia1 Psvch o1aey 33-44. Bruner, J.S.,& Rodrigues. J.S. (1953).Some determinants of apparent size. Journal gf Abnormal and Social Psvcholom. 17-24. Campbell, D.T. (1956).Enhancement of contrast as composite habit. Journal of

,u.

.a,

a,

Cantor, N., & Mischel. W. (1979).Prototypes in person perception. In L. Berkowitz ial Psv&r&gy , (Vol 12).New York (Ed.). A d v m x s in EXDerimental SOC Academic Press, 3-52. Cantor, N.. Mischel. W.. & Schwarz. J. (19821.Social knowledge: structure. content. use and abuse. In A. Hastorf. A. Isen (Eds.). Comitive Social Psvcholo-!. New York Elsevier North-Holland. Codol. J.P. (1984). Quand Dupond resemble a Dupont plus que Dupont a Dupond. Psvcholoeie Francaise. 29. 284-290. Codol, J.P. (1987).Comparability and incomparability between oneself and others: Means of differentiation and comparison reference points. Cahiers de Psvchologie Cognitive, 2. 87-105. Y

321

Social biases in categorization processes

Cohen. C.E. (1983).Inferring the characteristics of other people: categories and ,& 34I. attribute accessibility. Journal of Personalitv and Social Psv44. Dahlgren. K. (1985). The cognitive structure of social categories. Cormitive !Wen= 9. 379-398. Deschamps. J.C. (1977a).Effect of crossing category memberships on quantitative judgement. FuroDean Journal of S o c m v choloa, z. 517-521. Deschamps, J.C. (1977b).LAttribution et la Catei 1 . Berne: Peter 9

Social identity and relations of power between groups. In H. Deschamps. J.C. (1982). Tajfel, Social Identitv and IntererouD Relations, Cambridge: Cambridge University Press, 85-98. Deschamps, J.C. (1984). Identite sociale et differentiations categorielles. Cahiers de Psycholoeie Cormitive. 4,449-474. Deschamps, J.C., & Doise. W. (1978).Crossed categoIy memberships in intergroup relations. In H.Tajfe1 (Ed), Differentiation bet ween social eroum. London: Academic Press, 141-158. Deschamps. J.C., & Lorenzi-Cioldi, F. (1981)."Egocentrisme" et "Sociocentrisme" dans les relations entre groupes. & w e Suisse de Psvcholoeie, 108-131. Deschamps, J.C.. & Volpato. C. (1984).Identita sociale e identita individuale nelle relazioni tra gruppi. Giomale Italian0 di P s i c iw, 275-301. Doise, W. (1969).Strategies de jeu a l'interieur et entre des groupes de nationalltes differentes. Bulletin du C.E.RP.. Is.13-26. Doise. W. (1978).GrouDs and Individuals. EXDlanatiOnS in social Dsvcholom. Cambridge: Cambridge University Press. Doise. W. (1979a). J?mtriences entre erouneg. Paris: Mouton. Doise, W. (1979b).Contrastes sociaux dans les jugements. Les Cahiers de Psvcholoeie Sociale. 4. 1 - 12. Doise, W. (1984).Les relations entre groupes. In S. Moscovici (Ed.). Psvchologie Sociale. Paris: Presses Universitatres de France, 253-274. Doise. W. (1986). Levels of Ex0lanation in Socia1 Psvcholoa. Cambridge: Cambridge University Press. Doise, W. (1988).Individual and social identities in intergroup relations. EuroDean 1 Journal of Social Doise. W.. Csepeli. G . . Dann, H.D.. Gouge, C.. Larsen, K.. & Ostell, A. (1972). An experimental investigation into the formation of intergroup representations. FuroDean J ournal of SOC la1 Psvcholoa, 2.202-204. Doise. W.. Deschamps. J.C.. & Meyer. G. (1978). The accentuation of intra-category similarities. In H. Tajfel (Ed). D lfferentiation between soci a1 Cro u ~.gLondon: Academic Press, 159-168. Doise, W.. & Sinclair, A. (1973). The categorization process in intergroup relations. EuroDean Journal of Social Psvcholoa!.9. 145-157. Doise, W.. & Weinberger. M. (1972-73). Representations masculines dans dlfferentes situations de rencontres mixtes. Bulletin de PsvcholoiZie. 26. 649-657. Eiser. J.R. (1971).Enhancement of contrast in the absolute judgement of attitude statements. Journal of Personalitv & SOC ial PsvchoIQgy. 12. 1-10. Eiser. J.R. & Van der Pligt, J.V. (1984). Accentuation theory, polarization, and the judgement of attitude statements, In J.R. Reiser. (Ed),Attitudinal iudeement. New York, Springer. Fiske, S.T., & Cox. M.G.. (1979). Person concepts: The effect of target familiarity and descriptive purpose on the process of describing others. Journal of Personality, 42, 136-161. Forgas. J.P.. (1985). Person prototypes and cultural salience: the role of cognitive and cultural factors in impression formation. British Journal of Social p s v c h o l o ~24.3, 17.

a,

u,

.u.

W. Doise

322

Hovland. C.I.. & Sherif, M. (1952). judgemental phenomena and scales of attittude measurement: Item displacement in Thurstone scales. J o u r n a l r m a 1 and Socia1Psv . c h o l a , 42.822-832. Kennedy, J.. & Stephan. W.G. (1977).The effects of cooperation and competition on ingroup-outgroup bias. Journal of ADDlied Social Psvcholoa, 2. 115-130. Levine, RA., & Campbell, D.T. (1972).Ethnocentrism: theories of conflict. ethnic attitudes. and P-rouD b e h a w . London: John Wiley & Sons. Lorenzi-Cioldi, F. (1988).lndividus Dw s et GrouDes Domines. Imaculines et Feminines. Grenoble: Presses Universitaires. Value equivocality and goal availability. Journal Lysak, W.. & Gflchrlst. J.C. (1955). gf Personality. 23. 500-501. Ng, S.H., & Cram, F. (1987).Fairness and biases in intererouD relations: a studv on reward allomion and ~ O U dlfferenD . Dunedin: University of Otago. Psychology Department. Nigro. G., & Serino. C. (1985).Processi cognitivi in gruppi "dominanti" e "dominati". Giornale I m n o di ,P 199-233. Nisbett. R. & Ross, L. (1980).Human Inference: Strategies and Shortcomings of Social judgement. Englewood Cliffs, NJ: Prentice Hall. Park, B.. & Rothbart. M. (1982). Perception of out-group homogeneity and levels of social categorization: memory for the subordinate attributes of in-group and out-group members. Journal of Personalitv and SOC ial Psvcholoe-y, 42. 105 1-

u,

1068.

Pettigrew. T.F., Allport, G.W.. & Bamett. E.O. (1958).Binocular resolution and perception of race in South Africa. British Journal of Psvcholoizy. 49.265278. Quattrone. G.A. (1986). On the perception of a group's variability. In S. Worchel and W.G. Austin (Eds.). Psvcholo-uD Relatiom. Chicago: Nelson-Hall. Rabble, J.M. (1982).The effects of intergroup competition and cooperation on intragroup and intergroup relationships. In V.J. Derlega. J. Grzelak (Eds.), CooDeration and hehine behavior. London: Academic Press, 123-149. Roney, C.J.R., & Sorrentino. R.M. (1987).Uncertainty orientation and person perception: Individual differences in categorization. Social Comitioq. 5, 369382. Rosch. E. (1973). On the internal structure of perceptual and semantic categories. In T.E. Moore (Ed), Cognitive DeveloDment and Acauisition of Language. New York: Academic Press. Rosch. E. ( 1975). Cognitive representations of semantic categories. Journal of F n e m e n t a l Psvcholom: Generd, 1p4,192-233. Rosch. E. (1978).Principles of categorization, In E. Rosch et B.B. Lloyd (Eds.). Qgnition - a . Hillsdale. NJ: Erlbaum. Salmaso. P.. & Pombeni. L. (1986). Le concept de travail. In W. Doise, A. Palmonari (Eds.),L'ttude des remesentations sociales. Paris: Delachaux et Niestlt, 196207. Secord, P.F.. Bevan, W., & Katz, B. (1956).The Negro stereotype and perceptual accentuation. Journal of Abnormal and Social Psvcholoe-y, 5 .& 78-83. Semin, G.R., & Rosch, E. (1981).Activation of bipolar prototypes in attribute inferences. Journal of ExDerimental Social Psycholo&, 17.472-484. Serino, C. (1988). Strategies et structure de la comparaison sociale: Quelques aspects de l'asymttrie soi/autrui dans les relations entre groupes. Cahiers de psvcholoaie Cocnitive. to be published. Sherif, M. (1966). In common medicament. Boston: Houghton Mifflin. Tajfel. H. (1957).Value and the perceptual judgement of magnitude. Psvcholocical Review. 64,192-204. Tajfel, H. (1959a).Quantitative judgement in social perception. British Journal of psvcholoW. 16-29.

a,

Social biases in categorization processes

323

Tajfel, H. (1959b). A note on Lambert's evaluational reactions to spoken languages. Canadian Journa 1 of Psvcho1ocy 86-92. n ProUDS and social cateeories. Cambridge: Cambridge Tajfel, H. (1981).W University Press. Tajfel. H., & Cawasjee, S.D. (1959).Value and the accentuation of judged differences: 436-439. A confirmation. Journal of Abnormal and Social Psvcholo-, Tajfel. H..Sheikh, A.A., & Gardner. R.C. (1964).Content of stereotypes and the inference of similarity between members of stereotyped groups. psvcholo@l~. 22. 191-201. Tajfel, H.. & Turner, J.C. (1986). The social identity theory of intergroup behavior. In S. Worchel. W.G. Austin (Eds.). The Psvchologv of IntererouD Relations. Chicago: Nelson-Hall. Tajfel. H.. & Wilkes. A.L. (1963).Classifkation and quantitative judgement. British Journal of Psvchology. 101-114. Trentin, R. & Arcuri. L. (1987).Categorizing and evaluation through feelings and th concepts. Varna. Abstracts. VII- General MeetinP EurODean Association of ExDerlmental Social Psvcholom, May 19-23. Turner, J.C. (1987).Rediscoverine the social eTom. A self-cateeorization theorv. Oxford: Basil Blackwell. nersky. A. (1977).Features of similarity. psvcholoeical Review, 327-352. Vanbeselaere, N. (1987).The effects of Dichotomous and Crossed Social Categorizations upon Intergroup Discrimination. EurODean Journal of Social PsvcholoCv. 17.143-156. Wagner, V.. & SchBnbach. P. (1984). Links between educational status and prejudice: ethnic attitudes in West Germany. In N. Miller, M.B. Brewer, GrouDs in Contad. London: Academic Press, 29-52. Wilson. W.. Chun, N., & Kayatani. M. (1965).Projection. attraction and strategy choices in intergroup competition. Journal of Personalitv and Social PsvcholoW. 2,432-435. Zavalloni, M. (1964).m e s directs contre types indirects de jugement et echelles socio-psychologiques.Bulletin du C.E.R.P. , 199-210.

,u,

m.

a,

a,

u,

This Page Intentionally Left Blank

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez (Eds.) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

COGNITIVE BIASES IN SOCIAL CATEGORIZATION: PROCESS AND CONSEQUENCES C . Neil Macrae* and Miles R.C. Hewstone**

*

D e r a r t m e n t of Psychology,

**

U n i v e r s i t y of Aberdeen,

Scotland

Department of Psychology, U n i v e r s i t y of B r i s t o l , England

Abstract.

The present chapter outlines the process of social categorization and its resultant biases. In particular attention is directed to the effects of social categorization on the perception of others, causal attribution. and social memory. Recent research in each of these areas is critically reviewed and the need for a coherent theoretical framework implicating both the cognitive and motivational aspects of social Categorization is stressed.

In social psychology, the currently predominant metaphor characterizes the social perceiver as a "cognitive miser" (Fiske & Taylor, 1984). The central tenet of this characterization is that the human cognitive system has a limited capacity. Consequently, in order to avoid information overload, the social perceiver must utilize heuristics or shortcuts. The process of categorization is an example of such an information-processing shortcut, and it is pervasive in both object and person perception. Rather than try to process all the stimulus information available, the social perceiver simplifies matters by assigning individuals to social categories on the basis of their perceived similarities (Hamilton, 1979: Hamilton & Trolier, 1986). The process of social categorization, as such, is considered to be functionally expedient. As Hamilton (1979) noted, "to the extent that persons grouped into the same class are in fact similar, such categorization increases our ability to anticipate what a particular member of that group is like and how he is likely to behave" (p. 56). Therefore, to the extent that social categories capture and reflect real differences and similarities between people, then the process of categorization can be considered a valuable cognitive tool. A growing number of studies have, however, demonstrated biases which stem from the very process of social categorization itself. A critical consideration of some of this literature forms the basis of the present chapter. First, however, the process of social categorization will be outlined in more detail

32 6

C.N. Macrae

&

M.R.C. Hewstone

because of its specific inclusion in recent models of impression formation (Brewer, 1988; Fiske & Neuberg, in press). THE PROCESS OF SOCIAL CATEGORIZATION

Social categorization involves the matching of a target person to an existing social category, and it is suggested that the process is set in motion as soon as information sufficient to activate or cue a relevant social category has been encountered. The information necessary to cue an appropriate category can take several forms. For example, it may take the form of an observable feature (e.g., skin colour, clothing). Alternatively, and more common in psychological research, it may take the form of a written category label or a cluster of category-consistent attributes. Importantly, once a person has been classified into a particular social group, it is assumed that he or she possesses all the characteristics that define the group as a whole, and shares these characteristics with the other group members. That is, the person is deemed to possess a range of category-relevant attributes, despite the fact that none of these attributes may have served as the basis for the initial categorization. As we stated previously, the process of social categorization is thought to be driven by principles of cognitive economy. Once we have categorized a person into a particular social grouping, we can bring into play our vast store of generalized stereotypic knowledge. Fiske and Neuberg (in press) for example, state that because we possess limited cognitive resources, it is both simpler and more efficient to perceive others in this manner. This assertion receives empirical support from the findings that it takes longer to form an attribute-based impression than a category-based impression (Fiske, Neuberg, Beattie, & Milberg, 1987), and that attribute-based processes require more cognitive resources than do category-based processes (Bechtold, Naccarato, & Zanna, 1986). The actual process of social categorization itself raises some interesting questions. Messick and Mackie (1989) state that "cognitive representations of groups are assumed to be multipleelement structures with both horizontal links, connecting related concepts at a similar level of generality, and vertical ones, reflecting the hierarchical progression from more to less inclusive categories" (p. 46). Consequently, a t any given moment, a n individual could reasonably be classified into any one of a number of social categories.

Cogni t i ve b i a s e s i n soci a 1 ca t egori za t i on

327

Under s u c h conditions, what are the factors t h a t determine category activation? The available evidence suggests that category activation is affected by both cognitive and motivational factors: both the stimulus information available to the social perceiver and the social perceiver's processing objectives influence category selection (Srull & Wyer, 1986). Brewer (1988) summarizes the main determinants of category selection as falling within three classes. These are category accessibility, contextual effects, and processing goals.

Category Accessibility The likelihood that a specific category will be activated in the process of social categorization depends in part upon the recency and frequency with which it has been activated in the past. For example, if one has recently been burgled, a man seen loitering outside the house is more likely to be categorized as a thief than as a double glazing salesman. Thus, the recency with which a category has been activated in the past affects the likelihood of its future activation (Higgins, Rholes, & Jones, 1977; Srull & Wyer. 1979). Similarly, the frequency with which a category has been activated affects the activation process (Bargh, Bond, Lombardi, & Tota, 1986: Bargh & Pratto, 1986; Higgins & King, 1981). Frequently activated categories become chronically accessible in the social categorization process. Furthermore, the social information required to activate a chronically accessible category need not reach conscious awareness (Bargh & Retromonaco, 1982). Despite the wealth of empirical evidence showing t h a t recency and frequency of activation affect category accessibility, some cautions and caveats are in order. The majority of studies which have demonstrated the effects of recency and frequency on subsequent category accessibility have utilized trait concepts. For example, Higgins, Rholes, and Jones (1977) first presented subjects with a series of either favourable or unfavourable traits which they were supposed to associate with colours. Then, in a second, ostensibly different experiment, the same subjects had to form an impression of a person on the basis of some written information. Higgins et af. found that subjects' impressions of the person were more positive when the initial priming had involved a favourable, rather than a n unfavourable trait. Whether the same factors influence the accessibility of more complex social stereotypes,

328

C.N. Macrae

& M.R.C.

Hewstone

remains to be seen, however, and further experimental investigation is required.

Contextual EXfects

An individual's perceptual salience affects how he or she will be categorized. In particular, whichever feature is perceptually salient in relation to the surrounding environment is likely to be used as a basis for social categorization (Taylor, 1981). For example, Taylor, Fiske, Etcoff and Ruderman (1978) have shown that a solo group member (e.g., a single black in an otherwise all white group) is considered particularly salient, and this salience serves as the basis for the individual's social categorization. Similar effects are apparent even when the basis of the stimulus salience is quite meaningless, such as the colour of one's shirt (McArthur & Post, 1977). More recently, Oakes and Turner (1986) have questioned the existence of this perceptually-driven "distinctiveness effect" (Taylor & Fiske, 1978). Instead they suggest that the social perceiver's processing objectives determine the effects of stimulus salience upon social categorization. They state that "categorization simplifies perception by drawing attention to differences, and similarities, which are relevant for the purposes a t hand" (Oakes & Turner, 1986, p. 329). This view questions the prepotency of novel category memberships in social perception. According to McArthur (1982), a person's physical appearance serves as a n important cue in the process of category activation. This assertion is supported by the finding that when both visual and non-visual cues are presented simultaneously, visual cues dominate subjects' responses (Fiske & Cox, 1979; Posner, Nissen, & Klein, 1976). Therefore features which are physically manifested, such as a person's appearance, are likely to serve as the basis for subsequent social categorization. The Perceiver's Processing Goals

Category accessibility is also affected by motivational factors such as the social perceiver's current goals or processing objectives (Srull & Wyer, 1986). The social categories activated are likely to be goal specific. As Brewer (1988) states, "the perceiver being asked to make a psychiatric diagnosis of a new individual is going to start from a different set of categories than a perceiver trying to

Cognitive biases in social categorization

32 9

decide who to ask out for next Saturday night" (p. 18). Srull and Wyer (1986) provide a comprehensive account of how motivational factors such as current concerns, values, and beliefs can affect social information processing. Within the present context these factors are of relevance because they also influence the relative accessibility of social categories. Thus far, we have outlined the process of social categorization and the factors affecting the accessibility of social categories. Emphasis has been placed upon the utility of the categorization process as a means of simplifying a complex stimulus domain. More recently, however, the assumption t h a t social categorization proceeds in response to potential information overload has been questioned. Specifically, Medin (1988) suggests that rather than categorization occurring under conditions of excess information, the opposite is true: the process is initiated when the perceiver h a s available only limited amounts of information about a person. Under these conditions the activation of a relevant social category releases additional categorical information, which provides the perceiver with a richer knowledge base for subsequent judgements. Future empirical investigation is required to clarify Medin's (1988) claim. I t would appear, however, that social categorization may depend upon the perceiver's information processing objectives, with the implication that categorization may serve the perceiver in different ways. Under some conditions, it may operate as a cognitive tool for simplifying reality: while in others, it may enable the perceiver to generate additional information when the initial knowledge base is small (see Oakes & Turner, in press). The benefits of social categorization, however, cannot be considered in isolation from the costs which are incurred by the initiation of such a process, these costs being evident at all stages of social information processing. A critical evaluation of the literature on some of the consequences of social categorization is the topic of the remainder of this chapter, THE COGNITIVE CONSEQUENCES OF SOCIAL CATEGORIZATION

Once a social category has been activated, it plays an important part in subsequent information processing: that is, the available category affects the encoding, representation, and retrieval of social information (e.g., Srull & Wyer, 1989). Typically, the activated category is defined as a "cognitive structure that contains the

330

C.N.

Macrae

& M.R.C.

Hewstone

perceiver's knowledge, beliefs, and expectations about some human group" (Hamilton & Trolier, 1986. p. 133). This definition equates activated categories with schemata (Hamilton, 1981: Rumelhart, 1984: Taylor & Crocker. 1981), and prototypes (Cantor & Mischel, 1979). The assumption t h a t social categories, schemata, and prototypes are functionally identical is at best a n act of faith, however. with similarities between the concepts being definitional rather than empirically based (Fiske & Taylor, 1984). Most of the recent research within the social cognition tradition, however, is consistent with the view that, for experimental purposes, the three terms can be equated, their common feature being that they serve to organize knowledge (see Abelson & Black, 1986). In the case of social categories, this involves the organization of information about groups of people. Once a category becomes activated, the social perceiver has available a range of stereotypical preconceptions which can bias all stages of information processing. In his influential paper, Hamilton (1979) outlined a range of cognitive biases resulting from stereotypic conceptions of individuals. In the ten years since this paper was published, the range of experimental techniques available to the social cognition researcher h a s greatly increased. Consequently, these methodological improvements have further clarified some of the biases that Hamilton addressed. We shall now consider three of these biases in the light of recent research findings. Specifically, the effects of social categorization upon the perception of others, causal attribution, and social memory will be considered.

The Categorization of Others The process of social categorization is set in motion by apparent differences between individuals. The very a c t of categorization, however, can have pernicious consequences for individual group members, which extend beyond the actual differences between the groups. Furthermore, these effects are apparent even when the basis of the categorization is quite arbitrary and meaningless (Tajfel, 1969; Taylor, 1981). The basic problem with categorization is that once we have classified two people into different categories, we tend to ignore their similarities and exaggerate their differences (Tajfel & Wilkes, 1963). Similarly, when we have classified two people into the same category, we

Coyn i t i ve b i a s e s i n s o c i a 1 ca t egori za t i on

33 1

exaggerate their similarities and ignore their differences (Tajfel, Sheikh, & Gardner, 1964). I t would appear that the imposition of categorical boundaries upon a stimulus array affects the perception of group members. More specifically, it results in the accentuation of inter-category differences and intra-category similarities. These effects are further exaggerated if the social perceiver belongs to one of the groups in question (i.e.. ingroup or outgroup). In general, the perceiver shows greater differentiation between ingroup members. Consequently, ingroups and outgroups reflect an asymmetrical relationship, with ingroup heterogeneity being contrasted with outgroup homogeneity. Thus, the perceiver minimizes the variability between outgroup members, with this tendency generalizing to the perception of personality and behavioural characteristics (Park & Rothbart. 1982). In contrast to many experimental situations, the formation of naturally occurring groups in society is often based upon the fact that the group members do indeed have something in common which distinguishes them from other groups. Therefore, in some instances, diagnostic information which can serve as the basis of intergroup differentiation is readily available and applicable. Allen and Wilder (1979). however, have shown that differentiation between ingroup and outgroup occurs even when the basis of the group differentiation is quite arbitrary (e.g.. artistic preference). The existence of meaningful dimensions of comparison between groups probably only serves to accentuate this effect. A related phenomenon is the tendency for ingroup members to evaluate outgroup members more extremely on a series of personality traits (Linville, 1982; Linville & Jones, 1980). One possible explanation for the differential perception and evaluation of ingroup and outgroup members lies in the cognitive representation of categorical information (Linville & Jones, 1980). Linville. Salovey, and Fischer (1986) propose an exemplar model of group representation to account for ingroup heterogeneity and outgroup homogeneity, with the availability of individual exemplars from memory playing a crucial role. Greater contact with people increases the number and diversity, and therefore the availability of individual exemplars in memory. The predominance of withingroup, rather than between-group contact, however, results in the formation of a more complex and differentiated cognitive structure for the ingroup than for the outgroup. Consequently, greater variability is assumed to exist within the ingroup.

332

C.N.

Macrae

&

M.R.C.

Hewstone

However, Brewer and Kramer (1985) have drawn attention to some problems with this exclusively cognitive explanation. They point out that motivational factors may also be implicated in the perception of outgroup homogeneity. For example, while the perceived homogeneity of outgroup members may be attributed to a relatively undifferentiated cognitive representation, it may also be the case that distinctions between outgroup members are available, but the perceiver chooses simply to ignore them. A further limitation with the exemplar model is that outgroup homogeneity is apparent even within minimal groups, when the perceiver cannot have differential access to ingroup or outgroup exemplars in memory (Judd & Park, 1988). An alternative explanation, which also implicates intergroup contact, considers the manner in which intergroup information is processed and cognitively organized. In this regard the nature and structure of the groups are important. Distinctions can be drawn between cohesive and more diffuse social groups, with cohesive groups being characterized by an interdependence among group members. For example, for the group t o function effectively, individual members must interact and cooperate in the pursuit of shared goals (e.g.. work groups). In contrast, many social groups are more diffuse in nature, with the group members rarely interacting. Each of these group structures has important implications for social information processing. Simply stated, cohesive groups result in more person-based representations, while diffuse groups result in more group-based representations. Srull and Wyer (1989) report memory data which supports this contention (e.g., Srull. 1981; Wyer. Bodenhausen. & Srull, 1984). Typically perceivers share an affinity and interdependence with other ingroup members. Consequently, the ingroup is characterized by a cohesive structure with this resulting in a more person-based representation of the group. In contrast, contact between ingroup and outgroup members is relatively infrequent and no interdependence exists between individual members of each group. This results in the outgroup being characterized as more diffuse in structure with subsequent representations reflecting a group-based organization of information. Thus the perceived homogeneity of the outgroup can be attributed to the group-based organization of information, while ingroup heterogeneity reflects the organization of information on a n individual basis.

Cognitive biases in social categorization

333

Several motivational factors may also be implicated in the differentiation of group members. For example, future interaction and outcome dependence, may influence the cognitive representation of, and the information-processing about, social groups. Interaction and dependence may facilitate a more individuated or person-based representation of a group. The extent to which the perceiver's interactions are predominantly with other ingroup members, however, may work against the formation of a more person-based or differentiated conception of the outgroup. The nature of the intergroup interaction is also important. The instigation of intergroup competition, for example, may enhance the perception of outgroup homogeneity (Judd & Park, 1988). Intergroup differentiation appears to be the product of a n interaction between motivational and cognitive factors. The social identity theory approach to intergroup behaviour reflects an explicit awareness of such an interaction (Tajfel & Turner, 1979; Turner, 1982; 1984). Social categorization is seen to serve two distinct and important functions: (1) it simplifies the social perceiver's world: and (2) it provides a means of self-reference. The theory assumes that individuals strive to obtain a positive self-concept or social identity, this being partly achieved through the process of intergroup comparisons. Individuals strive to seek distinctiveness between their own and other groups on relevant and valued dimensions. Such distinctiveness on appropriate dimensions promotes ingroup favouritism and the maintenance of a positive selfidentity. The results of the so called "minimal group" studies appear particularly amenable to an interpretation in terms of social identity theory (e.g., Rabbie & Honvitz, 1969; Tajfel, Flament, Billig. & Bundy, 1971). Thus, intergroup differentiation is the consequence of both cognitive and motivational factors. Social Categorization and Causal Attribution Once a particular social category becomes activated it furnishes the perceiver with a wealth of category-related knowledge, including expectancies regarding the likely behaviour of group members. Functionally these expectancies serve to define what constitutes category-expected and unexpected behaviour. In recent years a n extensive literature has emerged concerning the effects of social categorization on causal attribution (for a comprehensive review see Hewstone, 1989, chapter 6). This

334

C.N. M a c r a e

&

M.R.C.

Hewstone

research has focussed upon the causal attributions provided by the perceiver for the same behaviour enacted by a n ingroup vs. outgroup member, and for behaviour that either confirms or violates categorical expectancies. The expectancies the perceiver holds about other persons can be derived from several sources. For example, they may be based on information that is provided by a third party, knowledge of the person's previous behaviour. or categorical assumptions derived from the person's group membership (Jones & McGillis, 1976). Furthermore, these expectancies are assumed to affect the process of causal attribution. Implicit in this argument is the assumption that a piece of behaviour may elicit different causal explanations, depending upon the category membership of the person performing the action. Intergroup attribution considers the causal explanations provided by the perceiver for ingroup and outgroup behaviour. Within this context several studies report a pervasive ethnocentric bias in the explanations provided for behaviour, with the perceiver adopting group-serving attributional strategies which favour his or her own group. For example, Duncan (1976) presented subjects, via close circuit television, with a scene of two men discussing possible solutions to a problem (one of the men was black, the other white). The behaviour that interested Duncan was a n ambiguous shove that one party gave the other at a point when their discussion had become heated. Duncan varied whether the black man or the white man was the 'victim' or the 'harm doer'. Amongst other things, Duncan was interested in the causal explanations subjects (who were all white) provided for the behaviour. He found that causal attributions were more internal when the protagonist was black, and external when the protagonist was white. These results appear to reflect a n ethnocentric bias in white subjects' causal attributions. There are, however, problems with this interpretation. Firstly, as Hamilton (1979) has noted, the two versions of the incident may not have been identical. Also, only white subjects' causal attributions were measured. The evidence for a n ethnocentric bias in attributions would have been stronger had black subjects also been used in the study. If self-serving attributional strategies were in operation, one would expect a reversal in the pattern of causal attributions obtained from whites when black subjects were used. Similar limitations are apparent in Taylor and Jaggi's (1974) study on ethnocentric attributions. They presented subjects, who

Cognitive biases in social categorization

335

were all Hindus, with a series of stimulus paragraphs describing the performance of a desirable or undesirable behaviour by either a Hindu or a Muslim actor. Subjects then received a series of alternative explanations for the behaviour. and were asked to select the one they considered most appropriate. Consistent with their experimental prediction, Taylor and Jaggi found that Hindu subjects attributed the desirable behaviour of Hindu actors to internal factors, and the undesirable behaviour to external factors. In contrast, for Muslim actors they attributed desirable behaviour to external factors, and undesirable behaviour to internal factors. These results again suggest a n ethnocentric bias in intergroup attributions. The failure to consider the attributions of outgroup (Muslim) members, however, limits the validity of this conclusion. Despite apparent methodological limitations, Pettigrew (1979) has systematized these findings in a general conception referred to as the "ultimate attribution error". Briefly stated, the ultimate attribution error can be summarized as follows. Positive ingroup behaviour tends to be attributed to internal factors, whereas similar outgroup behaviour tends to be attributed to external factors. Conversely, negative ingroup behaviour tends to be attributed externally, while the same behaviour performed by an outgroup member tends to be attributed internally. More recently, researchers have addressed and overcome the methodological problems apparent in earlier studies investigating intergroup attribution. For example, Hewstone and Ward (1985) carried out a conceptual replication of the Taylor and Jaggi (1974) study using Chinese and Malay students. However, on this occasion the causal attributions of both groups were assessed. The results of this study were important for two reasons: (1) they provided some s u p p o r t for t h e phenomenon of ethnocentric intergroup attributions; and (2) they illustrated the effects of cultural and political factors on causal attributions. Intergroup attribution does not occur within a cultural vacuum. Rather, as Hewstone and Ward have shown, it is the product of cognitive, motivational, and ideological factors. Further support for ethnocentric attributions comes from the explanations provided for success and failure in achievement situations. Extending Weiner's (1986) approach to the structure of perceived causality, several researchers have obtained evidence for group-serving attributional biases (e.g., Deaux & Emswiller, 1974; Feldman-Summers 81 Kiesler, 1974; Hewstone, Jaspars. & Lalljee,

336

C.N.

Macrae

& M.R.C.

Hewstone

1982). For example, Hewstone et al. (1982) investigated the attributions given for success and failure in exam performance by schoolboys from British "state" and "private" schools. Ethnocentric attributions were clearly apparent in the explanations provided for failure by the private schoolboys. They ascribed outgroup failure to lack of ability; ingroup failure was ascribed to a lack of effort. Thus, ethnocentric or group-serving attributional biases are apparent in intergroup attributions; different explanations are typically provided for identical behaviours performed by ingroup and outgroup members. According to social identity theory (Tajfel & Turner, 1979), intergroup attribution may function to enhance group distinctiveness. Tajfel (1969) stated that a person's system of causes can help provide a positive self-image. Thus, a t the intergroup level, group members provide causal explanations in a way that enables them to enhance their social identity. However, although the social identity framework provides a plausible motivational explanation for ethnocentric attributions, it h a s recently come under some criticism. More specifically, Messick and Mackie (1989) argue t h a t the theory's interpretation of intergroup differentiation is very difficult to test empirically. While people may want to maintain positive self-esteem, and they may view the groups to which they belong more positively, the establishment of a direct causal association with intergroup bias has not yet been achieved. The previously mentioned studies all considered the causal attributions preferred by the perceiver when providing explanations for identical ingroup and outgroup behaviour. A growing number of studies, however, have also investigated the causal attributions the perceiver provides for behaviour which either confirms or disconfirms categorical expectancies. S u c h categorical expectancies are considered to determine the perceiver's causal attributions (e.g., Deaux, 1976; Fiske & Taylor, 1984). Behaviour consistent with categorical expectancies tends to be attributed to stable internal causes, whereas behaviour discrepant with expectancies is attributed to more unstable external causes (e.g., Bell, Wicklund, Manko, & Larkin, 1976; Bodenhausen & Wyer, 1985; Deaux. 1976; Kulik. 1983; Macrae & Shepherd, 1989; Nesdale, 1983). For example, Deaux (1976) argues that the explanations given for the task performance of a person are mediated by the expectancies people hold regarding men and women. Specifically, Deaux proposed that behaviour that confirmed

Cognitive biases in social categorization

331

gender expectancies would tend to be attributed to internal factors, while disconfirming behaviour would tend to be attributed to external factors. In a review article, she cites several studies which support this hypothesis (e.g., Deaux & Emswiller. 1974; FeldmanSummers & Kiesler, 1974). More recently, Bodenhausen and Wyer (1985) and Macrae and Shepherd (1989) have shown similar effects in the explanations provided for criminal behaviour. When a criminal incident confirmed category-based expectancies, it was attributed more to internal factors than when it disconfirmed these expectancies. I t appears t h a t when perceivers explain expectancyconfirming behaviour. they simply rely upon internal factors implied by the social category activated. Ajzen (1977) suggests that peoples' judgements of the likelihood of occurrence of particular events are affected by the intuitive causal theories they hold regarding these events. Thus, when categorically-consistent or confirming behaviour occurs, individuals need look no further than their preexisting causal theory for a n explanation of why the behaviour occurred, with this theory implicating the actor's dispositions as a cause of the behaviour. In contrast, inconsistent or disconfirming behaviour instigates a new causal analysis because no prior theory exists which can account for the behaviour. In this case the perceiver's attention is directed to the environment in search of a causal explanation for the behaviour. Pyszczynski and Greenberg (1981). for example, have shown that the perceiver engages in more attributional processing when presented with expectancydisconfirming behaviour. Hamilton (1988). using a response-time methodology, has reported lower response times for trait and intention questions (rather than for causal attributions) following congruent behaviours. b u t for situation-cause a n d intention questions (rather than trait and person-cause questions) following incongruent behaviours. While the effects of cognitive factors, such as categorical expectancies, on intergroup attribution are compelling, more research is required. Specifically, researchers need to develop a means of empirically quantifying category-based expectancies. It seems unlikely that the effects of category activation on causal attribution will be universal. The strength with which a belief about a particular social category is held by the perceiver may be a n important determinant of subsequent effects on causal attributions. Also, a problem apparent in earlier studies concerned the selection

338

C . N . Macrae & M.R.C.

Hewstone

of behaviours which either confirmed or disconfirmed categorical expectancies. Rather than simply assuming that the performance of a particular behaviour disconfirms categorical expectancies, future research must establish this fact empirically. This empirical criterion for the selection of appropriate behaviours is particularly important. As Hastie (1981) has pointed out, there is an important distinction between behaviour which is not expected because it is irrelevant to the expectancy, and behaviour which is not expected because it disconfirms the expectancy.

Social Categorization and Memory Peoples' expectancies about what is likely to happen in a given situation play a critical role in the encoding of information into memory. Consequently, the process of social categorization exerts a n important influence on the information that the perceiver is likely to recall about an event or an individual. In recent years, memory for expected or unexpected behaviour has become a central issue in social cognition research (for a recent review see Higgins & Bargh, 1987). Typically, two issues are addressed: (1) do people erroneously remember category-consistent information which was absent at the time of encoding?: (2)do people preferentially recall category-consistent or inconsistent information? If categorical expectancies lead to the preferential encoding and retrieval of certain types of information, is it also the case that they result in the intrusion of unrepresented b u t consistent information in memory? Intrusions of such information have been evidenced in prose comprehension tasks (e.g., Bransford & Franks, 19711, and in memory for scenarios (e.g., Brewer & Treyens, 1981). This phenomenon is also claimed to exist in person memory. Cantor and Mischel (1977) suggest that cognitive structures, which they term "prototypes". operate as memory structures around which the social perceiver organizes social information. Subjects were presented with targets who were described as either introverts or extraverts by means of a series of trait terms. A third, control group were described in trait terms unrelated to introversion or extraversion. There followed a recognition task in which some of the original traits were presented, along with new terms, which were either related or unrelated to introversion and extraversion. Subjects then stated, on a four point rating scale, how confident they were that each term had been present in the original

Cogn i t i ve b i a s e s i n s o c i a 1 ca t e g o r i za t i on

33 9

description. Results showed that subjects in the two experimental groups were more confident in their ratings of prototype-consistent but new terms, than they were of prototype-inconsistent new terms. Cantor and Mischel (1977) concluded that subjects show a memory bias for information consistent with the prototype. As Higgins and Bargh (1987) point out, these results have been replicated on several occasions, thus adding support to the notion that people will falsely recognize prototype-consistent information as having been present, when in fact it was not (Taylor & Crocker, 1981; Fiske & Taylor, 1984; Tsujimoto, 1978). There are, however, problems with this interpretation. Hastie (1981) argues that there is little reason to suppose that subjects in the Cantor and Mischel (1977) study actually did believe that they had seen the prototype-consistent but new terms. This is apparent from the subjects' confidence rating scores. While these ratings were more confident for the prototype-consistent but new terms, the means for these terms still indicated a low confidence in previous presentation. An alternative explanation may be that subjects simply favour a guessing strategy which results in a bias for semantically similar items (Wyer & Gordon, 1982). Also, when free recall rather than a recognition task is used to measure memory, there is less evidence for the intrusion of category-consistent but new information (Cantor & Mischel, 1979; Hastie & Kumar, 1979; Srull, 1981; Srull & Brand, 1983; Srull & Wyer, 1983). Thus, while categorical expectancies can influence the manner in which information is encoded and retrieved, it seems unlikely that they can alter the dynamics of information processing such that people believe they recognize something which was not actually in the original display. People appear to be quite capable of distinguishing between internally and externally generated sources of information (Johnson & Raye. 1981). As was mentioned previously, categorical expectancies can affect the interpretation and explanations given for behaviour (e.g., Duncan, 1976). Typically, a behavioural episode is interpreted in accordance with prior expectancies. Thus, because expectancies affect the way in which we interpret and explain behaviour, it has also been suggested that they will influence the ease with which information can be recalled. The argument is as follows. Since expectancies appear to lead the perceiver to interpret behaviour in a schema-consistent manner, it might also be expected that the perceiver will show preferential recall for schema-consistent

340

C.N.

Macrae

&

M.R.C.

Hewstone

information. Such a process would facilitate the maintenance of erroneous stereotypical beliefs i n the face of potentially disconfirming information. The results of several studies appear to support this contention. Cohen (1981) showed subjects a videotape of a man and a woman eating dinner, in which the woman was described as either a waitress or a librarian. Included in the tape were equal numbers of stereotypical waitress and librarian behaviours. After viewing the interaction, subjects were asked a number of factual questions about its content. Cohen had predicted that subjects would show enhanced recall for information which was consistent with the initial occupational prime. However, only the results in the waitress condition provided support for this prediction. That is, subjects who thought they were watching a waitress were subsequently more accurate in remembering stereotypical waitress rather than librarian behaviours. However, an alternative explanation for these findings is available. I t may simply be the case that subjects who were informed that the woman was a waitress paid more attention to stereotypical waitress behaviours . Rothbart, Evans, & Fulero (1979) presented subjects with fifty behavioural descriptions, each of which described a different man who belonged to the same group. These descriptions either indicated friendliness or intelligence. Furthermore, prior to presentation of the information, half of the subjects were led to believe that members of the group were friendly, and the other half were led to believe that they were intellectual. On a subsequent memory task, subjects were better able to recall behaviours that confirmed their prior expectation. Thus, these results seem to suggest that information that is consistent with a n expectancy will be more readily recalled. However, within the literature, empirical support can also be found for the opposite claim, namely that expectancy-disconfirming information is more readily recalled (Crocker, Hannah, & Weber, 1983; Hastie & Kumar, 1979). Original support for this second view was obtained by Hastie and Kumar (1979). They presented their subjects with a description of a target by means of several adjectives which were related to the same personality trait. Subjects then read a list of behaviours performed by the target which were either consistent or inconsistent with the initial description, or else were irrelevant to it. Hastie and Kumar reported better recall of behaviours that were inconsistent with the initial trait description. Why should

Cogn i t i ve b i a s e s i n s o c i a 1 ca t egori za t i on

341

inconsistent information be more readily recalled? One possibility is that because this information violates the initial expectancy, it is processed more "deeply" (Craik & Lockhart. 1972) in a n attempt to reconcile it with the expectancy. In a sense, the additional processing is an attempt to explain the apparent inconsistency (Clary & Tesser, 1983; Hastie, 1984). Hastie (1980) accounts for the preferential recall of schema-inconsistent information in accordance with a n associative network model of memory. He suggests that inconsistent behaviours result in preferential recall because in attempting to reconcile the discrepancy, the perceiver forges more links between them and other behaviours in memory. Subsequent research has been largely supportive of this view (e.g., Srull. 1981; Wyer & Gordon, 1982; Wyer, Bodenhausen, & Srull, 1984). At first glance these various findings appear to contradict one another. One set suggests that category-confirming information is

more easily remembered, while the other suggests that categorydisconfirming information is more memorable. How can this apparent contradiction be reconciled? Firstly, attention must be directed to the nature of the information that is commonly manipulated in these studies. The important quality of the information presented is that it either confirms or disconfirms a prior expectancy. However, often the level of consistency is not empirically determined, with the selection of information proceeding on an a priori basis (Hastie, 1981). Further, Wyer and Gordon (1982) noted that in the Hastie & Kumar (1979) study, the inconsistent behaviours which were presented to subjects were evaluatively and denotatively confounded. In their own study they found evidence for the preferential recall of only evaluatively inconsistent information. Several other factors have been shown to affect the recall of category-disconfirming information. For example, Fiske, Kinder, and Larter (1983) found that experts were more likely than nonexperts to make use of inconsistent information when judging the political system of a fictitious country. Similarly, Bargh and Thein (1985) found that under conditions of information overload, only subjects with a chronically accessible relevant construct were able to notice inconsistencies in the information presented. The attributional options available to people have also been shown to affect the recall of information. Crocker, Hannah, and Weber (1983) found that if subjects were presented with a n external attribution

342

C.N.

Macrae

& M.R.C.

Hewstone

for an inconsistent behaviour this resulted in it being recalled at the same rate as consistent behaviour. These are some of the factors that have been demonstrated to affect the recall of category-disconflrming information. However. Higgins and Bargh (1987) have identified what they consider to be the most important determinant of whether schema-consistent or inconsistent information is preferentially recalled. They suggest that many of the contradictory findings in the literature can be attributed to the fact that in some studies, subjects are asked to form a n impression about a person, while in others, they are asked to test a previously held categorical expectancy. The different task objectives inherent in impression formation as opposed to impression testing account for the preferential recall of different types of information. For example, in the Hastie and Kumar (1979) study, subjects were required to form a n impression of the target. As such, the results of this study should be considered in conjunction with other studies which suggest that the presentation of relatively infrequent, novel, or salient behaviours will result in increased attention during impression formation, and as a consequence be more accessible in memory (Fiske, 1980; Hamilton & Gifford. 1976). Hence. these results should not be generalized to situations where established beliefs are being tested. Under such conditions it appears t h a t category-confirming or consistent information is more readily recalled. An important distinction can be drawn between category-based, and target-based expectancies (Jones & McGillis, 1976). The presentation of trait lists to subjects is likely to facilitate the formation of target-based expectancies. Therefore, the results of these studies may be inapplicable to situations where the social perceiver activates an established category-based expectancy. The previous qualifications have important implications for the process of social categorization. The categories normally activated by the social perceiver are typically rich in informational content, and can best be characterized as stores of established (although often erroneous) stereotypical beliefs. Consequently, the activation of a social category should result in the preferential recall of category-confirming information. Several recent studies appear to support this view (e.g., Bodenhausen & Wyer, 1985; Macrae & Shepherd, 1989; Ruble & Stangor, 1989). However, Hamilton and Trolier (1986) note that inconsistent information tends to be preferentially recalled only when stimulus targets are perceived as

Cogni t i ve b i a se s in so cia 1 ca te g o r i za t i o n

343

being members of groups. Conversely, when the targets are perceived as individuals, consistent information is more readily recalled (e.g., Rothbart. Evans, & Fulero, 1979). This however appears to contradict the contention that when testing established beliefs, category-confirming information is more likely to be recalled. One might expect that the established expectancies which can be derived from knowledge of a person's group membership would be likely to result in the preferential recall of confirming information. It remains the task for future research to clarify this issue. CONCLUSION

Social categorization appears to be driven by the demands of cognitive economy. However, despite the apparent utility of the process, the present chapter has outlined a number of biases which stem directly from its instigation. More importantly, these biases illustrate the need for social cognition researchers to consider the motivational and affective influences on social information processing. Early research in the social cognition tradition painted an overly cognitive picture of the social perceiver. Most phenomena were considered explicable in solely cognitive terms a n d motivational and affective considerations tended to be neglected in both theorizing and research. In more recent times, however, the emphasis has changed. Recent theorizing in social cognition reflects a growing awareness of the importance of motivational and affective factors. For example, the impression formation models of Brewer (1988), and of Fiske and Neuberg (in press), emphasize the interaction of cognitive and motivational factors in the processing of information about others. Similar considerations are also apparent in Hewstone's (1988) model of conflict reduction, and Pyszczynski and Greenberg's (1987) model of biased hypothesis testing. It is within s u c h a framework t h a t future research on social categorization must proceed. Only under these circumstances can a clearer understanding of the process and its resultant biases be obtained. REFERENCES Abelson. R.P.. & Black, J.B. (1986). Introduction. In J.A. Galambos. R P . Abelson. & J.B. Black (Eds.).Knowledge structures. Hillsdale. NJ: Erlbaum.

344

C.N.

Macrae

& M.R.C.

Hewstone

Ajzen, I. (1977).Intuitive theories of events and the effects of base rate information on prediction. J n o f i & y and &cial Psvcholw, 303-314. Allen, V.L.. & Wilder, D.A. (1979).Group categorization and attribution of belief similarity. Small GrouD Behavioa , U, 73-80. Bargh. JA.. Bond, R.N., Lombardi. W., & Tota. M. (1986).The additive nature of chronic and temporary sources of construct accessibility. Journal of personmocialPsvcm. 869-878. Bargh. J.A.. & Pietromonaco. P. (1982).Automatic information processing and social perception: The influence of trait information presented outside of conscious awareness on impression formation. 1of Perso437-449. Social Psvcholocy, Bargh, J.A.. & Pratto, F. (1986).Individual construct accessibility and perceptual selection. Journal sf mDerlmenta1 Social,-P 22. 293-311. Bargh. JA., & Thein, RD. (1985). Individual construct accessibility, person memory, and the recall-judgment link: The case of information overload. Journal of Personalitv and Social Psvcholo 1129-1146. Bechtold, A . Naccarato, M.E., & Zanra, M.P. (1986).m f o r structure and the preiudice-discrimination link. Paper presented at the annual meeting of the Canadian Psychological Association, Toronto, Ontario. Bell, L.G., Wicklund. R.A.. Manko, G.. & Larkin. C. (1976).When unexpected behaviour is attributed to the environment. Journal of Research i n personality. J.Q. 316-327. Bodenhausen, G.V., & Wyer. R.S.(1985).Effects of stereotypes on decision making - oJ and information-processing strategies. PSVCholO~,48.267-282. Bransford, J.D.. & Franks, J.J. (1971). The abstraction of linguistic ideas. Coenitive 2,331-350. Brewer, M.B. (1988). A dual process model of impression formation. In T.K. Srull & RS. Wyer (Eds.),Advances in social cornition (Vol.1). Hillsdale. NJ: Erlbaum. Brewer, M.B..& Kramer. R.M. (1985).The psychology of intergroup attitudes and behaviour. Annual Revlew of Psvcholoa, 219-243. Brewer, W.F.. & Treyens, J.C. (1981).Role of schemata in memory for places. CQgnitive Psvcbbgy, U. 207-230. Cantor, N.. & Mischel, W. (1977).Traits a s prototypes: Effects on recognition memory. & J J 38-48. Cantor, N.. & Mischel. W. (1979)Prototypes in person perception. In L. Berkowitz (Ed.). A d v a n c e s i n a g o c i a l D s v c h o l o P v, (V01.12).New York: Academic Press. Clary, E.G.. & Tesser, A. (1983).Reactions to unexpected events: The naive scientist $9. 609and interpretive activity. person-1 Psvcholom Bull620. Cohen, C. (1981)Person categories and social perception: Testing some boundaries of the processing effects of prior knowledge. Journal of Personalitv and SOCial P s v c ~ o ~ ~441-452. . Craik. F.I.M.. & Lockhart, RS. (1972).Levels of processing: A framework for memory research.1of Verbal Leamine and Verbal Behaviou€. U, 671-684. Crocker. J.. Hannah, D.B., & Weber. R. (1983).Person memory and causal attributions. Journal of Personalitv and Social Psvcholocy!. 44.55-66. Deaux, K. (1976). Sex:A perspective on the attribution process. In J.H. Harvey, W.J. Ickes & R F . Kidd (Eds.). New directions in attrlbution research (Vol. 1). Hillsdale. NJ: Erlbaum. Deaux. K.. & Emswffler. T. (1974).Explanations of successful performance on sexlinked tasks: What is skill for the male is luck for the female. journal of , a - P 29. 80-85.

a,

a,

a,

.*.

m.

a,

s.

a,

Cognitive biases in social categorization

345

Duncan, B.L. (1976).DifTerential social perception and the attribution of intergroup violence: Testing the lower limits of stereotyping of blacks. Journal of and Social Psvch&gy. 590-598. Those who are number two try harder: Feldman-Summers. S.. & Kiesler. S.B. (1974). The effect of sex on attributions of causality. Journal of Personalitv and 846-855. Social Psvcholoa, Fiske. S.T. (1980). Attention and weight in person perception: The impact of negative and extreme behaviour. Journal of P e r s o n a l i m d Social by€h&&y * 889-906. Fiske. S.T., & Cox, M.G. (1979). Person concepts: The effects of target familiarity and descriptive purpose on the process of describing others. Journal of Personality. 47,136-161. Fiske. S.T., Kinder, D.R.. & Larter. W.M. (1983).The novice and the expert: Knowledge-based strategies in political cognition. Journa 1 of Ernerimental 381-400. Social Psvcholom, Fiske. S.T.. & Neuberg. S.L. (in press). A continuum model of impression formation from category-based to individuating processes: Influences of information and motivation on attention and interpretation. In M.P. Zanna [Ed.). A d v a m s in exDerimenta1 social DSvChOlO& (Vo1.23).New York: Academic Press. Fiske. S.T.. Neuberg, S.L., Beattie, A.E.. & Milberg. S.J. (1987).Category-based and attribute-based reactions to others: Some informational conditions of stereotyping and individuating processes. Journal of Emerimental Social Psvcholo~. 399-427. Fiske, S.T.. &Taylor, S.E. (1984). Social cof!nitioa Reading, MA: Addison-Wesley. Hamilton, D.L. (1979).A cognitive-attributional analysis of stereotyping. In L. social Dsvchology, (Vol.12). New Berkowltz (Ed.), York: Academic Press. Hamilton. D.L. (Ed.) (1981).Cognitive Drocesses in stereotwine and intercrouo behaviour. Hillsdale, N J : Erlbaum. Hamilton. D.L. (1988).Causal attribution viewed from an information processing perspective. In D. Bar-Tal & A.W. Kruglanski (Eds.), The social DSVC holom of knowle&g. Cambridge: Cambridge University Press. Hamilton, D.L.. & Gifford, R.K. (1976).Illusory correlation in interpersonal perception: A cognitive basis of stereotypic judgments. Journal of ExDerimental Social Psvchology, 392-407. Hamilton, D.L., & Troller. T.K. (1986).Stereotypes and stereotyping: An overview of the cognitive approach. In J. Dovidio & S. Gaertner (Eds.), prejudice, discrimination. and racism. New York: Academic Press. Hastie, R. (1980).Memory for information that confirms or contradicts a personality impression. In R Hastie. T.M. Ostrom, E.B. Ebbesen, RS. Wyer. D.L. Hamilton, & D.E. Carlston (Eds.). Person memorv: The comitive basis of social Derceatim . Hfflsdale, NJ: Erlbaum. Hastie. R (1981).Schematic principles in human memory. In E.T. Higgins, C.P. Herman & M.P. Zanna (Eds.), J i m (Vol.1). Hillsdale. NJ: Erlbaum. Hastie. R (1984). Causes and effects of causal attribution. Journal of Personalitv and Social Psycholoa, S, 44-56. Hastie. R.. & Kumar. P. (1979).Person memory: Personality traits a s organizing principles in memory for behaviours. J J li n 1 pmchow, 25-30. Hewstone. M. (1988).Attributional bases of intergroup conflict. In W. Stroebe. A.W. Kruglanski, D. Bar-Tal & M. Hewstone (Eds.). The social Dsvcholoev of intercrour, conflict: Theorv. research and amlications. New York Springer. Hewstone, M. (1989).Causal attribution: From coen itive Drocesses to co11ect ive beliefs. Oxford: Basil Blackwell.

a,

a,

xi.

a.

a,

u,

x.

C . N . Macrae & M.R.C.

346

Hewstone

Hewstone, M.. Jaspars, J., & Lalljee. M. (1982).Social representations. social attribution and social identity: The intergroup images of "public" and "comprehensive" schoolboys. FuroDean Journal of Social PsvcholQgy* 12. 241-269. Hewstone, M.. &Ward, C. (1985). Ethnocentrism and causal attribution in Southeast Asia. Journal of Personalitv and Social Psvchology, 614-623. Higgins, E.T., & Bargh, J.A (1987).Social cognition and social perception. M n u a l 369-425. @view of Psvchology, Higgins. E.T., & King, G.A. (1981). Accessibility of social constructs: Informationprocessing consequences of individual and contextual variability. In N. Cantor & J.F. Kthlstrom (Eds.). Personalitv. cognition and social interaction. Hillsdale. NJ: Erlbaum. Higgins. E.T.. Rholes, W.S.. & Jones, C.R. (1977).Category accessibility and impression formation. J o u r n a l n t a l Social mvcholO-,u, 141154. Johnson, J.T., & Raye. C.L. (1981).Reality monitoring.1 Review.88.6785. Jones, E.E.. & McCillIs, D. (1976).Correspondent inferences and the attrlbution cube: A comparative reappraisal. In J.H. Harvey, W.J. Ickes & RF. Kidd (Eds.). w j o - t m U (Vol.1). Hillsdale. NJ: Erlbaum. Judd. C.M.. & Park, B. (1988).Outgroup homogeneity: Judgments of variability at the individual and group levels. m a 1 of Personality and m a 1 Psvcholorn. g,778-788. Kulik, J.A. (1983).Confirmatory attribution and the perpetuation of social beliefs. Journal of Persona litv and Social Psv,fi.1171-1181. Linville. P.W. (1982). The complexity-extremity effect and age-based stereotyping. Journal of Personalitv and Social Psvcho10,q ,42,193-211. Linville. P.W., & Jones. E.E. (1980).Polarized appraisals of out-group members. Journal of Personalitv and Social Psvchology, 689-703. Linville, P.W.. Salovey. P., & Fischer. C.W. (1986).Stereotyping and perceived distributions of social characteristics: An application to ingroup-outgroup perception. In J.F. Dovidio & S.L. Gaertner (Eds.). preiudice. discrimination and racism. New York Academic Press. Macrae. C.N.. & Shepherd, J.W. (1989).Stereotypes and social judgements. British Journal of Social Psvchology, B, 319-325. McArthur, L.Z. (1982).Judging a book by its cover: A cognitive analysis of the relationship between physical appearance and stereotyping. In A. Hastorf & A. Isen (Eds.), Comitive social . New York: North Holland. McArthur. L.Z., & Post, D.L. (1977).Figural emphasis and person perception. Journal o f t a l Social U, 520-535. Medin. D.L. (1988). Social categorization: Structures, processes and purposes. In T.K. Srull & Wyer. RS. (Eds.), Advances in social coanition (Vol.1). Hillsdale. N J : Erlbaum Messick. D.M., & Mackie. D.M. (1989).Intergroup relations. b n u a l Review of PSvChOlO~,4p.45-81. Nesdale. A.R (1983).Effects of person and situation expectancies on explanation seeking and causal attributions. British Journal of Social Psvcholom, 22, 9399. Oakes, P.J., &Turner. J.C. (1986). Distinctiveness and the salience of social category membership: I s there a n automatic perceptual bias towards novelty? EuroDean J oumal of Social Psvcholw. J& 325-344. Oakes. P.J. & Turner. J.C. (in press). Is limited information processing capacity the cause of social stereotyping? In W. Stroebe & M. Hewstone (Eds.). EuroDean Review of Social Psvchology (Vol. 1). Chichester: J. Wiley.

a,

a,

a.

m.

347

Cognitive biases in social categorization

Park, B., & Rothbart, M. (1982).Perception of out-group homogeneity and levels of social categorization: Memory of the subordinate attributes of in-group and out-group members. Journal of Personalitv and Socid Psvcholoq. G , 10511068. Pettigrew. T.F. (1979).The ultimate attribution error: Extending Allport's cognitive analysis of prejudice. Personalitv and Social Psvcholoe Bulletin, 5,461-476. Posner, M.I.. Nissen. M.J., & Klein. R M . (1976).Visual dominance: An information processing account of its origins and significance. Psvchological Review. B. 157-171. Pyszczynski, TA.. & Greenberg, J. (1981).Role of disconfinned expectancies in the instigation of attributional processing. Journal of Personalitv and Social P ~ C h O l OQ ~ ,, 31-38. Pyszczynski,T.A.. & Greenberg. J. (1987).Toward an integration of cognitive and motivational perspectives on social inference: A biased hypothesis testing model. In L. Berkowitz (Ed.), Advances in exDerimenta1 social Dsvcho1(V01.20). New York Academic Press. Rabble. J.M.. & Horwitz, M. (1969). Arousal of ingroup-outgroup bias by a chance win or loss. Journal of Personalitv and Social Psvchology, 13.269-277. Rothbart. M.. Evans. M.. & Fulero. S. (1979).Recall for confirming events: Memory processes and the maintenance of social stereotypes. Journal of , 343-355. Rumelhart. D.E. (1984). Schemata and the cognitive system. In RS. Wyer & T.K. Srull (Eds.), Handbook of social cognition (Vol.1). Hillsdale, NJ: Erlbaum. Srull. T.K. (1981).Person memory: Some tests of associative storage and retrieval models. Journal of Emerimental Psvcholorn: Human Learning and Memory. z. 440-463. Srull.T.K.. & Brand, J.F. (1983). Memory for information about persons: The effect of encoding operations on subsequent retrieval. m a 1 of V e r b e a r n i n e & Verbal Behaviour. 22,219-230. Srull, T.K.. & Wyer. R.S. (1979).The role of category accessibility in the interpretation of information about persons: Some determinants and implications. Journal of Personalitv and Social Psvchology, 1660-1672. Srull, T.K., & Wyer. R.S. (1983).The role of control processes and structural constraints in models of memory and social judgment. Journa 1 of p, 497-521. Srull. T.K., & Wyer, RS. (1986).The role of chronic and temporary goals in social information processing. In R M . Sorrentino & E.T. Hlggins (Eds.), Handbook gf motivation and coanitioq. New York: Academic Press, Srull. T.K.. Wyer, R.S. (1989). Person memory and judgment. Psvchological Review, 96.58-83. Stangor, C.. & Ruble, D.N. (1989).Strength of expectancies and memory for social information: What we remember depends on how much we know. Journal of j3merimental Social P s v c w, 18-35. Tajfel. H. (1969). Cognitive aspects of prejudice. Journal of Social Issues. 79-97. Tajfel. H..Flament, C.. Bfflig. M.G.. & Bundy. RP. (1971). Social categorisation and intergroup behaviour. EuroDean Jou ma1 of Social Psycholagy. 1,149-178. Tajfel, H.. & Turner, J.C. (1979). An integrative theory of intergroup conflict. In W. Austin & S.Worchel (Eds.). The social Dsvcholom of interarouD relations. Monterey. CA: Brooks/Cole. Tajfel. H., Sheikh, A.A.. & Gardner, R.C. (1964).Content of stereotypes and the inference of similarity between members of stereotyped groups. -A PSvCholo~i~. 22, 191-201. Tajfel. H..& Wilkes. A.L. (1963).Classification and quantitative judgment. British Journal of Psvchology, 54. 101-114. Taylor, D.M., & Jaggi, V. (1974).Ethnocentrism and causal attribution in a South Indian context. Journal of Cross Cultural Psvcholoq, 5,162-171.

s,

a,

u,

a.

z,

C.N.

348

Macrae

& M.R.C.

Hewstone

Taylor, S.E. (1981).A categorization approach to stereotyping. In D.L. Hamilton (Ed.). Coenitive Drocesses in stereotminc and intererouD behaviour. Hillsdale, NJ: Erlbaum. Taylor, S.E.. & Crocker. J. (1981). Schematic bases of social information processing. In E.T. Higgins. C.P. Herman & M.P. Zanna (Eds.). Social coPnition: The Ontario S Y ~ D O Sium (Vol. 1). Hillsdale, NJ: Erlbaum. Salience, attention and attribution: Top of the head Taylor, S.E..& Fiske. S.T. (1978). phenomena. In L. Berkowitz (Ed.), Advances in exDerimental social (Vol. 1 1). New York Academic Press. Taylor, S.E.. Fiske, S.T., Etcoff, N.L. & Ruderman. A. (1978).Categorical bases of person memory and stereotyping.J J PsvcholO@, B,778-793. -Tsujimoto. R.N. (1978).Memory bias toward normative and novel trait prototypes. Journal of Personalitv and Social P w h o l ~ g,yz,1391-1401. Turner, J.C. (1984).Social categorization and the self-concept: A social cognitive theory of group behavlour. In E.J. Lawler (Ed.),Advances in croU D process= n m (V01.2).Greenwich. CT:JAI. Weiner. B. (1986). An attributional theorv of motivation and emotion. New York: Springer. Wyer. RS.. Bodenhausen. G.V.. & Srull. T.K. (1984). The cognitive representation of persons and groups and its effect on recall and recognition memory. Journal pf ExDerimental Social Psvcholo , 445-469. Wyer. R.S.. & Gordon, S.E. (1982).z e recall of information about persons and groups. 128-164. ~

a,

u,

Acknowledgements. The authors would like to thank J o h n Shepherd and Colin Gray for their comments on an earlier version of this chapter. While the chapter was prepared the first author was supported by a n Economic and Social Research Council (U.K.) doctoral research grant.

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez (Eds.) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

BIASES IN CATEGORIZATION Edith A. Das-Smaal Department of C o g n i t i v e Psychology, V r i j e U n i v e r s i t y Amsterdam,

The N e t h e r l a n d s

Abstract. On what grounds can we conclude that an act of categorization is biased? In this chapter, it is contended that in the absence of objective norms of what categories actually are, biases in categorization can only be specified in relation to theoretical understandings of categorization. Therefore, the chapter starts with an overview of current theoretical positions regarding categorization. and the various constraints on learning and using categories that are specified by these theories. Several types of categorization bias follow from these descriptions. In this context, the role of prototypes, basic level categories, existing schemata, category width, context, feature frequency and correlation among features is discussed, a s well as the biasing influence of selective attention, hypothesis testing and task demands.

1. INTRODUCTION 1.1 On categories and reality

This discussion of biases in categorization will be preceded by a consideration of the question of whether biases in this area can be established at all. When can we state that an act of categorization is biased? Biases are defined as systematic error, or the difference between judgments and the true value (e.g. Anderson et al., 1981). In order to be able to claim that an observed categorization behavior is biased, it is therefore necessary to specify what categories, in reality, are like. However, features, objects and events of the real world can be categorized in a n endless number of ways. Besides, our perception is highly selective and thus biased already. Which features we do perceive will be influenced by factors like for instance our physical ability to perceive them, and by our needs, our motives and prior knowledge, as represented in our cognitive system. It is at present widely assumed that categories arise from interaction between events in the world and the perceiver with his own prior knowledge. The perceiver, in making sense out of the stimuli of the world, has his or her own contribution to the resulting representation of the world. As McCauley (1987) states, from their

350

E . Das-Smaal

mental constructs or models, people often impose categories on the world of their experience rather than induce them from this world, although some categories seem to have an ecological basis too. It is important to recognize that human categorization behavior is heavily dependent on the cognitive constructs, the models that are built up during a lifetime of experience. Specifylng the real world a s it is, not as we perceive and represent it, is simply beyond our reach. So in the case of categorization, the norm has to be a question of theory, because there is no way of being sure that the real world is structured as we perceive it, with categories existing a priori. Perception of the world is constrained by our mental constructs, which in turn influence our categorization performance. However, there are a variety of theories and models on categorization, and although there seems to be some consensus about which theories have to be abandoned by now, within these constraints there is much less consensus about the appropriate description of human categorization behavior. An example on the issue of prototype effects may serve to illustrate this. As we shall see in the next paragraph, from a classical point of view, all members of a category should be as representative of the category as any other. This follows from the assumption that the norm for category membership is sufficiently specified just by the presence or absence of a set of defining features. All other features, even if they are representative of the category, are considered irrelevant, and this makes all category members equivalent. However, advances in our understanding of categorization have made it clear that in many categories differences in typicality do exist among category exemplars. People do not judge all category members as equally representative of their category. Thus, systematic departures appear to occur from the definition of what constitutes a category in a classical point of view. However, in a prototype conception typicality effects are not biases in this sense. In this view they are in the core assumption of the theory, as will be outlined below. Prototype views are specially devised to account for typicality effects, trylng to capture why it is that membership is a matter of degree. So, in the process of theoretical development, categorization behavior that appeared to be biased may prove to be unbiased given another theoretical understanding. Theoretical development keeps going on. Again, the issue of typicality effects may illustrate this. Typicality effects, in a still more recent theoretical account, are considered as phenomena that are not

Biases in categorization

351

necessarily relevant to categorization. I t is contended that they need not show something direct about the nature of categorization. They are superficial effects that do not always mirror category structure directly. Prototype effects may result from many sources, and are to be accounted for in terms of deviations from one's cognitive models or "theories" for understanding the world (Lakoff, 1986). We shall return to this view later. The point here is that misinterpretations, or a t least different kinds of interpretations of categorization behavior, continue to occur, due to a n incomplete understanding of all the types of categories that occur, and to a lack of knowledge regarding all aspects of influence on categorization. One of the important aspects is peoplek prior knowledge. Another point to mention is that in categorization models, tasks and situations have not received much attention. They have nearly always implicitly been considered to be homogeneous. That is, models usually do not predict different categorization behavior in different tasks and contexts. It is true, however, that tasks as well as situations do influence categorization, as will be shown in this chapter. In sum, absolute or clearly established standards, from which departures can be described, unfortunately do not seem to be a t hand. However, there is a sense in which biases in categorization can be specified. I t is obvious that there are a variety of potential groupings of the things in the world around us. Likewise, there are many potential ways of representing such groupings, and ways of deciding on group membership. An important question, therefore, is which ones are actually done or preferred by u s and which ones are not. Preferential biases can be studied experimentally by investigating constraints in human categorization activities. To this goal, designing experimental categories so as to exclude ambiguous interpretations of categorization behavior can be profitable. Preferred ways of categorizing will provide important clues as to both the underlying structure and the process of categorization. For instance, people may show a "bias" favoring correlated features, as was suggested by Rosch (1978). Directing attention towards certain features and away from others can be another source of bias, pertaining to what features are represented when a category is learned. Or people may or may not consider the occurrence of features in contrast categories when making a categorization decision. If biases in categorization are conceived of as preferential ways of learning and using categories, both these ways and the

352

E.

Das-Smaal

factors of influence on them have to be described. As pointed out in this opening paragraph, it should be kept in mind that this conception of biases ultimately boils down to presenting a descriptive theory of human categorization behavior. Despite that, the approach that is taken in this chapter is not to present a full theory of categorization, but rather to describe biases as constraints with which models should not be inconsistent. Bearing this understanding of biases in mind, we shall deal with the subject of categorization in general first, and then turn to a discussion of a number of important biases in this matter. I t is our theoretical point of view that biases can result from limitations on aspects of category information that are stored in memory, or from computational constraints whenever that categorical knowledge is used. Both aspects will be considered in this chapter. 1.2

Categorization

One of the most fundamental aspects of thinking is the ability to perceive similarities and differences among the events that we experience. Events rarely repeat themselves in exactly the same way. And even if they did, it is still questionable whether our interpretation would be quite the same as before. In this respect, impressions are always new to us. I t is our conceptual system, however, that allows u s to perceive similarities between new and old experiences. We are inclined to relate incoming information to things that we know. One mental operation by which this can be accomplished is categorization. By categorizing, the continuous variation in environmental information is reduced to manageable and knowable proportions. Categorization provides a means of attaining cognitive economy (Rosch, 1978). In this way, adequate reactions are possible. Mental categories generally are formed by experience. Experience, however, usually encompasses only a reduced sample of the total set of exemplars of a category, and this raises a number of important questions. The main issue, of course, is the question of what these mental categories are like. What is the nature of the process of categorization? Are there general constraints regarding this process? How can limited experience lead to knowledge of the whole catego@ What are the constraints on this knowledge? And how, for instance, do we determine the boundaries of a category? How is experience related to category width? A variety of answers

Biases i n categorization

353

have been given to these and related questions on how we form a mental representation of a category on the basis of our experience. A short description of the historical development of views on categorization and the main current theoretical positions will be dealt with first. From that, it follows that categorization is biased in a number of different ways. We shall go into this matter subsequently. With respect to biases, in succession, the role of prototypes, basic level categories, existing schemata, the formation of category boundaries a n d the influence of context, the representation of feature frequency and correlation among features will be discussed, as well as the issues of selective attention, hypothesis testing and task demands. 2. THEORETICAL VIEWS ON CATEGORIZATION

2.1 Historical development

The study of categorization emerged within the field of psychological learning theory. A broad distinction can be made between earlier "associationistic" stimulus-response (S-R) theories and later hypothesis-testing theories. I t was the post- 1950's tradition of cognitive psychology that led to a preference for active hypothesis-testing models. The behaviorist tradition prior to the cognitive shift entailed a passive associationistic account of the category learning process. Because of the hypothesis-testing theories the historical development will be briefly described here. Hypothesis testing directs attention to certain aspects of the information encountered, and this can be assumed to have a biasing influence on the acquisition of categories. Besides, in hypothesis testing people may adopt various strategies, and show biases in this respect too. As for the S-R account, it was Hull (1920)who advanced the "common element" view of category concepts. In his theory, category learning involves the formation of a n association between a certain response and an element common to a class of different stimuli. The category learner is viewed as passively undergoing a series of experiences which gradually bring about these associations, by reinforcement. Hull's opinions were formed in the behavioristic climate of his time. In this tradition the unobservable internal processes of the organism were ignored and considered as a subject unsuitable for scientific enterprise. Throughout the 1950's,the

354

E . Das-Smaal

passive and mechanical associationistic account of the category learning process prevailed. S-R theories, however, appeared to be too simplistic to account for central or thought processes that may influence the category learner's response to the information presented in the stimulus. In about 1955, the S-R concept was to replaced by the inputoutput concept. I t was recognized t h a t a complex program controlled the input-output sequence (Miller. Galanter, & Pribram, 19601, and this opened the way for new approaches to the study of cognitive processes, leading to the discovery of some sources of bias in categorization. A theory was proposed according to which the organism is seen as actively seeking information. According to this view, categories are learned via a n active, strategic process of forming and testing hypotheses. Bruner, Goodnow and Austin (1956) advanced this view. They provided an analysis of the nature of categorization and its central role in cognition. They tried to externalize the thought processes of people when learning to categorize. They broke with the behavioristic tradition by verifymg the existence of hypothesis testing behavior. From the observed behavior of the subjects they inferred certain patterns of purposive behavior which they called strategies. The notion of strategies enabled them to describe what the subject does "internally" when learning new classifications. The work of Bruner et al. brought about fundamental changes in category learning theory, and stimulated a n intensive analysis of the variables affecting and the processes underlying category learning. The process by which subjects learn a category was accepted to be one of forming hypotheses, which are tested and revised in the light of information from experience. It was realized that categories generally are learned by examples. People often have no control over which instances are presented to them. Exemplars and non-exemplars are pointed out to them as they happen to occur. In that case, a logical way to learn the category is by following a kind of reception strategy. This involves adopting a hypothesis, based on either the whole or a part of the first category example, and knowing how and when to alter it whenever new information is presented. To study the strategies actually adopted by subjects, Bruner designed a task that has since been used as a standard procedure. Visual stimuli were constructed that subjects had to learn to classify into categories. Categories were defined by the experimenter. For example, the subject was shown a series of

Biases i n categorization

355

geometric figures that differed along the following dimensions: form (the values being circle and square), colour (red and blue) and size (small and large). A category could then be defined by all red circles, for instance, leaving size as an irrelevant dimension. The subject was asked to categorize each figure presented. In a reception paradigm, the experimenter determined which stimulus was shown, whereas in a selection paradigm the subjects were to choose the next figure themselves. Following each categorization the subject was told whether or not the response was correct. Learning took place from this feedback information. It was inferred that the subject had identified the category when no more categorization errors were made. Learning trials continued until the subject consistently responded correctly. Bruner et al. showed that in this task there is a bias regarding strategies of category learning. People do not behave randomly, but appear to prefer certain actions above others in a systematical way. Most people choose one of two strategies in formulating hypotheses and selecting subsequent stimuli for testing as far as a selection paradigm is concerned. A sizable number of subjects adopt a wholistic or focusing strategy. That is. they formulate a global, composite hypothesis based on the first category member they encounter, and use this hypothesized rule in making subsequent responses until they encounter a n instance that refutes the rule. Another strategy, adopted less frequently, is to formulate hypotheses involving only a limited number of features of the first example presented, rather than all of them. Such a strategy is called scanning. The former strategy seems to be the optimal one in this type of task, because the demands it makes on the subject's memory are not as high as the latter. In general, people show a preference for the strategy that requires the least memory load, that is, focusing. Nonetheless, the scanning strategy was also found to be used in Bruner's experiments. With the paradigm mentioned above, other issues were investigated too. The influence of stimulus factors like the number of relevant and/or irrelevant dimensions, the number of values on the dimensions, and the type and complexity of the rule defining the category, was studied, a s were procedural factors such as the order of presentation of the stimuli and the amount and nature of feedback to the subject. The investigations also dealt with the way in which hypotheses are revised on the basis of feedback.

356

E . Das-Smaal

Many of these variables and their combinations have been shown to affect categorization learning (Bourne, Dominowski, & Loftus, 1979). Categories defined by different rules relating the relevant features appeared to vary in difficulty. Also, it generally appeared that subjects enter a category learning problem with a bias favouring conjunctive (e.g. red and circle) over disjunctive (red and/or circle) rules. Subjects more frequently state conjunctive rather than disjunctive categorization rules when both are possible (Hunt & Hovland, 1960). and irrespective whether a conjunctive or a disjunctive rule applies, they tend to start testing conjunctive ones (De Swart & Das-Smaal, 1976). However, as we shall see with many factors in categorization, rule difficulty is not quite independent of the particular stimulus context in which it occurs. Pairs of features may or may not go together naturally, and this influences the ease with which either conjunctions or disjunctions are formed (Dominowski & Wetherick, 1976: Reznick & Richman, 1976). Regarding the search for category rules, Martin and Caramazza (1980) in a more recent experiment presented evidence t h a t subjects develop rule systems also when learning less well-defined categories. In their experiment, there was no simple set of features that could be used to determine membership of all exemplars of the category. In that case, subjects appeared to look for rules that would enable them to categorize the stimuli as easily as possible. Because the categories were structured so that no simple rule could serve to categorize the stimuli, subjects were forced to develop complex sets of rules if indeed they preferred to develop rules. Martin and Caramazza showed that this was what they actually did, even in a complex situation. We shall return to rule development and hypothesis testing as a source of bias later, in connection with selective attention. The studies mentioned above showed the existence of bias regarding strategies of category learning and the search for categorization rules. The question of the nature of the information that is stored in memory when a category is learned did not receive much attention in the earlier studies. It was taken for granted that categories simply consist of "defining" values and that categorization is based on rules for combining these values. The emphasis was placed on the learning process rather than on structure. In about the last fifteen years, the trend of research on categorization changed towards focusing on the issue of what is stored in memory, thereby questioning the idea of defining value representation. The

Biases in categorization

357

shift went together with more interest in natural, real-life categories. 2.2 Conceptions of categories

According to what was to be called the "classical" conception of categories mentioned above, it was assumed that all members of a category share one or more defining features. If there is more than one common feature, then a certain relationship among the features is specified in the definition. Thus, in this view, each category has its own definition that provides necessary and sufficient conditions for assigning category membership. Features that are not included in the definition are irrelevant to category membership. By experience we learn which features are the ones that define the category, and by what rule they are connected. Following learning. the categorization decision about new items is based exclusively on whether or not an item fulfills the conditions in the definition, and this is an all-or-none matter. This way, the category boundaries are determinate and well-defined. Also in this view, any one exemplar of the category should be as representative of the category as any other. An example of a well-defined category is the category of triangles, or sisters-in-law, or odd numbers. However, not all categories are that strictly definable. Many categories may not conform to the classical view. I t appears, for instance, that in all kind of tasks non-defining features of a category can also play an important role (e.g. Rosch, 1975; Das-Smaal & De Swart, 1984, 1986). Non-defining features may contribute to differences in typicality among category exemplars. An example is the category of fruit. An apple is, at least to a Westerner, a better example of the category of fruit than a mango. Furthermore, the category boundaries are not always quite clear. Not everyone is certain about whether a tomato, a coconut or a n olive should be counted as a fruit. The boundaries between categories like fruit and vegetables are vague. Vague boundaries and typicality differences are not in keeping with the classical view of categories. This has led many investigators to alternative views, mainly prototype ones. In the prototype approach. the representation of a category is also taken to be the result of an abstraction process, and it is generally assumed that different category exemplars are integrated into one memory structure, i.e. a summary representation of the category. However, in contrast to the traditional view, it is thought that

358

E . Das-Smaal

category representation is not restricted to a set of defining features. For a feature to be included in the summary, it need only be characteristic of the category. I t was Rosch's work (1973. 1975) t h a t applied prototype theory to natural categories, the implicit concepts of daily life. Rosch and her co-workers have demonstrated the existence of prototype effects in these categories. They showed that natural categories have a graded structure in that some members are usually regarded as prototypes or typical exemplars, whereas others are considered less typical of the category. More specifically, Rosch hypothesized that typicality of a category member is determined by its family resemblance to other members of the same category. A high family resemblance means that a large number of features are shared with the other members, while at the same time few features are shared with contrasting categories. Thus, the best exemplars of one category will not be good representatives of other comparable categories. Typicality effects not only occur in natural categories like fruit and vegetables, but also in ad hoc constructed and goal-derived categories (Barsalou, 1983, 1985). In addition, typicality effects are also reported to occur in well-defined categories. Das-Smaal & De Swart (1981) were able to show this for a number of different tasks. Categories that were constructed according to the traditional view - with membership being sufficiently specified j u s t by defining features -- were nevertheless learned in a way that favoured typical members over less typical ones. Bourne (1982). Armstrong, Gleitman, & Gleitman (1983) and Vandierendonck (1989) also demonstrated typicality effects in classically defined categories like for instance odd numbers. A view according to which no aspect of within-category variation is represented does not account for these effects. A prototype position offers a better explanation of the results. There are a number of different interpretations of prototype effects. A fairly common view, following the work of Rosch (1973, 1975). is that a prototype is a n abstraction, a n integration of exemplars in some kind of memory structure. Category membership can be gradual, and degree of membership of a n exemplar is determined by its similarity to the prototype. Later, Rosch (1978) herself became somewhat reserved on the issue of prototypes as a theory. Her position by 1978 was that prototype effects do indicate that prototypes, or best examples, must have

Biases in categorization

359

some place in psychological theories of representation, processing, and learning, but that they do not in themselves constitute any particular representation or process model. This point is stressed by Lakoff (1986). Lakoff, however, goes further and states that although he acknowledges that prototype effects are real and that even classically defined categories may give rise to prototype effects, such effects are mere shadows of cognitive models. They show nothing direct about the nature of categorization. Prototype effects are a result of the fact that knowledge is organized in terms of cognitive models. Cognitive models are used to structure and make sense out of our experience. There are various kinds of cognitive models, and hence prototype effects may come from different sources. Although direct correlations between conceptual structure and prototype effects do exist, in other cases the effects may arise indirectly. They do not necessarily correspond to degrees of membership of exemplars. A full study of category structure therefore must provide a n account of the details of the cognitive models that give rise to the effects. Other "post-Roschian" theoretical proposals also stress the importance of theories or mental models in categorization. In general, it can be stated that since Rosch's original publications, the theoretical thinking about categorization has been moving away from an emphasis on objective features and similarity to a n insistence on the role of cognitive models in categorization behavior. The influence of cognitive models provides a n obvious source of bias, which will be discussed later in the sections on prototypes and schemas. Suffice it to say here that theoretical development will undoubtedly lead to a richer view of intra- and inter-concept relationships than that advanced so far (e.g. Medin & Wattenmaker, 1987; Neisser. 1987). Besides the classical and prototype approaches, a third conception of categories is the exemplar view. According to this idea, categories are represented by the collection of individual exemplars someone knows. In this case there is no abstracted representation of a category. Rather, categorical knowledge consists of a n extensional description of the set of experienced exemplars. Categorization of an unknown exemplar is determined by the degree to which it is similar to a n already experienced exemplar. The weaknesses of the exemplar view were outlined by Das-Smaal (1986). One argument against exemplar representation is for instance the fact that specific exemplars are often poorly

3 60

E . Das-Smaal

remembered (e.g. Bourne & O'Banion, 1969). The exemplar view also encounters problems in representing generalized knowledge that pertains to exemplar features, such as knowledge on conjoint frequencies or relevancy of features to the categorization decision. As for the latter aspect, Das-Smaal & De Swart (1981) showed in a set of experiments that the degree of representativeness of features to their "central" value (e.g. the similarity of the shape of a flower leaf to the most typical form for that flower) influenced category learning, but that this effect was restricted to features that were relevant to the categorization decision. Relevancy was determined by frequency of occurrence in one category relative to occurrence in other categories. The discrepancy in effects of relevant versus irrelevant features was thought to be explained best by a n interaction of selective attention, guided by expectations, and the knowledge that is built up (Das-Smaal, 1986; Das-Smaal et al., 1987). In the course of learning, the information that has been gathered is used to direct attention to features deemed relevant. Features considered irrelevant to category abstraction may be represented incompletely or may not be remembered a t all. Without abstraction of relevant features this phenomenon is hard to explain. In fact, it is not clear what constitutes a category a t all when only exemplars are represented, as Smith and Medin (1981) pointed out. Also, there is a problem about what is done when only summary information is given and no exemplars are experienced. Categories may be learned by being stated rather than via examples. Some kind of abstraction seems to be required, although it may certainly be the case that some individual exemplars are also remembered and used in subsequent categorical judgments (see for instance Brooks, 1987). Perhaps a n exemplar view combined with a new approach to memory processes can overcome the objections mentioned above. Recently, Hintzman (1986) put forward such a theoretical approach. Although he does not preclude the existence of abstract memory representations, Hintzman maintains that they have no special status or function. It is assumed that each specific event to which one attends, is encoded as a new episodic memory trace. The theory differs from other approaches to memory in that information is abstracted from concrete experiences at the time of retrieval rather than during learning. A new experience gives rise to a process of activating or reminding of other similar experiences, and it is the summed contribution of these activated traces that is used

Biases in categorization

361

to categorize the new experience, by analogy. The point is that abstraction does occur, but only momentarily, as a result of activation by a new experience. I t need not be built up during learning, and stored explicitly in memory. A variant of this view was put forth by Das-Smaal (1986, pp. 121-123) to account for the effects of context and of the flexible use of categories found in her experiments. A distinction was suggested between the information that is stored to represent a category, and procedures for using that information. Categories do not exist in memory in a ready-made format. I t was contended that in order to be able to account for a number of phenomena regarding categorization behavior, the constituent parts of the category information should be separately available in memory. During learning, it appears that attention, guided by our schemas or theories and expectancies, does play a n important role in determining which aspects will be represented and which ones will not. Next, computational mechanisms make use of the information in a flexible way, as needed, dependent on task requirements or the utilization goal. For instance, a certain feature can be highlighted in a metaphor or primed by advance activation of a category to which it is related. Whenever a category member is experienced, it gives rise to a momentary activation pattern, which in turn results, in some specified way, in the activation of categories. Once a category is learned, excitatory and inhibitory relations among correlated features and contrasting categories, respectively, are assumed to contribute to the resulting activation pattern. Furthermore, due to contextual factors certain features may be temporarily more important than others. That is, these features receive some extra activation, while others may become simultaneously inhibited. Contextual factors are used in a broad sense here, and may also imply the "internal" context of features activated by thought or experience. Regarding context, it was Barsalou (1982) who made the distinction between context-independent and contextdependent information in categories. Context-independence arises automatically from a high frequency of occurrence of features in a category as experienced by a person. Context-independent information is activated on all occasions. Context-dependent information, on the other hand, refers to properties that are activated only in certain contexts. In his more recent work, Barsalou (1985, 1987) also stresses that representations should not be viewed as invariant structures, but as dynamic ones that vary

3 62

E . Das-Smaal

across contexts. Whenever a category is used, it is constructed in working memory from knowledge in long-term memory. Contextindependent information may automatically be part of it. What other information is used depends on the situation. We shall return to the biasing influence of context later in this chapter.

2.3 Contrasts and combinations The theoretical views described above were only summarily dealt with here. Many variants have been proposed. Prototype models may differ in what is assumed to be abstracted. This is a n important issue regarding biases. What exactly do we learn from experience with a category's members and nonmembers? One could assume, for instance, that feature experience frequencies are abstracted (e.g. Neumann, 1974). or distances among exemplars (Posner & Keele, 1970), or an integrated pattern of propositions plus an allowable set of transformations (Bransford & Franks, 1971). Among feature frequency models a further distinction is possible between representation of simply independent features and their frequencies (Kellogg, 1981). and representation of both simple and conjoint frequencies of features (Hayes-Roth & Hayes-Roth, 1977). In addition, combinations of the views have been proposed, for example, in Medin & Schaffer's context-theory (1978) in which both exemplar-representation and a more abstract representation are hypothesized. Frequency- and prototype-distance models appear not to be mutually exclusive (Das-Smaal & De Swart 1984, 1986). A continuous variable dimensional value like the color red may show u p within a category (e.g. apples) in several different variants. These variants of the value red differ in the degree to which they are typical of the category. However, there are also differences regarding the relative frequency with which the value red occurs within the category. The frequency variable has its own clear and specific effects on category learning and its results, independent of the effect of variant typicality. In short, frequencyand prototype-distance models that account for effects of frequency and variant typicality, respectively, may be complementary to one another. A frequency model that assumes continuous dimensions divided into intervals, and as with discrete values, representations of frequencies of these intervals, accounts for both effects mentioned (Das-Smaal & De Swart, 1986). Moreover, such a model unites the classical and the prototype approaches in the sense that

Biases in categorization

363

the s a m e principles of categorizing underlie both "classical" categories, with defining features, and categories with features that occur often b u t not necessarily always among category members. "Definingness" in a strict sense should then be replaced by some related but less rigid measure of variability, such as the degree to which a feature is representative of a category. This avoids the problem of whether or not people are able to specify defining features or whether or not they ultimately have defining features represented. At the same time it has the advantage of accounting for some cases of differences among category members as to how good a n example they are. The problem that then remains is j u s t what it is t h a t determines variation in representativeness or typicality. Differences in typicality have been found in a wide range of category types, and it appears to be a very important variable in relation to all kinds of tasks having to do with categories. It is therefore important to understand the constraints related to these phenomena. We shall turn to this subject now. In the next sections we take u p some basic issues in preferential ways of learning and using categories. 3. BIASES IN CATEGORIZATION

3.1 Prototypes From the preceding discussion it can be concluded that the work of Rosch and her associates h a s suggested very interesting issues that are relevant to preferential biases in categorization. Although it should be kept in mind that many other factors are involved i n learning and determining category membership, one very important factor h a s been described in the literature as the usage of prototypes. It has been argued that many categories are represented by prototypes, with exemplars varying in their similarity to the prototype. The implication is that people do not act equally toward all members of a category. A chair is a better example of furniture than a lamp or a piano. People are much faster at determining category membership for typical members than they are for atypical members. Also, when thinking about a category, typical members come to mind with a higher probability than less typical ones. Prototypes may furthermore serve as reference points in similarity judgments (Rosch, 1975). We tend to select the prototype as a referent, and the other item as a subject. The

3 64

E. Das-Smaal

reference point is the stimulus which other stimuli are seen "in relation to". For instance, we say that a panther resembles a cat rather than that a cat resembles a panther. "versky and Kahneman (1983) showed how the use of such reference points can lead to biases in reasoning. People employ typical cases as the basis for their judgments, and inferences are made from typical to nontypical members. Also the results of comparisons between items appear not to be invariant with respect to the directionality of the comparison. Thus, the choice of a reference point may influence one's judgment. The issue of biases in reasoning is discussed extensively elsewhere in this book. What sorts of biases are further implied by prototype theory? Followfng her observation that natural categories are organized around a prototype, two general and basic principles for the formation of categories were given by Rosch (1978). The first is a functional one and asserts that people are biased to develop category systems that provide maximum information with the least effort. The second regards the structure of the information thus provided, and implies that the perceived world comes to us as structured information rather than as arbitrary, or unpredictable features. The first principle states that categorizing is aimed at what Rosch calls "cognitive economy". This means on the one hand that categories should preserve information about the environment as much as possible, but on the other hand -- to minimize cognitive load -- that they should reduce the infinite differences among stimuli to manageable proportions. In other words, the aim is to maximize the information accounted for, and to keep at a minimum the number of categories that have to be distinguished. Economy also implies a maximization of intercategorical differences, so that the categories will be as distinctive as possible. In this context, the conception of hierarchical organization of categories is of relevance. Rosch claims that natural categories are hierarchically related. Larger categories usually contain a number of smaller ones. Three levels are distinguished. Superordinate categories, such as clothing, contain basic-level categories (e.g. trousers), which in turn contain subordinate categories (e.g. Levi's). Cognitive economy is maximized a t the basic level. The basic level is the level at which the objects share the most features with the other members of the category. At lower levels, the features of members of a particular category most frequently overlap with features of members of other categories. That is. most of them are distinctive,

Biases in categorization

36 5

whereas at superordinate levels there is less total similarity within a given category because there are relatively few features common to all members. Basic levels categories like car, table, or bird are the ones that languages tend to denote by single nouns, the ones that children learn first, and the ones we use most often in talking about the objects around us. Rosch maintains that her claims concerning a basic level of abstraction can be formalized in terms of cue validity (e.g. Beach, 1964). Cue validity is a probabilistic concept indicating the predictive validity of a feature of a category, and is based on the frequency of occurrence of features in both the focal category and other categories. A category exemplar with a high total cue validity is differentiated more from other categories than one of lower total cue validity. Cue validity is maximized at the basic level. Besides the first principle of categorization, cognitive economy, another principle was mentioned by Rosch. This second principle asserts that the environment is perceived as possessing a correlational structure. The combinations of what we perceive as features are not equiprobable. Rather, some features co-occur more than others. Rosch argues that we tend to form categories that mirror the structure perceived in the environment, although she recognizes in her later work that it may also be true that this structure is something that is imposed on regularities in nature by our conceptualizing minds (Rosch, 1978). Evidence that people indeed are sensitive to cue validity and correlated features will be presented separately in later sections. In t h e description of Rosch's basic principles of categorization, it was assumed that category representations can be decomposed into features. However, it should be mentioned that in the literature there is uncertainty as to exactly what features are. Features can be concrete or abstract, and they may be categories themselves too. Their perception may be the result of a built-in constraint of our perceptual system, or they may count as a feature due to theory-related constraints. That is, our theories determine them as relevant because of convenience to our way of thinking about the world (Medin & Wattenmaker, 1987). We will not go further into the question of what these features are. This, however, is an important issue that requires further clarification. Application of the prototype approach has not been limited to object categories. Research has extended it to different domains, where it serves as a useful organizational framework. Psychological

366

E . Das-Smaal

constructs like intelligence (Neisser, 1979) and emotion (Fehr & Russell, 1984) show a graded structure, as do categories of personality traits (Buss & Craik, 1983; Smid & Zwinderman, 1986) and categories of social perception (Cantor, Mischel, & Schwartz, 1982; Cohen, 1983). In applying prototypes to personality research, various traits, features, behaviors, or situations are distinguished according to the degree of typicality relative to a concept from this domain. It appears, for instance, that people consider excitement as a more typical feature of emotion than boredom. In this way, researchers try to put order in the contents of these constructs, which are often vague. In addition, the results are used to design questionnaires (e.g. Broughton. 1984; De Jong, 1988; Visser & Das-Smaal, in preparation). Here the prototype idea has also been shown to be a fruitful approach. The biases mentioned in the beginning of this paragraph concerning the non-equivalence of different category members are put into practice in these domains. To summarize, two important categorization principles mentioned by Rosch serve to bias or constrain the categories that we form. These are striving for cognitive economy, which is maximized a t the basic level, and sensitivity to correlated features. The implication of the principles of categorization is t h a t to increase distinctiveness among categories, categories tend to be formed around prototypes. Prototypes represent the features that are not only the most representative of the category members, but also the least representative of nonmembers. People do not consider all category members as equivalent, but are biased to focus on prototypes in a number of cognitive activities. 3.2 Schemas and levels of cognitive functioning A kind of bias stemming from prototypes that has not been mentioned is the development of specific expectations resulting from the activation of a prototype. In this respect, the term schema is often used. The use of a schema framework has become widespread in psychological research. At the same time interest is now growing in complex, composite categories (Millward, 1980; Mandler, 1984; Medin & Smith, 1984). Research on categorization has been dominated by the use of simple categories. However, these categories can be further integrated and related to each other to form more complex organizations of knowledge. Such elaborated,

Biases in categorization

3 67

interconnected knowledge structures are most generally referred to as schemas, although the schema notion implies also a n active organizing principle. Not just categories can be coded by schemas. Events, stories and scenes can also be represented by schematic forms of organization. Prototypes can be considered as a sort of schema. To a certain extent some variation in the objects that might fit a particular schema is allowed, and constraints as to what the typical features are also exist. Schemas in general can be conceived of as conceptually organized clusters of knowledge. They lead to expectancies that guide both the comprehension of what we perceive, and the planning and execution of our actions. People can instantiate schemas in the absence of external evidence, and these instantiations are referred to as their default values (see for instance Mandler, 1984). In making category judgments, people often rely on prototypical schemas. Social stereotypes can be considered to function in a n analogous way. Social stereotypes are frequently used to stand for a category as a whole, for instance people of a certain race. As such, they may lead one to jump to conclusions about individual members of that category. This may be viewed as the application of features to the entire category that actually apply only to the stereotype. In terms of schema activation it can be stated that only partial evidence can activate a whole stereotype schema. This also implies the activation of features that are not part of the current evidence, by default. Likewise, schemas in general are used to guide the interpretation of what we perceive. It is a commonplace observation that people tend to perceive what they expect to perceive. In this way, knowledge puts important constraints on perception and categorization. The idea t h a t stimulus analysis is guided by expectations is central to schema theories (Neisser, 1967). Current experiences are assimilated with past experiences, and what we subsequently perceive is influenced by these experiences. Thus, perception contains the memory of earlier encounters, as Arnold (1984) put it. As a result, the accounts of stimuli may be different from the stimuli that evoked them. Das-Smaal's (1986) findings on the effects of typicality-range experience, which will be described in the next paragraph on category boundaries, serve to illustrate this point. In a category learning experiment, two groups of subjects differed in the range of typicality over which the features of the stimuli were varied during learning. The groups subsequently

368

E. Das-Smaal

appeared to respond in different ways to the same inter-stimulus distance. The "psychological" distance between the stimuli from two categories was larger when a small rather than broad range of typical variants was experienced during learning. This shows that prior experience can make a significant difference in typicality judgments and in subjective intercategorical distance. The observation that people use schemas to infer that certain unobserved and unmentioned features must be present is a n obvious source of bias. I t was Bartlett (1932)who applied the concept of schema to the domain of memory, in particular in relation to text comprehension. He pointed out that part of what we remember is our understanding of what we have experienced. Perception and memory should be viewed as constructive and reconstructive activities, respectively. Bartlett showed that subjects recalling a rather bizarre story exhibit systematic inaccuracies in memory. They distort the story to fit their own stereotype schemas. It is currently assumed that schemas are produced by an interaction between what we experience in the actual world, and our expectations driven by other activated schemas. Both factors can be very personal, especially regarding the affective components. Therefore, one might expect especially this source of bias to have a personal aspect. However, in the case of socially determined stereotypes, as no personal experience is implied and the stereotype schema is typically shared by a group of persons, this personal aspect by definition pertains to certain groups of people. Recently, much attention has been paid to the importance of the relationship between category representation and other levels of cognitive functioning. For the higher levels, for instance, Murphy and Medin (1985)and Medin and Wattenmaker (1987)emphasize the influence of people's theories about the world on categorization. This idea has already been touched on earlier in this chapter. In their opinion, categories should be viewed as embedded in coherent, integrated knowledge systems, that in turn are based on perception, memory, and imagination. Categories are coherent insofar as they fit into someone's beliefs and knowledge. Implicit theories constrain our understanding of relationships both within and among categories to a high degree. That is not to say that these are the only constraints affecting categorization. Although theoretical factors are believed to be important, other factors, like the use of primarily perceptual information, play a role too. According to Murphy and Medin. feature models, in principle, can

Biases in categorization

369

encompass the important aspects of general knowledge, but this has not been accomplished yet. The schema notion h a s also been described in terms of activation patterns in a parallel distributed processing approach based on the brain metaphor (Norman, 1986). This approach has recently become increasingly popular. Low-level processing structures are assumed to result in higher order regularities in the following way. Information is represented by features, and each type of feature is represented by some unit in memory. Each stimulus attended to gives rise to a pattern of activation of units, and each time, this changes the strength of the connections among simultaneously occurring units. A unit may be involved in various events. Therefore, memory is said to be distributed across feature units. Units are functionally autonomous, but may show parallel activity. Categories are coded in terms of patterns of connections among units. A category has its own specific pattern of activation that mirrors the regularities of experience. Schemas or prototype patterns develop if many similar activation patterns are superimposed. This results in a composite trace that functions like an abstraction. Schemas are not fixed, permanent cognitive structures, but temporary patterns of activity. A similar pattern can be reinstated at another time. However, each re-creation may differ somewhat from the previous ones. The schema is constructed anew for each occasion by combining past experiences with biases and activation levels that result from the actual stimulus and context (Norman, 1986). Therefore, in this approach, category flexibility is accounted for, since the context is part of the constellation of activated units. This is an obvious advantage in view of the biasing influence of context. That is, categories may vary as the current context varies. For future theoretical development, it is a n interesting idea that the assumption of distributed memory representation is not necessarily at odds with models of categorization (Knapp & Anderson, 1984; Oden. 1987). 3.3 Category width and context

The description of categories in terms of prototypes with some degree of allowable variation among members raises the problem of defining category width. How much discrepancy is allowable among members of a category? Can constraints on boundaries between categories arise as a result of learning alone,

370

E . Das-Smaal

and if so, how? What are our biases in this respect? This issue has not received much attention. Some research has indicated that the category width variable experienced during learning does have a n impact on subsequent categorization. I t is also true that perceived boundaries are context-dependent. Furthermore, as we shall see, category ranges themselves can have a biasing influence on similarity judgments among items. Although categories are formed in bounded representations, boundaries are not always clear. I t was Rosch (e.g. 1975) who stressed this aspect of natural categories. On the other hand, in the prototype approach, the question of t h e ultimate range of discrepancies among category exemplars that a person is willing to accept was left open. Differences in degree of membership among exemplars was the main point dealt with in this approach. However, it is obvious that people may differ as to the ranges of their concepts of few, several, many, or a lot, for example. What are the minima, and what the maxima of each of these concepts? Context may be of relevance here. People do not always have to react to an object or a n event simply in ways typical of some category to which the object or event belongs. People also have to respond to new aspects of the total situation at hand. They must be able to adapt their behavior to specific circumstances. Context may provide a clue as to how the category should be apprehended in a particular situation. Context may cause certain features to be temporarily considered as more important than they would otherwise be. A study by Labov (1973) provides a n illustration of fuzzy boundaries and contextual influence. Labov studied the boundary between the cup and bowl categories. He was interested in which items subjects would call cups, and which ones they would call bowls, when given a series of items. The items increased in their ratio of width to depth. Subjects were asked to classify pictures of these objects. With increasing relative width, there was a gradual shift in responses from cup to bowl, but there was no clear-cut point where subjects stopped using cup. Even more interesting was that when subjects were asked to imagine the object placed on a table and filled with food, more bowl responses were given, although the same gradual shift appeared from cup to bowl. Thus, boundaries not only can be unclear, but they can also be influenced by the context in which something is placed. This shows that boundaries are flexible. Regarding individual variation in boundaries, McCloskey and Glucksberg (1978) showed that boundaries for categories like

Biases in categorization

371

fruit or disease vary among people. It even appeared that people themselves are not stable in their opinion. Many subjects changed their mind about the categorization of some atypical members when tested one month later. Although there seems to be some instability about category width both among and within people, a number of researchers have shown in a variety of tasks that, broadly, people can be qualified as either relatively narrow, or relatively wide categorizers. That is, some people consistently categorize atypical items in the same category, whereas others consistently categorize these same items in different categories (Detweiler, 1978). According to Detweiler, these individual differences are the result of developmental, cultural, and experiential factors. Detweiler also suggests some effects of category width as an individual difference variable of behavior. For instance, narrow categorizers show better face recall, adjust less well to different cultures, are more ethnocentric, and make different attributions to foreigners than to non-foreigners (Detweiler, 1975, 1978). Regarding biases, it is important to note that differences in category width may yield different categorizations of a particular object or event. Thus, the same stimulus may be interpreted differently by different persons. The foregoing remarks on context and individual differences concerned boundaries once they are formed. Das-Smaal and De Swart (1984, 1986) addressed the question of boundary formation. Their aim was not to ascertain individual differences due to styles of categorization. Rather, the possible influence of experience on category width was investigated. The results not only showed how category ranges can be determined by learning experience, but also how range experience influences the perceived within-category variation in typicality. In the learning phase, exemplars of a focal and a contrast category were composed of either a small range of typical variants only, or a broad range of both typical and atypical exemplars. As predicted, in contrast to narrow range experience, experience with a broad range of variants resulted in a relatively large extension of the focal category. It also resulted in better categorization of novel atypical focal category exemplars. This confirms Detweiler's idea that experience may determine the subsequent way of categorizing. Another effect shown in the studies by Das-Smaal and De Swart concerned the judged distance between categories and subjective typicality differences within categories. I t appeared that small as

372

E . Das-Smaal

compared with broad range experience had the effect of polarizing the focal and the contrast category. Thus, the same physical distance was judged differently, depending on range of experience. This polarization effect counts as a source of bias in category learning, caused by the category width experienced during learning. In addition, a recurrent finding in the experiments was the compression of the focal category following broad range experience. That is, members of the same category were felt to look more alike in this condition. To test whether this effect was indeed due to the width of the category, and not to the greater number of variants, in the later study a third range condition was added. This condition implied broad range experience, but with a limited number of differing variants. The results were essentially the same as with a broad range, indicating that the range was the crucial factor. To summarize, the influence of the category range experience factor can be characterized in the following way. Learning experience with a broad rather that narrow range of variants results in enhanced discriminability across a category boundary, and in decreased discrimination within a category. A similar range bias has been reported lately by Conner, Land and Booth (1987). 3.4 Feature frequency

In paragraph 3.1 on prototypes, it was stated that Rosch suggested maximization of cue validity to be a determining factor regarding the internal structure of a category. According to this principle, the best examples of categories are those exemplars that have the most in common with other members of the same category. and share the least with contrasting categories. I t is the latter addition that constitutes a n important research issue. When people learn a category, do they use information on contrasting categories at all when forming the focal category? And if so, then what information do they use, and what effect does it have on the category to be learned? I t is often assumed that in learning a category, the learner keeps track of the frequencies with which features occur within t h a t category. Indeed, there is substantial evidence for accumulation of frequency information during category learning (e.g. Neumann. 1974; Goldman & Homa. 1977; Hayes-Roth & HayesRoth, 1977; Chumbley et al.. 1978; Kellogg et al.. 1978). One could argue, then, that in a category representation the features

Biases in categorization

373

represented are the ones that occur most often among category members. This is in fact what Neumann (1974)for instance proposed in his attribute-frequency model. According to his model, a category prototype is formed that contains the maximal number of the most frequently experienced features. Occurrence of features within one category certainly is of relevance in the representation of that category. However, it could also be the case that precisely the same things that distinguish one category from other categories make u p the most important information. In this view, when a category is being learned, the learner pays particular attention to those features that provide the sharpest contrast with other related categories. These so-called distinctive features may then become central to the category representation. In that view, a category representation not only contains information on occurrence in the focal category, but also on occurrence in contrasting categories. The latter information can of course only be learned when contrasting categories are experienced in addition to the category to be learned. In the latter view, contrasting categories are useful in a learning phase because they provide information on the occurrence of features outside the focal category. Distinctiveness from other categories is not always accounted for in categorization models. I t is however accounted for in models that have cue validity as a critical term, such as the property-set model proposed by Hayes-Roth and Hayes-Roth (1977). As mentioned earlier, cue validity can be defined as the frequency with which a cue, or a feature, occurs in one category, divided by the total frequency of that cue across all categories. Cue validity thus takes into account resemblance within a category, as well as distinctiveness from contrasting categories. In the category learning tasks of several experiments by DasSmaal and De Swart (1984,1986).feature frequency was varied in both the focal and contrasting categories. As for occurrence in the focal category, a high frequency of features was assumed to facilitate categorization performance. In addition, the influence of occurrence in a contrasting category was investigated extensively on the hypothesis that the frequency of the features in the contrasting category would affect categorization performance a n d representativeness ratings. Category exemplars were expected to be categorized more easily and judged as more representative as the frequency of their composing features in the contrasting category was lower during learning. This effect was studied both by varying

314

E.

Das-Smaal

frequency in the contrasting category and by either including or not including a contrast category in the learning task. Frequency of occurrence of features in the focal category, a s compared to their occurrence in the contrasting category, was expressed in terms of cue validity. The question was what do subjects learn from this state of affairs about the representativeness of a feature: i.e. to what degree is it used and relied upon in subjective judgments following learning? The influence of the frequency of feature occurrence among contrasting category items on categorizing focal category exemplars was a very robust finding, coming from converging sources. It was demonstrated repeatedly, employing a variety of learning paradigms, testing methods, and dependent variables. In all experiments, high total cue validity of exemplars facilitated learning and classification following learning. It also appeared that when learning had to take place from focal category exemplars only, without experiencing a contrast category, categorization was guided by the focal frequency of the features. When features with the same cue validity but with different frequencies of occurrence in the focal category were compared, focal frequency showed its own influence, apart from cue validity. Features of higher frequency were judged as more typical. However, distinctive features -- that is, features that occur frequently and exclusively in the focal category -- appeared to be the most typical and most important ones to classification following learning. Delaying testing proved that this was all the more true with the passing of time. The apparent conclusion from the findings is that people collect considerable knowledge of the frequency with which features occur in particular categories during learning. This knowledge is not restricted to j u s t the focal category, b u t also extends to the contrast category. People are able to use this knowledge in order to evaluate relative frequencies, t h a t is. occurrence in one category a s compared with occurrence in a contrast category.

3.5 Correlation among features Another controversial issue regarding frequency of occurrence is whether or not subjects learn about each feature in a n

independent manner from the other features. The question is: Does

Biases in categorization

315

frequency of co-occurrence of features affect category formation? Furthermore, if people are sensitive to correlations among features, does this mean that all correlations are registered? Rosch et al. (1978) have claimed that the perceived world comes as structured information rather t h a n as arbitrary, equiprobable co-occurring features. Features that characterize the members of natural categories are often correlated, and these conjoint frequencies are mirrored in the categories that people form. The suggestion that conjoint frequency affects categorization was also made by Hayes-Roth and Hayes-Roth (1977) and by Medin and Schaffer (1978). Murphy and Medin (1985) also state that correlations are represented in mental categories, although they need not always be based on empirical relations: they can also mirror a person's ideas about the relationships. The point of correlational structure has also been emphasized by Anderson (1985). Anderson describes the structure of natural categories as a schematic structure, and asserts that schemas represent (among other things) our knowledge a b o u t how dimensional values tend to go together to define objects. Thus according to Anderson, it is the interrelational structure, the configuration of dimensional values, rather than j u s t a list of values that defines a category. On the other hand, Kellogg (1981) argues that stimulus features are independently represented, and that only simple frequencies are counted. Kellogg's study, a category learning experiment, failed to produce conjoint frequency effects. In his study, however, the features upon which conjoint frequency was vaned were all irrelevant to categorization. The possibility remains that conjoint frequency effects will show up when relevant features are concerned. This question was tackled by Das-Smaal & De Swart (1986). In that study, it appeared indeed that the effectiveness of conjoint frequency shows u p when at least one of the features is important to categorization. Conjoint frequency increases the judged typicality of a n irrelevant feature when it correlates with a feature that is important to categorization. This explains why Kellogg (1981) did not find any conjoint frequency effect using features that were all irrelevant to categorization. Moreover, this is another indication of a source of bias in category learning. The assumption t h a t effects of conjoint frequency are constrained to situations in which at least one of the joint features is relevant to or characteristic of the category also solves a major

37 6

E. Das-Smaal

problem noted both by Reed (1982)and by Medin and Smith (1984) and Medin and Wattenmaker (1987). They argue that it is not probable that all possible correlations among features are taken into account, because they are too numerous. The condition of relevancy of at least one of the conjoint features may in this respect appear to be a very important constraint on the correlations that are taken into account. Stated in terms of the view put forward by Das-Smaal (1986) mentioned at the end of paragraph 2.2 in this chapter, the following may apply. Excitatory relations between two irrelevant features do nothing significant to increase the activation of the category. However, when at least one relevant feature is concerned, the increased activation of the category is the result, facilitating categorization and increasing typicality judgments. The irrelevant feature activates the category by way of its relationship to the highly characteristic feature. Another interesting potential explanation bears on the Gestalt claim that "the whole is more than the sum of the parts". I t may be speculated that the formation of interrelations by conjoint frequency, and the resulting additional activation through mutual excitation, gives rise to some of these effects. The results of the present study indicate that correlation of features enhances the typicality of the "whole" stimulus containing the correlated features, relative to non-redundant combination. This shows that the conjoint occurrence of values during learning can cause the whole to be perceived as more than the sum of its parts. The existence of a predominant manner of organizing the parts of a stimulus may, in a comparable way, be due to special interrelations between features in memory, whether they are learned or physiologically "pre-wired". The effects of learning the interrelations have been established in the study by Das-Smaal and De Swart (1986). 3.6 Selective attention

From the study conducted by Das-Smaal and De Swart (1986) mentioned above, it appeared that typicality of an irrelevant feature increases only if this feature correlates with a feature that is important to categorization. This could explain why Kellogg (1981) found no effect of correlations, because in his experiment the features between which the correlations were varied were both irrelevant to the categorization decision. Results from other

Biases in categorization

371

experiments by Das-Smaal and De Swart (1981)indicate that with a well-defined category like the one Kellogg used, attention is directed at relevant features, not at irrelevant ones, or if so to a lesser degree. Typicality effects were restricted to features that were relevant to the categorization decision. I t should be noted here that typicality and relevance in these experiments were not the same thing. Typicality, as it was conceived of in these experiments, had to do with the range of a feature, whereas relevance had to do with its probability of occurrence. Features may vary in their frequency of occurrence, and this affects categorization as discussed previously. However, continuous features may also vary in the typicality of the variants with which they occur. For instance, the featore red may occur with a certain range of variants of this color, and these variants are not equally typical of red. I t appeared that category learning was facilitated by high rather than low typical feature variants when relevant features were concerned. No such difference was found with irrelevant features. The discrepancy may be explained by the existence of a n interaction between the selectivity of attention and the knowledge that is built up (e.g. DasSmaal et aL, 1987). In the beginning of category learning, there is no category information available in memory to direct attention t o specific features. The analysis of the stimulus is data driven, probably influenced by the salience of features. However, in the course of learning, the analysis becomes more "top down", guided by hypotheses or expectations. The information that h a s been gathered is used to direct attention to features deemed relevant, the latter being guided, e.g. by frequency. Finally, the learner will end up with detailed representations of information on at least the focused features. The information that is stored includes various details about frequencies as well as information on their typicality ranges, as has been shown in studies by Das-Smaal and De Swart (1984, 1986). Features considered irrelevant to category abstraction may be represented incompletely or may not be remembered at all. This is what a recognition experiment by Nickerson and Adams (1979) indicated. These authors showed that whereas knowledge of color and size of a U.S. penny is good, knowledge of its visual details is poor. Features such as color and size are relevant to the distinction between a penny and other coins. Additional details, on the other hand, are irrelevant to the purposes for which pennies are employed. Hence they apparently do not receive much attention.

378

E . Das-Smaal

3.7 Hypothesis testing

In the foregoing, selective attention was mentioned as a source of bias. Expectations or hypotheses contribute to directing attention to what is deemed relevant. Since this places a constraint on the kind of information that is gathered regarding a category, it specifies the bias of selective attention. The idea of hypothesis testing when learning categories was studied in experiments following the work by Bruner described earlier. The investigations dealt with the kind of hypotheses subjects entertained during learning and how these hypotheses were revised on the basis of feedback. A number of models have been developed regarding the way in which a subject might choose among a pool of hypotheses. Early models (Restle, 1962; Bower & Trabasso, 1964) assumed that the subject begins a learning task with a universe of hypotheses from which he draws one. This hypothesis dictates his response. The subject retains the hypothesis if his response turns out to be correct, but discards it and selects another one if his response is wrong. A refuted hypothesis is supposed to be returned to the pool. Later experiments refuted the assumption of no memory for previously tested hypotheses (e.g. Levine, 1966). A more elaborate version of hypothesis-testing theory was formulated by Levine (1969). This theory assumes that the subject begins a learning task by sampling a subject from the universe of hypotheses. He or she then takes one hypothesis from the subset a s the working hypothesis upon which his or her response is based. The working hypothesis is retained if the response is correct. If the response is wrong, the working hypothesis is discarded, and a new working hypothesis is chosen from the subset. If the subset is empty, the subject takes a new subset of several hypotheses and chooses a new working hypothesis from this subset. Other hypotheses from the subset are updated simultaneously after each feedback. These hypotheses are eliminated from the subset when they t u r n out to be wrong. In contrast to the no-memory assumption of the early models, Levine's theory assumed that subjects do eliminate hypotheses from the pool, at least for some time. In the course of time, subjects may forget that some of the hypotheses were not confirmed. These hypotheses then become part of the pool again. The weight of empirical evidence supports Levine's theory (e.g. Levine. 1975; De Swart & Das-Smaal, 1976,

Biases in categorization

379

1979a. 1979b; Bourne, Dominowski, & Loftus, 1979; Das-Smaal & De Swart, 1981; De Swart, Kok, & Das-Smaal. 1981). De Swart and Das-Smaal (1976, 1979a, 1979b) argued that Sokolov's (1969) model of the orienting reflex is a neural analogue of the cognitive process of hypothesis testing. The authors (1979b) found that the amplitude of the Skin Conductance Response (SCR) was higher when feedback during category learning confirmed a categorization response in which subjects had low confidence as compared with one on which they reported high confidence. Refutation revealed the opposite result: SCR was higher following a high-confidence categorization than following a low-confidence categorization. A study by De Swart, Kok and Das-Smaal (1981) replicated these results with the amplitude of P300 as a measure of changes in the probability of hypotheses. The findings of Das-Smaal (1986) mentioned earlier on the storage and use of information on frequency of occurrence in a categorization learning task can be reconciled with hypothesis testing theory by the assumption that frequency information is used to select hypotheses. Haygood et al. (1970) e.g., and more recently Kellogg (19801, have provided evidence of this view. One special theory on hypothesis formation, Levine's (1969) multi-hypothesis sampling theory, was supported in a study by Das-Smaal and De Swart (1981). An illustration of multi-hypothesis sampling in a reallife situation was given by Reed (1982). Reed describes how hypothesis testing is applied to a task that involves diagnosing medical problems. Results on how physicians attempt to diagnose a disease agree with Levine's theory. It appeared that physicians start to form hypotheses early in the examination. Then they monitor a subset of about three hypotheses a t a time, and this subset remains fairly constant through different stages of examination. The results show that more than one hypothesis or expectation can be active and evaluated simultaneously.

3.8 Task demands Ease of interpretation of new impressions is not the only function of categories. Categories are essential in thinking and problem solving. They also make conversation more easy, a t least if the involved categories have names and are subject to general agreement within a culture. Regarding categories, many cognitive judgments are possible. Das-Smaal and De Swart (1984, 1986)

380

E. Das-Smaal

presented their subjects, following category learning, with test tasks of different kinds. These were e.g. a categorization task and a task in which subjects were asked to choose the more representative one of two stimuli for the focal category. Differential performance on focal category exemplars, in particular those of lower total cue validity, was found consistently between these two tasks in all experiments. The results were taken to suggest that frequency in a contrast category is weighted more heavily in categorization tasks than in tasks that require judgments of the representativeness of exemplars relative to the focal category. Unlike in categorization decisions, differences among categories are not s o much the issue in the latter kind of task. Therefore, distinctiveness, as determined by occurrence in contrast categories, is deemed less important and receives less weight. This is not to say that the effect of occurrence in a contrast category is not important a t all in a paired comparisons important to a certain extent. task. Studies have shown that it However, the fact that the above disparity showed up repeatedly points to the relevance of task analysis. Comparable effects of task demands in categorization versus subsequent justification of a categorization decision are described by Medin and Smith (1984) and Landau (1984). Also, the results of Kemler Nelson (1984) and of Ward and Scott (1987) are very interesting in this respect. They found the same task effect as described above. In a traditional category learning task (intentional learning), subjects appeared especially to attend to the feature that maximized the difference between categories. For certain reasons, the intentional learning task in this study was compared with an incidental learning task. The latter task in fact boiled down to pairwise comparisons. The instruction was to determine which one of two exemplars was most typical of a category. In this task, in contrast to the categorization task, subjects granted no extra weight to the distinctive feature. A post-hoc interpretation of the findings on a more elementary level of theorizing may serve to integrate the results of Das-Smaal (1986) on frequency of occurrence in different tasks. Suppose the following. As a part of a momentary activation pattern, features activate categories in which they have occurred to the extent corresponding to the previously registered frequency of occurrence. Thus, a common feature occurring often in the focal category but also in a contrast category activates both the focal and the contrast category, although the focal category will be activated more. A distinctive feature, however, occurring often in the focal

Biases in categorization

381

but never in a contrast category, activates only the focal category. I t is furthermore assumed that contrasting categories inhibit each other when they are activated, a suggestion made by Wickelgren (1981)and Rumelhart and McClelland (1982). I t follows that the net activation of the focal category will be higher due to a distinctive feature than to a common feature, even if they occur with the same frequency in the focal category, as in the Das-Smaal (1986)studies. In a categorization task, the decision may then involve determination of which one of the categories at stake is activated most. A pairwise comparison task, on the other hand, concentrates on the focal category. It involves determining which one of the two exemplars is more strongly related to the focal category. In such a task the focal category may be primed, and this diminishes the relative influence of inhibition by way of the contrast category. The issue of task-dependence is, of course, not a new one in psychology. The point is that not much attention has been paid to the influence of task differences on the conclusions reached in much of the experimental research on categorization. One interesting implication of the findings on task demands mentioned above is that information on frequency in the focal category and information on frequency in contrast categories has to be separately accessible in memory, and not already joined in the measure of cue validity. Cue validity, then, may be either computed when necessary, or stored in addition to frequencies in the focal and contrast categories. Perhaps what is required is a distinction like the one made by Miller and Johnson-Laird (1976) between categories and procedures for using categories. Having knowledge of occurrence in contrasting categories does not imply that this knowledge is always used to perform. Thus the contention that a category consists of certain features is not enough. I t does not say how those features are weighted or integrated when people use that category in some particular situation. An account is needed of the manner in which a whole is constructed from the parts in each case. And as for storage, the information must be stored in such a way that it enables u s to use that information in a flexible way. Task and contextual factors may give rise to a kind of momentary relevant activation pattern in which certain values may temporarily be of more importance than others. The results indicate the importance of knowing which demands a task places on the subject. The effect of task difference

382

E. Das-Smaal

shows that category information can be used in a flexible way. I t is not very likely that all aspects of category representation can be investigated in one single type of task. Which demands are made by the various tasks in the current research paradigms is not yet clear. Systematic analysis and categorization of tasks is therefore necessary. In the future, this may prevent research partiality. 4. CONCLUSIONS

In this chapter, a number of important theoretical approaches have been presented regarding the question of how people are biased in the way they divide u p the world into categories. In addition, experimentally established sources of bias were analysed. Overall, perhaps the most striking observation in this overview is the fact that many of the constraints on human categorization behavior described in the principal theoretical alternatives seem to be implicated. The point is that they do not all occur under the same conditions, and there is a lack of knowledge about the way in which they are combined. There is evidence indicating the abstraction of schemas or prototypes, learning of feature frequencies both of the focal and contrast categories from the same domain, sensitivity to correlations among features, exemplar representation, category width effects, hypothesis testing, effects of selective attention, and for the very important role served by context, task and background knowledge and ideas. These processes serve to create a knowledge base regarding categories that can be activated in a flexible way. Categorization behavior then may be regarded as the result of some computational process which appears to be influenced by the cognitive constructs that we personally entertain in relation to the to-be-categorized object, the task at hand or the goal of the categorization activity, and the context in which it takes place. All of the factors mentioned above can bias our categorization behavior in certain directions. I t is a matter of further investigation to determine their boundary conditions. 5. REFERENCES Anderson. B.F.. Deane. D.H.. Harnmond. K.R..McClelland. G.H., & Shanteau. J.C. (1981). ConceDts in judgement and decision research. New York Praeger Publishers. Anderson, J.R. (1985). Coanitlve Dsvcholorn and its imdicatio- . New York W.H. Freeman and Company.

Biases in categorization

383

Armstrong. S.L., Gleitman, L.R. & Gleitman, H. (1983).What some concepts might 263-308. not be. Cormitlw, U, Arnold, M.B. (1984).Jvlemorv_and the B r u . Hillsdale. N J : Lawrence Erlbaum Associates. Barsalou. L.W. (1982).Concext-independent and context-dependent information in concepts. Memorv and ComUon. U, 82-93. Barsalou. L.W. (1983). dd hoc categories. Jvlemorv andCoenltio JJ, 21 1-217. Barsalou. L.W. (1985).Ideals. central tendency and frequency of instantiation a s determinants of graded structure in categories. Journal of m e r im e n t d rninu U. 629-654. Barsalou, L.W. (1987).The instability of graded structure. In U. Neisser (Ed.). ConceDts and conceDtual develoDment: Ecological and intellectual factors in rlzatiw. Cambridge: Cambridge University Press. Bartlett. F.C. (1932).p e m e m b e m . London and New York: Cambridge University Press. Beach, L.R. (1964).Cue probabilism and inference behavior. ESvcholaical Mono6raDhs. 28.NO. 582. Bourne. L.E!., & O'Banion, K. (1969).Memory for individual events in concept identification. Psvchonomic Science, Is.101-103. Bourne, L.E., Dominowski. RL.. & Loftus. E.F. (1979). Comitive DrOcesseS. Englewood Cliffs. N J : Prentice Hall. Bourne. L.E. (1982).Typicality effects in logically defined categories. Memorv and Cornition. 10.3-9. Bower. G . . & Trabasso, T. (1964).Concept identification. In R.C. Atkinson (Ed.). Studies in mathematical Dsvchofogy. Stanford, CA: Stanford University Press. Bransford. J.D.. & Franks, J.J. (1971).Abstraction of linguistic ideas. Comitive PSVCholoQ. 2.331-350. Brooks, L.R. (1987).Decentralized control of categorization: the role of prior processing eplsodes. In U. Neisser (Ed.), ConceDts a nd conceptual SleveloDment: Ecolouical and intellectual factors in categorization. Cambridge: Cambridge University Press. Broughton, R (1984). A prototype strategy for the construction of personality scales. Journal of Personalitv and S-I Psvcholo&, 42. 1334-1346. +4studv of thinking. New York Bruner, J.S., Goodnow, J.J., & Austin, GA. (1956). Wlley. Buss. D.M.. & CraLk. K.H. (1983).The act frequency approach to personality. psvcholog&d Review, 9p. 102-126. Cantor. N.. Mischel. W.. & Schwartz, J.C. (14).A prototype analysis of psychological situations. &c_nitive P s v c h o w . 45-77. Chumbley. J.L.. Sala. L.S.,& Bourne. L.E. (1978).Bases of acceptability ratings in quasi-naturalistic concept-tasks. Memorv and Cognition, 6. 217-226. Cohen. C.E. (1983).Inferring the characteristics of other people: categories and attribute accessibility. Journal of Personalltv and Social Psvcholoa, 44. 3444. Conner, M.T.. Land, D.G.. & Booth, D.A. (1987).Effect of stimulus range on judgements of sweetness intensity in a lime drink. w i s h Journal of Pecholo&, 28,357-364. Das-Smaal. E.A., & De Swart, J.H. (1981).Qpicality, physiological activity and 1-24. concept identiflcation. Acta Psvcholoaica. Das-Smaal, E.A.. & De Swart, J.H. (1984).Variation within categories. psvchologi~,z,165-192. Das-Smaal. EA.. & De Swart, J.H. (1986).Effects of contrasting category, conjoint frequency and typicality on categorization. Acta Psvcholoeicq. 15-40. Das-Smaal. EA. (1986).Variation within catecories. Amsterdam: Free University Press.

.-

a,

a.

384

E. Das-Smaal

Das-Smaal, E.A., De Leeuw. L.. & Orlebeke. J.F. (1987).Is er iets mis met de aandacht van het schoolkind.eSturn. 1-15. De Jong. P.F. (1988). An application of the prototype scale construction strategy to the assessment of student motivation., E, 487-508. De Swart, J.H.. & Das-Smaal. EA. (1976). Relationship between SCR heart rate and information processing. Bioloeical Pmcholo~!y~ 4, 41-49. De Swart, J.H.. & Das-Smaal, E.A. (1979a). Physiological activity in a n attribute learning task. Acta Psvcholoeiu, 57-69. De Swart, J.H.. & Das-Smaal, E.A. (1979b). Orienting reflex and uncertainty ni reduction in a concept learning task. In J.F. Orlebeke (Ed.), reflex in humans. Hillsdale. NJ: Lawrence Erlbaum Associates. De Swart, J.H., Kok. A , & Das-Smaal. E.A. (1981).P300 and uncertainty reduction in a concept identification task. &y&oDhvsiohgy. 6 19-629. Detweiler, RA. (1975). On inferring the intentions of a person from another culture. Jplunal of Personality, 591-611. Detweiler. RA. (1978). Culture, category width, and attributions: A model-building approach to the reasons for cultural effects. Journal of Cross-Cultural PWcholo&, 2, 3. 259-284. Dominowski. RL, & Wethenick, N.E. (1976). Inference processes in conceptual rule 1 PsvcholLea rnine and learning.

a,

s,

u,

a,

Memorv. 2. 1-10.

Fehr, B.. & Russell, J.A. (1984). Concept of emotion viewed from a prototype perspective. Journal of ExDerimental Psvcholom: General, 113.464-486. Goldman, D.. & Homan. D. (1977). Integrative and metric properties of abstracted information as a function of category discriminality. instance variability, and experience. tal PsvMo€w: Hum-

m, 2

,

3

7

5

-

3

7

Hayes-Roth. B.. & Hayes-Roth. F. (1977). Concept learning and the recognition and classification of exemplars. Journal of Verbal Learning and Verbal Behavior, J.@,32 1-338. Haygood. RC.. Harbert. T.L.. & Omlor, JA. (1970). Intradimensional variability and concept identification. Journal of m e n t a l Psvcholtgy, 216-219. Hintzman. D.L. (1986). "Schema abstraction" in a multiple-trace memory model. Psvcholoeical Review. 41 1-428. Hull. C.L. (1920). Quantitative aspects of the evolution of concepts. Pmcholodcal MonoeraDhS. 123. Hunt, E.B.. & Hovland. C.I. (1960). Order of consideration of different types of concepts. Journal of ExDemental Psv&&gy, 220-225. Kellogg, R.T.. Bourne. L.E.. & Ekstrand. B.R (1978). Feature frequency and the acquisition of natural concepts. American Journal of PsvchqLQey 212-

m,

s,

a,

,u.

222.

Kellogg. RT. (1980). Feature frequency and hypotheses testing in the acquisition of rule-govemed concepts. Memorv and C o m , 8. 297-303. Kellogg. R.T. (1982). Feature frequency in concept learning: What is counted? Memorv w e n i t i o n , 9,157-163. Kemler Nelson, D.G. (1984). The effect of intention of what concepts are acquired. Journal of Verbal Learning and Verbal Behavior, B, 734-759. Knam. A.G.. & Anderson. JA. I19841. Theorv of categorization based on distributed memory storage. Journi 1 of --me ntal Psvcholo@: Learnine. Memory and Coanition. la,616-637. Labov. W. 11973). The boundaries of words and their m e w s . In R.W. Shuv (Ed.). Pew wavs of arialvzina variation in E n e m . Washingon D.C.: Georgetown University Press. Lakoff, G. (1986). Women. fire. and dangerous things: What categories tell u s about ture of thought. Chicago: University of Chicago Press.

__

-

385

Biases in categorization

Landau, B. (1982).Will the real grandmother please stand up. Journal of 47-62. Psvcholimuistic Research , Levine, M. (1966).Hypothesis behavior by humans during discrimination learning. 1of be, 331-338. Levine, M. (1969).Latency-choice discrepancy in concept learning. Journal of Emerimental Psvcholom, 412. 1-3. Levine, M. (1975).J-imothesis Testin@:a -ti ve theorv of learn1%. Hillsdale, N J : Lawrence Erlbaum Associates. Mandler, G. (1984). Mind and body: u s v & & g v of emotion and s t r e a. New York W.W. Norton & Company. Mandler, J . M . (1984).Stories, scriuts and scenes: asuects of schema-theory. Hillsdale. N J : Lawrence Erlbaum Associates. Martin, R.C., & Caramazza. A. (1980).Classification in well-defined and ffl-defined categories: Evidence for common processing strategies. Journal of 1 . ,1Q9. 320-353. McCauley, R H . (1987). The role of theories in a theory of concepts. In U. Neisser (Ed.), Conceuts and conceutual develoument: Ecoloeical and intellectual factors in cateeorization. Cambridge: Cambridge University Press. McCloskey. M.E.. & Glucksberg, S. (1978).Natural categories. Well-defined or fuzzy sets?. M e m o m C o u m . 6,462-472. Medin. D.L., & Schaffer. M.M. (1978).Context theory of classification learning. Psvcholoaical Review, &, 207-238. Medin. D.L.. & Smith, E.E. (1984). Concepts and concept formation. Annual Review of Psvcholo@, 1 13-138. Medin, D.L.. & Wattenmaker, W.D. (1987).Category cohesiveness, theories and cognitive archeology. In U. Neisser (Ed.), Conceuts a nd conceutual m e n t : E w c a l and intellectual factors in catego rizatio n . Cambridge: Cambridge University Press. Miller, G.A., Galanter, E.H., & Pribram. K.H. (1960).Plans and the structure of behavior. New York Holt. Rtnehart &Winston. Mfller. G A . & Johnson-Laird, P.N. (1976).-erceut iqn. Cambridge. MA: Harvard University Press. Mfllward. RB. (1980). Models of concept formation. In W.E. Montagne (Ed.).Autitude, learnim and instruct.lpn, Vol. 2.Hillsdale, NJ: Lawrence Erlbaum Associates. Murphy, G.L., & Medin, D.L. (1985).The role of theories in conceptual coherence. Psvcholoeical Review, 289-316. Neisser. U. (1967). Cognitive usvchology. New York: Appleton-Century-Crofts. Neisser. U. (1979). The concept of intelligence. Lntelligence, 3,217-227. Neisser, U. (1987).Introduction: the ecological and intellectual bases of categorization. In U. Neisser (Ed.), Concepts a nd c o m u a1 develoument; Ecoloeical and intellectual factors in categorization. Cambridge: Cambridge University Press. Neumann. P.G. (1974).An attribute frequency model for the abstraction of prototypes. m o w and Coenitioq, 2. 241-248. Nickerson. R.S., & Adams, M.J. (1979).Long-term memory for a common object. w i t i v e Psv . cholo&. U, 287-307. Norman, D h . (1986).Reflections on cognition and parallel distributed processing. In The PDP Research Group (Ed.), Parallel distributed urocessing: Emlorations Jn the microstmcture of cognition. Cambridge: MIT Press. Oden, G.C. (1987).Concept, knowledge and thought. ,4nnual Review of Psvcholom, 33, 203-227. Posner. M.I.. & Keele. S.W. (1970).Retention of abstract ideas. Journal of ExDerimental Psvcholo-!. m, 304-308. Reed, S.K. (1982).Coanition. Theorv and auulications. Monterey: Brooks/Cole Publishing Company.

u,

n.

a,

s,

E. Das-Srnaal

386

Restle. F. (1962).The selection of strategies in cue learning. &L&Qkgical

Review. 69.

329-343. Reznick. J.S.. & Richman. C.L. (1976). Effects of class complexity, class frequency

and pre-experimental bias on rule learning.

Journal

2 in n M m ry.2.774- 782.

of Experimenta1

Rosch. E. (1973). On the internal structure of perceptual and semantic categories. In T.E. Moore (Ed.),W i t i v e devthe-tin uw.New York Academic Press. Rosch. E. (1975). Cognitive representation of semantic categories. Journal of

ExD-QGeneral.

3. 192-233.

Rosch, E. (1975). Cognitive reference points. CoenitWe Psvcholom. 2. 532-547. Rosch, E. (1978). hlnciples of categorization. In B.B. Lloyd [Ed.). m d .Hillsdale. NJ: LEA. Rosch. E.. Mervls. C.B.. Gray, W.D.. Johnson, D.M., & Boyes-Braen, P. (1978). Basic objects in natural categories. &miti ve Psvcholo-. 382-439. Rumelhart. D.E.. & McClelland. J.L. (1982).An interactive model of context effects in letter perception: 11. Psvchological Revlew. f&, 60-94. Smid. N.G.. & Zwinderman. A.H. (1986). Representatie van een eigenschappen-bijsituaties taxonornie in prototypische gedragsbeschriJvingen. Beder l a n d ~ ri!l voor de Psvchoha,fi.91-103. .Cambridge: Harvard Smith,=& Medin, D.L. (1981). University Press. Sokolov, E.N. (1969).The modeling properties of the nervous system. In 1. Maltzman (Ed.), A handbook of contemDorarv Sovlet Dsvcholofl. New York: Basic Books. Tversky. A.. & Kahneman, D. (1983). Probability. representativeness, and the conjunction fallacy. -1 RevleW. 293-315. Vandierendonck. A. ( 1988). Typicality gradient in well-defined artificial categories. Acta Psvcholoaica. &I.61-81. Visser, M.R.M., & Das-Smaal, E.A. (in preparation). Assessment of impulsiveness using the prototype scale construction strategy. Ward, T.B.. & Scott, J . (1987). Analytlc and holistic modes of learning familyresemblance concepts. &@mom and Corn. fi,42-54. Wickelgren. W.A. (1981). Human learning and memory. Bpnual Review of

a.

a,

m. 32, 21-52.

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e and M. G o n z a l e z ( E d s . ) E l s e v i e r S c i e n c e P u b l i s h e r s B . V. ( N o r t h - H o l l a n d ) ,

1990

STUDIES ON SELF-CENTERED ASSMILATION PROCESSES Jean-Pad Codol* CREPCO ( C e n t e r for R e s e a r c h

i n C o g n i t i v e Psychology)

Aix-en-Provence,

France

Abstract. The kind of research this paper deals with addresses the question of how referring to oneself modifies the way in which we perceive our physical and social environment. Stated in a more concept-oriented manner, it concerns asymmetry phenomena in perception and social comparison, phenomena which are induced from self-centering schemas. A few examples from some recent studies, either in the area of comparison or in the area of perception will be given. These examples are particular aspects of a much broader question, whose generalities will be introduced briefly.

ASYMMETRY PHENOMENA IN SOCIAL COMPARISON The simplest way to begin would be to s t a r t with the comparison process. Very generally speaking, the question may be presented as follows: when we compare any two objects, one of them always serves as a model to which the other is compared. For example, to say that "A is greater than B is to compare A to the model B. If we were to take A as the model, and compare B to it, the same reality would then be expressed as "B is less than A". Formally speaking - and especially if we are dealing with symbolic objects like those used in algebra - the distinction is irrelevant, and we establish the equality of "A is greater than B and "B is less than A". However, this logical equality is not generally accepted psychologically, particularly if the objects being compared are persons, and even more so if the subject making the comparison is himself one of the persons being compared. Indeed, using oneself as a comparison model (that is. comparing others to oneself), is not the same for a subject as using oneself as the object to be compared. In the first case, the subject "asserts" himself, defining others with

*

Jean-Paul Codol d i e d on August 13, 1 9 8 9 , b e f o r e he was a b l e t o review t h e f i n a l t e x t i n t h e p r e s e n t volume.

388

J.P.

Codol

respect to himself. In the second case, the subject "asserts" the others first, and then defines himself with respect to them. In a series of studies, I have found that such an asymmetry phenomenon occurs, for instance, in the perception of similarities and differences. By means of twenty or so experiments, conducted over the past six or seven years, I have shown that a person generally accepts a similarity relationship more easily if it is defined in reference to himself ("Other people are like me") than if it is defined in reference to others ('7 am like other people"). The opposing tendency exists for difference relationships: each of us considers him/herself more different from others than he considers others from him/herself (Codol, 1979, 1984a. 1986. 1987, 1988)*. Many variables were used to test the generality of the phenomenon, in particular those defining the other persons upon which the comparison bears, for instance others in general vs. a particular person, or an actual well-defined group of people vs. a wider social class, etc. Within this general framework, I shall simply point out now some new facts, brought out in recent investigations. Only a brief overview of the main tendencies observed in previously published studies will be given here: for each of the experiments discussed, the reader might wish to refer to the corresponding articles. Experimental illustration: evaluation and expression of similarities and differences between peers

The experimental situation was the following (Codol, 1986). After having specified the names of the two classmates they knew the best, 170 boys and girls, age 17 to 22, were asked both to compare themselves to each of these two classmates, and to compare the two classmates to each other, stating differences as well as similarities. In each case, the subject was to make the comparisons (a) by giving a n overall estimation of the degree of similarity (graded on a scale), and (b) byfreely expressing "whatever comes to your mind" in the way of similarities and differences between the persons being considered.

* F o r r e l a t e d l i t e r a t u r e , see a l s o Codol (1984 b and c ) ; E x t r a & Vermunt ( i n p r e s s ) ; Holyoak & Gordon ( 1 9 8 3 ) ; K a t o Maruno ( 1 9 8 5 ) ; S e r i n o ( i n p r e s s ) ; S r u l l & G a e l i c k ( 1 9 8 3 ) ; Tversky (1977).

&

389

Self-centered assimilation processes

For each situation, three experimental groups were defined according to the reference point proposed for the comparison. For example, when the subjects were to compare themselves to person P. the question was either "What makes you different from person P?" (the other person being the comparison reference point), or "What makes person P different from you?" (the subject himself is the reference point), or "What makes you and the other person different from each other? (no specified reference point). In agreement with previous results, the quantitative analysis of the data showed in particular that specifying a reference point affects the perception of similarities and differences when the subject himself is one of the persons being compared (Table 1). Table 1. Evaluation of similarities and differences between peers. Adapted from Codol 1986. SELF-OTHER

COMPARISON

ESTIMATION OF SIMILARITY REFERENT NO REFERENT SELF OTHER

3.65 3.47 2.77

5.59 4.92 4.23

COMPARISON BETWEEN

PERSON

1 AND PERSON

ESTIMATION OF SIMILARITY REFERENT NO REFERENT PERSON 1 PERSON 2

3.95 3.64 3.64

FREE EXPRESSION (number of i t e m s u s e d ) SIMILARITY DIFFERENCE 3.35 3.35 3.75

2

FREE EXPRESSION (number of i t e m s u s e d ) SIMILARITY DIFFERENCE 2.95 2.45 2.34

4.05 4.09 3.89

390

J.P.

Codol

In particular, compared to situations where the other person is the reference point, self-reference increases the degree of perceived similarity (4.92vs. 4.23).and the expression of resemblances (3.47 vs. 2.77);and decreases the expression of differences (3.35 vs. 3.75). (Compare, in Table 1. these data to those obtained when the subjects are not involved in the comparison). These phenomena have already been found elsewhere, granted, and can hardly be considered new. But in this line, the following results from a qualitative analysis are more interesting: using a factorial method, the qualitative analysis of the free descriptions made by the subjects revealed three main categories of responses: (1)Personalfty or behavioral aspects (ways of acting, personality traits, qualities, shortcomings, etc.) (2) Opinions, tastes, and aspirations (ways of looking at things, interests, ways of thinking and judging, etc.) (3)Objective, factual characteristics, usually physical (age, sex, skin color, height, etc.) The frequency with which each of these three categories was applied by each experimental group served as the dependent variable in a subsequent analysis of the data. Results indicate that personality and behavioral aspects almost always oppose opinions, tastes, and aspirations . They do not vary in the same way for each comparison reference point, and interfere with asymmetry phenomena. In fact, the former appear in particular to be more essential to the feeling of personal identity than the latter. For example, although subjects most often express their differences from others in terms of the way they are and act, they most often express similarities between people in terms of the way they think and judge (see also Marks, 1984). These results are interesting, since they reinforce the idea that behind asymmetry phenomena and self-centering schemas, personal identity is at stake. Another result of this study is worth mentioning: the lack of an explicit referent in comparisons involving the subjects produced effects similar to those produced by self-reference comparisons, effects which are generally the opposite of those produced during reference to others. This may be interpreted as a cue to a spontaneous self-centering schema. One final result, constantly found in this kind of research, is that the nature of the other people being compared is a determining

Self-centered assimilation processes

391

factor of the means of differentiation used by the subjects. The data gathered from various experiments may in particular be explained by the relative degree of categorical "typicality" attributed by the subjects to the other persons being considered. The asymmetry is even greater when, for the categories to which subjects relate themselves, the other people seem less typical of those categories. I shall not present other studies on these topics (see above references). For our purpose here, let us simply say that the results of these studies have been interpreted in the light of the general problem of personal identity: I am both digerent from others, and [yet) a model for them Moreover, a simple relationship can be established between this phenomenon and the assimilation and contrast processes resulting from categorical differentiation. Thus, if subjects are more likely to accept similarities between themselves and others when they are defined in reference to themselves than when defined in reference to others, this is most likely because they generally tend to perceive others as belonging to the same "category" as themselves, although they do not perceive themselves as belonging to the same category as others. My results may thus be explained as the manifestation of a basic self-centered assimilation process, so often found in the literature on social psychology. Each of us may somehow tend to have a "prototypical" conception of himself. ON SELF-CENTERED ASSIMILATION PROCESSES

Experimental illustration 1. Self and other descriptions: the influence of description order on categories used

This point can be illustrated by a very simple experiment (Hardoin & Codol, 1984). Each subject was asked to freely describe himself and a stimulus person, one after the other. Responses showed a marked asymmetry in the influence of the first description on the second, depending on whether the subjects described themselves first, or the stimulus person first. In the first case (self-description first), the subjects described the other person with traits of the same type as those used to describe themselves. In the second case (stimulus person described first), they used difierent types of characteristics for themselves than those previously used to describe the other person.

3 92

J.P.

Codol

Similarly. in another study (Codol, 1987), it was shown experimentally that individuals somehow have the tendency to make others comparable to themselves, while striving to make themselves incomparable to others. These findings have been interpreted as indicating that, by relating others to oneself, each of u s considers himself as a model, while attempting to make himself different from models of others.

Experimental illustration 2. The estimation of physical distance between people: Am I asfarfrom gou as you arefrom me? The example I would like to give now is of a different nature, and concerns personal spaces. By personal space, I mean the physical area that surrounds each individual. This space, whose boundaries are invisible, is in a certain sense prohibited to others. The establishment of this space, its defense by the individual, the fact that it is socially recognized within a given culture, its dependence on numerous psychological and social factors, etc. without a doubt, define the notion of personal space not only as a physical space, but also - and above all - as a psychological space, which appears to me to be linked to selfassertion, and to the establishment and defense of one's personal identity. I was interested here in how self-centering schemas alter the estimation of the physical distance between oneself and others. Concretely speaking, the idea was to find out whether the perception of the distance between a subject and other people is affected by the fact that the reference point for estimating is the subject himself ("Howfar are the other people from you?'), or the other people "How far are you from the other people?") (Codol, 1985).

The problem is less touchy than it might seem: the space within which the distance to estimate is located is defined differently according to how the question is asked. In the first case, the distance is somehow located within the subject's space (where others are defined in reference to him/her). In the second case, it is located within the other people's space (in which the subject is defined in reference to the others). We can see here that this problem is of particular interest to personal space theory. If it is true that such a space generally is formed, and is defended against the constant threat of intrusion by others, then we might in

Self-centered assimilation processes

3 93

particular hypothesize that distances will be perceived as shorter when they are related to oneself than when they are related to others. Social density is also known to play a n important role in maintaining and defending personal spaces. As a rule, the more people in a given space, the more the personal spaces are threatened. We may thus hypothesize that the reference point for estimating distances interacts with social density. The differences between self-referenced estimations and estimations referring to others, postulated to exist by the first hypothesis, should increase as social density increases. Three experiments were conducted to test these hypotheses. Two of them used a scaled layout on paper, the third, a concrete situation. The idea behind working with a layout on paper is that it would allow subjects to express how they represent, rather than how they behave, as well as what their expectations would be in well-defined situations. It also enables the experimenter to define stereotypes based on these expectations. Such a general expression is indeed rare in natural situations, which are always contingent and highly specific, containing numerous physical or psychological factors that may inhibit such expression. The third experiment, on the other hand, was conducted in a concrete situation, in a room where the subjects were actually brought together. The experiments using a layout on paper were very simple: 1. In the first experiment, 95 subjects were given a sheet of paper containing points that represented various people supposedly present in a public room. Each point was labelled by a letter of the alphabet. In half of the cases, one of the points, labelled "ME', was supposed to represent the subject himself. The scale was indicated on each sheet by a 3-cm line representing one meter in real life. The subject's task was to estimate, in centimeters, the actual distance (i.e. the distance in the room, not on the paper) between the people represented on the layout, taken two at a time, sometimes involving, sometimes not involving, the subject himself. The reference point for the estimation varied as follows: for any two people, say A and B, we had either "How far is A from B ? or "How far is B from A?". For estimations involving the subject, we had either "How far are you from person P?" or "How far is person P

394

J.P.

Codol

from you?". Four experimental groups were defined in this way, as shown in Table 2. Table 2. Estimation of physical distance by experimental group.

REFERENCE POINT FOR DISTANCE ESTIMATION PERSON OTHER THAN P

TYPE OF STIMULUS PERSON

ANONYMOUS 1.How f a r is P from t h e o t h e r s ? SUBJECT

3.How f a r a r e you from t h e o t h e r s ?

STIMULUS PERSON P 2.How f a r a r e t h e o t h e r s from P? 4.How f a r a r e t h e o t h e r s from you?

Each subject received several layouts which varied both in social density and in how close the people were placed to each other. Results of this experiment confirmed our hypotheses. In particular, as we can see in Figure 1. distances were underestimated in the group where the subject himself was the estimation reference point (compare 231.5 to any other value). Moreover, results showed that this effect increased as social density increased. 2. In the second experiment, the same techniques and layouts were used, but instead of representing people the points shown on the layout were supposed to represent material objects. None of the effects observed in the preceding experiment occurred here (except for the simple density effect, i.e. the more objects in the layout, the shorter the perceived distances. But this is only a very classical - almost trivial - finding that dates back to the Gestalt studies on perception). What is interesting here is the difference between the results obtained in these two experiments. This difference clearly shows that the schematic and artificial nature of the situation did not prevent the subjects from expressing specific expectations. Their responses were still dependent upon whether they were dealing with people or objects, and if dealing with people,

3 95

Self-centered assimilation processes

whether the subject him/herself was involved in the estimation, and if so. whether he/she was the estimation referent. 3. The third experiment, in which 72 subjects participated, took place in a rectangular room, where twelve wooden disks, 30 centimeters in diameter, were arranged at variable distances from each other. Each subject was to stand on one of the disks and estimate either how far he/she was from the other people, or how far the other people were from him/her. Here again, in agreement with the hypotheses, the results showed t h a t distances were underestimated to a much greater extent when the estimation reference point was the subject himself rather than the other people. 270 1

255

-

I HOW far are you I

I from the others? I from the others?

256.6

245 250

235 240

231.5 9-m

L " "

Other persons

Stimulus person

Reference point

Figure 1. Average estimation of the distances for the four experimental groups (actual mean: cm). Adapted from Codol 1985. Another interesting result of this experiment is that there was a significant correlation between the correctness of the estimations and the actual distances involved - the shorter the actual distance, the greater the underestimation. And this correlation was significantly stronger when the distance was defined in reference to oneself than when in reference to others.

396

J.P.

Codol

In other words, subjects are more likely to overestimate how close others actually are to them than how close they are to others. Relating this to the problem of personal spaces, it seems as if subjects consider others to be occupying their own personal space more than they themselves occupy the space of others. All of the above results indicate the effect of a self-centering schema, which, as I stated at the beginning, is known to play a role in other judgement and self-with-others comparison processes, and which has generally been interpreted as a sign of the assertion and defense of personal identity. The fact that we were able here to observe a self-centering schema effect during the cognitive activity of distance estimation (and that it was found not only in the concrete situation set up in experiment 3, but also in a situation as artificial as in the experiments using a layout on paper) clearly indicates to me both the importance of such a schema and the cognitive anchoring of personal spaces. In any case, this specific example does show how selfreference can modify the way in which we perceive our environment. SELF-JUDGEMENT AND ANTICIPATION OF THE JUDGEMENTS OF OTHERS

In conclusion, I would like to briefly discuss a line of research on self-centered assimilation processes that I feel is of particular interest: To what extent can individuals avoid such self-centered assimilation phenomena when judging similarity and difference? A pilot study was conducted to answer this question by asking subjects to put themselves in the place of another person when making a self-others comparison, and to predict what the other person's judgements of the subject would be. Would subjects imagine that the other person's answers would be guided by a self-centering schema (as their own generally are), and that the other person would thus make judgements that oppose the subjects' own feelings (being based on opposing self-centering)? Or on the contrary, would the subjects' self-centering schemas be so strong that they would prevent any process of this type from occurring, causing them to think that the other person would respond as they would? The experimental setup was quite simple (Codol, 1988). Each subject was given a photocopy of a self-description questionnaire

Self-centered assimilation processes

3 91

that was supposed to have been filled in by another subject (it had, in fact, been fabricated by the experimenter). The subjects were divided into two groups. In the control group, each subject was to state (for each item on a 9-point scale) to what extent he felt he and the other person were similar to, or different from. each other. The reference point for the comparison (oneself or the other person) was varied, making four situations in all. In the test group, each subject was to predict what the other person would say in the same four situations when shown the subject's answers to the selfcharacterization questionnaire. The results are given in Figure 2. They show among other things that: (a) When comparing themselves to others, the control group subjects were more sensitive to differences than to likenesses. This is a common finding. The interesting point is that in the test group, the subjects' predictions of the other person's answers lean in the opposite direction. That is, they felt that the other person would perceive more likenesses than differences. This discrepancy stresses how important the subjects' feeling of their own specificity is: in the area of similarity and difference perception, so sensitive to identity, subjects feel, above all, that they do not think like others. (bl The relationship between the reference point (oneself or the other person) and the type of assessment made (likeness or difference)was not the same in the two groups. In the control group, the usual effects were observed, i.e. subjects thought that the other person was more like them than they were like the other (4.77 and 3.40, respectively), and attested to how different they were from others more than to how different the others were from them (6.19 and 5.29). The predictions made by the test group concerning the other person's answers were different. Although they felt that the other person was going to respond like they did, when assessing likeness ("The other person is going to think that I am more like him than he is like me": 5.73 and 4.87). the same was not true for their difference assessments ("The other person is going to think that I am more different from him than he is porn me":4.90 and 3.92). In other words, it appears as though, from the subject's point of view, other people's answers are guided by a self-centering schema when they concern similarity relationships but not when they concern dflerence relationships. Comparing these results to the control group results, we see that subjects seem to be able to

J.P.

3 98

Codol

Subjects' estimations The other is different

5

4 -

I am similar to the other

3.4 3

I

I

Subject

Other persons

Reference point

Subjects' anticipation of others' estimations He is similar

3.92 from me

3

I

I

Subject

Other person

Reference point

Figure 2. Estimation means in control group (upper) and test group (lower).

Self-centered assimilation processes

399

put themselves in the other person's place (and thus are capable of decentering) when it comes to similarities but not to differences. One possible interpretation of this data relates the form of the question asked to the state of the relationships - inferred from the questionnaire by the subjects - that the subjects have with the other person. Being instructed to assess the degree of similarity, it is conceivable that subjects assume that some real similarity does exist between themselves and the other person, and that they are simply to indicate the extent of that similarity. In a similar manner, when asked to assess the degree of difference, a subject may assume a priori that some real differences do in fact exist between them. The existence of decentering in the first case but not in the second is thus understandable: identification with others is obviously easier between similar persons than between different persons. The diverging results of the two groups in this experiment bring up another interesting point. In the control group, it appears as though likeness and difference are two poles of one and the same dimension: the assessments made are indeed the opposite in these two cases. In the test group, however, the two types of assessments are parallel. This seems to indicate that difference and likeness represent two distinct dimensions. In this perspective, the more extreme answers the subjects thought the others would give when the other person was the comparison reference point may mean that, regardless of what dimension is involved, subjects assume that others consistently judge in accordance with a self-reference bias. If this is in fact the case (as other studies on other dimensions of comparison should be able to tell us), it remains to be explained why subjects systematically ascribe such a bias to others, while only attributing it to themselves for certain dimensions. In any case, what these results seem to indicate for sure is that subjects do not feel the word "difference" means the same thing to themselves as to others. Regardless of how we interpret these results, this seems to be a n interesting question that should be dealt with more thoroughly. To come back to more general considerations, the study of asymmetry phenomena as a function of the reference points used seems to me to be of particular interest to many lines of research, and should be further developed. The notion of "reference point" is indeed at the crux of processes involving the identification, recognition, and definition of all types of objects in our

400

J.P.

Codol

environment, be they material or social. I personally believe that this may become a new field in cognitive social psychology. REFERENCES Codol. J.P. (1979). et Dlffkrents, R e c h e sur la outte de la &nil itude la dfeir-. Aix-en-Provence: Universite de b v e n c e . 520 P. Codol, J.P. (1984a). La perception de la similitude interpersonnelle: influence de l'appartenance cattgorielle et du point de reference de la comparaison, L'Annee P s v c h o l o e i ~&, . 43-56. Codol, J.P. (1984b). L'asymttrie de la similitude percue entre des personnes de Psvcv ,4.605-610. diversement stbrtotypbes. -s Codol, J.P. (19844.Quand Dupont resemble a Dupond plus que Dupond a Dupont. L'asymttrie de la similitude percue entre personnes semblables. Psvcholoeie Francaise, 29.284-290. Codol. J.P. (1985).L'estimation des distances physiques entre personnes: suis-je aussi loin de vous que vous l'ttes de moi? M e Psv517-534. Codol, J.P. (1986).Estimation et expression de la ressemblance et de la difference entre pairs. w e e PsvchJ&gQgg 527-550. Codol, J.P. (1987). Comparability and incomparability between oneself and others: means of differentiation and comparison reference points. Cahiers de Psvcholoeie Coanltive-EuroDeanBulletin of Co@nitive Psvchology. 2. 87-105. Codol. J.P. (1988). Interpersonal similarity and difference: self-judgement and anticipation of the judgements of others. de Psvchol0gie SocialC (in presse). Codol. J.P.. & Jarymowicz. M. (1984). L'estimation de la ressemblance entre des personnes : quelques directions de recherches. Bulletin de Psvchologie, 37. 813-822. Extra, J., & Vermunt. R. (in press). Asymmetry phenomena in the comparison between persons. 1 V Hardoin. M.. & Codol. J . F ? ~ ~ & 4 ~ ~ ~ ~d h % ~ u e~n c~e de i I'ordre % des descriptions sur les cattgories de r€ponse utillstes. Psvcholou ie COrmitiVc. 5, 295-302. Holyoak, K.J.,& Gordon, P.C. (1983).Social Reference points. Journal of Personality and SUPsvchology, 5.881-887. Kato. K., & Maruno. S. (1985).What people employ a s a reference point when judging similarity between oneself and other? 5 e arch Bulletin of Education and 71-81. Marks, G. (1984). Thinking one's abilities are unique and one's opinions are common. Personalitv and Social Psvcholoe Bulletin, 1p.203-208. Serino. C. (in press). Biases egocentrici. confronto sC/altro et fenomeni di confronto asimmetria: alcune ipotesi di articolazione fra diversi live111 di analisi. i 1 . Sru11, T.K., & Gaellck. L. (1983).General principles and indivldual differences in the self as a habitual reference point. social Cormition. 2. 108-121. Tversky. A. (1977). Features of slmllarlty. . - P 84. 327-352.

,a,

,a.

mv. a,

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e a n d M . G o n z a l e z ( E d s . ) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. ( N o r t h - H o l l a n d ) ,

1990

SPECIFICITY AND CATEGORIZATION IN JUDGMENT: A COGNITIVE APPROACH TO STEREOTYPES Jean-Marc Fabre University of Franche-ComtC. Besancon. France

Abstract.

Judgment biases linked to Categorization effects were studied in two different theoretical frameworks. The first concerns the effects of context on absolute judgments, where speclflcity is emphasized a stlmulus is only an element of an effective context if it belongs to the same class of objects a s the elements of that context. p e other theoretical framework concerns categorical differentiation, by means of which the membership of two groups of stimuli in different classes determines both a n interclass differentiation effect and an lntraclass assimilation effect. Some experlmental studies illustrating these two theoretical frameworks are reviewed, and two new experiments on categorical differentiation are presented. It is shown that these two theoretical approaches are virtually contradictory, and that attempting to interrelate them poses serious problems.

Stereotypes are the source of one of the most well-known biases affecting judgments in everyday living. One method used to study their tremendous impact is to experimentally test the relationship between categorization and judgment. Research of this sort covers many areas ranging from social judgments to perception: by resolutely opting for an informational point of view we are led to question the soundness of the notion of bias, and to propose, as a general rule, t h a t the discovery of systematic divergence in judgments and even of distortions with respect to established norms is but a sign of how activities are organized. This sign must become intelligible. As Arnault de la Menardiere and De Montmollin (1985) stated, although stereotype effects, through which objects or individuals are attributed the real or supposed characteristics of the classes to which they are assumed to belong (sometimes at the expense of their own specific properties), have "long been considered as a n erroneous representation", they should be described as "categorization without which the environment cannot be processed" (ibid.). This is the perspective I shall take here, limiting the u s e of response modifications resulting from categorization to their role as cues likely to shed light on cognitive organization. The focal point of the conceptual framework for this study is the notion of differentiation, which provides u s with the key

402

J. M. F a b r e

to interpreting the functioning of absolute and comparative judgments, and more specifically to understanding how ordered rating scales work (Fabre, 1987) when applied to quantitative, symbolic, or social attributes. Differentiation is done between the objects in a group, between the objects and a norm, between objects belonging to different classes, and between classes. In the study of stereotypes, the kind of differentiation that interests us involves objects belonging to multiple classes. The theoretical background supporting such differentiation is predominantly marked by the model of the accentuation of interclass differences (Accentuation Theory. Tajfel, 1959) which has inspired many researchers in the fields of psychophysics (e.g. Tajfel & Wilkes. 1963) and social psychology (Eiser, 1971; Doise. 1976, 1979). The assumed generality of this theory will be reexamined in the light of recent research. In principle, this theory posits that when objects are members (either in fact or in fiction, provided the subject has knowledge of that membership, has recognized it, and is convinced of it) of classes that differ along the dimension upon which the judgment bears, the positioning of those objects along that dimension is affected in two different ways. First, objects are judged to be more similar to each other if they belong to the same class than if they do not belong to the same class (assimilation effect), and second, objects are judged to be more different from each other if they belong to different classes than if they do not (contrast effect). Another point of view can be adopted to consider the relationship between the classification of objects and their differentiation: that of specific restriction or "specificity". This point of view pertains to the differentiation of objects within the classes to which they belong. We know that objects are attributed a contrasting position along the judged dimension relative to representative parameters, or norms, inferred from a set of class instances (Helson, 1947. 1964; Johnson, 1949; Kahneman & Miller, 1986; Volkmann, 1951; Zoeke & Sarris, 1983). This contrasting value, which must be carefully distinguished from effects linked to the succession of stimuli over short periods, can be attributed to equilibrium principles governing the segmentation of the range of variation of each class and to the frequency of use of each response category (range-frequency principle, Parducci. 1965, 1983; Mellers, 1983; stabilization principle, Treisman, 1985). This point of view, as we shall see, leads to the following ascertainment:

Specificity and categorization in judgment

403

contrastive attribution implies common class membership. Taken literally. this conclusion opposes categorical differentiation, or at least limits it. After reminding the reader of some of the observations that have allowed u s to define specific restriction in all its complexity, and then presenting recent data that has provided incentive for the reexamination of the traditional understanding of categorical differentiation, I shall attempt to interrelate these two concepts. SPECIFICITY

The length of a piece of chalk or a blackboard is a value, measurable in millimeters. Yet the values used to estimate their respective lengths are taken from two separate functional dimensions. The difference between classes of objects is based on frames of reference, or perceptual habits, which determine the selection of relevant information for the assessment of the two lengths. The concept of specificity expresses the separation of the dimensions from the assessment. Two expressions are proposed above: specificity and specific restriction. Both deal with the same phenomenon, but refer to it from two different angles, that of object classes and stable judgment structures, and that of intraclass object variability and the relatihation of the assessment of those objects. Certain particularly demonstrative facts have been established. They suggest that when faced with objects to be judged, subjects are able to evoke several different reference contexts. Their responses regarding each object thus may stem from a primitive decision as to whether or not all objects pertain to the same context. Thus, the same response scale may apply to all objects, regardless of the decision, but there may be an exclusive interpretation of the responses only when the decision is affirmative. The distinctive marks that lead to specific restriction can either be quantitative (variation in the respective orders of magnitude of a series and an anchor stimulus) or qualitative (modification of a stimulus property). Order of magnitude of stirnull Using the anchor stimulus technique whereby a n extreme stimulus is added to a series, although without being subject to a judgment or assigned to a particular response category, Sarris

404

J. M. F a b r e

(1967, 1976) showed that an anchor stimulus produces a contrast effect which varies in magnitude as a function of the distance between the anchor and the series. Although the effect is greater when the anchor moves away from the series, there is a sort of critical threshold value beyond which the effect diminishes and finally disappears. I t looks as though a decision process occurs whereby the subject deems the anchor stimulus to belong to a different dimension than the to-be-assessed stimuli and therefore to not interfere with judgment organization. Sarris (1976) proposed a n algebraic model (SimilarityClassification Model), derived from the Adaptation Level Model, which accounts for the decrease in this effect. According to this model, short distances between the series and the anchor lead to marked contrast effects, whereas great distances lead to the gradual disappearance of the effects (asymptotic curve). The use of multiple anchor stimuli (Sarris & Parducci, 1978) has also provided evidence of a specific restriction effect, although the result pattern is more complex than with a single anchor due to the diversity of the positions occupied by the elements in the anchor stimulus relative to the series, and to the potential occurrence of a differentiation activity distinguishing those elements. These results prohibit any attempt to explain context effects on the basis of the undifferentiated adaptation of judgments to the entire set of stimuli. Roperties of stimuli Qualitative marks play a n even greater role in specific restriction, since the classification criterion is provided directly and therefore does not have to be inferred by quantitative ascertainment. The study by Bevan and Pritchard (1963) is a particularly good demonstration of this. With materials consisting of simple geometric shapes, these authors recorded judgments of object similarity by using the absolute judgment procedure and introducing a stimulus with an extreme value. Context effects have been shown to occur for similarity judgments of this type both with the magnitude estimation procedure (DiLollo & Kirkham, 1969) and with the "same" or "different" response times procedure (King, 1983). With similarity judgments, the anchor stimulus effect observed is comparable to the one obtained for non-explicitly comparative magnitudes, since "an anchor increases the perceived

Specificity and categorization in judgment

405

similarity of more similar stimuli" (King & Atef-Vahid, 1986, p. 104). For a series of squares and rectangles with the same perimeter, a contrast effect was shown to occur on the "rectangularity judgment" and to be caused by the introduction of an extreme anchor stimulus. But, not all the rectangles were effective anchors. Two experiments showed that their size and color had an impact on their effectiveness, and demonstrated "the lack of any necessary correlation between the presence of a widely deviant stimulus in the judgmental sequence and the occurrence of the judgmental shift" (p. 160). The judgment of the anchor was similar to that observed in the experiment demonstrating the contrast effect, but in comparison to the control condition, this extreme position on the scale was not accompanied by a significant shift in the judgments of series stimuli. On the curve plotting mean judgment by stimulus, a line can be approximately fitted to the points on the graph if the actual distance separating the anchor from the most extreme series stimulus is taken into account. We can see that in the absence of specific marking, the mean estimate for the anchor is very close to the extrapolated value. This is obviously no longer true when the anchor's size or color is modified. Thus, modifying the anchor stimulus introduces a distortion in the matching of the response scale to the values taken on by the stimuli. It is when the contrast effect does not occur that the outward responses do not conform with perceptual representations. We are led to consider that intra-specific effects (which in these experiments are "contrast" effects) result from the coherent adaptation of a n explicit scale to perceptual representations, whereas the disappearance of such effects due to specific restriction stems from the reinterpretation of the categories on the scale. While dealing with an uncommon magnitude, Bevan and Pritchard's experiment is obviously not the only convincing demonstration of a specificity effect. One of the first specificity effects was reported by Brown (1953) in the judgment of absolute weight. More recently, Parducci, Knoble, and Thomas (1976) established the role of specificity in frequency effects, and its potential cognitive control. Their experiment involved the judgment of two series of objects, squares and circles, mixed together to form a combined presentation sequence. The figures were chosen in such a way that the squares were generally smaller than the circles. The instructions either explicitly involved specific

406

J. M. F a b r e

restriction, in which case subjects were to assess the size of each figure relative to the whole set of sizes of like figures, or nonrestrictive, in which case the figures were mixed. The same response scale (ranging from "very small" to "very big") was used for both circles and squares: in the restriction condition, then, the interpretation of these categories was expected to vary across shapes. Some very conclusive results where obtained in this experiment. Not only was the rating scale in fact used in both ways when the instructions requested as such, but frequency effects induced by varylng the type of distribution of sizes were observed while specificity was maintained. For example, a n asymmetric distribution of squares led to a contrast effect on the assessment of squares analogous to that observed in simpler experimental conditions (frequency effects), but no contrast effect was found for circle assessment. Thus, specific restriction has not only been found in experiments involving one or more anchor stimuli outside a series (range effects or anchor effects). but also in the case of frequency effects when range is held constant. In addition, this restriction can in some cases occur when certain instructions are given (hypothetically "implicit instructions"): although the series effect is somehow replicated across situations, the identification of the relevant context reveals a more global cognitive apprehension of the situation. Nature of stimulus objects

The distinction between the to-be-estimated magnitudes, whether physical or symbolic, and the actual judgment dimensions, i.e. the context-dependent representations of specific sets, shows up very clearly in the Manis and Paskewitz work (1984). These authors studied a psychopathological judgment of "schizophrenia" applied to two different productions: term definitions and handwriting samples. The response scales used for separate stimuli (experiments 1 and 2) and pairs of stimuli (experiment 3) were degree scales. Each of the three experiments consisted of two phases, the first involved context induction wherein the experimental groups were given mainly extreme examples that were either weakly or strongly pathological (based on prior verification), and the second was a test phase in which intermediate items were used.

Specificity and categorization in judgment

401

The main result of the two first experiments is that inductive context and type of test phase object interacted. The context effect was highly significant when the inductive objects and the test objects were the same. Furthermore, specific restriction had a n effect on the first trials on the average, and a limited but equal contrast effect occurred for both types of stimuli. Specific restriction was limited at the point where the contrast effect was the most pronounced, an observation which clearly shows that the series effect was due to a value representation oriented towards a certain type of object with a n available trace in memory: when this representation changed, the readjustment was accompanied by a loss of specificity, thus causing the judgment dimension to which the response scale was matched to become more abstract. Manis and Paskewitz' third experiment (1984) provides direct evidence of specific restriction. This experiment involved the presentation of a stimulus in each class on each test phase trial, for subjects to compare. This technique is based on the intermodality comparison method, which we find in conjunction with the study of context effects in other research such as Manis (1967) and Mellers and Birnbaum (1982). On each trial, subjects were to decide who had the disorder, the handwriter or the term definer. This phase followed a context induction phase bearing on the definitions. The results were consistent with specific restriction: throughout the test, the definitions were chosen more often when the inductive context was "weakly pathological" than when it was "strongly pathological". The interpretation of this finding could be that comparison does not in fact bear on the stimuli themselves but on the relative position they implicitly occupy along the specified pathological dimension. The absence of an inductive context on the handwriting trials does not imply absence of a reference context, but rather of a balanced reference context. An adaptive function

The presentation of several facets of specific restriction by varying the type of judgment, along with the use of experimental designs aimed at producing context effects, has enabled us to better understand the consistency of this phenomenon. What functions does it fulfill? Perhaps a perceptual function: by restricting how the objects are positioned along the judgment dimension with respect

408

J. M. F a b r e

to perceptually homogeneous stimuli, specific restriction most likely makes this relative positioning easier to carry out. I t leads to a reduction in the information processing load: first, in the case of specificity linked to orders of magnitude, in the same manner as in marking with extreme stimuli, it reduces the field that must be covered to ensure stimulus differentiation and thus facilitates the differentiation of stimuli that are close to each other. Second, in the case of symbolic dimensions applied to different objects, it allows for the concrete apprehension of the judgment dimension, which need not be constructed abstractly in order to be applicable. The fundamental value of specificity nevertheless appears to lie elsewhere: that is. it enables a polysemy that is suited to the concerned judgment categories. Specificity is a n aspect of the cognitive structuring of the environment through which all objects are judged in reference to the class to which they belong on the basis of the currently relevant attribute: for the attribute in question, each object is situated with respect to a norm that is established on the basis of available information extracted from instances comparable to the object. An immediate consequence affecting judgment language (qualifications, estimations, rating scales) should be noted here: the same terms can be reused and applied to different classes of objects. The referent is what changes, but without infringing upon communication. How many elements would have to be added to the lexicon if context-dependent restriction of meanings did not exist?

CATEGORICAL DIFFERENTIATION According to the theory of categorical differentiation, when a mark superimposed upon a series of stimuli is correlated with the magnitude to be judged, judgments are affected in the following ways: accentuation of the differences between stimuli belonging to different classes relative to the marking, and a reduction in the differences between stimuli in the same class. The predictions in this theory can only pertain to "characteristics known or assumed to be systematically linked to membership in the different categories" (Tajfel, 1972,p. 277). In the Tajfel and Wilkes' experiment (1963).the subjects' task was to estimate the length of segments drawn on cards presented by an experimenter located in front of them. Stimulus classification was manipulated experimentally. A mark (the letter A or B)

Specificity and categorization in judgment

409

correlated to the to-be-judged dimension was written on the stimulus cards: the four longest segments had an A and the shortest had a B, or vice versa (disjoint groups). Compared to the control condition where a classification correlated with segment length was lacking, t h e correlation condition produced t h e expected differentiation effect. The difference between the mean estimates of segments 4 and 5 (which represented the boundary) was significantly greater in the correlation condition than in the other conditions. However, the intraclass slope reduction effect only occurred on the five trials added onto the second session for certain subjects, and did not obtain the conventional statistical significance level. The experiment was set up in such a way that categorization works like an added piece of information enabling the specific differentiation of the objects located near the middle of the values in the series, i.e. in the area most conducive to confusion. The categorical differentiation effect was thus identical to the effect of placing a distinctive mark at some point along the dimension. W a s the observed categorical differentiation effect as it was experimentally triggered a n d measured here simply t h e consequence of a response modification at the class boundary? With this hypothesis in mind, two experiments were conducted. In the first, run in collaboration with Sylvie Faure, two different response scales were compared, one consisting of discrete units (numerical system) and the other involving a continuum (analog system) offering subjects an unequal possibility of segmenting their responses into two disjoint classes. In the second experiment, the method chosen was the frequency effect method: U-shaped distribution versus rectangular distribution. The salience of the correlation was studied by having subjects identify the added cue. The categorization/response experiment The experiment was divided into two successive phases in such a way that all subjects went from the uncorrelated condition to the correlated condition via the introduction or modification of some added information. In the former case, no "secondary" information was presented during phase 1. whereas in the latter, subjects were given some secondary information during phase 1 that was not correlated with the stimuli and whose form was different from that in phase 2. In order to take the correlation into account,

4 10

J. M. F a b r e

subjects must be able to extract enough information from the stimuli, and the activity has to be organized to a certain degree. The main factor in this experiment, assumed to act upon activity organization, was the response scale used to contrast two conditions: a discrete system conducive to segmentation, and a n analog system. Task difficulty was manipulated by varying stimulus presentation time.

Methodology The experiment was conducted in visual perception conditions and the task was the absolute estimation of the length of line segments. The materials included a n Apple I1 computer and monitor. The stimuli were displayed horizontally in the center of the screen. Six stimuli with lengths ranging from 41 to 61 horizontal dots on the screen (in steps of four) were used. The superimposed signals were letters displayed on the screen. They were quite large (3 x 3 cm) and were blinking. For phase 1 of the uncorrelated signal condition, the letter was either a Y or a 2. depending on the trial, and was shown in the upper right-hand corner of the screen. For phase 2. the letters were A and B a n d were shown in the upper left-hand corner. Display time and stimulus onset time were controlled by the computer. Responding was done on pencil and paper. For each trial, subjects had a separate 21 x 15 cm sheet of paper for recording answers. In the discrete response condition, they were to write down their estimates of the length of the segment in centimeters, using decimal numbers if desired. In the analog response condition, there was a horizontal line dividing the sheet of paper in two. This line was to be used as a guide for reproducing a segment the same length as the stimulus. In addition to estimating stimulus length, in all conditions where a letter was displayed, subjects were to write the letter down in the upper right-hand corner of the answer sheet. Eight disjoint groups were set u p by crossing the following three dichotomous factors: (1) response scale: discrete or analog, (2) control phase 1: uncorrelated Y and 2, or no letters at all, and (3) stimulus display time: 500 ms or 1200 ms. The phase factor (uncorrelated, correlated) was crossed with subjects. The subjects were 32 undergraduate students. Each subject was tested individually in two distinct phases. Each phase consisted of 36 trials, and each stimulus was shown six times (in random order so as to avoid repetitions).

Specificity and categorization in judgment

411

The letters A and B used in phase 2 were associated with segments 1, 2, and 3 (A) and 4, 5, and 6 (B), respectively. The Y and 2 (phase 1) were distributed randomly, while maintaining the equal distribution of letters and stimuli across trials. Subjects were warned of the presence of the letters before each phase and of their location on the screen. They were instructed to write down the letter after having estimated the segment length on each trial. Each new trial began when the subject pressed the space bar on the computer. A preparation time of 1 second was allowed during which no signals were displayed on the screen. After this pause, subjects were warned by a beep. In the no-letter condition, the beep was followed by a 1-second pause after which the segment was displayed. In the letter conditions, the letter appeared immediately after the beep and lasted 1 second. The segment appeared one second after the letter disappeared. Subjects were requested to write down their answers after the segment had disappeared. The next trial was triggered by the subject when he or she pressed the space bar after having completely finished writing down the answer. In each phase, only the last 30 responses (five times for each stimulus) were analyzed. The differentiation index used, D/A, was the ratio of D - the index expressing interclass difference calculated after separate linear matching in each stimulus class (1,2,3 and 4,5,6) - to A, the index expressing mean differentiation between stimuli, the regression line between the mean responses and the stimulus series. A D/A value above 1 means greater differentiation at class boundaries than within classes, and thus may reflect a categorical differentiation effect.

Results Mean D/A by condition and by phase is given in Table 1. An outcome indicating a categorical differentiation effect (increase in D/A) was only observed for the discrete response scale, and only in three of the conditions. The analysis of variance yielded several interactions. In particular, the phase by response scale interaction was highly significant, and the Bayesian analysis (Upine & Rouanet, 1975; Lecoutre, 1983) confirms, with a unilateral guarantee of 0.90. that the difference between the interphase deviations obtained by response scale was at least equal to 0.48. This interaction cannot be interpreted as indicating the impossibility of expressing a difference

412

J. M. F a b r e

on median stimuli due to the use of the analog response scale. Indeed, we can see that in phase 1. all D/A ratios obtained with this response scale are greater than or equal to 1, and when the stimulus was preceded by a dichotomous signal, D/A is greater than 1.5. This high value for D/A on phase 1 was only maintained for the longer display time. In the numerical condition, the categorical differentiation effect did not occur when display time was short and no signal was presented in phase 1. This result suggests that at least in the deliberately uninformative conditions of this experiment (no prior presentation of the experimental correlation). the occurrence or non-occurrence of the effect is conditioned either by a sufficiently long display time or the presence of dichotomous signals in phase 1. Table 1. C a t e g o r i z a t i o n / r e s p o n s e e x p e r i m e n t : mean D / A by c o n d i t i o n and b y p h a s e .

Response s c a l e

Phase 1 Condition

Display time

Discrete

Analog

phase 1 phase 2

phase 1 phase 2

500

.95

1.93

1.57

-70

1200

1.34

1.86

1.66

1.67

500

.90

1.08

1.19

1.21

1200

1.33

1.70

1.08

.84

YZ

No letter

When at least one of these favorable conditions was satisfied, accentuation of interclass differences is manifested, but with the numerical response scale only. The joint positive effect of these two factors with the analog system ensured obtaining (in phase 1) and maintaining (in phase 2) a relatively high value for the differentiation index in the median zone, although it does not allow u s to conclude that categorical differentiation did occur since

Specificity and categorization in judgment

413

introducing the correlation did not in itself lead to a visible modification in this result. Categorization/distution experiment The method chosen in the categorization/distribution experiment was to compare the effects of a correlated salient signal with a U-shaped distribution to those obtained with a rectangular distribution. One initial assumption could be that a U-shaped distribution would more effectively transmit the correlation information since the majority of the stimuli are extreme in this case. The chances of confusion of stimuli in the two classes, which would occur for median stimuli, are relatively slight because few stimuli are involved. With a signal preceding the stimulus, as in the categorization/response experiment, and the attraction of the subject's attention to that signal, we can predict that the signal will favor the identification of the stimuli with the class to which they belong. This kind of reasoning does not take into account the specific role that the superimposed signal plays at class-boundaries. A U-shaped distribution generally contrasts the values of stimuli in the two classes of the partition more than a rectangular distribution, but not for class members occupying a median position in the series. Applying the frequency principle, it can be predicted that ( 1 ) if there is no categorical differentiation, there will be a context effect on the responses characterizing U-shaped distributions (Parducci, 1965; Parducci & Perret, 1971). i.e. the responses for the two classes will be closer to each other than with the rectangular distribution, and (2) if there is categorical differentiation, there will be a contrast effect in each category, due to the asymmetry of each of the display frequency distributions. In both cases, the assumed underlying mechanism should lead to a n effect that opposes the exaggeration of interclass differences. Methodology In this experiment, we attracted the subjects' attention to the superimposed signal by asking them to identify the signal aloud on each trial. With the rectangular distribution, two types of identification were used, only one of which was explicitly dichotomous. With the U-shaped distribution, only the dichotomous response was applied. The two factors crossed were the correlation

4 14

J. M. F a b r e

between the superimposed signal and stimulus (correlated, uncorrelated) and the type of signal identification (dichotomous, non-dichotomous) (disjoint groups). The 72 subjects were all undergraduates. Testing was done individually, and consisted of two identical phases separated by a short break. The experiment was conducted in visual perception conditions. The materials included an Apple I1 computer and a Philips monitor (46 x 36 cm screen). The stimuli were eight segments displayed horizontally in the center of the screen. In each block of 64 trials making up a phase, the number of times a given segment was presented was as follows: 8 times per segment for rectangular distribution: and 16 times (segments 1 and 8), 8 times (segments 2 and 7). and 4 times per segment for Ushaped distribution. The superimposed signals were series of beeps generated by the computer, one short signal (2 consecutive beeps) and one long signal (5 consecutive beeps). The segment presentation order was balanced across phases. The association of signals to stimuli was done as follows: in the correlated condition, the short signal was always associated with stimuli 1. 2. 3. and 4 (short) and the long signal, with signals 5. 6, 7. and 8 (long): in the uncorrelated condition, each signal was associated a n equal number of times with each of the segments. The computer keyboard was placed in front of the subjects, with the screen facing them 80 centimeters away. Subjects used the computer to indicate the beginning of each new trial and to respond. The response scale was a semi-open system based on 5 predefined ordered categories labelled A, B, C, D, and E. Subjects were told that the letter A corresponded to a short segment and the letter E, to a long segment. They also knew that if necessary, they could extend the response scale in two non-mutually exclusive manners: by adding more extreme categories coded A- and A-- on the one end and E+ and E++ on the other end, or by creating new intermediate categories as follows. A response involving two letters created a n intermediate category between the two. One A and one B, for example, meant that the answer was halfway between A and B, and two As and one B meant a n answer closer to A than to B, etc. Responding was done on the keyboard. The keys programmed for any given trial could be pressed more than once. The number keys in the top row of the Apple I1 keyboard were used. They were labelled with stickers (ranging from A-- on the 1 to E++ on the 9).

Specificity and categorization in judgment

415

A trial was defined as the time between two pressings of the return key by the subject. The beginning of each trial was signalled by a beep, followed one second later by the segment displayed on the screen. The time between the signal and the segment was used by the subject to identify the signal. Segment display time was left open. To record their answers, subjects were to erase the segment from the screen by pressing the space bar. They entered their responses on the keyboard and pressed the return key to initiate the next trial. The identification of the signal was done in one of two manners: in the dichotomous condition subjects used the words "short" and "long", and in the non-dichotomous condition they estimated how many beeps were in the sequence. The segment estimates were coded numerically. A scale ranging from 1 to 9 was used on which the 1 corresponded to A-and the 9. to E++. Intermediate responses were coded by a decimal number. For example, AB was coded 3.5, and AAB was coded 3.33. These data served as the basis for calculating a D/A index like the one used in the categorization/response experiment. In the present case, the differentiation studied opposed stimuli 1, 2, 3, and 4 to stimuli 5, 6,7.and 8.

Results The first outcome to check for was the possible effect of "short-long" signal identification. To make the control groups comparable to the groups in the correlated conditions, the index was calculated by taking into account only the responses following a short signal for stimuli 1 to 4 and only those following a long signal for stimuli 5 to 8. Mean D/A ratios are given in Table 2. The analysis of the results is broken down into three parts: effect of the correlation, short-long response effect, evolution across phases. Correlation effect The correlation effect had to be assessed separately for phases 1 and 2 since subjects were not familiarized with the material as they had been in the Tajfel and Wilkes experiment (1963). For phase 1, separate analyses of the U-shaped distribution and the rectangular distribution for the short-long and number-ofbeeps conditions yielded very clearly non-significant results (F < 1).

J. M. F a b r e

416

For phase 2,the outcome depended on the distribution. With the U-shaped distribution, the effect was still non-significant. With the rectangular distribution, however, the results indicate a difference between the correlated and uncorrelated conditions: for short-long responding, F = 2.871. p < .lo; for number-of-beeps responding, F = 5.621,p c .05. Pooling all rectangular distribution groups, the correlateduncorrelated comparison yielded F = 8.263. p < .01,and the difference between the two at the .90 unilateral guarantee level was always greater than or equal to .47. Table 2 . Categorization/distribution e x p e r i m e n t : mean D / A by c o n d i t i o n and b y p h a s e a n d d i f f e r e n c e ( p h .2 - p h . 1 ) . ph.1

ph.2

diff.

Correlated Uncorrelated

1.72 1.66

1.26 .87

-.46 -.79

Correlated Uncorrelated

1.47 1.34

1.44 .73

-.03 -.61

U-shaped distribution

Short/ long Rectangular d i s tr i b u t i o n Number Correlated of beeps U n c o r r e l a t e d

.86 .69

1.45 .45

+.59 -.24

Short-longresponse effect For phase 1, this effect can be assessed by opposing the set of eight groups giving one kind of response to the four groups giving the other, as well as by restricting the comparison to the control groups. On the whole, F = 14.28574,p < .01 (difference at least equal to .50). For control groups, F = 4.08542,p < .05 (difference at least equal to .29). For phase 2 and rectangular distribution, F < 1 for all groups pooled as well as for control groups. Thus, the marked effect found on phase 1 apparently disappeared on phase 2. This cannot be attributed to learning of the correlation since the control groups also exhibited this tendency.

Specificity and categorization in judgment

417

Evolution across phases The difference between phases 1 and 2 is negative for all control groups and all groups with a U-shaped distribution. An approximate correction which could account for the impact of the short-long response on the first phase would be responsible for the reduction or elimination of the difference in the U-shaped distribution case, and the equalization of the short-long and number-of-beeps groups in the rectangular distribution case. The reduction in the D/A ratio with U-shaped distribution, which is correlated with the virtual elimination of the long-short response effect, can be analyzed by studying the effect of the phase factor and by determining whether this factor varies as a function of the correlation. The findings were as follows: a significant drop in the ratio between phases 1 and 2 (F = 6.149, p c .02)and no significant interaction (F< 1). The decrease in D/A apparently does not depend on the presence or absence of a correlation. The main results of this experiment can be summarized as follows: 1. The association of a short-long response to the preparatory signal, which could also be implicitly associated to a stimulus, produced an effect under all conditions at the beginning of the experiment. But this effect dwindled as the experiment progressed, and was no longer found on phase 2. We might qualify the effect observed during the beginning trials as a "response availability" effect. Its systematic occurrence leads u s to wonder what, in situations where stereotypes and prejudices are applied without the repeated processing of relevant information, might be its role in the processes that cause effects of the categorical differentiation type. What is the real impact of such a response effect? 2. When the correlation was taken into account, it caused a categorical differentiation effect whenever the distribution of the stimulus series was rectangular, but apparently not when it was Ushaped. This conclusion must not be drawn hastily, insofar as no positive statistical testing was found to support it, but only the clearly non-significant nature of the correlated-uncorrelated difference on phase 2 for the U-shaped distribution. What is lacking in this "hollow" discovery is a negligible maximum bilateral difference at the .90 guarantee level, and a significant interaction between the correlated/uncorrelated and U-shaped/rectangular

418

J. M. F a b r e

oppositions on phase 2 (F < 1; in the correlation condition with the U-shaped distribution, D/A > 1). The results obtained for the U-shaped distribution are perhaps surprising since we could instead have predicted that the discrimination of stimuli belonging to different classes would increase as the classes became increasingly different from each other perceptually. Let me mention again two possibilities that are consistent with the finding of a reduced correlation effect: either the frequency effect increased the confusion in the median zone to the extent that it made the necessary information unavailable for the construction of classes (which means that the theory only applies in the case where the subject is able to establish a partition), or the categorization served as a setting for the occurrence of an intraclass differentiation effect in opposing directions in each class (which means that the theory is based on the presupposition that there is intraclass homogeneity). In both cases, the scope of the categorical differentiation theory is limited. DISCUSSION

Viewed separately, the concepts of specificity and categorical differentiation appear to go without saying, and to seem to meet the clearly defined goals of efficiency and economy. But when considered together, quite a more complex picture is painted. Beyond the manifest interactions that limit the generality of the effects, we might wonder whether the information provided by the experimental designs is all-encompassing. Starting from the wellknown experimental designs of Bevan and Pritchard (1963) and Tajfel and Wilkes (1963).we can easily conceive of experiments that could test these two approaches, the dependent variable being, for example, the response range corresponding to the elements of a given class. In the perspective of such a test, these designs become part of a more comprehensive organization. Let A be Bevan and Pritchard's control series. Let this series extend over a given range on the rating scale, which will be called r(A). Add an anchor stimulus: the range r'(A) covered by A decreases: r'(A) is less than r(A). Associate a distinctive mark to the anchor that triggers a specific restriction effect: the range r"(A) covered by A is equal to r(A). Let B be half of the range of the Tajfel and Wilkes series. From what we know about absolute judgments, presentation of the whole

Specificity and categorization in judgment

419

series decreases the range covered by B. Let r(B1 be the range covered by B presented alone, and r'(B1 the range it covers in Tajfel and Wilkes' control condition. Then ffB1 is less than r(B1. As to the effect of range on the assessment of €3. this control situation can be compared to the experimental condition in which the range of the series is extended (Bevan & Pritchard), or to the introduction of a n anchor stimulus as a standard (with multiple anchors, Sarris & Parducci. 1978). The experimental situation devised by Tajfel and Wilkes. in which a mark is introduced to distinguish B from the rest of the series, is analogous to Bevan and Pritchard's second situation (range f(A1 = r(A1). But it has the opposite effect: the range r"(B1 occupied by B is reduced even more, giving r"(B) c r'(B1. How can this contradiction be accounted for? For what reason would a difference in marking sometimes introduce a positive intraclass effect ("assimilation" of the elements that make u p the class), and sometimes a negative effect? The characteristic that distinguishes these experimental designs from those of more recent experiments is that they lead to hypotheses whose scope is not broad enough, whether we are speaking of a single anchor opposed to multiple classes, the magnitude of the difference between classes, or the relationship between the correlated signal and the value taken on by the to-beestimated stimulus, defined by whether the signal constitutes a property of the object or some associated piece of information. For example, a series of experiments by Manis, Paskewitz, and Cotler (1986) using the same type of material as Manis and Paskewitz (1984) provided evidence of potential intraclass negative effects with associated correlated information and with hardly contrasted test stimuli. In psychopathology judgments, a categorical induction by correlation learned with a n associated signal (source hospital) was shown to lead to a n intracategorical contrast effect rather than assimilation into the stereotype. Moreover, assimilation effects have also been found, but under specific conditions that are worth noting: either the superimposed information was the patient (rather than the group of patients), or the overall judgments of the patients themselves (and thus, of the classes) was added intermittently to the judgments of the elementary productions. Finally, the authors revealed another condition under which assimilation effects occurred: when the test items (occupying a median position on the judgment dimension) were mixed with the contextual items rather

420

J. M. F a b r e

than being presented on a subsequent phase. According to these authors, the induced contexts may thus have been less contrasting, favoring the assimilation of test elements. (Note that the conditional link between interclass differentiation and assistance in element differentiation is again found here). We cannot derive univocal explanations or explanatory elements from these particularities. The partial nature of the experimental designs used is evidently not the only cause. Perhaps we should give up on striving to make an old dream come true and admit that the psychophysical metaphor has but an illusion of generality! And admit that even by resolutely retaining a point of view that is both purely cognitive (information processing) and experimental (general concepts and tasks that are as analyzable as possible in terms of combinations of factors), complex phenomena must be studied on a scale that is closer to reality. For example, in the Manis, Paskewitz. and Cotler experiments intraclass assimilation was systematically brought about by adding a piece of information: the systematic qualification of the source hospital as psychiatric or general, regardless of the relationship between the hospital and the contextual stimuli. This is a nice example of a stereotype effect, but one which cannot be explained by the experimental paradigm (induction) used in Accentuation Theory. Obviously, although experimentally quite useful, distinguishing between the findings on the basis of the sign of the effects (positive or negative) is conceptually weak. Intraclass differentiation and intraclass assimilation both encompass processes that meet different functional goals. We can no longer legitimately deal with these issues without asking ourselves just what exactly the subject's task is, what type of information is dominant in performing that task, and what kind of verbalizable knowledge must the subject have available to make deductions. We can see that in psychology's chapter on the contribution of judgment biases to the study of cognitive processes, the page dealing with the relationships between stereotypes, language, and the diverse functions of cognitive activity such as differentiation is far from having been turned. REFERENCES Arnault de la Menardiere. M. & De Montmollin, G. (1985).La representation comme structure Cognitive en psychologie sociale. psvcholoeie Francaise. 239-244.

a,

Specificity and categorization in judgment

421

Bevan. W. & Pritchard. J.F. (1963).The anchor effect and the problem of relevance in . Choloa. 69.147-161. the judgment of shape. m a l of General Psv Brown, D.R. (1953). Stimulus similarity and the anchoring of subjective scales. American J o 66, 199-214. DFLollo, V. & Kirkham. R. (1969).Judgmental contrast effects in relation to range of stimulus values. Journal of ExDerimental Psvcholoev, 81,421-427. Doise, W. (1976). gartic-osociolocio ue et lese-er VrouDeg. Bruxelles: De Boeck. Doise. W. (Ed.) (1979). uDeS. Paris: Mouton. Eiser. J.R. (1971). Enhancement of contrast in the absolute judgment of attitude statements. Journal of Personality and Social Psvcholo&, 17. 1- 10. Fabre. J.-M. (1987). La relativisation des iuaements. Thtse d'Etat, Universite de Provence, Aix-en-Provence. Helson. H. (1947). Adaptation-Level a s frame of reference for prediction of psychophysical data. American Journal of PsvchOIO@Vv. 1-29. -Level Theory. New-York Harper & Row. Helson. H. (1964).-ion Johnson, D.M. (1949). Learning function for a change in the scale of judgment. Joumal of Emerimental Psvcholoey. 851-860. Kahneman. D. & Miller, D.T. (1986). Norm theory : Comparing reality to its alternatives. psvcholocical Review. 136- 153. King. D.L. (1983). Anchor research: Evidence for an increase in the perceived similarity of stimuli obtained with a speeded response paradigm. Journal of Emerimental Psvcholom: Human PerceDtion and Performance, 9,474-486. King. D.L. & Atef-Vahid. M.-K. (1986). Two extensions of the anchor-range effect. m D t i o n and Psvch ODhvsics, 96-104. Lecoutre. B. (1983). L'analvse buisienne des comDaraisong. Lille: Presses Universitaires de Lflle. Upine. D. & Rouanet. H. (1975). Introduction aux mCthodes fiduciaires: inference sur u n contraste entre moyennes. Cahiers de Psycholocie, 18.193-218. Manis, M. (1967). Context effects in communication. Journal of Personalitv and Social Psvchology, 5,325-334. Manis. M. & Paskewitz. J.R. (1984). Specificity in contrast effects: judgments of psychopathology. Journal of b erimental Social PsvcholQgy, 217-230. Manis, M., Paskewitz. J & Cotler, S. (1986). Stereotypes and social judgment. Journal of Personalitv and Social Psycholoa, 461-473. Mellers. B.A. (1983). Evidence against "absolute" scaling. PerceDtion and PsvchoDhvsics. 523-526. Mellers. B.A. & Birnbaum, M.H. (1982). Loci of contextual effects in judgment. Journal of Emerimental Psvchology: Human PerceDtion and Performance. 8.

v ,

a,

a, a,

a,

a.

m,

a,

582-601.

Parducci. A. ( 1965). Category judgment: A range-frequency model. Psvcholocical RevleW. 72,407-418.

Parducci, A. (1983). Category ratings and the relational character ofjudgment. In H.G. Geissler. H.F.J.M. Buffart, E.L.J. Leeuwenberg & V. Sarris (Eds.),Modem issues in DerceDtion (pp. 262-282). Amsterdam: North-Holland. Parducci. A.. Knoble. S. & Thomas, C. (1976). Independent contexts for category ratings: A range-frequency analysis. PerceDtion and PsvchoDhvsics. 20. 360366. Parducci. A. & Perrett. L.F. (19711. Cateaorv rating scales: Effects of relative sDacinc and frequency of stimulus values.-Jour;al of ExDerimental Psvcho loav -. Monoma 427-452. Sarrls. V. (1967). Adaptation-level theory: Two critical experiments on Helson's weighted-average model. American Journal of Psvcholoa, 80,33 1-344. Sarris. V. (1976). Effects of stimulus range and anchor value on psychophysical judgment. In H.-G. Geissler & Yu.M. Zabrodin (Eds.). Advances in BvchoDhvsicS. Berlin: VEB Deutscher Verlag der Wissenschaflen. 1

,a.

0

422

J. M. F a b r e

Sarris. V. & Parducci. A. (1978).Multiple anchoring of category rating scales. PerceDtion and PsvchoDhvsicS. 24. 35-39. Tajfel, H. (1959).Quantitative judgment in social perception. J oumal of 16-29. Tajfel. H. (1972).La cattgorisation sociale. In S. Moscovici (Ed.), Introduction a la psvcholoeie Sociale (Val. 1.272-300). Paris: LEUOUSS~. Tajfel, H. & Willces. AL. (1963).Classification and quantitative judgement. British Journal Of PSV&QJQ&. 101-114. Treisman. M. (1985). The magical number seven and some other features of categoIy scaling: properties of a model for absolute judgment. JDumal of Mathematical PWcholo@, B,175-230. Volkmann. J. (1951).Scales of judgment and their implications for social Psvcholom at the psychology. In J.H. Rohrer & M. Sherif (Eds.). Crossroads (pp. 273-294).New-York: Harper & Row. Zoeke, B. & Sarris. V. (1983).A comparison of "frame of reference" paradigms in human and animal psychophysics. In H.-G. Geissler, H.F.J.M. Buffart. E.L.J. Leeuwenberg & V. Sarris (Eds.), Modem issues in DerceDtion (pp. 283-318). Amsterdam: North-Holland.

m. a,

a,

a

SECTION 5 Biases in probabilistic judgment

This Page Intentionally Left Blank

Cognitive Biases C a v e r n i , J . -M. F a b r e a n d M . G o n z a l e z ( E d s . ) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. ( N o r t h - H o l l a n d ) , J . -p.

1990

BIASES IN PROBABILISTIC JUDGMENT A HISTORICAL PERSPECTIVE George Wright+ and Peter Ayton** **

* Bristol Business School, UK Decision Analysis Group, City of London Polytechnic, UK

Abstract.

This paper presents an overview of research that has been conducted on the human judgment of probability over the last thirty years. The last thirty years has been chosen because at the beginning of this period research began in earnest and over this period the view of human ability has changed from one of suboptimality and bias to one that is now less sure of the capabilities of human judgment.

Interest in subjective probability judgments by psychologists can be dated to a seminal paper by Ward Edwards entitled "The theory of decision making" which appeared in the Psvchological R i l l l e t i n in 1954. This article presented what was essentiaIly an economic decision theory as a psychological or subjective decision theory. Subiective exDected ut ility theory, as it became known over the next few years specified that normative decisions could be prescribed on the basis of two independent sources of information: subjective probabilities attached to the occurrence or nonoccurrence of future events, and utilities or subjective values attached to the possible outcomes of the interplay between human actions and events at some time in the future. (For an introduction see Wright, 1984.) In t h e 1960s this psychological decision theory was incorporated into a technique called decision analvsis and was pioneered by Howard Raiffa and Robert Schlaifer as a technique to improve decision making. However, these business-school-based proponents of decision analysis were more concerned with the computational aspects of the technology which implemented S E U and less with the subiective assessment of the inputs of probability and value. Also in the mid-1960s. Ward Edwards was working with Bayes' theorem which is a normative theory of the way in which subjective probabilities attached to the truth of hypotheses should be revised in the light of new information. (Phillips, 1973, introduces the

426

G. W r i g h t

&

P. A y t o n

statistical concepts and computational procedures.) To these studies we turn next.

Revision of probabilistic opinion In a number of studies, Edwards and his colleagues found that unaided human revision of opinion was often W than the theorem would prescribe. In other words, posterior opinion, given updated information, was not as extreme as that calculated by Bayes' theorem. This result h a s been termed "conservatism". Early research on the conservatism phenomenon used the "book-bag and poker-chip" paradigm. The basic paradigm was this: the experimenter holds three opaque book bags. Each contained one hundred poker chips in different, but stated, proportions of red to blue. The experimenter chooses one bag at random, shuffles the poker chips inside and successively draws a single poker chip from the bag. After each draw, the poker chip is replaced and all the poker chips are shuffled before the next drawing. The subjects' task is to say how confident (in probability terms) he or she is that the chosen bag is Bag 1, Bag 2 or Bag 4. The colour of the poker chip drawn on each occasion from the same bag is information on which to revise prior probabilities of 1 / 3 , 1 / 3 and 1 / 3 for the three bags. How well does Bayes' theorem describe h u m a n opinion revision? The data from a large number of laboratory studies, using tasks very similar to the one we have described shows that the amount of opinion revision is often less than the theorem would prescribe. However, the amount of "conservatism" shown in a particular task is highly situation specific. T h e degree of conservatism has been shown to vary with the diagnosticity of the data. For example, imagine that you were being shown samples drawn from one of two opaque bags each containing a hundred coloured balls. One of the bags contains forty-nine red balls and fifty-one blue balls, while the other contains fifty-one red and forty-nine blue. Clearly, two consecutive samples each of a blue ball would not be very diagnostic as to which bag was generating the data. Experiments have shown that the more diagnostic the data the less optimal the subject. When the data becomes very undiagnostic, as in the above example, human probability revision can become too extreme.

Biases in probabalistic judgment

421

Another variable which affects the amount of conservatism exhibited is the way in which data is presented. According to Bayes' theorem it should not matter whether a series of data is presented sequentially or all at once. However, Peterson, Schneider and Miller (1965)found that subjects' estimates were less conservative when a sample of data was presented one item at a time, with probability revisions required after each item, than they were when they made a single estimate of their posterior probabilities based on the information contained in the whole sample. A further finding is that, even when data is presented sequentially, the order or sequence of the data influences probability revision. Pitz, Downing and Reinhold (1967)described a n "inertia effect," where subjects tended not to revise their probabilities downward once the initial part of a sequence of data had favoured one of the hypotheses under consideration. In other words, subjects seemed unwilling to reduce their probabilities on a favoured hypothesis following disconfirming evidence. DuCharme and Peterson (1968)investigated human probability revision in a situation they considered to be nearer to real life than the book-bag and poker-chip paradigm. They argued that the datum in the latter paradigm is usually restricted to one of two different types, e.g. a red or a blue poker-chip, and that there are usually only two or three possible revisions that can be made on the information obtained. In real life, information may vary along a continuum rather than being discrete values. In their study, DuCharme and Peterson used a hypothesis set consisting of the population of male heights and the population of female heights. The subjects' task was to decide which population was being sampled on the basis of the data contained in randomly sampled heights from that population. Using this task, DuCharme and Peterson found conservatism half as great as with the typical book-bag and poker-chip task. They concluded that this effect was due to their subjects' greater familiarity with the data generating process underlying their task. Winkler and Murphy (1973)took DuCharme's arguments one stage further. In an article entitled "Experiments in the laboratory and the real world" they argued that, even though the typical bookbag and poker-chip paradigm seems outwardly simple, it differs in four major respects from the real world. First, the inference tasks so far discussed differ from everyday situations in that, in most of the laboratory tasks, samples of data are conditionally independent. That is, two or more pieces of

428

G. W r i g h t

&

P. Ayton

information have a n identical implication for the posterior opinion to be placed on a particular hypothesis no matter if the pieces of information are considered jointly or in any sequence. In the real world, Winkler and Murphy argue, successive items of information may be, to a degree, redundant. This would mean that the total impact of several pieces of information would be less than the sum of the impacts of each item, assuming the other data had not been observed. To quote Winkler and Murphy: Therefore, one possible explanation for conservatism in simple book-bag and poker-chip experiments is that the subject is behaving as he does in more familiar situations involving redundant information sources (p. 256). Second, in most experiments using the book-bag and pokerchip paradigm the data generators (the book-bags) are stationary. That is, the contents of the book-bag remain the same during the experiment. In the real world our hypotheses may not remain constant. Indeed, the nature of information obtained may change our hypotheses. Third, in the real world data may be unreliable and therefore be less diagnostic than perfectly reliable data, like the colour of a poker-chip. In many real-life opinion revisions the probability assessor not only has to determine the diagnosticity of a piece of data b u t also its reliability. In support of Winkler and Murphy's argument, Youssef and Peterson (1973) have shown that, when laboratory tasks include unreliable data, human inference tends to be less conservative, but still not in accordance with Bayes' theorem. Fourth, subjects in the book-bag and poker-chip experiment are typically given highly diagnostic data. In the real world, data may be relatively undiagnostic and so the result of subjects generalizing their experience of the real world to the novel laboratory task may result in conservatism. In summary, Winkler a n d Murphy concluded t h a t "conservatism may be an artifact caused by dissimilarities between the laboratory and the real world." However, despite these criticisms of the results of laboratory experiments, there has been a considerable research effort into the development of computeraided Probabilistic Information Processing Systems (PIP Systems) that implement Bayes' theorem. In these systems, the probability assessor makes the probability judgment after each item of

Biases i n probabalistic judgment

429

information arrives, but the computer aggregates these assessments. Of course, it is usually impossible in real life to check the veracity of prior opinion, likelihoods and posterior opinion against a suitable agreed-upon criterion, as it is in laboratory investigations of opinion revision. This fact accounts for the rarity of studies of opinion revision in the real world that are analogous to those studied so intensively in the laboratory and very few laboratory studies have appeared since the mid 1970s. Another line of research that has been concerned with the quality of human judgment of probability has stemmed from the work of Amos Tversky and Daniel Kahneman. In a series of wellwritten and accessible papers starting in the early 1970s they outlined some of the heuristics, or rules of thumb that people use for probability assessment (e.g. Tversky and Kahneman, 1974). Much of this research is now incorporated into general introductory texts on psychology, perhaps because the research studies are easily understood, non-technical. and appealing to the nonspecialist reader. Heuristics in probability assessment To demonstrate Tversky and Kahneman's approach to data collection we will illustrate their experimental material by presenting you with two questions which we would like you to consider. 1. Suppose I sample a word of three letters or more at random from a n English text. Is it more likely that the word starts with an 'r' or that 'r' is its third letter?

2. Which cause of death is more likely out of each pair: (i) Lung cancer or stomach cancer (ii) Murder or suicide (iii) Diabetes or a motor vehicle accident? If your responses are like most of Tversky and Kahneman's subjects you may have said that 'r' is more likely to start a word. However, in reality, 'r' is more frequent as the third letter. Tversky and Kahneman argue that people approach the problem by recalling words that begin with an 'r' (e.g. road) and words that have 'r' as the third letter, (e.g. care). Because it is much easier to search for words by their first letter than by their third letter most people,

430

G. W r i g h t

&

P. A y t o n

Tversky and Kahneman argue, judge that words beginning with 'r' are more likely. Tversky and Kahneman have also demonstrated that we judge the probability of a n event by the ease with which relevant information of that event is imagined. Instances of frequent events are typically easier to recall than instances of less frequent events, thus availability is often a valid cue for the assessment of frequency and probability. However, since availability is also influenced by factors unrelated to likelihood, such as familiarity, recency and emotional saliency, reliance on it result in systematic biases. In a convincing study, Lichtenstein et al. (1978) found that people overestimated the relative frequency of diseases or causes of death which are much publicized, such as murder or lung cancer, whereas the frequency of less publicized causes of death, such as stomach cancer and diabetes, were underestimated. In all parts of question 2 above, the second alternative is about one and a half times more likely than the first alternative. Most people think that death by lung cancer is much more likely than death by stomach cancer and death by murder is much more likely than suicide. Conversely, most people think that death by motor vehicle accident is many more times likely than death caused by diabetes. Another heuristic tending to bias probabilistic judgment is representativeness. Consider the following problem adapted from Kahneman and Tversky (1972). This is a brief personality description of Tom W. written by a psychologist when Tom was in his senior year at high school: Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Self-centred, he nonetheless has a deep moral sense. This personality description has been chosen, at random, from those of thirty engineers and seventy social scientists. What is your probability that Tom W. is an engineer? ...

Biases in probabalistic judgment

431

You have probably answered that Tom W. is more likely to be an engineer than a social scientist. However, Kahneman and Tversky argue that the base rate should have predominance over the low-reliability personality sketch, such that your probability response should be little different, if at all, from the base-rate probability of a 0.7 chance that Tom W. is a social scientist. Using similar problems Kahneman and Tversky found that when no individuating evidence is given, base rates are properly utilized, but when worthless information is given, as in the above example, base rates are ignored. Kahneman and Tversky coined the term representativeness to refer to the dominance of individuating information in intuitive prediction. This finding also h a s implications for the research investigating the psychological validity of Bayes' theorem. Ignoring the base rate is, in terms of Bayes' theorem, the opposite of conservatism, the evidence for which we discussed earlier. The discovery that subjects sometimes neglect base rate information in probability estimation tasks may be another reason why research using Bayes' theorem as a "bench mark" for studies of opinion revision has virtually ceased (cf. Fischhoff & Beyth-Marom, 1983). Kahneman & Tversky concluded: "In his evaluation of evidence, man is apparently not a conservative Bayesian: he is not a Bayesian at all" (Kahneman & Tversky, 1972, p. 450). Tversky and Kahneman have also identified a heuristic called anchoring and adjustment. To demonstrate this effect, subjects were asked to estimate various quantities, stated in percentages (e.g. the percentage of African countries in the United Nations). Before they made their estimates, the subjects were shown an arbitrary starting value between 0 and 100 given by the result of a spin of a wheel of fortune. Subjects were required to indicate whether they considered this value too high or too low and then to give their own estimate. Different groups of people were given different starting values. Surprisingly, the arbitrary starting values had a considerable influence on estimation. For example, median estimates for the question posed above were 25 per cent and 45 per cent, for groups which received 10 per cent and 65 per cent as starting points, respectively. Reward for accuracy did not reduce the anchoring and adjustment effect. Slovic (1972) gave another example of anchoring in the subjective valuation of gambles. He found that in making these judgments people who find a gamble basically attractive use the

432

G. W r i g h t

&

P. A y t o n

amount to win as a natural starting point. They then adjust the amount to win downward to take into account the less-than-perfect chance of winning and the possibility of losing a small amount. Typically, this adjustment is insufficient and Slovic argued that this is why people price gambles inconsistently with straight choices between pairs of gambles where a monetary response is not required, Yet another bias in judgment has been identified as misperception of regression towards the mean. Suppose a large number of children have been given an intelligence test. Some children scored above the mean, some below. If an equivalent test was then administered to the children, those children who scored above the mean on the first test would obtain a lower average score on the second test, whereas those children who scored below the mean on the first test would obtain a higher average score on the second test. Random fluctuations around the mean will produce sub-standard and above-standard performance which is highly likely to be followed by a n improvement and a worsening respectively. This phenomenon, known as regression towards the mean, was first documented by Galton a century ago. Failure to understand the importance of regression can have important consequences, as Tversky and Kahneman have documented. They found that flight instructors typically praised the trainee pilot after the successful execution of a flight manoeuvre and admonished him after a poor performance. Lack of understanding of regression towards the mean led the flight instructors to the erroneous conclusion that praise is detrimental to learning whereas punishment is beneficial. Tversky and Kahneman argue that people tend to use causal explanations to explain random variation. Tversky and Kahneman have devised many simple paper and pencil tests that have revealed heuristics people commonly use when making judgments about uncertain events. According to Tversky and Kahneman these heuristic principles may be quite useful in that they reduce the complexity of probability estimation, but they can lead to severe and systematic errors, analogous to the effects of the perceptual illusions. For example, perceptual psychology has documented that one cue to distance evaluation is the clarity of an object's image on the retina. The sharper the image of an object, the closer it appears to be. Clarity as a cue to distance has obvious general validity. However, when visibility is poor, distances are overestimated because the contours of an object are

Biases in probabalistic judgment

433

indistinct. Similarly, when visibility is good, distance is underestimated. How can the biases of the heuristic principles be reduced so that the process of probability estimation is improved? In a paper entitled "Intuitive prediction: Biases and corrective procedures" Kahneman and Tversky (1979) have attempted to answer this question. Their "strategy of debiasing" essentially concentrates on the elicitation and recognition of the importance of the usually neglected, but otherwise available, base-rate information for use in prediction. However, in the last five years, Tversky and Kahneman's research has become the subject of critical reaction. Tversky and Kahneman's demonstrations of judgmental fallibility, it is argued, should not be taken as having pejorative implications for the rationality of human reasoning (Cohen, 1981) and may not always occur in circumstances more akin to real-world decision making (Ebbeson & Konecni. 1980). In one attack on the status of the "most broadly accepted view that, in general, human judgment is seriously biased," Beach Christensen-Szalanski and Barnes (1987) argued that this view is not supported by convincing data. First, they identified what they termed the "citation bias." Of the 3,500 articles on judgment and reasoning published between 1972 and 1981 only eighty-four were empirical studies. Of these forty-seven obtained poor performance and thirty-seven obtained good performance. However, poor performance results were cited an average of six times more often than were good performance results. Second they argued that the "word problems" developed by Kahneman and Tversky may not be fully understood by subjects and further may not generalize to workaday judgment and reasoning. Recently Macdonald (1986. 1987) has argued that in natural verbal discourse questions are always motivated in the sense that they are only asked when there is some reason to expect a positive answer. Thus the social context of the request, the phrasing of the question and the perceived motivation of the questioner may influence the subject's perception of the answer to likelihood questions. Macdonald (1987) argues that subjects perceive questions as suggestions, and shows that these suggestions are enhanced with repetition. Further, when the experimental procedure is reorganised to reduce the need for such repetition there is a

434

G. W r i g h t

&

P. Ayton

consequent reduction in apparent fallacies of probabilistic reasoning. Pitz (1977) has demonstrated that very small changes in the experimenter's task can produce evidence of poor or good judgment. Pitz gives a n example of a situation in which poor probabilistic judgment is evident. He quotes a question from Tversky and Kahneman (1971, p. 107). The mean IQ of the population of eighth graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational achievements. The first child tested has an IQ of 150. What do you expect the mean IQ to be for the whole sample? Tversky and Kahneman reported that a large number of subjects responded with an estimate of 100. They concluded that these subjects expected samples in the long run to generate extreme values in one direction that would cancel extreme values in the other direction, one version of the gambler's fallacy. Pitz noted (p. 106) however, that it is possible that subjects are simply using a well-learned rule: "Given a known population mean and a randomly selected sample from that population, the expected sample mean is equal to the population mean.'' Pitz hypothesized that other rules may exist, for example, "Given two random samples, the expected mean of the second sample is independent of information about the first sample." but that they had a lower priority. Pitz (1977. p. 260) demonstrated this by rewording Tversky and Kahneman's example in this manner: The mean IQ of the population of eighth graders in a city is known to be 100. You have selected a random sample of 50 children for a study of educational achievements. The first child tested has an IQ of 150. What do you expect the mean IQ to be for the remaining 49 children?

As Pitz notes, people who have given an answer of anything other than 100 are rare, and usually turn out to have misunderstood the question. From this finding, Pitz developed a theory of a "production system" of decision rules within the individual that produces responses. In order to show the presence of a rule in a person's production system, it is, of course, necessary to prevent a

Biases in probabalistic judgment

435

response from being generated on the basis of higher order productions. Pitz's production system representation of contingent judgment is analogous to an "existence demonstration" which we shall return to in a later section on the calibration of subjective probability forecasts. Beach et al. also make the point that subjects who typically respond to the word problems and undergraduate students who are unrepresentative of the people are commonly regarded as qualified judges. In other words the experimental subjects would profess not special expertise in the subject matter of the word problems. A final point that Beach et al. make is that judgement should not be isolated from action since judgment is seldom an end in itself but a guide to subsequent action. Christensen-Szalanski (1986) has also questioned the generalizability of laboratory findings of cognitive bases in decision making to the real world -- and concluded that, even if the biases are prevalent, this knowledge by itself is of little use since we need to "determine the degree to which the decision outcome would improve once the cognitive bias was eliminated, a n d whether this improvement merits the expenditure of resources needed to accomplish it" (p. 399). Another approach to investigating the validity of human judgment has been that of calibration measurement. We turn to this topic next. Calibration of subjective probability forecasts For a person to be perfectly calibrated, assessed probability should equal percentage correct over a number of assessments of equal probability. For example, if you assign a probability of 0.7 to each of ten predictions, you should get seven of those predictions correct. Similarly, all events that you assess as being certain to occur (1.0 probability assessments) should occur. For a full review of this aspect of judgmental forecasting, see Wright and Ayton ( 1987a). In the 1970s studies of calibration used almost exclusively general knowledge items in the form of dichotomous questions such as: "Which canal is longer? (a) Suez Canal (b) Panama Canal". Subjects are required to indicate the answer they think is correct and then to assess a probability between 0.5 and 1 to indicate their degree of belief in its correctness. General knowledge a u e s t ions have been extensivelv used in studies of calibration because subiects'

436

G. W r i g h t

&

P. Ayton

. the answers ca n be immediatelv and convenientlv evaluated bv exDerimenter, This research has documented the generality of "over-confidence'' in probability assessment. Generally, for all propositions assessed as having a 0.XX probability of being correct less than XX percent actually are correct. (Lichtenstein 8t Fischhoff, 1977). Lichtenstein. Fischhoff and Phillips (1981)provide a full review of this general knowledge-based research. However, Wright (1982) h a s argued that probability assessments for future events involve different cognitive processes than those invoked in putting a probability to the veracity of one's own memory. Wright (1982).and Wright and Wisudha (1982),have shown differences in calibration and related measures for sets of questions where the answer is already known (general knowledge verification) and where the answer is not known at the time of the probability assessment (judgmental probability forecasting). In general, people do not use as many certainty assessments in judgmental probability forecasting and forecasts tend to be better calibrated (Wright & Ayton. 1987b: in press). One instance where judgmental probability forecasts are routinely generated is weatherforecasting. The official forecasts issued by the National Weather Service in the United States are subjective probability forecasts. Murphy and Brown (1985)have evaluated these subjective forecasts and found that, for certain predicted categories of weather, they were more accurate than the available objective statistical techniques. In this case the forecasters have a very large amount of information available, including the output from statistical techniques. They also receive detailed feedback and have the opportunity to gain experience of making forecasts under a wide range of meteorological conditions. Furthermore they have considerable practice in quantifying their internal state of uncertainty. These circumstances may well be ideal for the relatively successful application of judgmental, as compared with quantitative, forecasting, Additionally, good calibration has been demonstrated in several real-world forecasting situations apart from weatherforecasting. These situations include horse-racing (Hoerl & Falbin, 1974).prediction of future interest rates by bankers (Kabus. 19761, and prediction of the success of R & D projects (Balthasar. Boschi, & Menke, 1978). Phillips (1987)has argued that interest should now focus on what DeoDle can do under favourable conditions whereas the

Biases i n probabalistic judgment

431

research literature has tended to be dominated by reports of what people actually do without help, guidance or training. He makes the point that conditions need to be appropriate for the generation of precise reliable and accurate assessments of probability. He lists eight conditions which need to be satisfied. These include training in probabilistic thinking if the assessor is unfamiliar with probability concepts and the use of experts with substantive expertise in the area where judgments are required. We turn next to discuss issues regarding the concept of "exDertise". Expertise I t might be argued that much of the judgmental input to forecasting is made by people who would claim they benefited from the possession of some degree of expertise. A question then naturally arises: How good are experts ' judgments? Experts know more than novices but are their judgments less vulnerable to bias? Studies of the quality of 'expert' judgment, when the experts being evaluated are operating realistically in the domain of their expertise are, understandably, somewhat thin on the ground. Even when, individually or collectively, experts can be found who are willing to cooperate to the extent necessary the quality of judgments made by experts is not always readily amenable to analysis. In many cases it may not be possible to collect the kind of data that enable inferences concerning the validity of their judgments to be made. This may be because their judgments are undertaken in complex environments where there is no obvious or available index of judgmental quality. As to the quality of expert judgment studied in the laboratory there is a tension in the literature. Some researchers report evidence that experts are vulnerable to the same kinds of errors that might be predicted from Kahneman and Tversky's (and other) studies of students and paper and pencil tasks. Other researchers claim that experts are better judges than the research with paper and pencil tasks would imply. Likelihood estimation tasks - even when these are not explicitly concerned with probabilities for the future - are of particular interest because forecasts are often ideally expressed via the medium of probability. Christensen-Szalanski. Beck, Christensen-Szalanski and Koepsell (1983) studied the influence of availability on estimates of the risks (mortality rates) of various

438

G. W r i g h t

&

P. Ayton

diseases. They examined the relationship between reporting of diseases in medical journals and the estimates given by a group of experts (physicians) and nonexperts (college students). To their surprise, they found no statistically significant evidence for a link between the incidence of reporting in the journals influencing the judgments (though as the correlation between journal coverage and mortality is quite high, it is more difficult statistically to identify an effect). But they did find both groups overestimated the risks. In general, physicians were more accurate than students but the estimates of both groups were found to be biased by actual encounters with people with the disease. Christensen-Szalanski et al. suggest that experts and nonexperts may use similar thought processes but make differently biased judgments because of their different experience of the risky events. Northcraft and Neale (1987) report a study of real estate agents in Arizona which explored the influence of another heuristically driven bias discussed by Kahneman and 'hersky. that of anchoring and adjustment. Students and real estate agents toured properties and made pricing decisions: their judgments confirmed the hypothesis that manipulated valuations apparently made by the property owner would act as an "anchor" for the values assigned to properties and influence judgments accordingly. Northcraft and Neale conclude: ...the findings of this study provide strong evidence that previous laboratory research on decisional heuristics and biases is applicable to 'real world' information rich interactive estimation and decision contexts (p. 96). They further argue that: ...decision biases and heuristics are more than just parlour tricks and ... should play an important role in our understanding of every-day behaviour (p. 96). However, as we have seen in the previous section, on the "calibration of subjective probability forecasts," the quality of judgments made in work-a-day contexts can show good performance. Our view is that performance-demonstrated expertise in probability judgment is underpinned by practice and regular performance feedback. As Einhorn and Hogarth (1978) have argued, most judgmental forecasts are made without the benefit of accurate feedback. They trace these difficulties to two main factors. The first is a lack of search for and use of disconfirming evidence

Biases in probabalistic judgment

439

and the second is the use of unaided memory for coding, storing, and retrieving outcome information. In addition, predictions instigate actions to facilitate desired outcomes and indeed, outcome feedback can be irrelevant for correcting poor judgment. Einhorn and Hogarth give the following example: Imagine that you are a waiter in a busy restaurant and because you cannot give good service to all the people at your station, you make a judgment regarding which people will leave good or poor tips. You then give good or bad service depending on your judgment. If the quality of service, in itself, has an effect on the size of t h e tip, outcome feed-back will "confirm" t h e predictions ("they looked cheap and left no tip -- just as I thought"). The extent of such self-fulfilling prophecies is much greater t h a n we think and represents a considerable obstacle to learning from outcome feedback. It is clear that such "treatment effects" where actions in the world can determine subsequent outcomes may be more prevalent in some types of workaday situations than others. Clearly in weather forecasting, where the likelihood of events cannot be altered by the forecaster, forecasters are immune to the irrelevant outcome feedback cited in the example above. CONCLUSION

This paper has documented evidence for a shift in the pattern of studies of the revision of probabilistic opinion, heuristics in probability assessment and calibration of probability forecasts. Early research in each area showed evidence of subject bias or suboptimality. Later research questioned the ecological validity of the tasks used in the earlier studies. Notions of ecological validity link closely to subjects performing tasks about which they possess some expertise. Studies of informed subjects provide evidence of what Wallsten and Budescu (1983) call an "existence demonstration" in that there do exist conditions under which people can provide subjective assessments which are adequate by a variety of criteria. It follows that the question of whether there are severe and persistent biases in probabilistic judgment that occur in conditions that matter

G. W r i g h t

440

&

P. Ayton

and are produced by people who care is, as yet, a fairly open research question.

REFERENCES Balthasar. H.U.. Boschi, =A.

& Menke. M.M. (1978). Calling the shots in R and D. . 151-160. Beach, L.R. Christensen-Szalanski, J.J.J.. & Barnes, V. (1987).Assessing human

judgment: has it been done, can it be done, should it be done? In G. Wrlght & P. Ayton (Eds.). Judmnental Forecasting. Chichester: Wiley. Christensen-Szalanski. J.J.J. (1986).Improving the practical utility of judgment research. In B. Brehmer. H. Jungerman, P. Lowens, & G. Sevon (Eds.). pirecti-rch on D e c w a k i n g . New York: North-Holland. Christensen-Szalanski, J.J.J..Beck, D.E., Christensen-Szalanski, C.M., & Koepsell. T.D.(1983).Effects of expertise and experience on risk judgments. Journal of ADDlied Psvchology. 68,278-284 Cohen, L.J. (1981).Can human irrationality be experimentally demonstrated?. &T , -aheB 4.317-370. DuCharme. W.M.. & Peterson, C.R. (1968).Intuitive inference about normally distributed populations. 269-275. Ebbesen. E.G.. & Konecni. V . J . w decision making research: what do we know about decisions in the real world?. In T. S. Wallsten (Ed.). w ive Pr-s in Choice and Decision M&&g . Hfllsdale, N J : Erlbaum. The theory of decision making. -ical Bulletin. 5.l. 380Edwards, W. (1954). 417. Einhorn. H.J.. & Hogarth, R (1978). Overconfldence in judgment: persistence of the illusion of validity. Psvcholoeical Review. 395-416. Hoerl, A.. & Falbin, H.K. (1974).Reliability of subjective evaluations in a high incentive situation. J o u m a l v a l - S 227-230. W Review. Mav-June Kabus, I. (1976).You can bank on uncertainty. m a r d B 95-105. Kahneman, D.. & Tversky. A. (1972).Subjective probability: a judgment of representativeness. Cocnitive Psvcholoa. 3.430-454. Lichtenstein. S..Fischhoff, B.. & Phillips. L. D. (1980).In D. Kahneman. P. Slovic. & A. Tversky (Eds.). (1982). &&pent under uncertaintv: heurii . New York Cambridge University Press. Lichtenstein, S.,Slovic. P..Fischhoff, B., Layman, M.. & Coombs, B. (1978). Judged frequency of lethal events. Journal of ExDerimental Psvcholom: Human rnin an Memo .4.551-578. M a c d o n W R . w v o r e about or -lions in context. Paper presented at the eleventh research conference on subjective probability. utility and decision making. Cambridge. England. Murphy, A.H., & Brown, B.G. (1985).A comparative evaluation of objective and subjective weather forecasts in the United States. In G. Wright (Ed.). Behavioural Decision Making, New York Plenum. Northcraft. M.A.. & Neale, G.B. (1987).Experts, amateurs and real-estate: an achoring and adjust perspective in property pricing decisions. Qrpan isationaJ Behadour and H w Decision Pro, 84-97. Peterson, C.R.. Schneider. R.J.. & Miller. A.J. (1965).Sample size and the revision of subjective probability. , yg-d 69,522-527. Pitz, G.F. (1977). Decision making and cognition. In H. Jurgerman & G. de Zeew (Eds.), pecision Maw-f, m Amsterdam: D. Reidel. Pitz, G.F.. Downing, L.. & Reinhold. H. (1967).Sequential effects in the revision of subjective probabilities. Canadian Journal of Psvc?u&gy. 381-393.

.a.

a.

.m,

m.

a,

a.

Biases in probabalistic judgment

441

Phillips, L.D. (1973). .&yesian Statistics for Social Scientists. London: Nelson. Phillips. L.D. (1987).On the adequacy of judgmental forecasts. In G. Wright & P. Ayton (Eds.). Judmnental For e c m Chichester: Wfley. Slovlc. P. (1972). From Shakespeare to Simon: speculations -- and some evidence -about man's ability to process information. Research MonoUraDh, J2(12). Oregon Research Institute. Tversky. A.. & Kahneman. D. (1971).Belief in the law of small numbers. &ychological BulletQ. 26. 105-110. Tversky. A., & Kahneman. D. (1974).Judgment under uncertainty: heuristics and 1124-1131. biases. Science. Wallsten. T.S., & Budescu. D.V. (1983).Encoding subjective probabilities: a 151-173. psychological and psychometric review. Management Science Winkler. R.L.. & Murphy, A.M. (1973).Experiments in the laboratory and the real 252-270. world. w t ional Behaviour and Human Perfonnance, Wright, G. (1982). Changes in the realism and distribution of probability assessments as a function of question type. Acta P s v c w . 2,165-174. Wright, G. (1984).Behavioural decision theonr: an introduction. Harmondsworth: Penguin. Wright, G.. & Ayton. P. (1987a).Judmnental.-F New York Wfley. Wright, G., & Ayton. P. (1987b).Task influences on judgmental forecasting. Standinanvlan Journal of Psvch010@, 28,115-127. Wright, G.. & Ayton, P. Judgmental probability assessments for personal and impersonal events. International Journal of Forecastirw, in press. Wright, G., & Wisudha. A. (1982).Distribution of probability assessments for almanac and future event questions. Scandinavlan Journal of Pmchology, 219-224. Youssef, 2.1.. & Petersen, C.R (1973).Intuitive cascaded inferences. -sat ional m v i o u r a n w a n P d r m a n c e . JQ.349-358.

.

m.

a, m.

a

This Page Intentionally Left Blank

Cognitive Biases J. - P . C a v e r n i , J. - M . Fabre and M . G o n z a l e z ( E d s . ) 0 Elsevier S c i e n c e P u b l i s h e r s B . V . ( N o r t h - H o l l a n d ) , 1990

ARE TWO JUDGES BETTER THAN ONE? ON THE REALISM IN CONFIDENCE JUDGEMENTS BY PAIRSAND INDmuALs Carl Martin Allwood and Carl-Gustav Bj6rhag Department of Psychology, U n i v e r s i t y of Gothenburg Gothenburg, Sweden

Abstract. This study compares the confidence judgements of pairs of individuals (n = 2 1) to those of single individuals [n = 20). In both conditions subjects answered 80 general knowledge questions and rated their confidence in their selected answers. No difference was found between the pairs and the individuals in proportion of

correct answers, calibration and over/underconfidence. Drawing on previous research, it was concluded that several features of the social interaction in pairs may have contributed to these findings. For example, there was more argumentation among pairs than by individuals, suggesting more inferring in pairs. Moreover, the lack of correlation found between scores on the Need for Cognition test [Cacioppo & Petty, 1982) and calibration suggests that the greater amount of pair evaluation, for which we found some evidence, does not seem to help calibration. In addition, there was little negotiation within the pairs about the level of the confidence ratings and for about 20-25% of the items the second pair member did not make contributions that seemed to be of importance for the pair's response. Finally, the pairs showed better resolution than the single individuals. This was partly explained in terms of the greater spread of their confidence ratings [polarization) and the higher average level of their confidence ratings (risky shift) compared to the single individuals.

After retrieving knowledge from h i s / h e r memory, a n individual may attempt to judge whether or not the information retrieved is correct vis-a-vis some norm. The outcome of such confidence judgements often determines whether the person accepts the retrieved information as sufficient or continues to search his or her memory for more appropriate information. In many situations the realism of confidence judgements has important practical consequences. A common research result in the area of confidence judgements is that these judgements are unrealistic in the sense that people mostly overestimate or underestimate their knowledge in relation to some external conventionally accepted norm for the correct answer. To p u t this another way, people are poorly calibrated with respect to their own knowledge. When the proportion of correct answers is not very high, the usual finding is that people err in the direction of overconfidence. (For a review of

444

C.M.

Allwood & C.G. B j o r h a g

results, concepts and measures in this area, see Lichtenstein, Fischhoff, & Phillips, 1982). The extent to which the realism of confidence judgements is affected by different factors such as cognitive style, meta-cognitive factors, and emotional and social factors is not well researched. In the present study we investigate the importance of social interaction in pairs of individuals for the realism of their confidence ratings given for answers to general knowledge questions. The performance of pairs is compared to that of single individuals. The influence of social factors on confidence judgements is interesting to study for at least two reasons. First, it can be hypothesized that confidence judgements made by pairs to some extent will involve other processes than those involved in judgements made by single individuals. Comparing confidence judgements made by pairs with confidence judgements made by single individuals makes it possible to explore the extent to which processes activated in the social context influence the realism of the confidence judgements. Second, much decision making takes place in social settings and confidence judgements of the correctness of information pertaining to one or more of the decision alternatives often occur as part of the decision making. Accordingly, a better understanding of the effect of social factors on confidence judgements can be of great practical value. Koriat, Lichtenstein and Fischhoff (1980) have presented a three-stage model of the mental processes occurring in connection with confidence judgements. After the individual has decided on an answer to a knowledge question, he/she, in the first stage of the model, searches his/her memory for evidence of relevance to the confidence judgement. In the second stage of the model, the individual, on the basis of the evidence activated in the first stage, generates a general feeling of confidence about the answer. Finally, in the third stage, the individual "translates" his or her confidence feeling into a n appropriate response format, for example a qualitatively graded verbal response or a numerical confidence judgement. A general assumption made in the present study is that all information that is to some degree active in memory might affect the confidence feeling generated in stage two of the model of Koriat et al. Accordingly, not just information activated in stage one of the model, but also information activated when the individual attempts

Realism in confidence judgements

445

to find a n answer to the knowledge question might affect the individual's general feeling of confidence in the answer. At least two effects, relevant to confidence judgements, may occur in social interaction (Kaplan & Miller, 1983). First, new information that has not been considered by specific group members, or even any of the group members, may be generated as a result of the group interaction. Second, the opinions of some or a majority of the group members may act as a norm for other group members. Group interaction, even in groups as minute as pairs, can be expected to be complex. Different factors will influence the group decision in different ways. Drawing on earlier research, we hypothesize that the pair interaction will show various features with different implications for the realism of the pair's confidence judgements. Four such features are listed below together with their hypothesized implications for the realism of confidence judgements by pairs. Occurrence of the so-called "risky shift" phenomenon in groups, i.e. that groups tend to arrive at more risky decisions compared with individuals, could imply that pairs show poorer calibration (more overconfidence) than individuals. Given that people are usually found to be over-confident, a risky shift by itself would act to decrease calibration. However, research suggests that a more general description of this aspect of group effects is polarization, meaning that tendencies that occur in the group from the beginning will tend to be reinforced (Brehmer, 1977: Kaplan & Miller, 1983). Assuming a random distribution of group members, polarization will not have any clear consequences for the average calibration in pairs as compared with singles. However, the variation in the level of confidence judgements by pairs would be expected to be greater, compared to that of singles. This could result in better resolution in pairs. The "moderation" phenomenon, also reported in research on groups, complicates the analysis made above. The moderation effect means that initial tendencies in the group are moderated rather than reinforced. Previous research shows that the content of the group discussion and how it relates to the information already considered by the individuals in the group will decide whether polarization or moderation occurs in the group (see, Kaplan & Miller, 1983). Research reported by Kaplan (cited in Kaplan & Miller, 1983) shows that the moderation effect occurred when the

446

C.M.

A l l w o o d & C.G. B j o r h a g

shared information was in the opposite direction to the initial judgements of the group members. In contrast, the polarization effect occurred when the shared information was in the same direction as the initial tendency in the group. The general tendency of humans to seek verification evidence for their opinions would seem to suggest that the polarization effect would occur more often in groups than would the moderation effect. Another expected feature of the social interaction occurring in pairs is that more argumentation, and as a consequence of this, more inferring, may occur in a pair as compared with an individual. In a recent study testing individual subjects, Allwood and Montgomery (1987) showed that the strategy used to answer a knowledge question is important for the realism of the ensuing confidence judgement of the answer. The data showed that the subjects, for high confidence levels, gave more realistic confidence judgements when the chosen answer alternative immediatelv stood out as the correct one, than when they arrived at the chosen answer through different sorts of inferences. Generalizing from the results of Allwood and Montgomery (1987). the presence of more inferring in pairs might lower the calibration for subjects working in pairs. In contrast, a third feature of pair interaction might contribute to increasing pair calibration. Generally speaking, it may be easier for two persons together, compared to one person alone, to critically evaluate suggestions for answers or arguments indicating why one answer and not the other should be chosen. This suggests that two persons might have a better chance to detect weaknesses in the suggested answer compared to one. Given that pairs show more evaluative activity than singles, it seems likely that this may increase calibration among pairs. The fourth feature of pair interaction focuses on social factors. Arkes, Christensen. Lai and Blumer (1987) found that subjects who expected a group discussion of their answers to a set of knowledge questions showed less overconfidence in their answers compared with a control group. Since subjects working in pairs naturally expect to (and do) discuss their answers with their partner the tendencies reported by Arkes et al. (1987) might lower the degree of overconfidence in pairs, i.e. increase the degree of pair calibration. At the present state of research it is difficult to theoretically ground predictions of the quantitative strengths of the above effects. Thus, the extent to which these effects on the realism of confidence

Realism i n confidence judgements

447

judgements will cancel out, or whether one effect will be stronger than the others, is not clear. To our knowledge, the only research evaluating the effect of social interaction by comparing the realism of confidence judgements of pairs to that of individuals is a study by Sniezek and Henry (1989). In this study individuals made judgements concerning the frequencies of 15 causes of death (in the U.S. population of 230,000.000). The subjects were also asked to construct 99% confidence intervals around each frequency estimate. After each individual had carried out the tasks, groups of three participants each were created from the same individuals. Accordingly, Sniezek and Henry (1989) only analyzed the realism of very high confidence (99%) and each individual gave individual responses before he/she participated in a group response to the same tasks. The results show, among other things, that the groups were more accurate than the individuals and that both groups and individuals displayed "extreme overconfidence." Furthermore, groups set more narrow confidence intervals than individuals, i.e. in this sense displayed higher confidence, and simultaneously, the group confidence intervals more often contained the true frequency value. This is interpreted by Sniezek and Henry as meaning that the groups were better calibrated than the individuals. The extent to which this result is an effect of the so-called hard-easy effect (Lichtenstein & Fischhoff, 19771, i.e. that a higher percentage correct is usually (and at least partly for logical reasons) associated with less overconfidence, is not clear. Sniezek and Henry offer a similar argument: 'The groups knew more about risks but they did not necessarily know more about how much they knew" (p. 25). .As suggested above, two persons might have a better chance than one person to detect weaknesses in a suggested answer. In order to analyze the effect of evaluative processes on the realism in subjects' confidence judgements we asked the subjects to fill in a test developed by Cacioppo and Petty (1982) to measure the individual's "need for cognition" (henceforth the Nfc-test). Previous research by Pieters and Verplanken (1986) and Cacioppo, Petty and Morris (cited in Pieters & Verplanken, 1986) shows that individuals with high scores on the Nfc-test more often desired and searched for information which they did not have accessible in their own memories. Moreover, individuals with high Nfc-test scores made a more thorough evaluation of the arguments

448

C.M.

Allwood

& C.G.

Bjorhag

in a persuasive message than individuals with low scores on the Nfctest. By correlating subjects' Nfc-scores with their calibration scores we attempted to analyze the extent to which the described processes are important for high realism in confidence judgements. In the present study we analyze the realism of judgements by computing measures of calibration, overconfidence and resolution. Formulae and motivation for these measures a r e given by Lichtenstein. Fischhoff and Phillips (1982). Calibration reflects the relation between level of confidence ratings and accuracy. Calibration is computed by first dividing subjects' confidence ratings into a number of confidence classes. Next, for each confidence class, the difference is taken between the proportion of correct items at that confidence level and the mean confidence for the items in the particular confidence class. Finally, the squared differences are summed over confidence classes and divided by the total number of items. Over/underconfidence is computed in the same way, except that the differences are not squared. Both calibration and over/undercodidence are perfect when their value is zero. Resolution, finally, reflects the ability of the subject "to sort the events into subcategories for which the hit rate is maximally different from the overall hit rate" (Lichtenstein et al., 1982). A higher resolution score is better than a lower one.

METHOD Subjects

A total of 62 undergraduate psychology students participated in the study. Twenty subjects acted in the single subject condition and 42 subjects in the pair condition. The mean age in the single condition was 30 years (range 19-51 years). Fourteen women and six men participated in this condition. The mean age in the pair condition was 2 6 years (range 20-43). Thirty-three women and nine men participated in this condition. The subjects were randomized across conditions. Materials

General knowledge auestions. A total of 80 two-alternative general knowledge questions were given to the subjects. One

Realism i n c o n f i d e n c e judgements

449

question was not included in the analysis as the answer alternatives to this question were erroneously formulated. About half of the knowledge questions had previously been used by Allwood and Montgomery (1987). An attempt was made to see that the questions and their answer alternatives were not misleading to the subjects. For example, questions which subjects had found most difficult, in terms of proportion correct, in the Allwood and Montgomery study were not used in the present study. We also attempted to eliminate questions where the answer alternatives were on the same dimension, such as size items and geographical north-south items. This was supported by May's (1986a,b) examples which imply that subjects' knowledge plays a large role in determinirig their confidence at least for such items. An example of an item for which it is difficult to see the answer alternatives as lying on the same dimension is the question 'The most famous art gallery in Leningrad is:" with the two answer alternatives "The Hermitage" a n d "The Uffizi." Since we were interested in investigating the potential role of social interaction for the realism of confidence judgements we wanted to reduce the importance of the knowledge factor in this study. The questions covered topics including nature, society, film, history, geography and lexical knowledge. Need for Cognition Test. The subjects were also asked to fill in a Swedish translation of the Need for Cognition Test developed by Cacioppo and Petty (1982). This test contained 34 items.

Procedure Subjects in the single condition and pair conditions went through essentially the same procedure. First, subjects were asked to choose one of the two answer alternatives proposed for each of 80 general knowledge questions. After each choice, the subjects were asked to give a confidence judgement of how sure they were that their chosen answer to the question was correct. They were asked to use a scale ranging from 50% to 100%. It was explained to the subjects that 50% meant that the subject(s) could just as well have chosen the other answer alternative and 100% meant that the subject(s) was/were absolutely sure that the chosen alternative was correct. Subjects in the pair condition, i.e. subjects working together with another subject, were instructed to collaborate both when choosing a n answer alternative and when making the

450

C.M.

Allwood

&

C.G. B j o r h a g

confidence judgements. Thus. in contrast to the study by Sniezek and Henry (1989).the subjects in the pair group were not asked to answer questions or give confidence judgements individually before the joint effort of the pair. The instructions to the pairs especially stressed the importance of the subjects collaborating when making the confidence judgements. The subjects in the single subject condition were asked to think aloud both when choosing answer alternatives to the questions and when making their confidence judgements. This was done in order to produce more data about the cognitive processes of the subjects in the singles condition. The experimenter occasionally had to remind the subject to think aloud. The verbalizations of subjects in both conditions were tape-recorded with the subjects' consent. In the second phase of the experiment, the subjects were given a meta-cognitive question, asking them what things they thought one ought to consider when giving a confidence judgement of an answer to a knowledge question. These data will not be further considered here. Finally, the subjects were asked to answer the Nfc-test developed by Cacioppo and Petty (1982). The subjects in the pair condition filled in the Nfc-test individually. RESULTS

The main results are shown in Tables 1 and 2.Table 1 shows the frequency of confidence judgements, proportion of correct answers and mean confidence for different confidence levels and conditions. Calibration curves based on the data in Table 1 are shown in Figure 1. There was no significant difference between the conditions with respect to the proportion of correct answers out of the 79 items included in the analysis (t(39)= 1.23,p = .23). The mean proportion of correct items was .60 for the single condition and .63for the pair condition. Table 2 shows the results of measures of calibration, over/underconfidence and resolution for the two conditions. These measures were computed as described in Lichtenstein, Fischhoff. and Phillips (1982).As can be seen in Table 2 it was only resolution that differed significantly between the two conditions. The subjects working in pairs had better resolution than the subjects working alone.

Realism i n confidence judgements

451

Table 1. Frequency of confidence judgements, proportion of correct answers and mean confidence for different confidence levels in each condition.

Confidence l e v e l Condition S i n u l e (n = 20) Frequency Proportion correct Mean c o n f i d e n c e $hiT ( n

=

50-59

60-79

80-99

100

491 .54 .51

471 .56 .67

371 .60 .87

241 .77 1.0

3 65 .48 .51

4 53 * 57 .67

381 .62 .88

4 60 .82 1.0

21)

Frequency Proportion correct Mean c o n f i d e n c e

Table 2. Calibration, and other measures for the single and pair conditions: t-values for t-tests of differences between the two conditions.

Condition Measure Calibration O/u c o n f i d e n c e Resolut ion Confidence Mean Sd

Single

Pair

t-value

p-value

.050 118 ,021

.050 143 .032

.074 .895 2.55

.942 .376 .015

71.66 16.33

11.42 18.04

2.28 2.59

.028

.013

C.M.

452

40

.5

.6

.7

.a

A l l w o o d & C.G. B j o r h a g

.9

1.0

Confidence

Figure 1. Calibration curves for the single and pair conditions.

The difference between the conditions for the calibration score at the highest confidence level (100%) was not significant although nearly so (t(39)= 1.65, p = .11). However, the difference between the two conditions for over/underconfidence was f a r from significant at the 100% confidence level (t(39) = 1.04, p = .30). The difference between the single and the pair condition with respect to proportion correct at the 100% confidence level was close to significance (t(39)= 1.97, p = .06.). The last two lines of Table 2 show that there was a significant difference between the two conditions both with respect t o the level of the mean confidence rating and with respect to the standard deviations of the confidence ratings. The pairs were

Realism in confidence judgements

453

significantly more confident and had a greater spread of their confidence judgements than the singles. Number of words in protocols

Subjects' verbalizations in both conditions were transcribed from the tape recordings. In order to gain some insight into the processes behind the results reported above we first counted the words in the protocols for subjects in both conditions. This was done in order to gain a preliminary impression of the amount of activity going on in the two groups. These counts did not include subjects' reading of the knowledge questions. The mean number of words in the single condition was 2057 and in the pair condition was 4769 words, i.e. on the average, 26 and 60 words per item for each condition respectively. This suggests that more activity, possibly more inferring, went on in the pair condition compared with the single condition. Analysis of arguments

We next analyzed the protocols from the two conditions with respect to the amount of argumentation going on. The statements in the protocols were categorized with respect to the three following categories. Main argument. The subject states a main argument, i.e. an argument which directly argues for or against one of two answer alternatives, i.e. which is not only an argument for or against another argument. For example, in response to the question "If you see the nationality letter P on a car it is from:" (answer alternatives: Poland and Portugal) a subject said, "Poland, I think that is PL." The quality of the argument was not considered, and erroneous arguments counted as arguments. Arguments for or against another areument. For example in response to the question "In the old Nordic religion the God of the sea was called:" (answer alternatives: Balder and Agir) one subject said, "Balder, you a t least know that it is something God-like, but Agir. you don't know anything" (counted as Main argument). The other pair member then said, "Yes. I absolutely think that Balder was half God and half giant, ..." (counted as argument for another argument).

454

C.M.

A l l w o o d & C.G. B j o r h a g

Sure. believe and guess. The subject gives no argument but simply states that he/she is sure, or believes, or guesses that a specific answer alternative is the correct answer. For example, one subject read the question "Punjab is:" (answer alternatives: a people of India and an area in India and Pakistan), and the other pair member said, "It is the latter." This category was only used once for a question and it was not used If either of the two other categories applied to a question. The results from this coding are shown in Table 3. This table shows that in general more arguments were given in the pair condition than in the single condition. Table 3 also shows that in the pair condition there were many more arguments given for and against other arguments. The interjudge reliability for this coding, computed on a random selection of 25% of the subjects/pairs in each condition, was 85%. i.e. 85% of the codings were identical between two coders. In this percentage, we also included as nonidentical codings occasions where one coder used one of the above three categories for a segment in the protocol and the other coder did not apply any of the three categories to the same segment in the protocol. Table 3. Number of arguments and other types of statements given in the pair and single conditions. (In parentheses, the equivalent number adjusted for difference in n between the two conditions).

Condition S i n g l e (n = 20) (n = 21)

Pair

Main argument 851 1577 (1523)

Argument for/against o t h e r argument 43 372 ( 3 5 4 )

Sure, Believe, Guess 888 612 ( 5 8 3 )

The types of arguments categorized as Main arguments varied. In order to analyze potential differences between the two conditions with respect to the types of arguments in the Main argument category we coded the arguments in this group into a number of categories. The following categories were used (after each category,

R e a l i s m i n confidence j u d g e m e n t s

455

the percentage of the Main arguments classified in that category are given for the single and pair conditions, respectively). Knowledge authoritv. As a way to argue that a n answer alternative is correct or incorrect the subject claims to be knowledgeable in the content domain, for example due to prior studies or other type of prior experience. The category was also used when one subject in the pair claimed the other subject in the pair to be knowledgeable in the content domain as an argument for a specific answer alternative. For example, a subject claiming Tomtabacken to be the highest mountain in the county of SmAland, Sweden said, "I have lived in SmAland" (3% and 2%). Would have heard. The subject argues for/against a n answer alternative by claiming that he/she would have heard of it (seen it, read it, etc.) if the alternative were an answer to the question. For example, a subject argued against the idea that Rumania participated in the Second World War on the side of the Axis Powers saying, " ... if it had been the Axis Powers then one would have heard about it" (2% and 2%). Is pos sible. The subject argues for/against a n answer alternative by arguing that for some general reason, the alternative isjis not a possible answer to the question. For example, arguing that Obote is president of Kenya, one subject said, "I have heard of Obote, but Arap I've never heard of." Another example is a subject who argued against The Uffizi being the famous art gallery in Leningrad by saying, "I really don't think it sounds like serfdom's Russia either" ( 22% and 16%). Pragmatic. The subject argues for a given answer alternative by bringing attention to the context of the experiment. For example, one subject argued against an answer which in the pair interaction had gained some credibility by saying, "Maybe they are trying to fool u s completely" (1% and 1%). Other t w e s. If a Main argument was not classified in any of the previous four categories it was coded in the present category. The arguments in this category usually contained more explicit domain-specific content than arguments in the above categories. For example, arguing that Sn is the chemical symbol for tin and not for lead a subject said, "Pb is lead, ... so it must be tin" (72% and 79%).

The percentages given after the five categories coded among the Main arguments indicate that there was no marked difference between the single and the pair condition with respect to the type

456

C.M.

Allwood

& C.G.

Bjorhag

of arguments given in this group of arguments. The interjudge reliability for this coding, based on a random selection of subjects in the same way as above, was 87%, i.e. 87% of the codings within the Main argument category were identical between two coders. Only arguments that both coders in the previous coding had coded as Main arguments were included in the computation of the interjudge reliability figure given above. All discrepancies between the coders were resolved in a later discussion between the two coders.

Participation in the interaction in the pair condition In order to investigate the degree to which social interaction really occurred among the pair members when they decided on which answer alternative to choose, a further analysis was made. This analysis investigated the contribution of the two pair members to the decision process leading to the choice of a n answer. The interaction was categorized for each question item with respect to the degree of participation of the two pair members in the decision process. The following three categories were used. (After each category label, the number of question items coded in that category is given.) Interaction (1245). Both pair members make some substantial content contribution to the interaction. Still, of course, one member might make more contributions than the other. Both members know (105). This category was used for items where it was clear that both members in the pair independently knew the answer to the question from the beginning. To some extent this category can be viewed as a subcategory to the Interaction category just described. Another interpretation of this category, in contradiction to the one j u s t offered, is that the subjects did not interact in any substantial way, i.e. each member would have given the same answer a n d confidence rating irrespective of the social interaction. For example, in response to the question "Lapidus is a well known trademark in:" (answer alternatives: the ready-made clothing industry and the cosmetics industry) the pair said: S1: "Lapidus is a well known trademark in" S1 and S2: "the ready-made clothing industry." S 1: "Yes, there we answered directly."

Realism i n c o n f i d e n c e judgements

451

Dominance of one member (309). One member dominates the social interaction. The other member at the most only claims ignorance about the topic in the question. For example, the interaction in a pair in response to the question 'The agency of the United Nations for customs and foreign trade is:" (answer alternatives: GATT and IMF) was:

... GATT. I'm sure about it." S2:"You are sure about it?" S1: "Mm." S2: "Then we'll trust that." S1 (reads question): "

The number of instances coded into each category, given above after each category label, shows that both members of the pair made substantial content contributions in response to the large majority of the items (75%). Furthermore, there was a n explicit conflict (each member arguing for a different alternative) between the two pair members concerning which answer should be chosen for only 20 items, all in the Interaction category. For 27 of the items in the Interaction category there was an explicit compromise made between the pair members with respect to the level of the confidence rating. Two pairs were responsible for 23 of the 27 compromises. For 10 of the items in the Dominance category the "passive" subject affected (raised or lowered) the final level of the confidence judgement given by the dominant member. Furthermore, for 40 of the items coded as belonging to the Dominance category there was really too little data to make a coding with a n acceptable degree of certainty. The interjudge reliability, computed as above, for the coding of the questions into the three categories described above was .87.

Decision about confldence rating We were also interested in the extent of negotiation which took place in the pair condition in connection with the giving of confidence judgements. An analysis was made of the numerical values of the confidence judgements mentioned for each of the 1659 questions answered in the pair condition. The results show that in 1379 (83%) of the 1659 analyzed questions the subjects accepted the first given numerical suggestion for confidence

458

C.M.

dllwood

&

C.G. B j o r h a g

judgement as the final confidence judgement for that question. In 239 of these cases the confidence judgement accepted was not even verbalized and in the great majority (1076) of the remaining 1140 cases the initial numerical suggestion was not discussed at all. In these cases it seems that the subjects had reached some kind of mutual understanding of the confidence rating while they answered the knowledge question. In 84 of the 1659 ratings (5%)one of the two subjects initially suggested a confidence interval for the other subject to choose from. The average width of the interval offered was 8.4 units on the scale from 50 to 100 used by the subjects. In only 11 of the 84 cases discussed did the final confidence rating fall outside the interval initially suggested. Finally, in 196 of the 1659 ratlngs (12%) one subject first suggested an initial number for the confidence rating but the final rating differed by some amount from the initial suggestion. When coding the suggestions for confidence ratings in the order they appeared for a specific item we found that in 107 cases the final confidence rating exceeded the initial bid and in 89 cases the initial bid was lowered in the fmal rating given. The average amount by which the final rating differed from the initial rating was 10.1 units when the final rating exceeded the initial rating, and 10.6 units when the final rating was lower than the initial rating. Accordingly, the net effect of changes made to the initial level was negligible. The net effect of the changes made to the initial rating for the above 27 items, where explicit compromises were made between pair members with respect to the final confidence rating, was also negligible. The changes in Confidence ratings made in the single condition were analyzed so as to compare them with those made in the pair condition. The analysis utilized the think-aloud protocols from the single condition. Each time a subject mentioned more than one numerical value of the confidence rating to be given we noted the amount and direction of the change in value. The result shows that the subjects in the single condition changed their confidence rating in an upward direction for a total of only 20 questions (with a n average increase of 8.35 for each of the 20 questions) and in a downward direction for a total of only 13 questions (with an average decrease of 9.76 for each of the 13 questions). Thus, as in the pair condition, the net effect of the changes was negligible.

Realism in confidence judgements

459

Relation of Nfc-test to calibration and resolution

As described above, the subjects were also asked to fill in the Nfc-test developed by Cacioppo and Petty (1982). This test contained 34 items and our subjects filled in a Swedish version of the original scale. An abridged version of the test containing 18 items was published by Cacioppo. Petty and Kao (1984). A Principal Components Analysis (PCA) was performed both on the original 34item test and on the 18 items making up the abridged version from 1984. For the 34-item test we found one component which explained 21.48% of the total variance (eigenvalue = 7.30). A second component explained 7.5 1% of the total variance (eigenvalue = 2.55). When only the 18 items in the abridged version were included in the PCA the largest component explained 29.52% of the total variance (eigenvalue = 5.31). One further item was dropped from the final analysis since it differed substantially from the other 17 items both with respect to its low loading on the main component (.24, the second and third lowest items having loads of .38 and .41) and with respect to its low item-total Correlation (.30, compared to .38 and -43 for the second and third lowest correlated items). A PCA analysis on the remaining 17 items showed one component explaining 30.99% of the total variance (eigenvalue = 5.27). In these analyses the Nfc data from all 62 subjects were used. Using the scores from the 17-item Nfc-test. we next correlated the Nfc-test scores of the 20 subjects in the single condition with the same subjects' calibration and resolution scores. Both correlations were far from significant (r = .12, for the correlation involving calibration and r = .30 for the correlation involving resolution). No substantial differences in correlations were found when subjects' scores on the full 34-item test were used. DISCUSSION

The results in the present study show no significant differences with respect to calibration and over/underconfidence between subjects working alone and subjects working in pairs when they made confidence judgements of their answers to knowledge questions. In this connection it is of interest that we found no

4 60

C.M. Allwood

&

C.G. B j o r h a y

significant difference between the two conditions with respect to the proportion of correctly answered questions. I t seems likely that the results showing no difference for calibration and over/underconfldence are the effect of a number of interacting factors. The finding that the subjects in the pair condition verbalized more arguments compared with the single condition indicates that they more often decided on an answer alternative on the basis of inference. Given that the results by Allwood and Montgomery (1987) discussed above hold also for pairs, the greater amount of argumentation should have a negative effect on calibration in the pair condition. In the introduction it was suggested that more evaluation and critique of suggested answer alternatives or of arguments for answer alternatives would take place in the pair condition than in the single condition, and furthermore t h a t this would contribute to better calibration. The results show that more verbalization, and more argumentation, also in the form of arguments for and against other arguments, in fact took place in the pair condition compared to the single condition. These results serve to support the hypothesis that more critical and evaluative activity went on in the pair condition. However, from the fact that there was more evaluative activity in the pair condition it does not follow that more evaluative activity leads to better calibration. The finding that the subjects' scores on the Need for Cognition Test (Cacioppo & Petty, 1982) did not correlate significantly with the subjects' calibration and resolution is relevant in this respect. As noted in the introduction, the Nfc-test appears to be indicative of the degree to which subjects make explicit evaluations of arguments and the degree to which they tend to be aware of and search for information missing in memory. Although surprising, the lack of correlation found between scores on the Nfc-test and degree of calibration suggests that the type of evaluative processes of which the Nfc-test is indicative are not particularly helpful in themselves for achieving good calibration. The question of the relation between the presence of evaluative processes and realism of confidence judgements deserves more research. The average confidence level was significantly higher for the pairs than for the singles. This finding would seem to argue against the suggestion made in the introduction that subjects may lower their confidence ratings if they expect a discussion of their suggested answer with the other pair member. However, from this

Realism i n c o n f i d e n c e judgements

461

study we can only conclude that if this tendency existed it was not pervasive enough to survive the competition with other tendencies also occurring as effects of the social interaction in the pair condition. The fact that one pair member completely dominated the interaction as to the contribution of domain-specific information for about 20% of all questions probably contributed towards diminishing any potential differences between the pair and the single condition. Likewise, the questions where both members appear to have known the answer independently of each other (about 5% of all questions) may have contributed in the same manner. In contrast to t h e findings for calibration a n d over/underconfidence, the results for resolution show a difference between singles and pairs. That is, subjects working in pairs exhibited a greater ability than singles to classify items into subcategories, with a correct proportion that was maximally different from the overall correct proportion. The fact that the spread in the level of their confidence judgements was greater for subjects in the pair condition may have contributed to their better resolution. The larger spread of confidence judgements in pairs as compared with individuals can be seen as a polarization effect in the pairs. Considering that the overall correct proportion was about .6 in both groups, the pairs' higher level of confidence ratings, compared to the singles, may have increased the difference between the correct proportion in different classes of confidence judgements and the overall correct proportion, i.e. it may have contributed to their better resolution. The finding that the average confidence level was higher in the pair condition than in the single condition in spite of the fact that there was no significant difference between the conditions in the proportion of correctly answered questions can be viewed as a sign of the risky shift phenomenon. Two further results give some information about when the events leading to the risky shift phenomenon took place. These results are, first, the fact that there was very little negotiation in the pairs for the final confidence rating (for the great majority of the ratings the subjects accepted the first suggestion made by one of the pair members), and second, the fact that the negotiation that took place did not have any clear net effects on the level of the final confidence rating. These two results together suggest that the

4 62

C.M.

A l l w o o d & C.G. B j o r h a g

events leading to a higher average confidence level in pairs took place before the initial suggestions for confidence ratings were given. A more controlled approach is desirable in order to more precisely determine what features of the interaction lead to the presence of polarization and a risky shift in the pair condition. Our results differ from those reported by Sniezek and Henry (1989) in that these authors found improved realism in the confidence judgements (given in the form of confidence intervals) made by groups as compared with individuals, whereas we did not find this to be so for pairs as compared with individuals. As noted above, Sniezek and Henry only studied confidence intervals at the 99% level. Thus, a comparison between the two studies can only be made for very high confidence levels. However, our analysis shows that the difference in results noted above between the two studies holds also at the highest confidence levels. In the introduction we also reported that the single and group conditions in the Sniezek and Henry study differed in accuracy. This opens up the possibility that the difference in realism of confidence judgements found in that study could be explained by the difference in accuracy. The facts that the single and pair conditions in our study did not differ in accuracy and the fact that we found no difference between the conditions with respect to the realism of the given confidence judgements adds some support to this explanation. Furthermore, at the highest confidence levels in our study there was a tendency for the pairs to attain higher accuracy than the singles, and at this level there was also a (somewhat weaker) tendency for higher calibration in the pairs in comparison to the singles. A final comment on the Sniezek and Henry study is that the subjects in that study first acted as individuals and then as group members, whereas this was not the case in our study. We do not know to what extent this difference may have contributed to the inconsistency of the results across the two studies. Further research should be carried out to clarify these issues. I t is of interest to investigate to what extent more social activity in the pair condition than that found in the present study would have an effect on the realism of the given confidence ratings. It would be especially interesting to study the effect of increased activity when the pair discuss the level of the confidence judgements. Furthermore, it may be of interest to study differences

463

Realism in confidence judgements

in interaction and realism of confidence judgements between pairs and groups consisting of three or more members. REFERENCES Allwood. C.M.. & Montgomery. H. (1987). Response selection strategies and realism of confidence judgements. m a t i o n a l Behavior and H w n Decision Processes. 365-383. Arkes. H.R. Christensen. C.. Lai.. C.. & Blumer. C. (1987). Two methods of reducing overconfldence. 3 .33. 133-144. Brehmer. B. (1977). Decision makine in arouDs. Project report 12-77. Project title: "Risk generation and risk assessment in a social perspective". Unpublished manuscript. Department of Psychology, University of Uppsala, Sweden. Cacioppo. J.T..& Petty, RE. (1982). The need for cognition. Journal of Personalitv 42.116-131. Cacioppo, J.T.. Petty, & Kao, R E . (1984). The efficient assessment of need for cognition. Journal of Personalitv Assessment. 306-307. Kaplan, M.F., & Miller. C.E. (1983).Group discussion and judgement. In P.B. Paulus Drocesm. New York Sprlnger Verlag. (Ed.), Korlat. A.. Lichtenstein. S.. & Fischhoff. B. (1980).Reasons for confidence. Jwrnal Qf olom: Human and Memog. 6.107-118. L i c h t e n n o f f , B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and H u m a n perfo-. 159-183. Lichtenstein. S.. Fischhoff. B.. & PhilliDs. L.D. (1982). Calibration of mobabilities: The state of the art of 1980. In D. Kahneman. P. Slovic, & A. s e r s k y (Eds.). uric-tics and b i w . Cambridge: Cambridge University Press. May, R.S. (1986a). Inferences, subjective probability and frequency of correct answers. A cognitive approach to the overconfidence phenomenon. In B. Brehmer. H. Jungermann. P. Lourens. & G. Sevon (Eds.). New directions in .Amsterdam: Elsevier Science Publishers. May, RS. (1986b). Overconfldence as a result of incomplete and wrong knowledge. In R.W. Scholz (Ed.), Current issues in West German decision research. Frankfurt: P. Lang Publisher. Pieters. RG.M.. & Verplanken. B. (1986). of thinking about nuclear energy. Paper presented at the 1l t h Colloquium of the International Association for Research in Economic Psychology. Haifa, Israel. Sniezek. J.A. & Henry. RA. (1989).Accuracy and confldence in group judgement. Organizational Behavior and Human Decision Processes. G , 1-28.

a,

a,

a.

Acknowledgements. The authors wish to thank Regine May, Henry Montgomery and two anonymous reviewers for their comments on an earlier version of the manuscript. This research was supported by a grant from the Swedish Council for Research in the Humanities and Social Sciences.

This Page Intentionally Left Blank

Cognitive Biases J . - P . C a v e r n i , J. -M. F a b r e and M. G o n z a l e z ( E d s . ) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

UNCERTAIN MEMORIES:EVALUATING THE COMPETENCE OF PROBABILISTIC COGNITION Peter Ayton* and George Wright** *

Decision Analysis Group, Department of Psychology City of London Polytechnic, London, England ** Bristol Business School, Bristol, England.

Abstract.

Although much research has established that subjects typically overestimate the probability that they have selected the correct answer to a general knowledge question there is a dearth of psychological theory to explain this phenomenon. Here we present the rationale and results of a study designed to test a simple model concerning the origins of subjective uncertainty in semantic memory. Subjects were asked questions such as Which is the longest canal in the world?' and provided with one, two or no possible answers and required to rate their confidence in their answer. Our experiment tested the notion that internal states of uncertalnty arise from, and are calibrated by, subjects monitoring the number of alternative answers they are able to generate but cannot refute. The results indicate that providing more candidate answers to subjects improves their ability to calibrate their confidence that they have selected the correct alternative. These results lend qualified support to the model for subjective probability. The sensitivity and appropriateness of different dependent measures of probabilistic competence are briefly considered.

INTRODUCTION

Expressions of subjective uncertainty regarding the truth of general knowledge propositions and the likelihood of possible future events have been the focus of considerable investigative effort by experimental psychologists over recent years. Lichtenstein. Fischoff and Phillips (1982) have provided a review of this research which is principally concerned with the calibration of subjective degrees of belief. Calibration is one method for assessing the veracity of a set of subjective probabilities. A mathematical coefficient of calibration (developed by Murphy, 1973) can be computed to check the degree to which the probabilities given by a subject as a measure of their confidence in a set of propositions correspond to to the actual probabilities of the propositions being correct -- as measured by relative frequency. So, for example, a subject expressing 70% confidence in each of a set of twenty propositions being correct should get fourteen correct (70%) to be perfectly calibrated. The

466

P. A y t o n

&

G. W r i g h t

degree to which their subjective probabilities do not correspond to the hit rate will be reflected in the size of the calibration measure. The research investigating the calibration of subjective uncertainty has found that, typically, individuals are overconfident. Thus, than 70% of a set of statements rated with a subjective probability of 70% of being correct will actually be correct. Although this finding has been replicated many times, as yet no detailed psychological explanation of this phenomenon has been proposed. Pitz (1974) has suggested that the judgemental mechanism is overconfident because the information processing heuristics employed for probability judgement rely on short cut methods, but individuals wrongly assume that the inferences based on these heuristics have no possibility of error. Thus the system ignores or forgets its own inefficiency. No corroborative evidence has been offered as support for this hypothesis however. A major problem for researchers in this field is the lack of any theoretical framework for explaining the basis of subjective uncertainty. Psychological models of the processes implicated in the task of verifying a general knowledge proposition concentrate on how successful performance is achieved and do not attempt to represent uncertain or probabilistic responses (cf. Collins & Quillian. 1969; Smith, Shoben, & Rips, 1974). The lack of any coherent theory to explain the locus of uncertainty in knowledge verification can be viewed as particularly unsatisfactory when it is considered that although probabilistic judgements are usually badly calibrated they do, nonetheless, exhibit some of the features necessary for the responses to be considered valid probabilities. Typically the proportion correct varies as a function of subjective confidence and in a manner that preserves transitivity. Thus the proportion correct of all the judgements rated with a subjective probability of 70% will be less than that for judgements of 80% which, in turn, will be less than that for judgements of 90%. The empirically observed finding that human subjective probability assessments are poorly calibrated and usually overconfident might be attributable to a relatively trivial inability to accurately translate states of uncertainty into an accurate numerical label. The fact that the same responses are usually correlated with the veridical probabilities (as inferred from the relative frequencies of 'hits') to some extent indicates that the subjective states of uncertainty are somewhat valid. In short there is evidence for a

Probabilistic cognition

4 67

cognitive ability: subjects can give some useful information about their subjective uncertainty for their own knowledge. Presumably this uncertainty arises from the performance of computation on stored information; clearly it is not plausible that individuals store all propositions they may ever confront together with some uncertainty coefficient. In this sense, subjective probabilities for general knowledge questions can be viewed as being entirely a product of meta-cognition. Considered individually, the states of uncertainty generated by a subject for their own knowledge have no external corollary or reference to justify their numerical value in the sense that mathematical probabilities for, say, dice or roulette wheel outcomes do. Subjective probabilities for general knowledge propositions reflect an individuals awareness of the limitations of their own stored knowledge and the plausibility of inferences they may derive from it. The experimental evidence shows that individuals possess the competence to scale internal states of uncertainty relative to each other so that they bear some correspondence to the externally measured hit rate. So how could someone obtain information about the efficiency of the inferences and guesses they are making in order to determine how likely it is that a given proposition is true? Pitz (1974) has suggested that one way this might be achieved is for individuals to count how many different ways they can arrive at a n answer. So. for example, someone attempting to verify the proposition 'The Suez Canal is the longest canal in the world' would attempt to generate other possible candidates. The more candidates which an individual can generate which they are unable to rule out the more uncertain they will be. Some support for this theory comes from a study performed by Koriat. Lichtenstein and Fischhoff (1980). They found that if experimental subjects were explicitly required to generate reasons why their decision about a proposition might be wrong calibration was improved -- overconfidence being reduced. This theory is useful because not only does it provide a n account of how uncertainty arises in general knowledge but it also provides a plausible basis for explaining why probabilistic judgements preserve transitivity in the manner described above but are poorly calibrated. Memory search and retrieval processes will tend to generate more items when the subjective probability of being correct is relatively low but, plausibly, they will not generate sufficient items for any internal sampling strategy to be accurately

4 68

P. Ayton

& G.

Wright

calibrated. To the extent that too few items are generated the subjective probability would be overconfident. In an attempt to empirically test this model of probabilistic judgement we designed an experiment to investigate the effect of providing a varying number of specific candidate alternatives on subjective probability judgements concerning general knowledge propositions. Subjects encountered the propositions as questions of three different types. Two-cue type: Which is the longest canal in the world? (a) Suez (b) Panama One-cue type: Which is the longest canal in the world? (a) Suez (b) Other No-cue type: Which is the longest canal in the world? (a) * (b) Other * In this case subjects have to generate and provide their own candidate answer. The internal sampling model predicts that the Two-cue questions should be better calibrated than the One-cue questions because two rather than one specific possibilities are explicitly offered for consideration. The cues may have more than a purely intrinsic function in the sense that they may also serve to generate yet further candidates. The No-cue questions should be worse calibrated than Twocue or One-cue questions because here no items are specifically offered for consideration. Thus the number of cues available to aid the interrogation of semantic memory should, according to this model, have a direct impact on the subjective probability assessments.

_________

METHOD

Subjects Forty-eight subjects volunteered to participate in the study. They were all students at the City of London Polytechnic. They were each paid 32.50.

Probabilist i c cognition

469

Design and Procedure

Two hundred and twenty-five general knowledge questions were generated. All questions were presented in the same order. However t h e three response modes (question types) were counterbalanced over blocks of seventy-five questions and over subjects. Thus all subjects answered seventy-five (one third) of the questions in each response mode and all six possible orders of response mode occurred equally often over subjects. For the Two-cue and One-cue question types the correct answer (a) or (b) was fixed randomly for a given question and to eliminate the possibility of a response bias affecting the hit rate there were two versions of each such that if in one version, for a given question, the correct answer was alternative (a) then in the other version (b) would be the correct answer. Note that the Nocue response mode has no a priori correct alternative: the correct alternative depends on what the subject generates as a candidate. All subjects completed the questions which were divided into three booklets (one for each question type) in one session. Instructions were given at the beginning of the session by the experimenter and were also printed on the cover of each booklet. They were reminded that they should never give less than a 50% confidence rating as this would imply greater confidence in the unselected alternative. Particular emphasis was stressed that, in the No-cue condition, they should do their best to generate a candidate answer that they felt was plausible to some extent. Thus they should only very rarely either fail to write any candidate answer or write one which left them 100% sure that the response "Other" was correct. The subjects worked under examination conditions in a quiet room and all completed the task within an hour and a half.

RESULTS We collated the data so that we could compute measures of calibration, for each subject, for each response mode. We also decided to compute another measure of probabilistic performance, namely resolution. Resolution, as Yates (1982, p. 150) notes refers to "...the ability of the forecaster to discriminate individual occasions on which the event of interest will and will not take place". Yates h a s documented the considerable confusion that h a s often accompanied the use of this measure and points out that a measure

P. Ayton

470

&

G. W r i g h t

of resolution can be computed and interpreted even if the responses are completely non-numerical. In fact as resolution reflects the ability of the assessor to sort events into categories for which the hit rates are either 100% or 0% a well calibrated judge who used responses other than 1 or 0 would not be well resolved. Thus for example, judgements of .6 and .7, if well calibrated will have hit rates of .6 and .7 and so will be imperfectly resolved. However, if judgements of .6 and .7 had hit rates of 1 and 0 they would be perfectly resolved but. because the numerical labels do not accurately reflect the hit rate, they would be miscalibrated. We decided to include this measure of probabilistic judgement since, unlike calibration, it is independent of the exact numerical label the judge gives as a probability judgement. As such, to the extent that the independent variable merely affects the translation of states of uncertainty into numerical responses resolution may be less influenced. Murphy (1973) has presented formulae which allow the computation of a calibration score and a resolution score. 1

T

-

c

N

t=l

Calibration =

n t (rt

-

ct)

L

where N = total number of responses, nt = number of times response r t was used, ct is the proportion correct for all items assigned probability r t and T is the total number of different response categories used (e.g. T = 2 for subjects who use only responses of 0.5 and 1.0). A perfectly calibrated person would score 0 on this measure. 1 Resolution

=

T

N

n t (ct

-

n

c )L

t=l

where c = the proportion of items for which the correct alternative was selected. The higher the resolution score the better resolved are the responses. All responses were grouped into six categories: 50 to 59, 60 to 69, 70 to 79, 80 to 89, 90 to 99 and 100. The mean response in each category was used as rt and the proportion correct across the whole category was used for ct. (This method is in accordance with previous investigations e.g. Lichtenstein & Fischhoff, 1980; Wright, 1983.) Table 1 presents the mean resolution and calibration scores taken across all subjects separately for the three question types

471

Probabilistic cognition

together with the overall proportion of questions answered correctly under each response mode. Repeated measures 't' tests were computed to compare the means for each question type with every other within each dependent measure presented. None of the means of the calibration scores were significantly different from each other. However the resolution scores do show some differences. Although the means for the Two-cue and One-cue questions do not signifkantly differ, The No-cue questions are better resolved than both these types of question (respectively t = -7.557, p c .001 and t = -7.024,p c .001). A similar pattern emerges with the percentage of questions answered correctly across the three question types. The Two-cue and One-cue questions are not significantly different while the No-cue questions are answered significantly better than Two-cue (t = -2.279.p c .025)and One-cue (t = -3.159, p c .005) questions.

Table 1. Mean Resolution and Calibration measures for each question type.

Question type Two-cue Calibration Resolution % Correct

.061 .027 59.3

One-cue .077 .025 57.3

No-cue .070 .065 64.6

Although, as we have reported, subjects were instructed that for the No-cue questions they should not write in a candidate answer that they had no confidence in, we noted a considerable number of responses where subjects had signalled 100% confidence in the "Other" alternative. We considered that such responses might have a contaminating effect on the resolution and calibration scores as subjects could avoid errors of judgement by writing in a "pseudo candidate" -- that is one they thought could not possibly be correct and, thereby, select the "Other" alternative with 100% confidence. Consequently we re-collated the data omitting all

472

P. Ayton

& G.

Wright

responses of that type from the No-cue condition. Similarly we reasoned that subjects use of 50% responses for the other two question types (indicating complete uncertainty) ought to be deleted from a new analysis. These responses, as with those deleted in the No-cue condition, suggest total uncertainty as to the correct answer. Moreover, subjects using the 50% response for the Two-cue and One-cue questions or the 100% "Other" combination for the No-cue questions can be perfectly calibrated merely by taking advantage of a feature of the experimental design. By omitting these responses from a re-analysis we consider only those occasions when subjects expressed at least some degree of confidence in a specific proposition. We would argue that this is the appropriate focus for a study of subjective confidence in knowledge. The condition means of our new edited set of responses are presented in Table 2. Table 2. Mean Resolution and Calibration measures for each question type for the edited data set.

Quest ion type Two-cue Calibration Resolution % Correct

.074 .029 60.9

One-cue .091 .028 58.7

No-cue .114 .052 52.6

Again we computed repeated measures 't' tests on the measures of calibration, resolution and percent correct to compare performance across the three response modes. As we hypothesised, subjects are significantly better calibrated for the Two-cue questions than the One-cue questions (t = -1.732.p c .05,one tailed test). The Two-cue questions are also better calibrated than the No-cue questions (t = 3.304,p c .005). The One-cue questions are also better calibrated than the No-cue questions (t = -1.79,p < .05). On the measure of resolution responses to the Two-cue and One-cue questions do not significantly differ b u t they are both

Probabilistic cognition

413

significantly worse resolved than the No-cue questions (Two-cue vs No-cue: t = -4.94. p < .001; One-cue vs No-cue: t = -5.125, p < .OOl). The analogous 't' tests conducted on the percent correct scores shows that the Two-cue and One-cue question types do not differ while they are both significantly higher than the No-cue question types (Two-cue vs No-cue: t = 3.304, p < .002; One-cue vs No-cue: t = 2.501, p c .013). All tests are two tailed unless stated otherwise. DISCUSSION

The analysis of the calibration measures on the edited data set support the hypothesis derived from the internal sampling model of uncertainty outlined in the introduction. The greater the number of specific alternatives presented to subjects the better calibrated their judgements of uncertainty were. We have argued that the full data set, including responses where subjects indicate they have no idea what the answer is, is inappropriate for testing this model because, in effect, subjects are completely uncertain and, because of the design of the experiment. those responses can strongly influence the dependent measures we have examined. Our results indicate that the form of general knowledge questions is a n important factor influencing subjects' ability to calibrate their expressed confidence in their own general knowledge. The pattern of the effect is consistent with the view that the more specific cues that subjects have with which to search their semantic memories the better is their ability to retrieve plausible alternatives to use as a measure of the likelihood of any specific answer being correct. I t h a s been noted (e.g. Lichtenstein et al., 1982) that calibration is apparently influenced by the difficulty of the questions as measured by the overall percentage correct. The difficulty levels for the Two-cue and the One-cue conditions are similar s o the difference in calibration here cannot be attributed to differences in difficulty. However the No-cue questions are significantly more difficult than both of the other question types. As this study used the same propositions in the three response modes we can attribute the increase in difficulty here to the experimental manipulation. The lack of any specific cues in the No-cue condition might well be expected to increase the difficulty for subjects attempting to retrieve plausible answers. A further consideration is that in the

P. A y t o n

474

&

G. W r i g h t

No-cue condition the probability of getting the correct answer by guessing is clearly far less than 50% -- the rate for the other two question types. Examination of the plotted calibration curves (see Figure 1) prompts further consideration of the probabilistic responses to the questions. 1.0

two cues 44

0

2

.7-

L

one cue

0 0

no cues

c -6-

0 .+ L

0 Q

2

-5-

Q

-

.4

-

probability assigned

-

.3

Figure 1. Calibration curves for the three response modes (edited data). The figure reveals that the Two-cue responses are closer to the diagonal (perfect calibration) than the One-cue responses at every point bar one, and are thereby better calibrated. The No-cue responses are furthest from the diagonal for the probability responses between .5 and .79 and, except for responses of 1.0, nearest for responses above this interval. That the degree of miscalibration for No-cue responses close to 0.5 is far worse than for the other two conditions can be attributed to a feature of the design of the experiment which restricts the potential for miscalibration in the other two conditions. Subjects responding close to .5 in the Two-cue and

Probabilistic cognition

415

One-cue response modes are unlikely to be badly calibrated because even if they have no knowledge their expected proportion correct will be 50% -- the chance performance level is 50% correct. But in the No-cue response mode there is no probability response that a subject can choose that will minimise miscalibration. In this condition there is no natural anchor point that will serve as a base rate response. The best a subject with very little or no knowledge could do to minimise miscalibration in the No-cue condition would be to give a very high confidence rating that their self-generated candidate item was not the true answer -- an "Other" response. Recall that the 100% "Other" responses are omitted from the edited data which presumably explains why the 100% responses are badly calibrated in this condition -- here subjects are risking a commitment to their self-generated answer. S u c h considerations highlight that the extent of miscalibration in general knowledge can be a function of the method of eliciting probability responses as much as a subject's uncertainty concerning a general knowledge proposition. Yates (1982) h a s argued that too much emphasis has been placed on calibration assessments and that the evident appeal of calibration to researchers is largely aesthetic. He claims that resolution is a more important measure pertaining to a more fundamental skill because, unlike calibration, resolution is logically dependent on having knowledge of the answers to the questions. We have some sympathy for the argument particularly in view of the fact that that it is possible to be perfectly calibrated merely by identifylng the base rate probability for a whole set of judgements and that in a good deal of reported research (including two of the conditions here) subjects may quite easily deduce the base rate (usually .5) and moderate their judgements accordingly. Though, of course, accurate identification of the base rate in some tasks (e.g. the No-cue condition) will not be aided by spurious experimental cues and will therefore reflect judgemental skill. Recall that there was no difference between the resolution scores for the Two-cue and No-cue conditions but that the No-cue condition produced superior resolution. So, on this criterion, one could argue t h a t t h e No-cue condition produces the best probabilistic judgement -- a conclusion quite contrary to that suggested by consideration of calibration. However the propriety of the resolution statistic as a measure of probabilistic judgement is also questionable. Recall t h a t

416

P. Ayton

&

G. W r i g h t

resolution is maximised when the propositions are sorted into categories for which the hit rate is either 1 or 0 -- regardless of the judged probability. Consequently resolution provides an assessment of the deterministic validity of judgements and does not reward probabilistic accuracy. The superior resolution scores for the Nocue condition can be explained by the greater variance in hit rate for this condition which, in turn. can be attributed to the potential for a low (less than 50%) hit rate. Determining the relative quality of probabilistic competence on what appear to be even quite simple tasks is actually a somewhat deceptively complex matter. The choice of task demands and dependent measure must be carefully considered Wagenaar & Keren. L985: Wright & Ayton. 1986). Our experiment provides some support for one theory of how different states of uncertainty are engineered from limited knowledge. I t strongly suggests the need for close consideration of further methods for evaluating the competence of probabilistic cognition. REFERENCES Collins, A.M., & Qufllian. M.R. (1969).Retrieval tlme from semantic memory. Journal of Verbal Learning and Verbal Behavior. @, 240-247. Koriat. A.. Lichtenstein. S.. & Fischhoff, B. (1980).Reasons for confidence. Journal of ntal Psvcholom: Human Learnlm and M e m a , 6. 107-118. Lichtenstein. S., & Fischhoff. B. (1980).Training for calibration. -t ional Behavior and H u nW,%, 149-171. Lichtenstein, S..Fischhoff. B.. & Phillips. L.D. (1982).Calibration of probabilities: the state of the art to 1980.In D. Kahneman. P. Slwic. & A. Tvemky (Eds). Judement under uncertaintv : heuristics and biases. Cambridge: Cambridge University Press. Murphy, A.H. (1973). A new vector partition of the probability score. Journal of ADDlied MeteorolQgy,u.595-600. Pitz, G.F. (1974).Subjective probability distributions for imperfectly known quantities. In L.W. Gregg (Ed). Knowledce and Cormition. New York Wfley. Smith,E.E.. Shoben. E.J., & Rips, L.J. (1974).Structure and process in semantic memory: a featural model for semantic decisions. Psvcholoaical Review, 81, 214-241. Wagenaar. W.A.. & Keren. G.B. (1985).Calibration of probability assessments by professional blackjack dealers, statistical experts and lay people. Owanisational Behavior and Human Decision Processes, 406-416. Wright, G. (1982).Changes in the realism and distribution of probability assessments a s a function of question type. Acta Psvchologia. 2, 165-174. Wright, G., & Ayton, P. (1986).Subjective confidence in forecasts: A response to Fischhoff and Macgregor. Journal of Forecas-. 6. 117-123. Yates. J.F. (1982). External correspondence: decompositions of the mean probability score. Oraanisatlonal Behavior and Human Performance, 30.132-156.

s,

Cognitive Biases J . - P . C a v e r n i , J . -M. F a b r e and M . G o n z a l e z ( E d s . ) 0 E l s e v i e r S c i e n c e P u b l i s h e r s 8.V . ( N o r t h - H o l l a n d ) ,

1990

THEORIES OF BIAS IN PROBABILITY JUDGMENT Michael H. Bimbaum* Carolyn J. Anderson**and Linda G. Hynan** * **

California State University, Fullerton, CA, USA University of Illinois, Urbana-Champaign, IL, USA

Abstract. When psychologists study human judgments of probability, judged probabilities unfortunately do not conform to the equations of probability theory. Because probability theory offers such a convenient and compelling structure for discussing beliefs about ambiguous and uncertain events, many scholars have found it disturbing to think that humans might have been rational enough to invent probability theory but not rational enough to use it in their daily thought. This chapter will explore explanations of the discrepancies between judged probabilities and the implications of probability theory.

Many decisions in the modem world are based on subjective probabilities. Subjective probabilities guide discussions of critical issues because objective probabilities are often undefined in many situations in which people wish to use the language of probability to plan for future events. For example, the "Star Wars", strategic defense initiative would seem a good idea if it appeared likely to reduce the likelihood of nuclear war or to increase the chances of survival given accidental nuclear attack. On the other hand, some argue that if one side were to develop a defense system that was perceived to be 50% effective, the other side might decide to double their arsenal of weapons to restore the balance of terror. Such increases in arms could increase the likelihood that the weapons would be used and therefore increase the chance of nuclear war and thereby increase the chance that the world will be destroyed by nuclear war. Notice that these arguments involve intuitive probabilities that the weapons will work, that certain actions would be taken by one side or the other, or that an accident might occur. These intuitive probabilities and the decisions they engender will determine the future of the planet, yet they cannot be checked for their accuracy. With enough replicas of the earth and enough time, perhaps a very powerful experimenter could observe the proportion of earth replicas annihilated by nuclear war with or without any given policy.

478

M.H.

Birnbaum, C . J . A n d e r s o n ,

&

L.G.

Hynan

Such experiments could in principle provide an empirical basis for decision making. However, in the absence of such experiments, such likelihoods are left to be determined by human judgment. The survival of the planet now rests on the capability of humans to evaluate probability and to make proper decisions in the face of uncertainty. Unfortunately, when psychologists study human judgments of probability, judged probabilities do not conform to the equations of probability theory. Because probability theory offers such a convenient and compelling structure for discussing beliefs about ambiguous and uncertain events, many scholars have found it disturbing to think that humans might have been rational enough to invent probability theory but not rational enough to use it in their daily thought. This chapter will explore explanations of the discrepancies between judged probabilities and the implications of probability theory. Probability Theory

Although many different philosophical underpinnings have been proposed for the concept of probability (Krantz, Luce, Suppes, & Tversky. 1971; Kyburg & Smokler, 1964; Savage, 1954; von Winterfeldt & Edwards, 1986). this chapter will consider the "standard" probability theory presented in the introductory texts on statistics and probability (e.g.. Hogg & Craig, 1965; Mosteller, Rourke, & Thomas, 1961). Probabilities will be numbers assigned to the beliefs that events will occur, and the same language will be employed for unique events as, "there will be a new champion in boxing this year" and for events that are replicable such as, "five coins will come up heads when ten are flipped." Events are represented by sets (and set algebra), and probability is a measure on the sets, taking values between zero (for the null set) and one (for the universal set), that satisfies the following equations: P(AuB) = P(A) + P(B) - P ( h B )

where P(A) and P(B) are the probabilities of events A and B; P(AuB) is the probability of the union (either A or B, denoted, u); and P(AnB) is the probability of the conjunction, or intersection of events (both A and B, denoted, n). If the intersection is the null set. probability is additive across mutually exclusive events.

479

Probability judgment

The probability of an event, A, and Its complement, A (where A' is the complement of A, or not-A), sum to one, because an event and its complement are defined to be mutually exclusive and exhaustive: hence, P(A) = 1 - P(A).

(2)

If A is the null set, it is impossible and A is a certainty, so P(A) = 0 and P(A) = 1. The probability of the conjunction of events can be written as follows:

P(AnB) = P(AIB)P(B) = P(BIA)P(A),

(3)

where P(A I B) and P(B IA) are the conditional probabilities of A given B, and of B given A, respectively. The probability of a simple event, A, can also be expressed as follows: P(A) = P(AnB) + P(AnB').

(4)

because A can either occur with B or without B (there are no other ways), and the intersection of B and B' is empty, by definition. For two constituent events, A and B, one could ask twenty simple probability questions, including 4 probabilities of the events and their complements [P(A), P(A). P(B), P(B')I. 4 conjunctions [P(AnB), P(AnB'), P(AnB). P(AnB')]. 4 unions [P(AuB). P(AuB'), P(A'uB), P(AuB')], and 8 conditionals [P(AIB), P(AI B'), P(A' I B). P(A I B'), P(B IA), P(B' IA), P(B IA), P(B' IA)]. However, among these 20 probabilities, there are only 3 degrees of freedom, because once three values are known [for example, P(A). P(B), and P(AnB)]. the remaining 17 can be calculated from the equations. Furthermore, even these three values are constrained, because P(AnB) must be less than or equal to P(A), less than or equal to P(B), and greater than or equal to P(A) + P(B) - 1. For example, if P(A) is .7 and P(B) is .6. then P(A) = .3 and P(B') = .4, by Equation 2. From Equations 3, P(AnB) must be less than or equal to .6. Additionally, P(AnB) must also be greater than or equal to .3 [by Equation 4, P(AnB) + P(AnB') = .7 and P(AnB') + P(A'nB') = .4; therefore, P(AnB) = .3 + P(A'nB'), which is greater than or equal to .3 because P(AnB') 2 01. This constraint also implies that P(AIB) 2 .5, since P(AIB) = P(AnB)/P(B). Suppose

480

M.H. Birnbaum, C.J. A n d e r s o n ,

&

L.G.

Hynan

P ( A n B ) = .4; from Equation 4, it follows that P ( A n B ' ) = .3, P(A'nB) = .2, and P(A'nB') = .l. Equation 1 can then be used to calculate all of the unions, and Equations 3 can be used to calculate

all of the conditionals. Figure 1 illustrates some of the relationships imposed by Equations 3. Probability of the conjunction of A and B should be proportional to P(A). with the slope equal to P(B IA). Specifymg two of the values determines the third. The conjunction could also be analyzed as the product of P(B) and P(AIB). The conjunction is therefore constrained by the following four expressions: P(AnB) I P(A) P(AnB) 5 P(B) P(AnB) I P(AIB) P(A) - P(B') I P(AnB) I P(BIA)

(5) (6)

(7) (8)

which can be seen as consequences of the requirement that all of the values in Equations 3 and 4 must be between 0 and 1. = 1.0 = .8

= .6

= .4

= .2

0.0

0.2

0.4

0.6

0.8

1. 0

P(A)

Figure 1. Probability of conjunction of A and B, plotted as a function of the probability of A, with separate curves for different levels of the probability of B given A. Note that P(AnB) is always less than or equal to P(A), and it is also less than or equal to P(B I A).

Probability judgment

Judgments of probability violate these inequalities.

481

Wyer

(1976) found many instances in which the judgment of P(AnB) was greater than the judgment of either P(A) or P(B). Tversky and Kahneman (1983) reviewed this literature, presented additional

examples, and used the term "conjunction fallacy" to describe violations of Expressions 5 and 6 that occur in within-subjects designs. Judged probabilities have systematically violated probability theory in a number of studies involving a variety of tasks. Reviews of this literature from different points of view can be found in Wyer (19741, Kahneman. Slovic, and Tversky (1982). Birnbaum and Mellers (1983).and other chapters in this volume. As yet, no theory has been developed to explain all of the phenomena involving judged probabilities. Subjective and Judged Probability Deviations between implications of probability theory and judgments of probability may occur because subjective events do not obey the algebra of sets, because subjective probabilities do not obey the calculus of probability theory, or because probability judgments have not been properly scaled. Figure 2 helps to clarify these theoretical distinctions. In the outline in Figure 2, events and information influence subjective probabilities (e.g., s(A) in Figure 2). These values may have an organization of their own that differs from that of probability theory. These impressions are mapped into overt judgments of probability by judgment functions, which assign numerical responses to the subjective impressions. In addition, errors may enter the system in the subjective stages or in the responses. The presence of random errors cause judgments of probability (q(A) in Figure 2) to violate the constraints in Expressions 5-8,even if the subjective probabilities conformed to probability theory. The first issue is whether people use the same representation of events as the investigator. For example, the English words, "and" and "or". which are assumed to refer to conjunctions and unions, have different meanings in different sentences. For instance, in the sentence, "to enter this bar, you must have either a driver's licence or a military I.D. showing you are older than 21 years," people correctly understand that both 1.D.s would be acceptable. Similarly, people take "and" to denote the conjunction in the sentence, "to cash a check here you need a driver's licence and a credit card."

4 02

M.H. Birnbaurn, C.J. Anderson,

&

L.G.

Hynan

Figure 2. Outline of objective, subjective, and judged probabilities. Objective events, probabilities, and information combine to produce subjective events and their probabilities, which in turn, are mapped into judged probabilities of events. However, sometimes "and' and "of' have other meanings. In the sentence, "either you return my wallet or I will punch you in the nose," people would be surprised if the wallet were returned and the punch were delivered, presumably because " o f ' in some cases implies the union excluding the intersection, as it does in the sentence, "an assistant professor must either publish or perish". The term, "and' sometimes refers to the union, as in the sentence, "only women and children may enter the lifeboats." Because the probability of a union often exceeds the probability of the constituents, miscommunication concerning the word "and" could explain conjunction fallacies. Tversky and Kahneman (1983) discussed this possibility and concluded that subjects indeed interpreted their conjunction problems in the fashion intended. In this chapter, the issue of subjective algebra of events will also receive less emphasis, but a complete theory of probabilistic reasoning will eventually need to include a theory of the subjective analysis of propositions (Wason & Johnson-Laird, 1972). The second issue is to determine the calculus of subjective probability. If people do not obey probability theory, what laws would explain probability judgments? Perhaps another coherent structure interlocks judgments of probability in some consistent, if strange, fashion. I t is possible that intuitive probability judgments

Probability judgment

483

form a “workable” structure that differs from the system in probability theory. Perhaps they form a system that would be considered acceptable for certain situations, but unacceptable for others. For example, people might be fairly accurate at estimating the value of a basket of groceries, but when cash is to be exchanged, both parties feel better when intuitive judgments can replaced by explicit calculations in the supermarket. Nevertheless, investors will buy and sell real estate where the calculations by necessity are much cruder. Similarly, an engineer may have expertise enough to realize whether the design of a freeway overpass will withstand earthquakes of expected magnitudes, but people want to see (and the law requires) explicit calculations, because expert intuitions are sometimes wrong. The point here is that we need to explain the facts that people can be fairly accurate and consistent and yet can still make systematic deviations. The third issue is to determine the relationship between subjective and judged probability. This issue is analogous to the concept of the judgment function in psychophysical and social judgment (Birnbaum. 1982). Some investigators have implicitly assumed that judged probabilities can be interpreted as subjective probabilities. However, Varey, Mellers, and Birnbaum (in press) observed that judged proportions in visual displays show the same sorts of contextual effects as do other types of psychophysical stimuli. For example, when the actual proportion of open dots in a display of open and solid dots was .17,the judged proportion could be either .21 or .32, depending on the distribution of other proportions presented. Findings such as these indicate that theoreticians need to distinguish between subjective probabilities, which might obey certain consistency properties, and judged probabilities, which might not. Contextual effects in judgments of proportions indicate that between-subject comparisons of probability judgments need a theory of the context for their interpretation. Birnbaum and Mellers (1983) found that within- and between-subject investigations of the “base rate fallacy’’in Bayesian inference problems give very different results. In between-subject studies, the data led some to conclude that subjects underweight the base rate (Kahneman. et. al., 1982). Birnbaum and Mellers (1983) found that when subjects were asked to make a single judgment, in a between-subject design, they appeared to choose one of the stimulus values as their response: however, in within-subject designs, subjects appeared to give weight

484

M.H. Birnbaum, C.J. A n d e r s o n ,

&

L.G.

Hynan

to the base rate, although they did not combine it with the evidence in the appropriate fashion. Instead of using Bayes Theorem, they appeared to combine the information by a configurally weighted, scale-adjustment averaging model (Birnbaum & Stegner, 1979; 1981). Birnbaum (1982) similarly found that the judged fault of a rape victim who was described as a "virgin" was greater than that of a "divorcee" in a between-subject design, but not when the subjects judged both types of victims or even judged the fault of the defendant in a within-subject design. Birnbaum argued that Parducci's (1965;1983) range-frequency theory could explain these effects if it is postulated that in a between-subject design, the stimulus and the context are confounded, because the stimulus brings its own context. I t may be that the subjective "fault" of these victims never changed, only the judgments. Suppose probability theory were an accurate representation of intuitive reasoning, but that judged probabilities were linear functions of subjective probabilities, with errors that produce regression. This concept is similar to the general idea of factor analysis, in which there are "true" values of psychological constructs, and observed measures are assumed to be correlated with them. Let us suppose also that the marginal distributions of true scores and observed scores (judgments) are constant, consistent with rangefrequency theory. If so, then statistical regression will influence the slopes and intercepts of the functions relating average judgments to the "true" scores, and therefore also influence the relationships between different judgments. [In the diagram in Figure 2. notice that errors could be attributed to the psychological processing as well as to the response stages. This distinction is analogous to the distinction made in theories of scaling and signal detection between variability in the perception of the stimuli and variability in the criteria, or limens for judgment (see e.g.. Torgerson, 1958: Thurstone, 1927). This distinction allows the subject to have uncertainty about a stimulus (fuzziness in the subjective probability) as well as uncertainty about what to call it.] Conjunction errors can be explained by simply allowing that the regression of judgments of conjunction probabilities has a lower slope relative to other probability judgments. Taking the picture of Figure 1 into account, the idea of regression suggests a simple recipe for finding violations of Expressions 5 and 6 (conjunction fallacies): combine small values of P(A) with large values of P(B IA) or small values of P(B) with large values of P(AI B). One might expect to

Probability judgment

485

find violations of Expressions 7 and 8 when the probability of one event is high and the conditional probability of the other event given that event is low. EXPERIMENTAL INVESTIGATIONS OF CONJUNCTIONS

Wyer (1976) proposed a type of averaging model for conjunction probabilities. However, Tversky and Kahneman ( 1983) argued against averaging models on the basis of their results with the "Peter" problem. Instead, they concluded that psychological "representativeness" introduces a bias into judgments of probability. Birnbaum. C. Anderson, and Hynan (1989) designed a study of the "Peter" problem to address three issues: First, they used variations of the "Peter" problem, to check whether or not the "conjunction fallacy" would behave like the "base rate fallacy", and change drastically when the subject makes many judgments. Second, each subject was requested to judge events, conjunctions, and conditionals to allow tests of Expressions 5-8 and also, presumably, to keep the subject clear on the distinctions among these different concepts. Third, structure was imposed on the problems t o facilitate tests of models of the conjunction fallacy. In one of their experiments, Birnbaum, et al. (1989) asked 60 university students to evaluate 56 variations of the "Peter" problem (Tversky & Kahneman, 1983, p. 306). which are all based on the following information: Peter is a junior in college who is training to run the mile in a regional meet. In his best race, earlier this season, Peter ran the mile in 4:06 min. The subjects were asked to judge the probabilities of various outcomes. For example, what is the probability that Peter will run the second half-mile under 2:04 min. and will complete the mile under 4:09 min.? They judged the probabilities that Peter will complete the whole Mile in under 4:00, 4:03, 4:06, or 4:09 min. (Event A); they also judged the probabilities for second Half Mile times under 1:55, 1:58, 2:01, or 2:04 min. (Event B). Each event was defined as Peter's time being under the time specified. Subjects also judged all of the 16 (4 by 4) conjunctions of these times, and all 16 conditional probabilities of the Mile times given Half mile times, and also all of the 16 conditional probabilities of Half mile times given whole Mile times. After warm up trials, subjects rated all of these 56 events, conjunctions, and conditionals,

M.H. Birnbaum, C.J. Anderson,

486

&

L.G.

Hynan

randomly ordered (8 Events. A, B; 16 conjunctions, 16 conditionals of A I B and 16 conditionals of B IA). Table 1 shows the mean judgments of the conjunction probabilities, with the mean judgments of the events in the last row and column. If Expressions 5 and 6 described judgments, then each judged probability of a conjunction should be less than the corresponding judgments for the constituent events. Instead, all of the entries in the first and second rows of conjunctions exceed the judged probability for the second Half mile (last column). Similarly, all of the conjunctions to finish the Mile under 4:OO (first column) with any Half mile exceed the probability of the 4:OO min. Mile (last row of first column). Table 2 shows that mean judgments of conditional probabilities can also be exceeded by conjunctions, contrary to Expressions 7 and 8. For example, all conjunctions involving the 4:09 min Mile exceed the corresponding judgments of conditional probabilities of finishing the Half mile times given the slow, 4:09 time for the Mile (last column of lower portion of Table 2).

Table 1. Mean Judgments of Events and Conjunctions. Half M i l e

B

1:55 1:58 2:Ol 2:04 q (A)

Whole Mile

4:OO .29 .27 .26 .29 .23

4:03 .33 .36 .44 f38 .36

4:06 .38 .48 .46 .57 .58

q (B)

4:09 .37 .42 .57 .61 .81

.22 .25 .42 .65

Note: A = P e t e r f i n i s h e s t h e whole M i l e under t i m e l i s t e d ; B = P e t e r ' s time f o r t h e second Half m i l e of t h e same r a c e i s l e s s than time l i s t e d . From Birnbaum et a l . (1989).

Two situations are highlighted in Table 3 that appear inconsistent with the account of Tversky and Kahnernan (1983). The representativeness interpretation suggests that two fast times (4 min Mile with a 1:55 Second Half) should produce a conjunction fallacy because that combination would be representative of a winning race. However, the 4:09 Mile and fast 1:55 Half mile, though not representative, produce a greater conjunction judgment,

487

Probability judgment

and a greater "fallacy"when compared with the Half mile event. As will be shown below, these results are consistent with an algebraic explanation that uses the concept of subjective conditional probability instead of representativeness.

Table 2. Mean Judgments of Conditional Probabilities. Conditionals: Whole Mile given Half Mile [q(AIB)3 A B 1:55 1:58 2:Ol 2:04

4:OO .68 .60 .57 .36

4:03 .67 .62 .58 .35

4:06 .78 .76 .74 .56

4:09 .82 .78 .82 .75

Conditionals: Half Mile given whole Mile [q (BIA) I 1:55 1:58 2:Ol 2:04

.49 .54 .60 .55

.36 .43 .60 .63

Note: see note to Table 1.

.30 .37 .53 .66

.27 .32 .40 .65

From Birnbaum et al. (1989).

Table 3. Comparison of Probability Judgments for Two Situations. Judgment

P (Mile) P (Half) P (Mile1 Half) P (HalfI Mile) P (MilenHalf)

Mile < 4:OO min Half < 1:55 min. .23 .22 .68 .49 .29

(45%) (48%) (15%) (13%)

Mile < 4:09 min. Half < 1:55 min. .81 .22 .82 .27 .37

(7%) (55%) (2%) (47%)

Note: Percentages of subjects who judged conjunction more probable than each term (violating Expressions 5-8) are shown in parentheses. From Birnbaum, et al. (1989).

488

M.H.

B i r n b a u m , C . J . Anderson,

&

L.G. Hynan

Table 3 also shows the percentage of subjects whose conjunction judgments were greater than their judgments of the events and conditionals. For example, for the 4:OO min. Mile and 1:55 min. Half mile, the conjunction was judged greater than the probability of the 4 min. Mile by 45% of the subjects and greater than the 1:55 min. Half by 48% of the subjects, in violation of Expressions 5 and 6; 38% judged the conjunction more probable than both of the marginal events. When the Mile time was increased to 4:09 min. with the same Half mile time, 55% of the subjects rated the conjunction more probable than the 1:55 Half mile, 7% judged it more probable than the 4:09 Mile, and only 5% judged it more probable than both marginal events. This conjunction was also judged more probable than the judged conditional probability of finishing the Half under 1:55 given the Mile is under 4:09 by 47% of the subjects, in violation of Expression 8. For the two slowest times (4:09 Mile and 2:04 Half), 25%. 60%, 32%. and 35% of the subjects violated Expressions 5, 6. 7. and 8. respectively: 18% judged the conjunction more probable than both marginal events. Fit of Probability Theory

The standard probability theory was fit to the data by estimating the values of subjective probabilities for the events and the conditionals to minimize the sum of squared errors in the following set of equations: q(A) = s(A) + eA q(B) = s(B) + eg q(AIB) = S(AIB) + eAiB q(BIA) = S(B)S(AIB)/s(A)+ eBlA q(AnB) = s(B)s(AI B) + e h B

(9) (10) (11)

(12) (13)

where q(A), s(A). and eA denote the judged probability of A, the subjective probability of A, and random error in the judgment of A, respectively. Note that in Equations 9, 10, and 11, there are as many subjective values as judgments, but the same parameters reappear in Equations 12 and 13. To fit the model to the data, a special computer program was written to select 24 scale values [4 values of s(A), 4 values of s(B), and 16 values of s(AIB)] so as to minimize the sum of squared errors. The program utilized the STEPIT subroutine (Chandler, 1969) to minimize the following index of fit: F = C(qi -q',)2, where F is the

Probability judgment

409

function to be minimized: qi is the mean judgment: qi is the predicted judgment: and the summation is over the 56 judgments of probability. This version of probability theory achieved a rather poor index of fit of .33, using 24 scale values to approximate 56 judgments. Apparently, the equations of probability theory do not fit the judgments well, even when the subjects make a variety of judgments in a within-subjects design, and even when the data are fit in a fashion that allows the errors to occur in all dependent variables. The problem is that this model predicts that conjunction judgments should be less than their constituents, as in Expressions 5-8, but contrary to the data. When probability theory is modified to allow the dependent variables to be only linearly related to subjective probability, (allowing judgment functions to intervene between subjective probability and response), then the algebra of probability theory gives a much better fit. Two additional parameters can be introduced into Equation 13 as follows:

where c and d are constants that allow judgments of conjunctions to be linearly related to the other judgments of probability. This model h a s an overall sum of squared errors of .06. which is a vast improvement over the unmodified probability theory. This model accounts for the general pattern of violations of the Expressions 5-8. Wyer (1976) proposed a model that can also be interpreted as probability theory with different scales for the event and conditional probabilities, although he interpreted it as a compromise between the average of the event and conditional probability and the product of these terms. (If the additive constants differ for different scales, then the equation for conjunctions will be a linear combination of the product and the components.) Wyer's model was generalized and fit using the approach described above, and it achieved a fit comparable to that of the modified probability theory. The conditional probabilities and event probabilities showed a fair degree of consistency with the equations of probability theory. For example, for each subject's data, the relationship between the products, q(A)q(BIA) and q(B)q(AI B), was examined: interestingly, these two products fell close to the identity line in most cases, as would be predicted from Equation 12. This consistency should be regarded with caution, because if there were distortions in the

4 90

M.H. Birnbaum, C . J .

Anderson,

&

L.G. Hynan

conditional probability judgments, such tests of consistency might allow biases to go undetected since they appear on both sides of the expression under investigation. Examination of the data, however, indicated t h a t the conjunctions still showed systematic deviations from the predictions of the modified probability theory that appeared to indicate that conjunction judgments should be represented as a geometric average of s(A) and s(B IA). Furthermore, it seems unattractive to postulate different judgment functions for the different types of probability judgments since they were obtained under the same conditions with the same stimuli.

A General Theory The following theory was developed to include as special cases both probability theory and geometric averaging theory, while retaining a consistency of the conditional probabilities, analogous t o Equations 3. Suppose the subjective analog to Equations 3 can be written as follows: s(AnB) = s(A I B)as(B)P= s(B IA)%(A)P

(14)

where a a n d p are the weights in the geometric average determining the intuitive probability of the conjunction. These equations lead to a generalized theory in which the conditional probabilities are proportional to each other, and the proportion is a function of the ratio of event probabilities, as in probability theory, b u t a power function of the ratio connects the two conditionals, as follows: q(BI A) = S(Al B)[s(B)/s(A)]P/a +eg 1~ q(AnB) = s(A)Ps(B IA)a + e h B

(15) 116)

where Equations 15 and 16 replaced Equations 12 and 13, and Equations 9-11 were kept the same. When 01 = p = 1, Equations 14 reduce to the subjective counterpart of Equations 3, and Equations 15 and 16 are the same as Equations 12 and 13. When a = 1 - p, Equation 14 is a geometric averaging model. The fit of several special cases of this model are shown in Table 4. When a = p and a + p = 2, the theory reduces to probability theory, and the fit is -33. When both parameters were free to vary, the best-fit values were a = .68 and p = .54. and the overall fit was

4 91

Probability judgment

.05. Examination of the predictions indicated that this version of the model gave a better approximation to the data than the modified probability theory, which used the same number of parameters and allowed a different judgment function for conjunctions. Table 4. Fit of Special Cases of the Generalized Model. C o n s t r a i n t s on t h e Parameters

a+P=2

a= p .33

a, .33

a+p=1

.ll

a +p free

.06

.10 .05

Note: Index of f i t i s sum of means and model p r e d i c t i o n s .

p

unrestricted

s q u a r e d d e v i a t i o n s between

MODELS. HEURISTICS, AND BIASES

The algebraic modelling approach does not make use of the heuristic of representativeness, or other content-specific cognitive biases. The approach makes the strong prediction that the content of the problem makes no difference, except to affect the parameters of the model. In using an algebraic model, the present chapter is compatible with the approach of Wyer (1976). In contrast, Tversky and Kahneman (1983)approach conjunction fallacies in terms of the specific content of the problems, citing the biasing effects of "representativeness" on probability judgments (see also Kahneman & Tversky, 1972: 1980: Kahneman, et al., 1982: Slovic, Lichtenstein, & Fischhoff, 1988). Representativeness is postulated to be a psychological construct that co-exists with subjective probability, that is "naturally assessed", and that contaminates subjective probabilities, which presumably might otherwise be unbiased. Representativeness is analogous to, but distinct from similarity or prototypicality. and it can be measured by asking subjects to rate the "representativeness" of certain events. The theory of representativeness can be depicted as in Figure 3. Events and stimulus information affect two constructs, subjective probabilities and representativeness, which in

492

M.H.

Birnbaurn,

C.J.

Anderson,

&

L.G. Hynan

turn both influence the dependent variables, subjective probability and judged representativeness. A complete theory would specify how the independent variables affect each construct, how the constructs influence each other, and how they affect the two types of judgments. Judged

Information

Representativeness

Representativeness

/

Judged

Probability

Figure 3. Representativeness as a psychological construct, independent of subjective probability. The open arrow is used to highlight the open question whether or not representativeness has a causal effect on subjective probabilities. Among the obvious null hypotheses to test are the onemediator model, which states that both types of judgments are mediated by a single intervening construct, and the two-mediator, segregated theory that there are indeed two mediators of the judgments, b u t there is no causal effect of representativeness on subjective or judged probability. Single mediator theories have had surprising success in situations in which it had been taken for granted that multiple mediators were a t work (Birnbaum, 1982, 1985). For example, even when subjects are instructed to judge "ratios" and "differences" between stimuli, the data can be explained by the theory that both judgments are mediated by the same comparison operation on the same subjective values. Had subjects actually used two different operations to compare the stimuli, the judgments would not have been monotonically related, b u t instead would have shown a particular ordinal pattern. It should be noted that single mediator theories do not predict that the dependent variables will be perfectly correlated, nor do they require that the

Probability judgment

4 93

partial correlation between a n independent variable and one dependent variable with the others partialed out will be zero: instead, these partials are predicted to have the same signs as the original correlations (Birnbaum & Hynan, 1986; Birnbaum, 1985). The evidence presented does not yet appear to require rejection of such simpler null hypotheses in favor of the theory that the construct of representativeness is distinct enough from subjective probability to be given status as a separate, causal mediator. What has been demonstrated is that judgments of representativeness and probability are correlated, and that both demonstrate certain properties that are not consistent with "standard" theories of probability. We will now consider some examples of conjunction fallacies to contrast heuristic interpretations as opposed to algebraic interpretations in terms of subjective conditional probabilities. The Linda problem, the Bill problem, and the other experiments cited as examples of the M->A paradigm by Tversky and Kahneman (1983. p. 305) are consistent with the model presented here, assuming plausible values for the parameters. Linda is described as follows: "Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations." People rank the probability that Linda is a feminist and a bank teller (AnB) higher than the probability that Linda is a bank teller (B). The representativeness interpretation assumes that the description produces a "model" of Linda that resembles a feminist and adding feminist to bank teller makes the conjunction more plausible than leaving it out. The weakness of this explanation is that it "passes the buck" from probability to another psychological construct, representativeness, without explaining why that construct has the properties that it does. For the present algebraic model, the interpretation would be that the description of Linda produces high values of s(A), low values of s(B), and a high value of s(AIB). For example, suppose s(A) = .85 (she is probably a feminist), s(B) = .1 (she is probably not a bank teller), and s(AIB) = .80 (given Linda is a bank teller, it is still quite

4 94

M.H.

B i r n b a u m , C.J. A n d e r s o n ,

& L.G.

Hynan

likely that she is a feminist). Assuming a = .68 and p = .54 (the values estimated for the Peter problem), Equation 14 implies a value of s(AnB) = .25,which exceeds P(B). The weakness of this explanation is that the parameters are left to be determined from the data: however, the strength of such a n explanation is that it makes particular predictions and can be refuted by evidence, if the experiments are designed to constrain the parameters. The pattern predicted by Equation 14 is shown in Figure 4, plotted for comparison with Figure 1. This figure shows the situations in which judgments of the probability of conjunctions are predicted to exceed judgments of the probability of events or conditionals. = 1

1.0

= .8

0.8

= .6 n

p?

0.6

C 4 W

0.4

= .4 = .2

v1

0.2

0.0 0.0

0.2

0.4

0.6

0.8

1.0

s(A) Figure 4. Subjective probability of a conjunction, according to Equation 14, using parameters estimates from Birnbaum, et al. (1989). Values are plotted to permit comparison with probability theory, as in Figure 1. Curves above the identity line represent values for which subjective probability of a conjunction exceeds the event probability listed on the abscissa. Tversky and Kahneman (1983) cited the “Peter” Problem as evidence against an averaging rule. Their argument was based on the finding that many subjects judged the conjunction as more probable than either constituent. However, such a result would disprove only the simple average of the event probabilities, and it

Probability judgment

495

would be consistent with any averaging model with an initial impression, even with independent events (Yates & Carlson, 1986). Furthermore, the result is consistent with the generalized averaging model of the event and conditional probability, as in Equation 14. In general, the concept of conditional probability may permit a more tractable theoretical construct and perhaps more testable theory than the construct of representativeness. The "Peter" problem, the health survey problem, and the other problems cited by Tversky and Kahneman (1983) as examples of the A -> B paradigm can also be explained by the present model. According to the representativeness argument, event A provides a possible cause, explanation, or motive for B and therefore, it makes the conjunction more plausible. By the present model, the link between A and B is represented by subjective conditional probabilities. In these problems, the values of s(A) and s(B) might both be small, b u t the values of s(AIB) and s(BIA) are high. The Peter problem, with 4 min. Mile and 1:55 min. Half, is a n example of this situation. As shown in Table 3, the judged probabilities of these events are low (.23 and .22); however, the conditional probabilities of finishing the Mile in less than 4:OO min., given Peter finishes the second Half under 1:55 min., is high (.68). The judged probability that Peter will finish the Half mile under 1:55, given a sub-4:00 min. Mile, is also high (.49). Figure 4 shows that the predicted value of the conjunction is higher in this situation than both of subjective probabilities of the separate events. In the health survey problem, Mr. F. was selected randomly from the participants in a survey of adult males in a certain population. Subjects ranked the probability that Mr. F has had a heart attack and is over 55 years old (AnB) higher than the probability that Mr. F has had a heart attack (A). However, subjects tended to rank the conjunction that Mr. F. has had a heart attack and Mr. G. (another randomly selected participant) is over 55 years old (AnC) as lower than the probability that Mr. F. has had a heart attack. These two data are both consistent with the present model. Suppose the subjective probability of being over 55 years old is .4 [s(B) = s(C) = .4], and the probability of having a heart attack is . 3 [s(A)]. Suppose the probability that Mr. F is over 55. given he has had a heart attack is .8 [s(BIA) = .8]. It seems reasonable to suppose that the probability that Mr. G. is over 55, given Mr. F has had a heart attack is still .4 [s(CIA) = s(C)]. Hence, the two conjunctions are predicted to be s(AnB) = .45, which exceeds s(A), and

496

M.H.

B i r n b a u m , C . J . Anderson,

&

L.G. Hynan

s(AnC) = .28, which is (properly) less than both s(A) and s(B). These situations are illustrated in Figure 4, where it can be seen that s(AnB) can be greater than or less than s(A), depending on the value of s(BIA). In each of these cases, the concept of subjective

conditional probability seems a natural replacement for the concept of representativeness. Furthermore, because it can be quantified and specified in a model, it allows an explicit prediction of when the conjunction will violate the inequalities imposed by probability theory. If a concept is not required to deduce the phenomena to be explained, by Ockham's razor, the construct can be eliminated from the theory of the phenomena without loss. I t will be useful to devise experiments in which the construct of representativeness is required and is also testable. The theory of representativeness appears to be one that can be stated in a form that can be tested, but it remains as yet untested. Yates and Carlson (1986) elaborated the theory of Tversky and Kahneman (1983) that subjects make "natural assessments" of similarity or representativeness: they postulated that there could be a variety of strategies or procedures that might be evoked by different situations. To test this hypothesis would require demonstrating that some variable (such as a change in instructions or "set") produced a change that could not be explained by a change in parameters of an unchanging algebraic rule, but instead required a change in algebra. There are subtle changes in wording that seem likely to induce the subject to utilize different rules. For example, consider the following problems: 1 . If you were taking an examination, which of the following choices would most probably be marked as correct? a. men tend to be taller than women. b. men tend to be heavier than women. c. both a and b.

2. In a survey of adults, M. and F., a male and female, were selected at random: which statement is most probably correct? Which

statement would you bet on? a. M. is taller than F. b. M. is heavier than F. c. M. is both taller and heavier than F.

Probability judgment

4 97

According to the conventions of testing alluded to in the first problem, when both choices are separately "true", then the choice of "both" is considered correct. However, in the second form of the problem, c is the least probable. If the answers to different forms of a single problem produced different solutions, it would not necessarily imply that the subject's rule changed, since a change in response could be produced by different values of the parameters in an unchanging algebra. Therefore, to demonstrate the locus of the effect of the instructional "set", it would be necessary to conduct a complete investigation t h a t would allow one to estimate the parameters and test the algebraic model of each type of problem. It remains to be demonstrated that changes in the problem produce changes in the strategies used by subjects. Such investigations may reveal how subjective and judged probabilities are formed in different contexts. There may exist a set of procedures that would allow a n investigator to elicit judgments of probabilities that would obey a consistent algebra, so that the subjective probabilities, defined as parameters in the model, might have some use in guiding decisions. If subjective probabilities are to determine the future, we have a right to know what their properties are, how they can be measured, and how they can be properly employed in making better decisions.

REFERENCES Birnbaum. M.H. (1982). Controversies in psychological measurement. In B. Wegener (Ed.), Socia1 attitudes a nd DSVC hoDhvsica1 measurement. Hillsdale, N J : Erlbaum, 401-485. Birnbaum, M.H. (1985).Relationships among models of salary bias. American PSvChOlO~iSt,40.862-866. Birnbaum, M.H., Anderson, C.J., & Hynan, L.G. (1989). Conditional Probability and the Confunction Fallacy. Unpublished manuscript. Available from author, Fullerton, CA 92634. Dept. of Psychology, C.S.U.F., Birnbaum, M.H., & Hynan. L.G.(1986). Judgments of salary bias and test bias from statistical evidence. Qreanizational Behavior and Human Decision Processes, 37,266-278. Birnbarm. M.H.. & Mellers. B.A. (1983).Bayesian inference: Combining base rates with reports of sources. Journal of Personalitv and Social Psvcholocy. 792-804. Birnbaum. M.H., & Stegner. S.E. (1979).Source credibility in social judgment: Bias, expertise, and the judge's point of view. Journal of Personality and Socid Psvcholo~,37.48-74. Birnbaum. M.H., & Stegner. S.E. (1981).Measuring the importance of cues in judgment of individuals: Subjective theories of IQ a s a function of heredity and environment. Journal of Emerimental and Social Psvcholos!y, u.159182. Chandler, J.P. (1969).STEPIT: Finds local mlnima of a smooth function of several parameters. Behavioral Science, 14.81-82.

s,

4 98

M.H.

Birnbaum, C . J .

Anderson,

&

L.G. H y m n

Hogg. RV.. & Craig, A.T. (1965). Introduction to rnathemat 1cal s m. New York: Macmillan. Kahneman. D., & Tversky. A. (1972). Subjective probability: A judgment of 430-454. representativeness. nitive Psvc 1 Kahneman. D.. & TverskyTA. (1980)%sal schemas in judgments under uncertainty. In M. Fishbein (Ed.), Procress in social DsvcholoPv. Hillsdale. NJ: Erlbaum. Kahneman. D., Slovic. P.. & Tversky. A. (1982). Judement under uncerta intv; .New York Cambridge University Press. Krantz, D.H.. Luce. R.D.. Suppes, P., & Tversky. A. (1971). measurement (Vol. 1). New York Academic Press. Kyburg, H.E.. & Smokler. H.E. (1964). (Eds.) Studies in subjective Drobability. New York: Wiley. Mosteller. F.. Rourke. RE.K.. & Thomas, C.B. (1961). p r o w v and statisticg. Reading. MA: Addison-Wesley. Parducci. A. (1965). Category judgment: A range-frequency model. psvcholo€ficd Revlew. 72.408-418. Parducci. A. (1983). Category ratings and the relational character of judgment. In H.C. Geissler (Ed.) Modem issues in Dercw.Berlin: VEB Deutscher Verlag der WissenschaRen. p. 262-280. Savage, L.J. (1954).m e foundations of statistics. New York Wiley. Slovic. P.. Lichtenstein. S., & Fischhoff. B. (1988). Decision making. In R.C. Atklnson, RJ. Hemstein, G. Lindzey. & R.D. Luce (Eds.), Stevens' handbook of emerlmental Dsvcholo (Vol 2). New York Wiley. Thursto-comparative judgment. ical Review. 273-286. Torgerson, W. (1958).Psvcholocical scaling. New York Wiley. Tversky. A.. & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. PsvcholoPical Review, m, 293315. von Winterfeldt. D. & Edwards, W. (1986).Decision analvsis and behavioral resea rch. Cambridge: Cambridge University Press. Varey. C.. Mellers. B.A.. & Birnbaum, M.H. (in press). Judgments of proportions. urn 1 rimn 1 1 . H m r i n P Wason,?.C.. %l%&-L.&~ ~ ~ ~ ~ ~ 2 ~C:=b: P s ~ Harvard University Press. Wyer. R.S. (1974). Coenitive orion and chance: An information -Drocessing UDroach. Hfllsdale, NJ: Lawrence Erlbaum. Wyer. R.S. (1976). A n investigation of the relations among probability estimates. j, ational Behavior and Human Performance, JJ1-18. Yates. J.F., & Carlson. B.W. (1986). Conjunction errors: Evidence for multiple judgment procedures, including "signed summation." Orcanizational 230-253. Behavior and Human Decislon P r o c w

,a.

x,

.=,

c

Cognitive Biases J.-P. C a v e r n i , J.-M. F a b r e a n d M . G o n z a l e z ( E d s . ) 0 E l s e v i e r Science P u b l i s h e r s B. V. (North-Holland),

1990

OVERCONFIDENCE IN SELF-ASSESSMENT OF MOTOR SKILL PERFORMANCE Nigel Harvey Department of Psychology, U n i v e r s i t y C o l l e g e London, UK

Abstract. Subjects used a joystick to move

a point on a screen to intercept a moving target a s fast as possible. On each trial, they assessed whether interception had been faster than on the previous trial and estimated the probability that this assessment was correct. These estimates showed that subjects were overconfident in their self-assessments. Also, a s interception performance improved, selfassessment performance declined and overconfidence in self-assessment increased.

INTRODUCTION

Many cognitive psychologists feel that Anderson's (1976) distinction between declarative knowledge (i.e. knowing that) and procedural knowledge (i.e. knowing how) is a fundamental one. Other chapters on biases in probabilistic judgement discuss experiments in which subjects assess the probability of correctness of their answers to questions designed to test their declarative knowledge. In general, results from these tests of semantic and episodic memory have shown that people tend to be overconfident in their declarative knowledge (e.g. Lichtenstein, Fischhoff, & Phillips, 1981; Wagenaar, 1988; Wright & Phillips, 1980). Recently, other studies have been performed to determine whether this same cognitive bias is present for self-assessments of procedural knowledge. Subjects have been asked to assess the probability that their attempts to perform some skill have been or will be effective. Studies of problem solving (Metcalfe, 1986). judgemental forecasting (Fischhoff & MacGregor, 1982; Harvey, 1988) and judgemental control (Harvey, in press) have all shown that people are indeed overconfident in their ability to perform skilled tasks. Whether the knowledge tested is declarative or procedural, the same cognitive bias towards overconfidence is present. In this chapter, I will discuss work that I have done to generalize this conclusion to cover another type of procedural knowledge - that involved in performing a motor skill. The skill chosen for study was target interception. After each trial, subjects

500

N. Harvey

decided whether or not the time that they took to intercept the target was greater or less than on the previous trial and then they estimated the probability that this decision would be correct. Overestimation of this probability was taken to indicate overconfidence whereas its underestimation was taken t o demonstrate underconfidence. Before I describe the experimental predictions in detail, I would like to outline briefly why the study of biased confidence in skilled tasks is of practical as well as of theoretical interest. Degree of confidence prior to performance affects a n individual's decision of whether to participate in some activity (e.g. Griffin & Keogh, 1982). Biased confidence can have various deleterious effects. Overconfident individuals will decide t o participate when they are insufficiently skilled. This may cause accidents or other disruption. Underconfident individuals may decide not to participate when they have a good chance of achieving success or contributing to success in some joint activity. This may cause people to be unemployed when they need not be. Because of this, training schemes for the unemployed in Britain have included acquisition of metacognitive skills as one of their goals (Hayes, Fonda, Pope, Stuart, & Townsend. 1983). Degree of confidence after performance but prior to receiving information about its success can affect an individual's decision of whether or not to seek that information. Again biased confidence can have deleterious effects. Individuals who are highly overconfident in their self-assessments may not attend to or seek the information about their performance that would facilitate further learning. For instance, a surgeon who is highly overconfident in a particular technique may fail to perform the follow-up survey that would reveal its deficiencies. I t is well known that as skilled Performance improves, confidence in that performance increases (e.g. Schmidt & White, 1972). However, all experiments have used confidence ratings. While these allow changes and differences in confidence levels to be examined, they do not allow a bias towards overconfidence or underconfidence to be identified. I t is possible that as people become more skilled, they do not just become more confident in their performance but become (increasingly more) overconfident in it. Now let us turn to the experimental predictions. Obviously, I expected that findings from other procedural skills would

Overconfidence i n self-assessment of performance

501

generalize. In other words, a n overall cognitive bias towards overconfidence would be present. However, in addition to this, I made two rather more specific predictions based on work that has shown that overconfidence in performance declines as task difficulty decreases. (This work has been reviewed by Lichtenstein et al., 1981.) First, as people get better at the interception task, the variance of the times they take to intercept the target will decrease. On the whole, therefore, interception times on successive trials will become increasingly similar. Thus it will be increasingly difficult to decide whether interception time was greater or less than on the previous trial. This should result in increasing overconfidence in the self-assessment decision as the experiment progresses. The second specific prediction concerns the effect of difficulty of the interception task. Two versions of the task were examined. In one, the target's movement was such that it was easy to intercept: in the other, its movement made it difficult to intercept. When interception is easy, means and variances of interception times should be less than when interception is difficult. Hence interception times on successive trials should be more similar, self-assessment should be more difficult and overconfidence should be greater when the interception task is easy than when it is difficult.

METHOD Apparatus and stimuli

The experiment was controlled by computer. Subjects sat in a quiet room in front of a fast phosphor colour monitor. Stimuli appeared on the screen and subjects made their responses by moving a joystick and pressing a button on its handset. Targets were small white squares with sides subtending .2 minutes of visual angle. They moved on a horizontal axis 3 minutes of visual angle down from the top edge of the screen. Easy targets moved in simple harmonic motion. Difficult targets moved in complex harmonic motion formed by the addition of the second harmonic to the fundamental frequency used in the easy task. (The harmonic had twice the amplitude of the fundamental and its phase was 90 degrees in advance of it.) In both versions of the task, the

502

N. Harvey

target moved with a fundamental frequency of ten cycles per minute (.166 Hz) and the oscillating movement spanned 4 minutes of visual angle. Subjects controlled the position of a point on the screen by moving a joystick with a gain of .2. At the start of each trial, subjects moved the point into a square start box located on the central vertical axis of the screen with its midpoint 2.6 minutes of visual angle u p from the bottom edge and 10 minutes of visual angle down from the point where the central vertical axis of the screen intersected the horizontal axis along which targets moved. The start box was hollow with an open top. Its sides and bottom subtended -75minutes of visual angle. After intercepting the target, subjects received feedback messages, self-assessment questions and probability estimation questions on the screen. All this information was presented in white upper-case characters that subtended horizontal and vertical visual angles of .3 minutes and .4 minutes respectively and was, therefore, clearly legible to all subjects. Subjects responded to questions by using the joystick to move the point that it controlled into one of a number of response boxes that were displayed on the screen. After the point had been placed in the chosen box, subjects pressed a button on the joystick handset to register the label displayed below that box as their chosen response. All response boxes were of the same dimensions as the start box. Subjects

Forty subjects aged between 18 and 30 years each served for one session of approximately 60 minutes duration. Experimental design

The experimental session was divided into two blocks of 80 trials. On the first trial of each block, subjects just intercepted the target. On subsequent trials they first intercepted the target then assessed whether their interception was faster than on the previous trial and finally estimated the probability that this self-assessment was correct. To ensure that the target was not predictable from trial to trial and therefore capable of being intercepted with a stereotyped response, the centre position of its oscillatory movement, the time

Overconfidence i n s e l f-assessment o f performance

503

for which the subject previewed it before being allowed out of the start box and the phase of its movement at which it first appeared were all varied randomly across trials. Ten equally spaced points on the axis of target movement were chosen as central positions for the oscillation. Each one of the ten positions was used on a randomly selected eight trials in each block. The eight trials that shared the same central position for target movement differed in terms of the time for which subjects had to preview the target before being allowed out of the start box. The eight different preview times were 0.3, 0.6, 0.9, 1.2, 1.5. 1.8, 2.1 and 2.4 seconds. Thus in each block of 80 trials there was one trial with each combination of preview time and centre position of target movement. The phase of movement at which the target first appeared was chosen randomly on each trial from 64 different possible phases of the fundamental frequency. These were equally spaced and ranged between -90 degrees and +90 degrees. The 40 subjects were divided into four groups of ten each. The first group (EE) intercepted easy targets in both trial blocks: the second group (DD) intercepted difficult targets in both trial blocks: the third group (DE) intercepted difficult targets in the first trial block and easy ones in the second: the fourth group (ED) intercepted easy targets in the first trial block and difficult ones in the second. The purpose of this was to determine whether performance and its self-assessment are affected in any way by difficulty of a previous task as well as by difficulty of a current one. In other words, the aim was to examine whether conditions of practice have any long-term effect on later performance. In fact, as the results show below, there was no evidence of a n effect of previous task difficulty. Procedure

Each trial started with the message -Move into the start box but do not leave it until you hear the tone'. The entry of the subject's point into the box made the target appear. (If the subject left the start box before the tone signalled the end of the preview period, the target vanished. It only reappeared when the point reentered the start box.) After hearing the tone, the subject intercepted the target as quickly as possible. When interception occurred, the target vanished and another tone sounded. After intercepting the target

504

N. Harvey

on the first trial of each block, subjects just received a message on the screen informing them of how long (in ms) it had taken them to intercept the target. On all other trials in each block, they received a message asking them whether they were faster at intercepting the target than on the previous trial. Below this message were two response boxes. Under the lefthand one was the word 'NO' and under the righthand one was the word 'YES'. After subjects had selected between these alternatives, another message appeared lower down on t h e screen. It asked them to estimate the probability (in Yo)that the decision that they had j u s t made was correct. Below this message were another six response boxes. From left to right, the labels under each box read 50-59Y0, 60-69%, 70-79%,80-89%, 90-99% and 100%. Once they had selected their response, they received feedback informing them of whether their response was actually faster than on the previous trial and also of how long it took (in ms). Half-way through the experiment, between the two trial blocks, the subjects were given a short break. RESULTS

Four aspects of subjects' performance were analysed: median interception times, median differences between successive interception times, percentage of self-assessment decisions that were correct and confidence bias in the probability estimates. So that changes in these variables could be examined within as well as across the blocks, the 80 trials within each block were split into five runs of 16 successive trials. Each of the four above-mentioned dependent variables was subjected to three analyses of variance. The first of these was on the first block only. Subjects in the DE group and the DD group were treated as belonging to a single group. Similarly those in the EE group and the ED group were treated as belonging to a single group. Thus, the analysis had one between-subjects variable (interception task difficulty) and one within-subjects variable (trial run). The second analysis of variance was on the second block only. There were two between-subjects variables -interception task difficulty and whether difficulty level was the same or different from that in the first block. Trial run was again a within-subjects variable. The third analysis of variance included data from both trial blocks.

Overconfidence i n self-assessment of performance

505

There were two between-subjects variables - interception task difficulty in the first block and whether interception task difficulty was the same or different in the two blocks. There were also two within-subjects variables -trial block and trial runs within a block.

Median interception times (9) averaged over subjects

B l o c k 1 Trial Runs

Group

1

B l o c k 2 Trial Runs

2

3

4

5

1

2

3

4

5

EE

4.4

3.5

2.9

2.9

2.6

3.1

2.9

3.1

2.9

2.7

DD

4.9

4.6

4.3

3.7

4.1

4.2

3.9

3.9

3.4

3.4

DE

4.0

3.7

3.6

3.3

3.3

2.7

2.7

2.6

2.6

2.7

ED

3.9

3.4

3.2

3.3

3.0

3.7

3.4

3.3

3.0

3.2

First block analysis of the above data showed that it took longer to intercept difficult targets (F(1,38)= 4.19; p < ,051and that performance improved over trial runs (F(4.152) = 12.38; p < .01). Second block analysis also showed that difficult targets took longer to intercept (F(1.36) = 8.14; p < .01) and t h a t there was improvement over trials (F(4,144)= 4.25; p < .01). Analysis of data from both blocks showed that performance improved over trial blocks (F(1.36) = 28.61; p < . O l ) and over trial runs within a block (F(4.144) = 18.23; p < .01) and that there was greater improvement within the first block than within the second one (F(4.144) = 2.72, p c .05). An interaction between the two between-subjects variables showed that speed of interception depended on subject group (F(1,36)= 4.61; p < .05). On average, it was slower in the DD (4.0 s ) and ED (3.3 s) groups than in the EE (3.1 s) and DE (3.1 s) groups. Finally, a n interaction between first block difficulty and trial block (F(1,36)= 8.63; p < 0.01) arose because median interception time averaged over DD and DE groups was very much greater than

N. Harvey

506

median interception time averaged over EE and ED groups in the first block but not in the second one. Median differences between successive interception times averaged over subjects

Block 1 T r i a l Runs Group EE DD DE ED

Block 2 T r i a l R u n s

1

2

3

4

5

1

2

3

4

5

2.0 2.8 1.7 1.6

2.2 2.8 2.2 1.7

1.6 2.3 1.8 1.9

1.4 2.4 1.5 2.1

1.4 3.2 1.8 1.7

1.4 1.7 1.0 1.7

1.3 2.1 1.0 2.4

2.2 2.3 1.3 1.8

1.5 1.7 1.0 1.8

1.4 1.6 1.2 1.6

When blocks were analysed separately, it was found that, although the difference between successive interception times was numerically greater for the difficult than for the easy task in both blocks, this effect only reached significance in the second block (F(1.36)= 14.02; p < .01). The analysis of both blocks together showed that the difference between successive interception times was less in the second than in the first block (F(1,36)= 10.74: p < .01). Also, a n interaction between the two between-subjects variables arose because, on average, the difference between successive interception times was larger for the DD (2.3 s) and ED (1.8 s) groups than for the EE (1.6 s) and DE (1.5 s) groups (F(1.36)= 5.71; p < -05). Finally, a n interaction between first block difficulty and trial block (F(1.36) = 8.96; p < .01) occurred because the median difference between successive interception times averaged over the DD and DE groups was very much greater than that difference averaged over the EE and ED groups in the first trial block but not in the second one. Self-assessment decisions (940 correct averaged over subjects)

The data below were subjected to an arcsine transformation prior to analysis. First block analysis revealed two effects. An effect

Overconfidence i n self-assessment o f performance

507

of trial run (F(1.152) = 5.62;p c .01)showed that after an initial rise, veracity of self-assessment declined slightly over the block. An effect of difficulty (F(1.38) = 4.14: p c .05)showed that selfassessment was worse when the interception task was easy than when it was difficult. Second block analysis showed an effect of trial run (F(4.144) = 2.82;p c .05)- again, after an initial rise, selfassessment became worse over the trial block. Analysis of both blocks together showed a n effect of trial run (F(4.144) = 4.11; p < .01)and an effect of block (F(1.36) = 7.09; p < -01).The latter indicates that self-assessment was worse in the second half of the experiment than in the first half. An interaction between block and trial run (F(4,144) = 4.25;p c .01)appears to arise because the decline in veracity of self-assessment was larger but more gradual in the second block than in the first.

B l o c k 1 T r i a l Runs

B l o c k 2 T r i a l Runs

Group

1

2

3

4

5

1

2

3

4

5

EE

82

91

78

85

85

83

84

81

82

76

DD

87

94

78

86

91

84

87

85

82

82

DE

86

89

87

88

91

84

86

90

86

85

ED

87

87

85

90

82

85

90

85

81

77

Overconfidence The difference between the mean of the probability responses on each trial (expressed as a value between zero and unity) and the overall proportion correct in that run gives a measure of confidence bias (Lichtenstein et al., 1981). Overconfidence is indicated by a positive difference and underconfidence by a negative one. First block analysis of the data below revealed two effects. The effect of difficulty showed that overconfidence was greater when the interception task was easy than when it was difficult (F(1,38) = 4.3; p < .05). The effect of trial run showed that overconfidence first

N. Harvey

508

increased and then declined slightly over the block (F(4.152)= 4.3; p c .01). In the second block analysis, there was just evidence of a n increase in overconfidence over the block (F(1,144)= 2.6; p c.05). Analysis of both blocks together also showed effects of first block difficulty and trial r u n b u t , in addition, it revealed t h a t overconfidence was greater in the second half of the experiment (F(1.36)= 9.55; p < .01).

B l o c k 2 T r i a l Runs

B l o c k 1 T r i a l Runs Group

1

2

3

4

5

1

2

3

4

5

~~

EE

.09

DD

.03

.16

.08

.04

.11

.10

.12

.12

.15

. 0 7 -.lo

.12

.06

.03

.10

.04

.07

.10

.12

DE

.05

.05

.05

.04

.03

.09

.06

.03

.07

.09

ED

.05

.06

.09

.04

.11

.06

.05

.08

.12

.13

DISCUSSION

The predictions outlined in the introduction fared reasonably well. First, the bias towards overconfidence that has been previously found in a variety of tests of both declarative and procedural knowledge was present here also. Findings from other areas generalize to motor skill. Second, it is clear from the effects of trial block (and rather less clear from the effects of trial r u n within a block) that changes in self-assessment over time were much as predicted. Interception performance (measured by median interception time) improved. As it did so, difficulty of the selfassessment t a s k (measured by median difference between successive interception times) increased. Consequent on this, selfassessment deteriorated and overconfidence in self-assessment increased. Finally, in the first half of the experiment at least, it is clear that self-assessment was worse and overconfidence in this judgment was greater when the interception task was easier. However, it was not apparent how these effects were mediated by

Overconfidence i n s e l f -a ssessmen t o f p e r f orman c e

509

the difficulty of the self-assessment task (as measured by median difference between successive interception times). REFERENCES Anderson, J.R (1976).LanQuaQe,memorv and thought. Hillsdale, NJ: Erlbaum. Subjective confidence in forecasts. Journal of Fischhoff. B.. & MacGregor. D. (1982). Forecasting,I,155-172. G M i n . N.S., & Keogh, J.F. (1982). A model for movement confidence. In J.A.S. Kelso & J.E. Clark (Eds.), The develoDment of movement control and coordination. New York Wiley. Harvey, N. (1988).Judgmental forecasting of univariate time series. Journal of n ,A.95-110. Harvey, N. (1990).Effects of difficulty on judgmental probability forecasting of control response efficacy. Journal of Forecastirg. 9.in press. Hayes, L.. Fonda, N., Pope, M.. Stuart, R , & Townsend. D.T. (1983).Training for skill ownershiD -learning to take it with vou: A reDort to the ManDower Services Commission. Sussex: Institute of Manpower Studies. Lichtenstein, S.. Fischhoff. B., & Phillips, L.D.(1981).Calibration of probabilities: State of the art to 1980. In D. Kahneman. P. Slovic, & A. Tversky. (Eds.), Judgment under uncerta ink: Heuristics a nd b w . New York: Cambridge University Press. Metcalfe, J. (1986).Feeling of knowing in memory and problem solving. Journal of Emerimental Psvcholom: Learninc. Memorv and Cognition. 12.288-294. Schmidt, R.A.. & White, T.L. (1972).Evidence for a n error-detection mechanism in motor skills: A test of Adams' closed-loop theory. Journal of Motor Behaviour, 4.143-153. Wagenaar. W.A. (1988). Calibration and the effects of knowledge and reconstruction on retrieval from memory. Coenition, 28. 277-296. Wright. G.,& Phillips, L.D. (1980).Cultural variation in probabilistic thinking: Alternative ways of dealing with uncertainty. International Journal of psvcholo&. 239-257. ~

s,

This Page Intentionally Left Blank

Cognitive Biases J . - P . C a v e r n i , J . - M . F a b r e and M. Gonzalez ( E d s . ) 8 E l s e v i e r S c i e n c e P u b l i s h e r s B . V. (North-Holland),

1990

BIAS IN =A-MEMORY

PERFORMANCE AND ITS IMPLICATIONS FOR MODELS OF lMEMORY STRUCTURE Alastair G.R. McClelland*, Andrew S. CouLson** and Sarah E. Icke* *

N a t i o n a l Hospitals C o l l e g e o f Speech S c i e n c e s ,

London

and U n i v e r s i t y C o l l e g e London, UK

**

U n i v e r s i t y C o l l e g e , C a r d i f f , UK

Abstract. This paper is concerned with the effect of memory task on meta-memory performance, a s assessed by calibration accuracy. In the first experiment overconfidence was evident in the general knowledge (semantic memory) task, but was virtually absent in a recognition (episodic memory) task. In the second experiment subjects showed slight overconfidence on a vocabulary test, but marked underconfidence when later asked to make recognition judgements about the vocabulary test items. In both experiments, there was no correlation between subjects' calibration scores on the two tasks, and in the second experiment stochastic independence was evident between actual performance on the two tasks. The results are discussed with reference to the episodic/semantic memory distinction proposed by Tulving (1972). and it is concluded that the findings are consistent with the operation of independent memory systems.

INTRODUCTION

It is now well established that when individuals are asked to give ratings of their confidence in answers to general knowledge questions, they are badly calibrated, due to marked overconfidence in their performance (Lichtenstein, Fischhoff, 8r Phillips, 1982). An interesting question is whether or not this phenomenon is observed in other memory tasks, such as recognition, and, more generally, what is the relationship between meta-memory performance across different memory tasks. The comparison of tasks such as general knowledge and recognition is of particular importance as, by Tulving's (1972) definition, these tasks represent prototypical semantic and episodic memory tasks respectively. Tulving proposed that episodic memory deals with personal, temporally dated events which have been witnessed by the observer, whereas semantic memory is concerned with the storage and retrieval of general, abstract knowledge about the world. He suggested that episodic and semantic memory represent distinct memory systems,

512

A. McClelland, A. Coulson,

&

S. Icke

their function being "governed at least partially by different principles" (Tulving, 1983, p. 66). The majority of calibration experiments investigating memory performance have used semantic memory tasks, but a recent study by Wagenaar (1988) is a notable exception. Wagenaar argued that when subjects base their responses on inference or reconstruction, rather than on information retrieved directly from memory store, they will show overconfidence, reflecting an unwarranted trust in their reasoning abilities. In support of his claims, Wagenaar (1988, Experiment 1) showed that calibration was good for old items and poor for new items when subjects were required to make old/new recognition judgements about items they had learned incidentally, and in a second experiment, he showed that there was less overconfidence for visual questions concerning a television programme than for factual questions. Although Wagenaar made no reference to multiple memory systems, his results are certainly compatible with the episodic/semantic distinction. According to Tulving (1983), the episodic memory system is very limited In its inferential powers, whereas the semantic memory system h a s a rich inferential capability. Thus, following Wagenaar (19881, one would predict that when subjects perform an episodic memory task they should reveal less overconfidence than when performing a semantic memory task. Further, if Tulving is correct in believing that episodic and semantic memory represent separate systems, one might predict that there would be no correlation between subjects' calibration performance on an episodic and semantic memory task. The aim of the first experiment was to investigate these two possibilities. EXPERIMENT 1

Method Subjects The subjects were 29 undergraduate students at University College London. Materials Two types of memory questionnaire, one semantic and the other episodic, were compiled. The semantic questionnaire took the form of a general knowledge test. I t consisted of 74 twoalternative questions covering a wide range of topics, e.g. "What

B i a s i n meta-memory performance

513

colour are sapphires? (a) red (b) blue". For each question, the position of the correct answer was determined at random. For the episodic memory task, 74 high frequency (100 or more occurrences per million) concrete nouns were selected from the Thorndike and Lorge (1944) word norms. In the learning phase of the experiment, these items were presented to subjects sequentially, each subject receiving a different random order. The episodic task took the form of a two-alternative forced choice recognition test. The questionnaire consisted of 74 target-distractor pairs. Each target was paired with a semantically related distractor, e.g. "(a) town (b) city", the position of target and distractor in each pair being determined a t random. Procedure The subjects were told that they would be presented with a series of words to learn in preparation for a subsequent memory test. Each subject was seated in an experimental cubicle, in front of a computer monitor, and instructed that the words would be presented in the centre of the screen a t a rate of one item every 1.5 seconds. At the end of the presentation sequence, the subjects returned to the main laboratory and completed a number of unrelated memory tasks. After a short break, and one and a half hours after the presentation of the recognition-test target items, the subjects completed two questionnaires. They were requested to circle one of the two alternatives for each question and then to give a percentage rating of confidence in their answer. A value of 50% would indicate a complete guess, a value of 100% absolute certainty. Subjects were encouraged to use the full range of percentages between these two values, and reminded that they should not give a rating of less than 50%. as this would imply greater confidence in the unselected alternative. Results

Each subject's responses were grouped into six categories on the basis of confidence level. These were 50-59, 60-69, 70-79. 8089, 90-99. and 100. The mean confidence rating and the percentage correct a t each confidence level, were computed for each subject on each task. On the basis of these values a measure of calibration was computed (as described by Lichtenstein, Fischhoff, & Phillips, 1982) for each subject on each task. In addition, the

A. McClelland, A. Coulson,

514

&

S. Icke

overall mean confidence rating and the mean percentage correct for each category were computed for the two tasks, and these values were used to produce the calibration curves which are displayed in Figure 1. Percentage correct 100

90

80 70 60 50 40

50

I

I

I

I

I

60

70

80

90

100

Confidence rating

Figure 1. Calibration curves for Experiment 1. Inspection of these results reveals that overall, subjects were far better calibrated for the recognition task than for the general knowledge task, the latter revealing marked overconfidence. I t was also evident that the percentage of items correct was considerably higher for recognition than general knowledge. These findings were confirmed in the statistical analysis. Mean calibration performance was significantly better for recognition than for general knowledge, t(28) = 4.42. p < .001, and an analysis of the percentage correct scores revealed significantly better performance on the recognition task than on the general knowledge task, t(28) = 6.66, p c .001. A by-subjects analysis revealed a modest negative correlation between the subjects' percentage correct scores for the two task (r = -.16), and essentially no correlation between the subjects' Calibration scares for the two tasks (r = -.06). To ensure that the lack of correlation between the calibration scores was not

B i a s in m e t a - m e m o r y p e r f o r m a n c e

515

simply due to unreliability in the calibration measure, split-half correlations were computed for both tasks. For each subject a calibration score was computed on the basis of answers to the even numbered questions and a second score on the basis of the odd numbered questions. The split-half correlations were found to be .45 and .43 for the semantic and episodic tasks respectively. Discussion

This experiment yielded two main findings. First, calibration for the episodic memory task (recognition) was significantly better than for the semantic memory task (general knowledge), the latter revealing the overconfidence commonly associated with general knowledge items (cf. Lichtenstein, Fischhoff, & Phillips, 1977). Thus on a task which gives scope for the subjects to use inference (general knowledge), there is marked overconfidence, whereas subjects are well calibrated on the recognition task when they are likely to retrieve information directly from memory. However, there is one serious problem with this interpretation. I t is well known that overconfidence is more extreme the more difficult the task, and in this experiment subjects found the general knowledge task more difficult than the recognition task. Of course, it is possible that when a task is more difficult subjects tend to rely increasingly on inference, and hence are more overconfident, but this is conjecture: one ideally needs to compare tasks which differ in the degree of inference they are likely to engender, b u t are of equivalent difficulty. The second major finding was the lack of association between calibration performance on the recognition and general knowledge tasks. This (coupled with the reasonably high correlations within tasks) suggests that calibration is not a general ability possessed by subjects, but is task specific. This lack of correlation between the two memory tasks is consistent with the notion t h a t semantic memory a n d episodic memory are independent systems, as Tulving has suggested (Tulving, 1983). However, a stronger case could be made for the episodic/semantic distinction if the same target items were used in both tasks. This would counter the argument that the lack of correlation observed between tasks was due to the difference in stimulus materials employed. Thus, the aim of the second experiment was to compare calibration performance in a semantic and episodic memory task whilst holding both materials and overall performance levels

516

A. McClelland, A. Coulson,

&

S. Icke

constant. In addition, by using the same target items and same subjects, the stochastic relationship between the tasks could also be investigated. EXPERIMENT 2

Method Subjects The subjects were 26 female undergraduate students reading Speech Sciences at University College London. Materials The target items were 32 words drawn from the Mill Hill Vocabulary Scale (Raven, Court, & Raven, 1982). In the standard version of this test, each target item is presented along with six other words, one of which is a synonym of the target. In the version of the test used here as the semantic memory task, the correct alternative was paired with one of the five distractors, s o that the subject chose between two, rather than among six items. Each target was printed in capital letters with the alternatives printed below it in lowercase. For example, the target "VERIFY' appeared with the alternatives "confirm" and "correct". Four addition items were added to the beginning of the list to give subjects practice at the task. Each subject received the same targets and distractors, but in different random orders. The episodic task was a two-choice recognition task. Each item consisted of a target drawn from the vocabulary test together with a distractor drawn from the Mill Hill Scale. For example, the target "VERIFY' appeared with the distractor "ELEVATE". The order of each target and distractor was randomly determined, and each subject received a different order of items, again determined at random. Procedure The experiment was conducted in a large lecture room. All subjects were handed a vocabulary test sheet, and were then given verbal instructions, similar to those used in the first experiment. They were asked to study each item, and select from the two alternatives the word they thought was a synonym of the target. In addition, they were asked to give a percentage rating of confidence in their answer. The subjects then worked through the vocabulary

517

Bias in meta-memory performance

test at their own pace, and all completed the task within 10 minutes. They were not informed that they would subsequently be given a recognition test, which was administered two weeks later. Subjects were told that one member of each word pair had appeared as a target item in the vocabulary test. Their task was to circle the target item, and to again give a percentage rating of confidence in their answer. The subjects were self-paced, and all completed the recognition task within 10 minutes. Results

The subjects' responses were treated in a similar manner to that described in Experiment 1, but slightly different confidence level groupings were used. Because of the very small number of responses in the range 51-59. and reasonably low number of responses in the range 60-69, these two ranges were combined. The mean confidence rating and mean percentage correct within each category were computed for the two tasks, and these values were used to produce the calibration curves displayed in Figure 2.

-

Percentage correct

.

..... ....

100

90

-

Recognition Vocabulary

80

70 60 50 40

50

I

I

I

I

60

70

80

90

Confidence rating Figure 2. Calibration curves for Experiment 2.

100

518

A . McClelland, A . Coulson,

&

S. Icke

Inspection of Figure 2 reveals that subjects were slightly overconfident on the vocabulary test, but for the recognition task they were extremely underconfident. Unlike the first experiment, this marked difference in confidence cannot be attributed to differences in overall performance level. There was no significant difference between the tasks t(25) = 1.20, p = .24. Surprisingly, statistical analysis revealed that there was no difference in calibration between tasks, t(25) = 1.60, p = .12, but it is clear from Figure 2 that subjects were poorly calibrated on the two tasks for different reasons. An examination of over/underconfidence scores (Lichtenstein, Fischhoff, & Phillips, 1982) revealed that although subjects were overconfident on the vocabulary test this was not significant, t(25) = 1.34, p = -19. but they were significantly underconfident on the recognition test, t(25) = 3.86, p c .001. This was also reflected in the positive correlation between calibration and over/underconfidence for the vocabulary test (r = .37) and the negative correlation for the recognition test (r = -.46). In addition, the correlation between calibration performance on the two tasks was small and negative, (r = -.lo). Finally, in terms of actual performance rather than rated confidence, a phi coefficient of .03 was observed when the data were cast into a 2 x 2 contingency table, indicating almost complete stochastic independence between the tasks. GENERAL DISCUSSION

The results from the second experiment partially replicate the results from Experiment 1, and extend the findings of Wagenaar (1988). In the first experiment the semantic task (general knowledge) showed marked overconfidence whereas the episodic task (recognition) was well calibrated. In the second experiment, the semantic task (vocabulary) still showed some overconfidence, b u t t h e episodic t a s k (recognition) revealed extreme underconfidence. Taking the semantic tasks first, it seems reasonable to suppose that a general knowledge task gives subjects greater scope for inference than a vocabulary task, and on the basis of Wagenaar's argument, this could explain the greater overconfidence in the former. Unfortunately, the results from the two experiments are not directly comparable because of the difference in overall performance level. The underconfidence

B i a s i n meta-memory p e r f o r m a n c e

519

revealed on the recognition task in Experiment 2 is more puzzling. Recognition memory is extremely powerful, particularly for low frequency words and even after a delay of two weeks, the subjects were able to recognize (on average) over 80% of the target items. Their calibration scores suggest that subjects did not trust their powers of recognition, and this led to underconfidence. Whatever the mechanisms underlying calibration bias, it appears that for semantic memory tasks, any bias tends to be in the direction of overconfidence, whereas for episodic memory tasks, it tends to be in the direction of underconfidence, indicating a dissociation between meta-memory performance on the two types of task. In both experiments, the calibration measures failed to show a positive correlation between memory tasks. This suggests t h a t the information upon which confidence judgements are made in a n episodic memory task is quite different from that used in a semantic memory task, even when the judgements are being made about the same target items. In conclusion, subjects had a tendency to be overconfident with respect to their semantic memory ability and underconfident with respect to their episodic memory ability. This finding, taken together with the lack of correlation between metamemory performance as indexed by calibration, and the finding of stochastic independence between actual performance on the recognition and vocabulary tasks, is consistent with the notion that episodic and semantic memory performance reflects the operation of separate memory systems.

REFERENCES Lichtenstein, S., Fischhoff, B..& Phillips, L.D. [ 1982).Calibration of probabilities: The state of the art to 1980.In D. Kahneman. P. Slovic, & A. Tversky [Eds.). Judaement under uncertaintv: Heuristics and biases. Cambridge: Cambridge University Press. Raven, J.C.. Court, J.H.. & Raven. J. (1982). The Mill Hill Vocabularv Scale: 1982 Revision. London: H.K. Lewis & Co. Ltd. Thorndike. E.L., & Lorge, I. (1944). m e teacher's word book of 30.000 word& New York: Columbia University. Tulving, E. (1972).Episodic and semantic memory. In E. Tulving & W. Donaldson (Eds.). . New York Academic Press. Tulving, E. (1983). Elements of eDisodic memorv. Oxford: Clarendon Press. Wagenaar. W.A. [ 1988).Calibration and the effects of knowledge and reconstruction in retrieval from memory. Coenition. 28. 277-296.

This Page Intentionally Left Blank

SECTION 6 Biases and cognitive aids

This Page Intentionally Left Blank

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M. Gonzalez ( E d s . ) Q E l s e v i e r S c i e n c e P u b l i s h e r s B.V. (North-Holland),

1990

COGNITIVE AIDS AND DEBIASING METHODS: CAN COGNITIVE PILLS CURE COGNITIVE ILLS? Cideon Keren U n i v e r s i t y of U t r e c h t , The N e t h e r l a n d s

Abstract. The purpose of the present chapter is to examine the effectiveness of cognitive aids in reducing and eliminating cognitive biases and errors of reasoning. Several classifications of cognitive aids are briefly discussed. In particular, it is proposed to distinguish between two general types of cognitive aids, procedural and structure modifving. Procedural aids are based on a specific prescription or algorithm, and do not require a deeper structural understanding. While such mechanical aids may often lead to performance improvement, their effect is limited to a particular task and cannot be generalized. In contrast, structure modLfying aids require more effort and a change in the internal representation of the problem or task at hand. Such aids are more effective and permit generalizations beyond a particular task. The effectiveness of these two types of aids are further discussed in the context of improving the calibration of probabilistic assessments.

Few would challenge the assertion that a correct diagnosis is an essential first step for curing an illness. A necessary requirement for being a good physician is the ability to make a n accurate diagnosis: only then would we expect the physician to prescribe the right treatment and monitor the recovery, eventually changing the treatment if necessary. The employment of cognitive aids for modifymg and reducing potential cognitive biases should follow a similar route: First, the existence and nature of a potential bias have to be identified. The identification stage should focus on understanding both the environment and conditions under which the bias may occur, and the nature of the cognitive processes that may induce the bias. As a second step, we may consider different alternative means for reducing or eliminating a potential bias. Finally, once a certain debiasing technique has been chosen, it should be carefully monitored to assess its effectiveness, and at the same time assure that it does not lead to unwanted side effects (such as creating new biases). The present paper adopts the above prescription. The first section briefly covers some of the controversial issues of what is a bias (and consequently the errors it may lead to). Next, the diagnostic problems involved in identifymg and establishing the

524

G.

Keren

existence of a bias are discussed. Following. different possible classifications of biases (and errors) and potential corresponding cognitive aids are presented. In the fourth section the difficulties involved in evaluating and assessing the effectiveness of a given cognitive aid are presented. Finally, a particular bias, namely that of overconfidence (e.g., Lichtenstein, Fischhoff. & Phillips, 19821, is thoroughly analyzed within the proposed framework, and possible cognitive aids to modify it are critically evaluated. Cognitive biases have been studied in different research areas, such as decision making, reasoning, and social cognition. Though an attempt is made in this article to treat cognitive biases in general, the examples discussed are mainly drawn from the decision making and reasoning literature. INTERPRETING PERSPECTIVE

BIASES AND

ERRORS:

A

FUNCTIONAL

For identification and eventual modification of a potential bias, it is essential to define what a bias is. Jungermann (1983)proposed to distinguish between three types of biases, depending on their source. Judgmental biases are said to be deviations from the normative canons of logic and the principles of statistical theory. Supposedly, they reflect inappropriate or suboptimal reasoning compared with what is dictated by the normative theory, and therefore considered to be incongruent with rational behavior. The second type of biases, termed by Jungermann renresentational faults, stem from different formulations or frames of a decision problem. Indeed, there are several striking demonstrations showing that different framings of the same formal problem may lead to different (and often even conflicting) decisions and choice behavior (e.g., Slovic, Fischhoff, & Lichtenstein, 1982; Tversky & Kahneman, 1981). The third type, called copinP defects, are motivational and emotional in nature, and reflect the use of inadequate and suboptimal strategies and responses due to emotional stress and or lack of motivation (Janis & Mann, 1977). Not undermining the importance of such potential deficiencies, the present paper is limited to the first two types of errors that are mainly cognitive in nature. There is a deep disagreement among different researchers as to the meaning of cognitive biases and reasoning errors. One view asserts that our limited memory and processing capacity often lead

C o g n i t i v e a i d s a n d d e b i a s i n q methods

525

to the adoption of different simplifying strategies (usually referred to as heuristics), that under certain conditions may yield prevalent systematic errors (e.g., Kahneman, Slovic. & Tversky, 1982). The opposing view, simply denies the existence of judgmental errors (Cohen, 1981; Edwards, 1984), or in a weaker form claims that empirically it is impossible to establish unequivocally the existence of such errors (Cohen, 1981: Kruglanski & Ajzen, 1982). Some claim that the reported biases do not reflect inadequacies of human behavior but rather a researcher's bias (e.g.. Berkeley & Humphreys, 1982). I t is beyond the scope of this chapter to present in detail the various arguments raised in this context, and the different interpretations that researchers have attached to the concepts of error and bias (e.g,, Cohen 1981 and following peer commentary: Jungermann, 1983 Kahneman & Tversky, 1982). The following discussion is limited to three major arguments that are essential in the present context of debiasing and correction techniques. One claim, often raised in the context of representational faults, is that cognitive deficiencies are more apparent than real: They may simply represent incompatibility between the internal representational structure of different people. This is most pronounced (but not limited to) in the experimental laboratory where experimenter and subjects may not share the same representation of the problem space. Indeed, care should be taken not to infer an error when linguistic ambiguities are concerned. Notwithstanding, not all "framing" effects are necessarily due to linguistic ambiguities. As I will claim in the next section, there are genuine framing effects that may indeed lead to a biased representation and consequently to the use of unwarranted strategies or algorithms. Such framing effects do not necessarily lead to errors in the strict sense of the word, yet may result in inconsistent and unwanted decisions or preferences. 1 A second claim concerns the perspective from which biases (and eventual consequent errors) are judged. Hogarth (1981) 1. The word e r r o r i s o f t e n a s s o c i a t e d w i t h n e g a t i v e c o n n o t a t i o n s and i m p l i e s ( i m p l i c i t l y o r e x p l i c i t l y ) a s h o r t c o m i n g . The f r a m i n g e f f e c t s t h a t w e a r e c o n c e r n e d w i t h s h o u l d n o t be i n t e r p r e t e d i n t h i s manner. N e v e r t h e l e s s , t h e y may l e a d t o d e c i s i o n s t h a t d e c i s i o n makers would have l i k e d t o p r e v e n t would t h e y be aware o f t h e framing e f f e c t .

52 6

G. Keren

proposed that most decision research, and in particular that on heuristics and biases, has been conducted in a static environment, whereas most real life problems occur in a dynamic and changing environment. He suggested that many of the heuristics that supposedly lead to biases and errors in static conditions, may be justified and considered functional when viewed as moments in a continuous process and changing environment. Moreover, the static perspective, according to Hogarth, ignores the potential corrective nature of feedback. Indeed, under circumstances where optimal behavior may indeed differ under static and dynamic conditions, one should be careful to infer a bias (and a possible consequent error). On the other hand, it may be argued that part of an ideal decision making process is to identify the exact conditions under which a decision has to take place, and then adopt the optimal strategy or decision scheme that is appropriate for the given conditions. A bias is usually defined as a deviation from a normative correct answer, and thus implies the assumption of a single correct answer (Gigerenzer. Swijtink. Porter, Daston, Beatty, & Kruger, 1989). A major claim of those challenging the existence of biases and reasoning errors, and the most controversial one, concerns the lack of a universal and acceptable criterion from which a unique 'correct' solution or decision can be derived with certainty, and against which biases and errors can be assessed (Funder, 1987). It is true, especially under real life conditions, that a strict criterion for what is 'correct' may frequently not exist. Under s u c h conditions, it may often be difficult to judge among alternative decisions (or solutions to a problem) which one is best or optimal. Incidentally, t h a t may be one justification for adopting the "satisfying" principle (Simon, 1957). This, however, does not necessarily imply that certain decisions or solutions cannot be assessed as wrong. Solutions that are incongruent with given specified goals or that lead to internal contradictions with respect to these goals, could be classified as incorrect. For example, any decision rule that would lead to a "Dutch book' is unacceptable according to that criterion. Much of the debate on the meaning of errors and biases has centered around the appropriateness of the basic logic and fundamental reasoning processes underlying different solutions. The assertion h a s been made (e.g., Cohen, 1981) that different logical a n d reasoning systems can be formalized, and that no criterion exists for judging which of these systems is the 'correct'

Cognitive a i d s and debiasing methods

527

one. In other words, the claim is that at the final analysis, no absolute justification can be granted to any normative system. There are, however, strong arguments to challenge such a claim. Perhaps the most important one, in our context, is that the rules of a logical system (like for instance probability theory) are not arbitrary and, as proposed by Baron (1989),"can be justified in terms of arguments concerning their suitability to the purpose". Such a functional approach is adopted in the present paper. Consider for instance Euclidean geometry in contrast with Reman's geometry. As is well known, each of these two geometries is based on different axiomatic systems, and theorems that can proved to be correct in one system do not necessarily hold in the other system, and vice versa. Moreover, the question as to which of these two abstract systems is the 'true' or 'correct' one is a meaningless question that cannot be answered. The situation may be somewhat different, however, when one of the existing systems has to be chosen for specified purposes or concrete applications. For example, for most applied objectives of architecture or engineering t h e superiority of the Euclidian system is unquestionable. Unlike in the abstract case, in this concrete context there are sound and valid arguments to favor the Euclidean model. In general, for purposes of application, the criterion for preference should be pragmatic and functional: Apply the system (or model) that will best serve to achieve certain specified goals. As long as the practical aims and goals are well-defined.2 it should be possible to determine criteria for applying the most appropriate system. Once it has been established that a given system is the most suitable one for achieving certain goals, then any statement or conclusion t h a t is incongruent with it should be considered erroneous (even if they are valid within a different system). An analysis similar to the one offered above, is applicable to the assessments of cognitive biases and reasoning fallacies. For example, different probability calculi are possible, each of which may be based on a different axiomatic system. In addition, various 2 . I n my o p i n i o n , t h e main problem ( i n p a r t i c u l a r , under r e a l l i f e c o n d i t i o n s ) f o r i d e n t i f y i n g b i a s e s and e r r o r s i s n o t t h e l a c k of an a c c e p t a b l e c r i t e r i o n f o r ' c o r r e c t n e s s ' , b u t r a t h e r t h e f a i l u r e ( o r sometimes t h e i m p o s s i b i l i t y ) t o s p e c i f y unambiguously t h e problem s p a c e and t h e g o a l s t o be I t i s d i f f i c u l t t o determine what i s an e r r o r reached. when i l l - d e f i n e d problems a r e i n v o l v e d .

528

G. Keren

semantic characterizations of probability have been offered (e.g., Cohen, 1981). The question, however, is not whether classical probability theory is more 'correct' t h a n Cohen's Baconian probability (Cohen, 1977). or any other conceivable system. In the present context the main issue is what system can best serve the achievement of a certain goal or goals at any given time. The viewpoint advocated here is functional in nature, suggesting that the organism should adopt the most effective system for survival in the environment. Many of the recent discussions on the issue, notably the provocative article by Cohen (1981) and the peer commentary that followed it, have centered too much on abstract and theoretical considerations, disregarding the functional aspects that are the most important ones from a descriptive psychological viewpoint. For the remainder of this chapter it will be assumed that "deductive inference, because of its ability to preserve truth" (Margalit & Bar-Hillel, 1981. p. 347) is usually the most effective system to guide our judgments, decisions, and problem solving efforts, and is therefore the most acceptable and valid normative theory. To challenge t h a t assumption, one would have to demonstrate that any alternative normative model would be more constructive and functional in assisting us to solve our problems and achieve specified goals. Such a demonstration should not be on a n abstract and theoretical level, but should rather be in terms of pragmatic and practical examples. In a similar manner, given that we have to cope with and function in a probabilistic environment3 (Brunswik. 1952). it will be assumed that probability theory is the most useful one to describe our empirical world (Diaconis & Reedman, 1981). It is important to emphasize that acceptance of a certain normative theory does not imply that this theory is also adequate from a descriptive viewpoint. For example, Gigerenzer et al. (1989) claimed that many psychologists have adopted the view that (implicitly or explicitly) probability theory is a n adequate model for personal knowledge, and that the operation of the mind can best be described as that of an intuitive statistician. This assumption is certainly not shared in the present paper. The functional criteria mentioned above only suggest that the organism may best serve its goals and survival needs by adopting the most utilitarian normative 3. For the present discussion it is irrelevant whether this probabilistic nature is inherent in the environment, or just reflects our limited knowledge of the world.

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

52 9

system, and may thus strive to identify the rules of such a system. Whether such normative rules are indeed adopted, and the extent to which they differ from observed behavior, is mainly an empirical question. In summary, I propose that it is possible to establish a valid normative framework against which potential biases and errors can be assessed and evaluated. The validation of such a framework should mainly rest on functional and pragmatic criteria. As a n example for such a test, consider a set of probability statements that are incoherent. Under such conditions, a series of bets can be created that would lead to a so-called 'Dutch book namely the creation of a series of bets that would lead to a loss independent of the outcome. A probability system that leads to a 'Dutch book' should be rejected if just on utilitarian grounds. ESTABLISHING THE EXISTENCE OF BIASES

Given the underlying assumption that cognitive biases and corresponding errors can occur, the question still remains as to how they should be identified and corroborated. Once a bias has been identified, and assuming that it is reliable (i.e., can be replicated under identical conditions), there are at least three criteria to test the validity 4 of an alleged bias. 1. The representational issue

Cognitive biases and errors, according to the view presented here, result from fundamental faulty reasoning and inappropriate inferential rules (Kahneman & Tversky, 1982), and from failure to identify correctly the relevant problem space (e.g.. Keren, 1984). Excluded from this definition are errors or failures of performance that result purely from language misinterpretations. A linguistic problem statement must be translated into a meaningful cognitive representation: this translation process is inherently subjective such that many different (and often conflicting) guidelines may apply, but for which there is seldom any objective well-defined rule or algorithm with which all would agree. Indeed, different wording of the same problem may lead to different interpretations, and it is 4 . By v a l i d i t y i s meant h e r e t h e e x t e n t t o which t h e r e i s sound j u s t i f i c a t i o n f o r l a b e l i n g a c e r t a i n t y p e of b e h a v i o r as biased.

530

G.

Keren

often the case that subjects and experimenter derive different interpretations from the same problem due to linguistic ambiguities. For example, Olson (1976) used a replication and extension of an experiment reported by Kahneman and Tversky (1972); using several group treatments that differed only slightly in the wording of the problem (keeping the substance and structure of the problem unaltered), Olson reported very different patterns of results for the different groups. Similar 'wording' effects were demonstrated by Evans and Dusoir (1977) who noted that "The construction of their (Kahneman & Tversky) problems seems unnecessarily complex" (p. 13). Keren (1984) claimed that one of the examples used by Kahneman and Tversky (1972) was sufficiently vague to justify different interpretations of the problem. In particular, the problem was phrased such that distribution and class of distributions could be easily confused. A similar concern has also been expressed in the literature on reasoning by Henle (1962). who proposed that subjects may reason logically from misinterpreted premises (see also Revlin & Leirer. 1978). One should be careful, however, not to dismiss every bias or error by claiming that it is due to language misinterpretation. A clear distinction should be made between biases (and corresponding potential errors) that result from ambiguous or confusing wording, and between what has been termed framing. A frame contains "the decision-maker's conception of the acts, outcomes, and contingencies associated with a particular choice" (Tversky & Kahneman, 1981; p. 453), and determines the internal representation of the 'problem space'. Consider for instance the framing effect obtained by Tversky and Kahneman (1981) in which they present subjects with two alternative rescue programs to fight a hypothetical outbreak of an unusual Asian disease. Describing the two programs in terms of number of survivors for each program (survival frame), or number of deaths (mortality frame), produced a complete reversal in subjects' choice (A similar effect was demonstrated by the same authors [Tversky 8r Kahneman, 19861 by referring to information about the outcomes of two lung cancer treatments). No one could claim that the formulation of the problem was ambiguous or unclear. Apparently, describing the outcome of a program in terms of survivors or number of deaths invokes different internal representations (and different reference points) of the problem, and leads to different considerations (and strategies) in the decision process. In particular, the survival frame

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

531

results in risk aversiveness, whereas the mortality frame yields risk taking behavior. Another example of framing, that cannot be explained by language ambiguities, is offered by Slovic, Lichtenstein, and Fischhoff (1982). They framed the probabilities of being involved in a car accident for one group by indicating the probability for each single trip (trip-by-trip frame), and to another group by presenting the corresponding probability during a n average life time (life-span frame). The two groups showed a remarkably different pattern of responses regarding the use of seat belts, supposedly because each frame invoked different representations that lead to different judgment considerations. Most of the biases and errors reported in the recent literature (see a list by Hogarth, 1980) cannot be solely accounted for by pure language ambiguities. Nevertheless, each potential bias should be carefully examined in this respect, and any possible language ambiguity should be removed. Language effects that do not result from ambiguity or obscure wording should not be classified as artifacts. In addition to framing, there is growing evidence (both in the reasoning and decision making literature) that the cognitive tools by which we process and analyze incoming information are not always adequate. Humans' limited processing capacity often requires the use of simplified strategies (often referred to as heuristics) that may potentially lead to different biases. The literature on reasoning and problem solving portrays a similar picture. In order to cope with potential biases, it is important to comprehend the corresponding cognitive processes. As summarized by Lopes (1987). "serious engineering in any domain rests on knowledge of the medium to be engineered" (p. 184). 2. Robustness

Early work on judgmental biases and errors, originated by Kahneman and Tversky (1972, 1973). had the tendency t o generalize empirical findings beyond the particular conditions at which they were observed. Subsequent research (e.g., Bar-Hillel, 1980; Olson, 1976; Kahneman & Tversky, 1982) demonstrated that not all the findings were sufficiently robust to warrant generalization beyond the specific experimental conditions under which they were obtained. A preliminary task in the treatment of cognitive biases, therefore. is to delineate the exact boundaries and conditions under

532

G. Keren

which a particular bias may take place. This, however, may often turn out to be quite difficult, as exemplified in the case of the selection task (e.g., Griggs, 1983). The selection task, first introduced by Wason (1968), has been studied extensively yet its interpretation remains controversial. In its basic form, subjects are presented with four cards on which one letter or number appear (e.g., E, K. 4. 7). They are told that each card has a number on one side and a letter on the other. Their task is to turn over the minimum number of cards necessary to verify the statement (rule) "If a card has a vowel on one side, then it has an even number on the other side". A robust finding. replicated in several studies, is that the most frequent answers are either turning the E and the 4, or the E only. The essential point is that the large majority of subjects do not ask to turn over the 7. thus exhibiting the failure to apply the important step of disconfirming the consequent. Besides demonstrating the difficulties subjects encounter in reasoning about conditionals, the selection task exhibits the bias shared by many people to focus their attention on confirming evidence and overlooking the important role of disconfirming evidence. The failure to search for disconfirming evidence or neglect it, often causes the formation of another bias, namely the overconfidence people have in their fallible judgments, or what has been termed the illusion of validity (Einhorn & Hogarth, 1978). A recent paper by Griggs (1983) offers an insightful analysis of

the selection task. As he noted, a major criticism of the selection task was that it involved a highly abstract and theoretical problem and consequently generalization of the results was questioned. Several studies published during the 1970's have employed realistic and concrete stimuli and reported a much higher level of performance. The implication from this Thematic-material effect5 was to suggest that the earlier results were restricted to the original abstract form, and should not be generalized to alternative (more concrete) formulations. However, Griggs (1983) and Griggs and Cox (1982) pointed out that all the studies supporting the thematicmaterial hypothesis were contaminated with methodological problems. Indeed, subsequent studies that employed thematic material failed to replicate the earlier results (i,e., improvement in 5 . Despite t h e l a c k of an a c c u r a t e d e f i n i t i o n , by thematicm a t e r i a l s it i s meant t h a t t h e s t i m u l i a r e c o n c r e t e , r e a l i s t i c , and not a r b i t r a r y and a b s t r a c t .

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

533

performance due to thematic material). For example, in one of the earlier studies, Johnson-Laird, Legrenzi, and Sonino Legrenzi (1972) instructed their subjects to imagine they were postal workers engaged in sorting letters. They were shown four envelops: A closed one, a n open one, a 5d envelop, and a 4d envelop, and were asked to verify the statement "If a letter is sealed, then it has a 5d stamp on it" by turning over the least number of envelops necessary. Performance observed by Johnson-Laird et al. (1972) was very high (81%) thus supporting the thematic-material effect. I t is important to notice, however, that at the time of the study, there was in fact a n actual British postal rule of this sort: A sealed envelop signified first-class mail which required more postage. Griggs and Cox (1982) administered aversion of the postal problem to American students (who were not familiar with the rule) and completely failed to replicate the Johnson-Laird et al. findings. Consequently, Griggs and Cox (1982) claimed that direct prior experience might be essential to produce facilitation in the postal version, a conclusion that was further supported in a subsequent study by Golding (1981). Griggs and Cox (1982) reported other studies, all of which used thematic material, yet facilitation was observed only when subjects had previous experience with the particular stimuli. The experimental history of the selection task demonstrates the difficulties involved in judging the generality of a potential bias, and the extent to which it can be generalized to the real world, a criterion to which we turn next.

3. Generalization to the real world The question regarding the justification for generalizing results from simple and artificial laboratory experiments to the real world is not new (e.g., Ebbeson & Konecni. 1980; Winkler & Murphy, 1973). Indeed, several authors have questioned the extent to which biases demonstrated in the laboratory are applicable to real life situations. Various different claims have been made in this context. Edwards (1984) has questioned the ecological validity of studies on biases and errors. He maintained that laboratory experiments are unrepresentative of both tasks as well as subjects who might perform these tasks. A different argument was made by Hogarth (198l), who pointed out that most biases have been identified under discrete

534

G. Keren

conditions, whereas judgment and choice in real life situations are usually characterized by a continuous interactive process. Hogarth claimed that several biases that were demonstrated under discrete conditions result from heuristics that are functional in the more natural continuous environment. Funder (1987) proposed a distinction between a n "error". that he defined as a deviation from a formal model of the judgment process and is applicable to the laboratory setting, and a "mistake" which is a n incorrect judgment of a real-world stimulus. According to Funder, not only is it more difficult to determine a mistake because often a normative correct answer is not available b u t in addition, errors "are not necessarily relevant to the content or accuracy of particular judgments, because errors in a laboratory may not be mistakes with respect to a broader, more realistic frame of reference and the processes that produce such errors might lead to correct decisions and adaptive outcomes in real life" (p. 75). Undoubtedly, our knowledge concerning the mechanisms underlying real life decisions and reasoning is still limited. We often lack an appropriate normative yardstick for evaluation purposes, and consequently not all the findings of the artificial laboratory are necessarily applicable to real life. Notwithstanding, generalizing and inferring that biases do not occur under real-life circumstances is an unwarranted conclusion. Some of the judgment and reasoning biases are so deeply rooted that most likely they are present under both laboratory and more realistic circumstances. Recent empirical studies support this assertion. For instance, overconfidence in probability assessments (Lichtenstein, Fischhoff, & Phillips, 1982) is a phenomenon that is not restricted to laboratory conditions. Christensen-Szalanski and Bushyhead ( 198 1) reported physicians' assessments of the probability of pneumonia who exhibited poor calibration and extreme overconfidence, and similar results were reported by Lusted( 1977) with regard to diagnosis of skull fracture and pneumonia. Keren and Wagenaar (1985) studied gambler's behavior in real life casinos and observed several biases that were previously reported in the literature. Northcraft and Neal (1987) studied the anchoring and adjustment heuristic in property pricing decisions. They used both amateur (undergraduate students) and expert (real estate agent) subjects, and showed that both groups employed anchoring-and-adjustment estimation strategies. They concluded that their study provides "strong evidence that previous laboratory research on decisional heuristics and biases is applicable

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

535

to 'real world', information-rich, interactive estimation and decision contexts" (p. 96). There are also studies that have been conducted in a laboratory setting, but used real-life stimuli, for which there is no reason to believe that the results are not applicable to the real world. For example, Keren (1984) used undergraduate students (both in Canada and Israel) and showed that subjects grossly underestimated the exponential growth of inflation rates. There are no a priori good reasons to claim why most people would not exhibit the same bias in similar realistic situations. Finally, the discussion concerning the selection task (in particular, the recent studies that yielded poor performance despite the use of thematic material) offers another example in which the finding of a bias to prefer confirming evidence is most likely not restricted to laboratory conditions. In summary, it is recommended that the applicability of a potential bias to the real world should be tested, and not rejected on a priori grounds. Current evidence suggests that a t least some of the biases that were originally revealed in the laboratory are equally potent in real life. APPLYING COGNITIVE AIDS

Once a bias is identified and established, and the conditions and boundaries under which it may occur have been determined, the application of cognitive aids to reduce or eliminate the bias can be considered. A major obstacle in designing appropriate aids is the lack of a rigorous theory on judgmental heuristics and biases (Wallsten. 1983). implying that our understanding of the underlying mechanisms governing these biases is still limited. Pitz and Sachs (1984) suggested that the development of decision aids requires a n understanding of the process involved in performing the task. Similarly, Fischhoff (1982) encouraged the development of process models that could lead to more powerful aids, and help identify when and why engineering approaches will or will not work. As in medicine, however, the search for a cure cannot wait until full understanding of the phenomenon is reached. Hence, the development of suitable cognitive aids should be pursued along a corresponding attempt to better understand the nature of biases. Fischhoff ( 1982) offered a comprehensive discussion on debiasing methods. He proposed a classification of debiasing methods

536

G. Keren

according to the assumed underlying source of the bias. In particular, he distinguished between biases due to Faulty tasks in which supposedly the source of the bias lies in the nature of the task: 6 biases that result from Faulty judges namely where the source of bias (according to the investigator's interpretation) lies primarily in the judge: and biases that seem to result from a mismatch between judges and tasks. Another taxonomy, based on the nature of the bias and the corresponding judgmental errors, was suggested by Kahneman and Tversky (1982). They proposed separating two types of errors: comprehension errors, in which people fail to understand and recognize the (normatively) appropriate rule indicating lack of competence: and application errors, where it is assumed that comprehension exists and the failure is limited to the particular situation, a performance failure. For a cognitive aid to be effective, it would be important to determine which of these two types of errors it is supposed to correct. As noted however by Kahneman and Tversky (1982). it is often methodologically difficult, if not impossible, to determine the exact nature of the error. A n alternative classification is one that is based on the nature of the cognitive aid. For instance, Arkes, Christensen, Lai, and Blumer (1987)discussed methods for reducing overconfidence in probability assessments and distinguished between what they called direct and indirect approaches. In a direct approach, a person is explicitly presented with the alleged bias and is instructed to make an attempt to avoid it. In contrast, indirect methods are aimed a t reducing the bias in a subtle implicit way, manipulating variables that are supposedly closely related to the bias. Another way to classify different aids is the distinction between what I termed procedural or mechanical m e t h o d s contrasted with structure modifying techniques. Mechanical methods supply the user with certain instructions or algorithms that attempt to manipulate the bias at hand. The instructions may be directly related to the task such as a n explicit warning of the bias, telling the person to avoid certain operations (e.g., don't use too high estimates), or may be indirect such as offering higher 6 . According t o F i s c h h o f f , i n c l u d e d under f a u l t y t a s k s are a l s o t h o s e t h a t a r e m i s u n d e r s t o o d by t h e s u b j e c t ( a n d are t h u s a r t i f a c t s ) o r are i n t e r p r e t e d d i f f e r e n t l y b y s u b j e c t and experimenter. These are excluded from t h e d e f i n i t i o n of biases i n t h e c u r r e n t t e x t .

Cognitive aids a n d debiasing methods

531

incentives (a procedure that usually has little effect), discouraging guessing, or offering an alternative response mode that leads to bias reduction. The common characteristics of such procedural methods is that the user is not necessarily forced to understand the internal structure of the problem and the source of the bias, nor is he or she required to comprehend the logic and design that are underlying the particular aid. In certain respects the user is asked to follow the procedure blindly, in which case the method assures some "technical" correction resulting in improved performance as assessed by some quantitative measures. I t is important to emphasize that focusing entirely on such quantitative measures may be misleading (Lopes, 1987), since such a test does not guarantee qualitative improvement. The question is not just whether the particular bias, as exhibited in a certain specific task, can be reduced in some technical way. The essential issue is to what degree can the assistance and knowledge offered by a cognitive aid (1) be applicable even if environmental conditions have somewhat changed, and (2) be applied beyond the particular situation, and generalized to similar relevant tasks. Structure modifying aids should be constructed such that the person should be able to modify (if necessary) the internal representation of the task or the problem space. Such aids may require more cognitive effort and a deeper understanding, the results of which could be generalized beyond the particular problem or task at hand. In contrast, most procedural methods provide little knowledge, are confined to a certain task and are limited to a relatively fixed set of conditions and environmental constraints. The essential difference between procedural and structure modifjmg aids can further be clarified by referring again to the selection task. As was noted, current evidence suggests that it is not the use of thematic material per se that facilitates performance in the selection task. Rather, it is the presentation environment of the task (and stimuli) that enable subjects to recall past experience with the content of the problem, the rule and relationships expressed, and a counter example to the rule. This explanation is referred to as the memory-cuing or reasoning by analogy hypothesis (Griggs, 1983). The importance of this hypothesis is that it "does not propose that 'logical' reasoning is facilitated. It only proposes that cued relevant experience leads the subject to make a correct selection. In fact, all the available experimental evidence indicates that logical reasoning is not facilitated. A good criterion for the

538

G. Keren

presence of logical reasoning is positive transfer to the basic-form abstract version of the problem following a thematic version. There is absolutely no evidence for such transfer" (Griggs, 1983. p. 28). Using the terminology of the present paper, the use of thematic material provides procedural aid that is limited to a particular environment and set of stimuli. Since the subjects have been exposed to the procedural solution before, they are able to apply it by analogy, but current evidence suggests that fundamental structural properties (e.g., preference for confirming evidence) remain unchanged. An essential aspect of problem solving behavior is the cognitive representation of the task that has been termed by Newel1 and Simon (1972) the "problem space". Similarly, with regard to decision making processes, Einhorn and Hogarth (1981) concluded that "it is now clear that the process of representation, and the factors that affect it, are of major importance in judgment and choice" (p. 57). Structure modifying methods (similar to what Fischhoff (1982) refers to as restructuring methods) attempt to make knowledge explicit, and guide the person to a n appropriate representation of the relevant task and problem space. As such, these methods focus more on qualitative aspects of the process. rather than solely concentrating on the end result. Such methods require more effort from both the designer of an aid as well as the user, and consequently are rather rare. The main benefit of such methods is in the deeper understanding they achieve, in their generality beyond the specific condition, and in the flexibility they afford in accommodating to modified conditions and changing environments. Earlier, I claimed that mechanical methods require blind adherence to a procedure or algorithm prescribed by the aid. The word blind has not been used incidentally. In his seminal work on productive thinking, Wertheimer ( 1959) describes the manner by which children solve simple geometrical problems, such as calculating the area of a parallelogram. Wertheimer separates what he called blind solutions and genuine constructive solutions. Some of the children he observed followed blindly the procedure they were taught by their teacher, and were successful in computing the area of a parallelogram as long as it was presented in the same way, namely in the same orientation. Once the parallelogram was rotated such that its base was the short side, they continue to apply the same procedure they memorized by rote learning and fail. In

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

53 9

c o n t r a s t , o t h e r children who d e m o n s t r a t e d s t r u c t u r a l understanding of the problem were quick in modifying the procedure and correctly applied it to obtain the correct answer. It should be emphasized that the distinction between procedural and structure modifying aids is not dichotomous, and should be viewed as two extremes on a continuum. Current debiasing techniques, however, can be characterized as lying closer to the procedural end of the scale, which probably explains why many of them have failed or at best have limited success (Fischhoff, 1982). EVALUATING THE EFFECTIVENESS OF COGNITIVE AIDS A major obstacle, often encountered in the evaluation of cognitive aids, is the lack of a n acceptable criterion (Pitz & Sachs, 1984). A central question in decision theory is what constitutes a good decision and how could it be measured (e.g. Vlek. Edwards, Kiss, Majone, & Toda, 1984). In particular, the question has been raised whether the quality of decisions should be assessed by the processes in which they were reached, or by the corresponding outcomes (Keren, 1986). A similar question can be posed concerning the evaluation of cognitive aids: Should they be appraised according to process or outcome? Most researchers suggest, at least implicitly, that evaluation of a decision should be based on the inferred process by which it was made. For instance, Simon (1978) claimed that "economics h a s largely been preoccupied with the results of rational choice rather than the process of choice" (p. 2). There are several reasons for avoiding the judgment of decisions by outcomes: First, evaluating by outcome presupposes the existence of an objective and unambiguous criterion of evaluation that often does not exist. In addition, given that all real decisions are made under uncertainty, a good decision cannot guarantee a good outcome (Edwards, in Vlek et al., 1984). Consequently, inferences based on outcomes are vulnerable to two types of potential errors: A favorable outcome can occur by chance which does not necessarily imply that the preceding decision process was indeed the appropriate one, and similarly a n unfavorable outcome cannot be necessarily interpreted as evidence for an inappropriate decision process.

540

G. Keren

Evaluating a decision by the underlying process is, unfortunately, also not problem free. In particular, there are no common standards and criteria for a decision process, and such an analysis can finally be based only on clinical judgment (Fischoff, 1983). In order to improve the quality of such clinical judgments and increase their validity, we need a better understanding of the cognitive processes underlying existing biases. The design of debiasing methods as well as testing and evaluating those methods, depends in the first place on one's theory of why and how those biases occur. I elaborated on the differences between judgment by process and judgment by outcome not only because of its direct relevance to the evaluation of cognitive aids, but also because it is intimately related to the distinction made earlier between procedural and structure modifying aids. Whereas procedural methods focus on the outcome (and supposedly use it as the main evaluation criterion), the latter methods are primarily concerned with the process itself. As such, structure modifylng aids have to be evaluated not just by outcomes, but also by some qualitative assessments that admittedly may often be clinical and subjective in nature. APPLICATIONS: ANALYSIS OF OVERCONFIDENCE

In the first part of the present paper I proposed a framework for assessing possible biases, and evaluating the efficiency of potential aids (or debiasing methods) to reduce or eliminate them. In the present section this framework is applied to analyze a particular bias, namely the bias of overconfidence in the context of calibration studies. Calibration studies are concerned with the appropriateness of assessors' subjective probability estimates, or confidence in their judgments. For discrete probability statements, judgments are well calibrated "if on the long run, for all propositions assigned a given probability, the proportion that is true is equal to the probability assigned" (Lichtenstein. Fischhoff. & Phillips, 1982). A common way of analyzing confidence ratings or probability assessments of discrete propositions is via the use of a calibration curve (see for example Figure 1) in which the hit rate (percentage

541

Cognitive a i d s and d e b i a s i n g methods

correct) is plotted for each confidence r e ~ p o n s e . The ~ 45" line represents perfect calibration. Any point below this line reflects overconfidence, whereas any point above it represents underconfidence. Under- or overconfidence can be further assessed by the difference between the mean of the probability responses and the corresponding overall proportion correct for each category (Lichtenstein & Fischhoff, 1977), or: 1 / N X nt(rt - ct) 11) where stands for the total number of responses, f is the number of response categories, gt is the number of times the response xt has been used, and ct the corresponding proportion of correct items. Any positive result of the above measure indicates overconfidence (the larger the number, the higher t h e overconfidence), and a negative score implies underconfidence.

90

-

.......0

Group 1 Group 2

w

0

800 0

Q)

I ?

!!

70-

a

I

40

50

60

.

I

70

.

I

80

.

1

90

.

I

-

100

confidence ratings

Figure 1. Two calibration curves adopted from Keren 1985. 7. It is customary t o group all responses in the range .50.59, .60-.69, . . . , . 9 0 - . 9 9 , and 1 . 0 0 . The mean percentage correct for each response group is then plotted against the corresponding mean probability assessment in that category.

542

G . Keren

Calibration regarding uncertain continuous quantities is often measured by the interquartile index, and the surprise index. The former is the percentage of items for which the true value (actual outcome) falls inside the interquartile range (i.e., between the -25 and the .75 fractiles), and perfect calibration is indicated by a n index of 50. Any value lower than 50 would imply overconfidence, and values above 50 are interpreted as underconfidence. The surprise index represents the percentage of true values falling outside the most extreme fractiles assessed. For instance, a surprise index of 2% refers to the extreme values that fall below .01 and above the .99 fractiles. Ideal calibration would lead to a surprise index of 2, and any value above it would represent overconfidence. A pervasive and robust finding in the calibration literature is a strong tendency of overconfidence (Lichtenstein, Fischhoff, & Phillips, 1982). With few exceptions (e.g., Keren, 1987; Murphy & Winkler, 1977), most studies report a bias of overconfidence that becomes larger as the task becomes more difficult. For example, Figure 1 depicts two calibration curves showing a strong bias of overconfidence. The curves are based on a general knowledge task (using 50 discrete binary questions with two options) administered to 70 (group 1) and 74 (group 2) subjects, respectively. The overall percent of correct responses was approximately 66% for each of the two groups, and the corresponding overconfidence as measured by (1) was .0591 and .0134 for group 1 and 2 respectively (these results are further discussed below). Since overconfidence has been reported by several researchers, using different tasks and different subjects, there is little doubt about the robustness of the phenomenon. However, in accordance with the discussion above, three questions should still be addressed: Can we be certain that this bias is not just an artifact? Is the phenomenon applicable to real life situations? And, if the answer to the first two questions is positive, what are the possible mechanisms that lead to the bias? These three issues are discussed in the following paragraphs. 1. Is the bias real? Concerning the first question, there are several points to be made. First, all calibration curves reported in the literature, even those that exhibit strong overconfidence, are monotonically increasing functions.8 This suggests that subjects are 8 . T h e r e a r e few e x c e p t i o n s i n which t h i s m o n o t o n i c i t y i s v i o l a t e d a t a c e r t a i n point on t h e curve, but t h i s i s u s u a l l y a c c o u n t e d f o r by t h e s m a l l number of o b s e r v a t i o n s

C o g n i t i v e a i d s and d e b i a s i n g methods

543

able (at least) to discriminate between more or less likely events, b u t not on a n absolute scale. In other words, much better calibration could be achieved if subjects' responses were rescaled. While this argument is certainly not without merits, two reservations can be made. One is that as the difficulty of the task (as measured by percent of correct predictions or responses) increases, overconfidence increases and the calibration curve gets flatter (Lichtenstein, Fischhoff, & Phillips, 1982) which, according to the above argument, would imply t h a t different scales are used depending on the difficulty of the task. Second, the 100% confidence may serve as a "natural" anchor, since it clearly implies absolutely no errors. The fact that the large majority of studies show that subjects use this confidence rating quite frequently yet with accuracy that falls much below 1.0 suggests that overconfidence is not likely to be a simple artifact. Another issue concerns the fact that most calibration curves and calibration measures are usually applied to group data, since with few exceptions (i.e., weather forecasters) it is extremely difficult to obtain a large number of observations from one individual. Indeed, informal analyses t h a t were made on several of my calibration studies suggest large individual differences. This is in particular important when considering cognitive aids, since a given debiasing method may have different effects on different individuals. The most important problem concerning the interpretation of calibration studies (and the overconfidence phenomenon) is related to the loss function assumed by the subjects and the use of scoring rules. Since subjective probability assessments exist solely in the assessor's minds (Murphy & Winkler, 1970), there is no way to determine whether they agree with the reported probabilities. For example, in certain contexts, the assessor may believe that deviating from the true judgments may be advantageous: or, the assessor may not be sufficiently motivated to make the effort for producing his or her best assessments (that may often be the case in laboratory experiments). A scoring rule provides the assessor with a 'payoff that depends on the stated probability assessments and the corresponding outcomes. A proper scoring rule is one which contains a reward (or a penalty) function such that the objectively expected score is maximized if and only if the "true" judgment is

a t t h i s p o i n t , which make t h e e s t i m a t e e x t r e m e l y unreliable.

544

G.

Keren

reported. Hence, a proper scoring rule encourages the assessor to be honest. Despite some problems involved in the construction and use of proper scoring rules,' it is difficult, if not impossible, to interpret and evaluate probability assessments without knowing the underlying scoring rule that has been used by the assessor. Unfortunately, most calibration studies (in particular those conducted in the artificial laboratory) fail to specify a n explicit scoring rule. Consequently, each assessor is implicitly forced to construct and use his or her own rule. Not only is this rule unknown to the experimenter, b u t it is also most unlikely that all assessors will arrive at the same scoring rule. The failure to employ explicitly a proper scoring rule has several consequences. For instance, most of the studies using discrete propositions employ for evaluation purposes the so-called Brier score (Brier, 1950; Lichtenstein & Fischhoff, 1977, 19801, which is a quadratic proper scoring rule. However, if the assessor is not aware that he or she will be evaluated by the Brier scoring rule, there are obviously many other possible scoring rules that he or she may employ. In fact, there is the possibility that a subject follows properly a certain scoring rule and does not show a n y overconfidence, whereas the same data analyzed by the traditional way and using the Brier score will indicate poor calibration and a strong bias of overconfidence. Finally, and perhaps most important for the present context, it is difficult if not impossible to construct efficient and successful debiasing methods without the explicit use of a scoring rule. Given that different assessors implicitly use different scoring rules, the effect of a certain debiasing technique may differentially affect different assessors. A more general issue concerns the method by which a bias is measured. Often, disagreement may exist as to how the bias should be measured (e.g., Cohen, 1981). Obviously, the design and construction of debiasing techniques are largely dependent on the way in which a n alleged bias is measured. 9 . S c o r i n g r u l e s assume t h e e x i s t e n c e o f one " t r u e " u n d e r l y i n g p r o b a b i l i t y d i s t r i b u t i o n i n t h e a s s e s s o r u s mind, an assumption t h a t i s q u e s t i o n a b l e (Hogarth, 1 9 7 5 ) . A l s o , a s c o r i n g r u l e c a n be e f f e c t i v e only t o t h e e x t e n t t h a t it is w e l l understood, and t h a t t h e a s s e s s o r s are c o n s i s t e n t l y f o l l o w i n g t h i s r u l e . T h e r e is u n f o r t u n a t e l y n o e m p i r i c a l way t o v a l i d a t e w h e t h e r a s s e s s o r s i n d e e d employ ( a n d c o r r e c t l y ) a given scoring r u l e .

C o g n i t i v e a i d s and d e b i a s i n g methods

545

2. Is the bias applicable to the real world? Despite the above comments, there is nevertheless sufficient evidence to indicate that the overconfidence phenomenon is probably not a n artifact. Most of this evidence, however, comes from the artificial laboratory and the question still remains as to the extent to which this bias is also apparent in the real world. The answer is not unequivocal: Some studies reported overconfidence, especially for different types of diagnosis in the medical field (e.g. Christensen-Szalanski, 198 1: Lusted, 1977). In contrast, several studies in other fields resulted in good calibration showing little if any overconfidence (occasionally even some underconfidence). This was true for weather forecasters (Murphy & Winkler, 1977), accountants (Tomassini, Slomon, Romney, & Krogstad. 1982). students predicting their course grade (Sieber. 1974), and professional bridge players (Keren. 1987). The picture that emerges from studies that are closer to real life is thus mixed: Some studies produce the familiar overconfidence while many others do not. A plausible explanation could be that the phenomenon of overconfidence is task dependent. For instance, Lichtenstein et al. (1982) suggest that the excellent calibration exhibited by meteorologists is d u e to several favorable characteristics of the task: long experience with probabilistic forecasting: repetitiveness (the question is always the same, e.g., Will it rain?); feedback is accurate and promptly received. In contrast, there are several factors that hinder physicians from being well calibrated: they have no experience with probabilistic assessments: there may be a n implicit expectation from patients that physicians should be confident in their judgment: they may overestimate the likelihood of a malady because of malpractice worries (Fischhoff, 1982); and finally, feedback is frequently delayed (or not received at all), and its accuracy may often be questionable. If the quality of calibration is task dependent, it would suggest that a debiasing method should be geared for a specific task. Constructing appropriate aids for improving calibration should therefore depend on the particular characteristics of the task a t hand. Whether a general debiasing technique exists, that would lead to improved calibration independent of the specific task, is highly questionable. 3. What are Dossible mechanisms that lead to the bias? To design efficient debiasing aids, it is necessary to obtain knowledge about the possible underlying mechanisms that may lead to the bias. This applies especially for the construction of what was termed in

54 6

G . Keren

this paper as structure modifying aids. What are possible cognitive accounts for the overconfidence bias? Dawes (1980) noted that evidence for poor calibration and overconfidence has been mainly obtained from tasks that require judgments about intellectual knowledge. He claimed that we grossly overestimate our intellectual capacities and mentioned several reasons that may account for it. Pitz (1974) proposed that estimation of uncertainty may often require several steps that are performed serially. The final probability estimate should be a function of the uncertainties associated with each step, but due to limited processing and working memory capacity people often ignore the uncertainties at the initial stages of the process. Pitz also proposed that the degree of uncertainty would be a function of the different possible solutions or answers that one can produce, and the broader the base of relevant knowledge, and that the larger the set of possible alternative structures, the smaller should be the tendency for overconfidence. Several recent studies (Allwood & Montgomery, 1987; Keren, 1987; May, 1986) suggest that when confronted with uncertainty and lack of knowledge, subjects will often rely on inferential strategies to fill in missing information. Many subjects, however, fail to realize that their inferences are probabilistic and error prone and fail to discount their confidence ratings accordingly. This explanation is congruent with Pitz's account, indicating more specifically the stage a t which overconfidence may start. Finally, a most viable account of overconfidence may be in terms of scoring rules. As noted above, most calibration studies have failed to u s e a n explicit scoring rule. Under s u c h circumstances subjects have to follow their own scoring rule, and it is not inconceivable that many have chosen a rule that yields overconfidence when assessed, for instance, by the Brier score. Unfortunately, this explanation has not yet been tested empirically. This last hypothesis is of special importance in the present context, since biases other then overconfidence can occur due to the lack of a common scoring rule, or more generally a loss function, that are shared by both subjects and experimenter.

CAN OVERCONFIDENCE BE REDUCED? In light of the above analysis we now examine some possible aids that were proposed for reducing or eliminating overconfidence.

Cognitive a i d s and d e b i a s i n g methods

541

Several attempts, using different methods, have been made to reduce overconfidence and are summarized by Fischhoff (1982). The major conclusion from this review is that overconfidence is a rather robust phenomenon, and that current debiasing methods are at best moderately successful in reducing overconfidence. The purpose here is not to present an updated exhaustive literature review (Fischhoff covered the literature only up to 1982). but rather to analyze selectively some examples in order to highlight the distinction between procedural and structure modifylng techniques. For a typical example of a procedural method consider again Figure 1 which is taken from Keren (1985). Both group 1 and 2 received an identical set of 50 general knowledge questions and, as mentioned, their performance (in terms of percentage correct) was almost identical. The only difference between the two groups was that group 2 was explicitly instructed not to use frequently confidence ratings of 100% and limit them only to those cases "that you are absolutelv sure and that you do not have the slightest doubt". Since part of the overconfidence phenomenon is due to the it was relatively large proportion of 100% confidence ratings, anticipated that the additional instructions would profoundly reduce the amount of overconfidence. The experimental manipulation was effective: Confidence ratings of 100% accounted for 9.1% in group 2 compared with 15.7% for group 1, a difference that was statistically highly significant (p c .005). As a consequence, there was also a significant reduction in overconfidence from .0591 in group 1 to .0134 in group 2. Inspection of Figure 1, however, shows that the pattern of the two calibration curves is very similar. The only reason for the large reduction in overconfidence in group 2 is due to a reduction in the frequency of 100% confidence ratings, which consequently receives a lower weight for this group. In other words, subjects in group 2 are not more accurate -- they only 1 0 . T h e r e were s e v e r a l r e a s o n s t o t r y a n d m a n i p u l a t e t h e 1 0 0 % c o n f i d e n c e r a t i n g s : F i r s t , b e c a u s e many s t u d i e s h a v e shown t h a t s u b j e c t s u s e t h i s r a t i n g w i t h a h i g h f r e q u e n c y even though t h e corresponding accuracy i s f a r from b e i n g p e r f e c t . Second, t h e l a r g e r t h e d e v i a t i o n between t h e o b s e r v e d p o i n t on t h e c a l i b r a t i o n c u r v e a n d t h e c o r r e s p o n d i n g p o i n t on t h e 45' l i n e , t h e l a r g e r t h e c o n t r i b u t i o n t o overconfidence as measured by ( 1 ) . Consequently, a r e d u c t i o n i n t h e r e l a t i v e frequency of 100% r a t i n g s should have a profound e f f e c t i n reducing overconfidence.

548

G.

Xeren

reduced (artificially) the number of responses of 100% which are a major source for overconfidence. The procedural instructions result in a lower measure of overconfidence, yet there is no justification to claim that subjects in group 2 are now better assessors (and are less biased) than subjects in group 1. A s another example for a procedural method, consider the debiasing method proposed by Arkes et aZ. (1987). In their study, one group of subjects received several practice questions that appeared to be easy but were in fact quite difficu1t.l' and then received feedback on the accuracy of their answers prior to the experimental test. These subjects were less confident in the experimental session (in fact they exhibited underconfidence) than several other control groups. The group exhibiting the highest level of overconfidence received the apparently easy (but misleading) items, but did not receive any feedback. There are several problems with this type of procedural debiasing: First, the manipulation of the practice trials apparently leads to the establishment of a certain anchor, which subjects continue to adopt during the experimental session. In fact, that pattern is just a manifestation of yet another bias, namely that of "adjustment and anchoring" (Kahneman & Tversky, 1973). Second, under certain conditions (including to a certain extent the study by Arkes et aZ.) one may get a n opposite bias, namely underconfidence. Third, the applicability of the method is restricted, since under real life conditions it may be impossible to create a practice session with so-called misleading items. In short, the method does not lead to any qualitative changes, and is not generalizable to other conditions. A method that resembles a structure modifying technique has been proposed by Koriat, Lichtenstein, and Fischhoff (1980). They hypothesized that overconfidence stems from an attempt to justify one's chosen answer, that leads to a biased search of confirming evidence, and disregard for negative and contradicting evidence (see also Einhorn & Hogarth, 1978). Consequently, Koriat et al. forced their subjects to list reasons for and aPainst their chosen answer, before they made a probability assessment. This procedure resulted in a marked improvement in the appropriateness of confidence judgments. Although subjects were not told explicitly about the overconfidence bias, the above procedure supposedly 11. I r z f e r t o such items a s " m i s l e a d i n g " items (Keren, 1987).

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

549

changed the pattern of search for information and eventually the structure by which they assessed probabilities. SOME FINAL COMMENTS

Judgment and reasoning errors can reflect either failure in performance or in competence. The former are incidental in nature and are accounted for by factors such as lack of attention, fatigue, stress, etc. The latter are more fundamental and systematic and reflect a deficiency at a much deeper level. A somewhat similar distinction has been proposed in this paper regarding debiasing methods and cognitive aids: procedural methods affect the surface level of processing, whereas structure modifying techniques are supposed to alter a deeper and more fundamental level of judgment and reasoning processes. Indeed, the construction of effective structure modifying techniques requires a n understanding of the underlying cognitive mechanisms and a comprehensive theory. Currently, such a theoretical framework is often missing, which explains why modifymg structure methods are relatively rare. In the introduction, the analogy was made between medical treatments and cognitive aids. Frequently, the use of some medical treatments may reduce the surface symptoms yet have little effect in curing the actual disease. Similarly, the claim was made that procedural aids may affect some symptoms as measured by certain external quantitative measures, but they do not alter fundamental erroneous reasoning and judgment processes. We certainly need a better theoretical understanding of the mechanisms underlying cognitive biases. Until then, we may often have to rely on procedural and mechanical aids provided that they do not lead to unwanted side effects.

REFERENCES Allwood. C.M.. & Montgomery, H. (1987).Response selection strategies and realism of confidence judgments. Qreanizational Behavior and Human Decision RocessB 365-383. Arkes. H.R.. Christensen, C.. h i . C.. & Blumer, C. (1987). l k o methods of reducing overconfidence. Oraanizational Behavior and Human Decision Processes 3% 133-144. Bar-Hillel M. (1980).The base-rate fallacy in probability judgments. A d Psvcholoaica. 44,21 1-233. Baron, J. (1988). Trmktne and deciding. Cambridge: Cambridge University Press. Berkeley, D.. & Humphreys, P. (1982).Structuring decision problems and the 'bias heuristic'. Acta Psvcholoei@, 201-252.

,a,

7

a.

G. Keren

5 50

Brunswik. E. (1952).The conceptual framework of psychology. In J n t e r n a U u m1 E;ncvcloDe& of Unified Science, (Vol. 1. no. 10).Chicago: Chicago University Press. Christensen-Szalanski. J.J.J.. & Bushyhead. J.B. (1981).Physicians’ use of probabilistic information in a real clinical setting. Journal of ExDerimental Psvcholom: Human PerceDtion and Performance. z, 928-935. Cohen. L.J. (1981).Can human irrationality be experimentally demonstrated? Behavioral and Brain S c i a, A . 317-331. le and the Provable. Oxford: Oxford University Press. Cohen. L.J. (1977). Dawes. R.M. (1980).Confidence in intellectual judgments vs. confidence in perceptual judgments. In E.D. Lantermann & H. Feger (Eds.). Similaritv and Choice: PaDers in honor of Chrde CoombS. Bern: Hans Huber. Diaconis. P.. & Freedman, D. (1981).The persistence of cognitive illusions. Behavioral and Brain SciencQ, 4. 533-534. Ebbeson, E.B., & Konecni. V.J. (1980).On the external validity of decision making research: What do we know about decisions in the real world. In S . Wallsten (Ed.). Cognitive Processes in Choice and Decision Behavior. Hillsdale. N J : Erlbaum. Edwards, W.(1984).Human cognitive capabilities, representativeness, and ground rules for research. In P. Humphreys, 0. Svenson, & A. Vari (Eds.),Malvzing and Aidin6 decision Drocesm. Budapest: Hungarian Academy of Sciences. Einhorn. H., & Hogarth. R.M. (1978).Confidence in Judgment: Persistence in the illusion of validity. Psvchological Review, 5. 395-416. Einhorn. H.. & Hogarth. R.M. (1981).Behavioral decision theory: Processes of judgment and choice. m u a l Review of Psvcholocy. 53-88. Evans, J.St.B.T.. & Dosoir. A.E. (1977). Proportionality and sample size a s factors in intuitive statistical judgments. Acta Psvch o l o a i ~ , 129-137. Fischhoff. B. (1982).Debiasing. In D. Kahneman. P. Slovic, & A. n e r s k y (Eds.), Judgment Under Uncertaintv: Heuristics and Biases. Cambridge: Cambridge University Press. Fischhoff. B. (1983).Decision analysis: Clinical art or clinical science? In L. Sjoberg. T. Tyszka,& J. Wise (Eds.).Human Decision Making. Bodafors: Doxa. Fischhoff, B., Slovic, P.. & Lichtenstein, S. (1979).Subjective sensitivity analysis. Qraanizattonal Behavior and Human Performance, 1919. 339-359. Funder. D. (1987). Errors and mistakes: Evaluating the accuracy of social judgments. Psvcholocical Bulletin. 101.75-90. Goldlng. E. (1981). m e effect of uast ernerience on uroblem solving. Paper presented at the annual conference of the British Psychological Society, Surrey University. April. Griggs. R.A (1983).The role of problem content in the selection task and THOC problem. In J 3 . B . T . Evans (Ed.),Thinking and Reasoning: Psvchological Amroaches. London: Routledge and Kegan Paul. Griggs, R.A.. & Cox, J.R. (1982). The ellusive thematic-materials effect in Wason selection task. British Journal of Psvcholocy.3. 407-420. Cigerenzer. G..Swijtink, 2.. Porter, T., Daston. L.. Beatty, J.. & Kruger. L. (1989). EmDire of Chance. Cambridge: Cambridge University Press. Henle, M. (1962). On the relation between logic and thinking. Psvchological Review. @, 366-378. Hogarth. R. (1980). Judaement and Choice. New York: Wiley. Hogarth. R. (1981). Beyond discrete biases: Functional and disfunctional aspects of judgmental heuristics. Psvchological Bulletin, 197-217. Johnson-Laird, P.N.. Legremi, P.. & Sonino Legremi, M. (1972).Reasoning and a sense of reality. British Journal of Psvcholom, 63.395-400. Kahneman. D.. & Tversky. A. (1972).Subjective probability: A judgment of representativeness. aenitive Psvcho1ocy.Q. 430-454.

a, a.

a,

a,

551

C o g n i t i v e a i d s and d e b i a s i n g m e t h o d s

Kahneman. D.. & Tversky, A. (1973). On the Psychology of prediction. psvcholoaical Review. 8Q,237-251. Kahneman. D., & Tversky. A. (1982). On the study of statistical intuitions. Coanition. 11.123-141. Keren. G. (1983). Cultural Differences in the misperception of exponential growth. PerceDtion and Psvchoohvsics. 34,289-293. Keren, G. (1984). On the importance of identifying the correct 'problem space'. Copnition. 18. 12 1-128. Keren, G. (1985).On the calibration of emerts and Iav-DeoDlC. Paper presented at the 10th conference on subjective probability. utility, and decision making. Helsinki, Finland. Keren. G. (1986).On the judgment and measurement of good decisions. Technical reDort IZF-TNQ. 1986. Keren. G. (1987). Facing uncertainty in the game of bridge: A calibration study. 98-114. a c a n i z a tional Behavior and Human Decision Procesm , Keren, G., & Wagenaar, W.A. (1985). On the psychology of playing Blackjack: Normative and descriptive considerations with implications for decision theory. Journal of ExDerimental Psvchologv: General. 114.133-158. Koriat, A.. Lichtenstein. S . . & Fischhoff, B. (1980).Reasons for confidence. Journal of ExDerimental Psvcholom: Human Learning and Memory, 2.107- 118. Kruglanski. A.W., & Ajzen. I. (1983). Bias and error in human judgment. FuroDean Journal of Social PsvcholQgy,*, 503-518. Lichtenstein. S . . & Fischhoff, B. (1977). Do those who know more also know more about how much they know? Organizational Behavior and H u m a n Performance. 159-183. Lichtenstein, S.,& Fischhoff. B. (1980). Training for calibration. Qraanizational Behavior and Human Performance. 252. 159-183. Lichtenstein. S..Fischhoff. B.. & Phillips. L.D. (1982). Calibration of probabilities: The state of the art to 1980. In D. Kahneman, P. Slovic, & A. Tversky (Eds.). Judgment Under Uncertaintv: Heuristics and Biases. Hillsdale. N J : Lawrence Erlbaum. Lopes, L. (1987). Procedural debiasing. Acts Psvcholoaica, 167-185. Margalit, A,, & Bar-Hillel, M. (1981). The irrational, the unreasonable, and the wrong. The Behavioral and Brain Sciences, 3. 346-348. May, R.S. (1986).Inferences, subjective probability and frequency of correct answers: A cognitive approach to the overconfidence phenomenon. In B. Brehmer, H. Jungerman. P. Lourens, & G. Seron (Eds.), pew Directions in Research on Decision Making. Amsterdam: North-Holland. Murphy, A.H.. & Winkler. R.L. (1970). Scoring rules in probability assessments and evaluation. Acta Psychologim, 273-286. Murphy, A.H., & Winkler. R.L. (1977). Can weather forecasters formulate reliable probability forecasts of precipitation and temperature? National Weather 2,2-9. Newell. A.. & Simon, H.A. (1972). Human Problem Solving. Englewood Cliffs. N J : Prentice-Hall. Northcraft. G.B.. & Neale. M.A. (1987). Experts, amateurs, and real estate: An anchoring-and-adjustment perspective on property pricing decisions. Orpanizational Behavior and Human Decision Processes. 39.84-97. Olson, C.L. (1976). Some apparent violations of the representativeness heuristic in human judgment. Journal of ExDerimental Psvcholoav: Human PerceDtion and Performan%, 2.599-608. Pitz. G. (1974). Subjective probability distributions for imperfectly known quantities. In L.W. Gregg (Ed.),Knowledge and Cognition. New York Wiley. Pitz. G., & Sachs. N.J. (1984). Judgment and Decision: Theory and Application. Annual Review of Psvchology, 139-163.

a.

a,

a.

a,

a,

-.

a,

G . Keren

552

Revlin, R., & Leirer, V.O. (1978).The effect of personal biases on syllogistic reasoning: rational decisions from personalized representations. In R. Revlin & R E . Mayer (Eds.),Human Reasoning. New York: Wiley. Sieber. J.E. Effects of decision importance on ability to generate warranted subjective uncertainty. Journal of Personalitv and Social Psvcholoeu, 30, 688-694.

Simon, H.A. (1957). Models of Man: Social and R a t i o d . New York: Wiley. Simon. H.A. (1978). Rationality a s process and product of thought. American Cconomic RevieW. 68. 1-16. Slovic. P. (1972). From Shakespeare to Simon: Speculations - and some evidence about man's ability to process information. ORI research monoeraDh. s ( 2 ) . Eugene: Oregon Research Institute. Slovic. P.. Fischhoff. B.. & Lichtenstein. S. (1982). Response mode, framing, and information processing effects in risk assessment. In R M . Hogarth (Ed.).N w pfrections for Methodolorn of Social and Behavioral Science: The framine of guestions and the consistency of resDonse. San Francisco: Jossy-Bass. Tomassini, L.A., Solomon, I., Romney. M.B., & Krogstad, J.L. (1982). Calibration of auditors' probabilistic judgments: Some empirical evidence. Orr~anizational Behavior and Human Performance. 391-406. Tversky. A.. & Kahneman. D. (1981).The framing of decisions and the psychology of choice. Science. u. 453-458. Tversky. A . & Kahneman, D. (1986). Rational choice and the framing- of decisions. The Journal of Business, 59.251-284. Wallsten. T.S. [1983).The theoretical status of iudmental heuristics. In RW. Scholz (Ed.), Decision Making Under Uncerta inty.Tkmsterdam: Elsevier. Wason, P.C. (1968). Reasoning about a rule. Quarterlv Journal of ExDerimental 273-281. Psvcholw. Wertheimer, M. (1959). Productive Thinking. New York: Harper and Row. Winkler, R.L., & Murphy, A.H. (1973). Experiments in the laboratory and the real world. Oreanizational Behavior and Human Performance, J.Q. 252-270. Vlek, C.. Edwards, W.. Kiss, I., Majone. G., & Toda. M. (1984).What constitutes a good decision? Acta Psvcholoeic~,56, 5-27.

a,

m,

Cognitive Biases J.-P. Caverni, J.-M. F a b r e and M . Gonzalez ( E d s . ) Q E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

CONFIRMATION BIAS, PROBLEM-SOLVING AND COGNITIVE MODELS David W.Green Department o f Psychology,

U n i v e r s i t y C o l l e g e London, UK

Abstract. In a study exploring the nature of rule-discovery in a conceptual task, it is shown that confirmation bias is eliminated when individuals build a model of the problem through a process of questioning. Such models support the process of discovery in a number of ways without altering the nature and order of the hypotheses being tested.

INTRODUCTION

We construct our social and physical worlds and use information and opinion to revise our hypotheses about them. A task devised by Wason (19601 provides a window on such processes. Individuals were required to discover a rule relating three numbers (2-4-6). On each trial a n individual proposed three numbers and was told whether they conformed to the rule that the experimenter had in mind. Subjects tended to form a hypothesis based on the initial exemplar (e.g. 'increasing intervals of two') and would then test this hypothesis with a triple (8-10-12) that was a positive instance of it. Further examples (14- 16-18) would reinforce their commitment to this belief and they would announce the rule 'increasing intervals of two'. In Wason's experiment the rule was 'ascending numbers' hence the announced rule was incorrect. Subjects showed a bias towards confirmation that led them to propose rules that were merely sufficient, rather than ones that were necessary and sufficient. Of course, a number of subjects did announce the rule correctly on first announcement and did eliminate merely sufficient hypotheses. Other phenomena were also observed. For instance, some individuals after being told that a n announced rule was incorrect, would continue to generate triples consistent with it and then announce a reformulation of the same hypothesis. Similar phenomena were observed by Mynatt, Doherty, and Tweney (1978) in a n environment modelled after a real research setting. Moreover, the findings seem generalisable to

554

D.W. Green

different types of rules (see Klayman & Ha, 1985). Recent work has sought to explore factors that might improve testing. Neither the requirement that subjects entertain two hypotheses on successive tests, nor specific instructions to disconfirm exerted any effects on subjects' ability to discover the rule. However, when required to discover two complementary rules, 60% of the subjects correctly identified the ascending numbers rule on first announcement (Tweney, Doherty, Worner. Pliske, & Mynatt, 1980; see also Gorman, Stafford, & Gorman, 1987; and Green, 1988 for a n overview). Such findings suggest the importance of the mental organisation or model of the problem. Organised representations have been shown to be crucial in facilitating solutions to deductive inference problems such as the selection task or variants of it (see, for example, Wason & Green, 1984: Cheng & Holyoak. 1985: Girotto, Light, & Colbourn. 1988). More generally, Johnson-Laird (1983) has cogently demonstrated the utility of the notion of a mental model for accounts of the functioning of mind. One of the aims of the present study therefore was to explore the role of cognitive models in the rule-discovery task. An appropriate model in the present case would involve a structured set of hypotheses (i.e. the explicit representation of two or more hypotheses and their interconnection). Of course, a model of a problem is ineffective without a procedure for searching the set of hypotheses and a means to interpret the results of those searches. The model condition of the present experiment aims to help individuals construct an appropriate representation and devise procedures for testing it by asking them a series of questions to which they write down replies. I t was supposed that these models and testing procedures would provide a key input to the discovery process by, for example, providing the rudiments of a problemsolving plan. The model building phase itself preceded the standard 2-4-6 task. I t was predicted that more subjects in the model condition would announce rules correctly the first time, compared to subjects participating only in the standard condition. The number of trials needed for individuals to converge on the correct rule is also likely to be a function of their testing procedure. Accordingly, an attempt was made to characterise a subject's testing procedure and to relate it to task performance. A final aim was to explore how subjects construed the problem and moved from hypothesis to hypothesis.

Confirmation bias

555

METHOD Subjects and Design The subjects were 16 undergraduate psychology students of University College London (6 men, 1 0 women) with no previous experience of the task. They were unpaid volunteers. Eight were assigned either to the standard or to the model condition. Procedure Subjects were tested individually. In the standard condition they were instructed as per Wason (1960). On each trial, individuals wrote down on a record sheet, three numbers and their reasons for choosing them. They were then told, and duly recorded, whether the numbers conformed or did not conform to the rule the experimenter hand in mind. The instructions stressed (1) that the rule concerned a relation between any three cumbers and not their absolute magnitude (2) that the aim was to discover the rule itself and not to find numbers which merely conformed to it and (3) that subjects should write down the rule and tell it to the experimenter only when they felt highly confident that they had discovered it and not before. Subjects were not under any time pressure. Identical instructions were given to subjects in the model condition. In addition, though, and prior to carrying out the task, they were required on a separate sheet of paper to answer a series of questions. These asked them to (1) write down two or three (or more) possible rules [In fact, four subjects in the standard condition were also required to do this as a control.] (2) to describe the relationship they saw between these rules (i.e. their similarities and differences) (3) to write which triples they would use to test them and (4) to write down what they would do if all their triples conformed to the rule. Having answered these questions, they were asked to give a precise description of the example triple (2-4-6) and to s a y how they would determine whether t h a t description corresponded to the rule.

RESULTS Quantitative Findings A number of indices were computed. In all cases there was no evidence of any sex differences, nor differences due to educational background. Neither was there any difference between subjects in

556

D.W. Green

the standard condition required to produce possible rules in advance compared to those who were not so required. One subject in the model condition withdrew before reading through the instructions. All subjects in the model condition announced the rule correctly whereas only 3 out of the 8 subjects did so in the standard condition (p c 0.05, Fisher-Yates, exact test, one-tailed). Hence the model condition appears to have encouraged subjects to search for necessary and sufficient, rather than merely sufficient rules. Subjects in the standard condition, whose first announcement was incorrect, all announced correct rules on second announcement. In order to look at the kind of thinking used, an eliminative/enumerative index (see Wason. 1960) was constructed for each subject, up to the first announcement, by considering the nature of each of the triples in relation to the reasons for choosing them. Each 'reason for choice' was coded as either compatible or incompatible, with all the subsequent triples. It was assumed that thinking was eliminative if there was a high proportion of incompatible triples and enumerative if the proportion of compatible instances was high. The mean ratio was 1.51 for those subjects whose first announcement was correct (n = 10) and 0.44 for those whose first announcement was incorrect (n = 5). As expected, this difference was significant (z = 2.32, p < 0.025, Jonckheere trend test, one-tailed. see Leach, 1979). Furthermore, subjects whose first announcement was correct tended to generate more triples, 6.8 vs 3.2 [z = 2.17, p < 0.025, one-tailed test]. In short, subjects announcing incorrect first rules did so on the basis of a relatively few confirming instances while those announcing necessary and sufficient rules tended to eliminate merely sufficient ones over a longer sequence of trials. The model manipulation supported eliminative testing. In terms of the type of triples produced, these were invariably positive instances of the hypotheses under test. [Such tests accounted for an average of 78% and 79% of the tests in the model and standard conditions, respectively.] However, as would be expected if subjects in the model condition were guided by their model, the conditions differed in the nature of the opening triple. In the standard condition this matched the exemplar (2-4-6) in the sense that it was a positive, even number with eaual intervals of two, in an ascending order. Using this classification, the mean match was 5 features in the standard condition and 3.7 features in the model condition [z = 2.49. p < 0.01, one-tailed].

Confirmation bias

551

Qualitative Findings For the model condition, three categories of model were discerned based on the relationships described by the subjects themselves. Four descriptions were classified as partitioned (e.g. 'one rule is additive, the second is multiplicative'). ?tyo were classified as general (e.g. 'the first rule is more general than the second rule which relates specifically to the sequence of numbers'). Finally, one description was classified as overlap ('the factor of two is involved in all of these (rules); the increment is the same'). One might expect that subjects with a partitioned model or branching structure might converge on the rule more rapidly since alternatives could be clearly eliminated. The median trials for solution for such subjects was 5 compared to a median of 8 for subjects with the other models. In order to explore the kind of testing procedure in more detail two sources of information were consulted, viz: the kind of instances proposed as tests of the rules and the ways subjects proposed to test their precise description of the exemplar. All subjects, sought at some point to eliminate hypotheses but it was clear that subjects differed in their sensitivity to the need to eliminate and to seek falsifymg tests before the experimental phase. From the weakest to the strongest form of test these procedures were (1) positive testing, n = 2 (i.e. subjects cited positive instances of each rule): positive testing together with the notion of elimination, n = 1 (the subject provided a positive instance of the rule and wrote of the need to 'narrow down'): negative testing together with the notion of eliminating alternatives, n = 3 (e.g. a subject referred to testing each part of the descriptions and cited falsifying instances to do so); strong inference, n = 1 (this procedure was identified through the use of triples, i.e. '4-6-8 upholds the first but not the second rule, 1-2-3 upholds the second, not the first'). This ordering corresponded to the median number of trials to solution (positive testing 9; positive testing and elimination 8; negative testing +elimination 6; strong inference 3). Both the nature of the model and the kind of test procedure seem to play a role in the efficiency with which individuals discover the rule. Pattern of Testing Individuals in both conditions approached the task in a systematic fashion and varied their hypotheses. As a n illustration,

558

D.W. Green

consider the following protocol from a subject in the model condition. The subject first proposed a triple 3-5-7 to see whether the rule 'has a sequence of odd numbers conforming to it'. This triple holds the interval and order of magnitude of the exemplar constant. As a second triple, 4-2-6 was proposed in order to see whether the rule is 'merely a sequence of numbers with increasing magnitude'. Obviously, this triple uses the original example and is confounded with a change in interval. On the next trial, the subject ruled out the relevance of interval, having previously eliminated the relevance of evenness, by testing with the triple 1-5-8. The rule was then announced correctly. A similar, though less rigorous approach is shown in the protocol from a subject in the standard condition. The subject opened with the triple 8-10-12 that matches the exemplar, citing the hypothesis 'even numbers in ascending order'. On the next trial, 12-8-10, was proposed to see if the rule was even numbers in any order. Since this triple inevitably varies interval as well, the subject needed to eliminate this feature. However, in the next trial the subject generated the triple 3-5-7 which retains the interval and ascending order features of the original example, and shows the irrelevance of evenness per se. For the subject, this test confirmed the relevance of ascending, even intervals. The rule was announced: 'two is added to the first number and then two to the sum of that addition etc.' Informed that the rule was incorrect, the subject noticed that all the ascending triples were correct, and that the interval feature had not been ruled out. Accordingly the triple 1-2-3 was proposed to preserve order of magnitude but to change the interval. A second announcement followed which was correct. Broadly speaking, subjects in the standard condition, although operating in an incremental way, relied on one or more of the following: the use of announcement, noticing differences, and the generation of 'random numbers' t o provide a non-conforming triple. They also checked previous results using effectively the same hypothesis. Nature of Hypotheses Generated In terms of the actual hypotheses generated -- their frequency of mention and order of appearance -- these were similar in the two conditions. Prior to the experimental phase, some of the possible rules cited were even numbers: numbers increasing in a series of two: numbers divisible by or multiples of 2. The most cited also appeared before the least cited in order of mention. As a possible

Confirmation b i a s

559

rule 'even numbers' is salient. In contrast, 'ascending order of magnitude' is a less salient rule and was always tied to another property of the exemplar triple as in the rule, 'increasing intervals of two'. The possible rules also seem to vary in their specificity. More specific rules (e.g 'the numbers when added together sum to 12') are mentioned infrequently and only after more general rules. Frequency of mention prior to test was mirrored in the order of testing with the two conditions showing a similar pattern. Individuals invariably considered whether the rule involved even numbers before they considered whether it concerned the size of the interval or the order of magnitude. In summary, the findings are that when subjects are asked to think up, and to structure, a set of hypotheses (i.e. create a model of the problem space) and to envisage testing these hypotheses (i.e. simulate tests of the model) before carrying out the discovery task then the tendency to announce merely sufficient hypotheses (or rules) is eliminated. In announcing the rule correctly first time subjects propose more triples and eliminate more hypotheses. The data also suggest that some models (where hypotheses form a partitioned structure) and some procedures for testing them (e.g. strong inference) are more efficient than other models and procedures. Qualitatively, subjects in the model condition appear to operate in a more systematic and hypothesis-driven fashion compared to subjects in the standard condition. There are also commonalities between the two conditions. Subjects generally tested hypotheses with positive instances and proposed similar hypotheses in the same order. In addition, they sought to generate hypotheses by selectively varying their features (e.g.: maintaining the interval but varying whether the numbers were odd or even). DISCUSSION

The standard condition of this experiment indicates, in line with other research in both concept formation (see Bourne, Dominowski. & Loftus, 1979) and diagnosticity (e.g.: Doherty, Mynatt, Tweney, & Schiavo, 1979), that individuals tend to focus on a specific hypothesis and only gather evidence relevant to it. In the present case this results in a failure to test its generality and in the announcement of a merely sufficient rule. This failure can be described as a bias towards confirmation, which carries the connotation of a motivation to verify rather than to falsify

560

D. W. Green

hypotheses, or it can be described as a tendency to focus on positive information (see Evans, 1989). Individuals, it seems, have problems selecting and organizing relevant information. Prior work shows, however, that individuals can make use of relevant information in concept formation when it is presented to them (i.e. they can revise hypotheses) and they can also be led to consider base rate information in diagnosticity studies if they are expressly alerted to it (see, for example, Beyth-Marom & Fischhoff, 1983). Bias is also entirely eliminated in the model condition of the present experiment. Individuals established a necessary and sufficient rule before announcing it. Unlike previous attempts to facilitate performance in the rule discovery task, the model condition does not prescribe behaviour: the precise structures that subjects build are free to vary as are the interpretations they place on feedback (cf. the DAX-MED study). Nor does the condition delimit the testing procedures. The questioning procedure functions rather as a cognitive aid to discovery. I t ensures that subjects realise that there can be alternative hypotheses and that these are equally relevant (they are asked to state them and to interrelate them). It also allows them to appreciate that conformity to experimenter defined rules does not by itself entail the necessity of the hypotheses currently being entertained (they are asked what they would do if all their tests proved positive). In addition, in the 'on-line' phase it permits subjects to test a pre-established structure of alternatives, rather than needing to 'guess' a triple or to notice a pattern in order to develop such alternatives. In this sense prior questioning serves to provide a plan and some options for exploring and testing possibilities. It may be t h a t certain plans and procedures are more effective than others. Individuals whose initial model partitioned the set of rules were able to discover the rule in fewer trials but the current data are insufficient to warrant strong conclusions on this matter. With respect to the nature of the hypotheses generated and tested individuals generate hypotheses on the basis of the salient features of the exemplar and test them in a similar order, The relevant level a t which hypotheses are formulated is presumably determined by a number of factors common across conditions. The instruction stressed that the rule was simple and relational. Subjects may also be guided by heuristics such as 'keep it simple' as well as by constraints imposed by a lack of knowledge. It is noteworthy, too, that subjects in both conditions (after the first

Confirmation bias

561

announcement for those in the standard condition) sought t o proceed in an incremental and systematic fashion. How generalizable are the present findings on the elimination of bias? Ignoring the obvious factor of sample size, there are properties of the problem that may limit generality. For both conditions the target rule was in some sense available since the feature 'ascending' was part of the evoked set of alternatives, though not one which was separately considered. It is natural to wonder whether subjects in the model condition would fare better than those in the standard condition when the target rule is not in this evoked set, i.e. when the envisaged alternatives have been exhausted. This possibility is open to test by requiring subjects to discover a more difficult rule such as 'three different numbers' (see Gorman, Stafford, & Gorman. 1987). I t would also be useful to extend the findings by selectively varying the nature of the questions asked. At the moment, subjects are asked both to construct a model and to simulate tests of it. I t may be that certain questions are critical. Finally, if we accept the importance of a cognitive model as an aid to discovery and in the elimination of bias then the existence of such a structure should preclude confirmation bias in other areas. In social cognition individuals have a model involving a contrast between introvert and extravert and this structure does produce unbiased information-gathering (cf. Trope, Bassok, & Alon, 1984). It should also yield unbiased evaluation of the evidence collected. REFERENCES Beyth-Marom. R., & Fischhoff, B. (1983).Diagnosticity and pseudo-diagnosticity. J a u m a1 of Personalitv and Social Psvcholom. 1185-1 197. Bourne, L.E., Dominowski. R.L.. & Loftus. E.F. (1979).Cognitive Processes. Englewood Clms, NJ: Prentice-Hall. Cheng. P.W.. & Holyoak, K.J. (1985).Pragmatic reasoning schemas. Cognitive Psvcholofi, 12.391-416. Evans, J.St.B.T. (1989). Bias in human reasoning: causes and conseauencea. Hove and London (UK)and Hillsdale (USA): Lawrence Erlbaum Associates. Doherty, M.E.. Mynatt, C.R.. Tweney. R.D.. & Schiavo. M.D. (1979).Pseudodiagnosticity. Acta Psvcholow. Q, 11-21. Girotto. V.,Light. P.J., & Colbourn, C. (1988).Pragmatic schemas and conditional reasoning in children. Quarterlv Journal of ExDerimental Psvchologv. m, 342-359. Gorman, M.E., Stafford. A.. & Gorman. M.E. (1987).Disconfirmation and dual hypotheses in a more difficult version of Wason's 2-4-6task. Quarterly Journal of Emerimental PsvcholoPy, 1-28. Green, D.W. (1988).Problem-solving: representation and discovely. In G. Claxton (Ed). Growth Points in Cognition. London and New York Routledge. Johnson-Laird, P.N. (1983).Mental Models. Cambridge: Cambridge University Press.

s,

m,

5 62

D.W. Green

Klayman. J.. & Ha, Y-W. (19851. l i V ’ and Structure. Paper presented at the Tenth Research Conference on Subjective Probability. Utility and Decision Making. Helsinki, Finland. Klayman. J., & Ha, Y-W. (1987).Confirmation, disconfirmation and information in hypothesis-testing. Psvcholoaical Revlew. 21 1-228. Leach, C. (1979). Introduction to statistics. A non-Darametrlc aDDrOaCh for the social Sciences. Chichester: John Wiley & Sons. Mynatt, C.R., Doherty, M.E.,& ISveney. R.D. (1978).Consequences of confirmation a n d disconfirmation in a simulated research environment. Q u a rterly J . h I-, 395-406. Trope, Y.,Bassok, M.. & Alon, E. (1984). The questions lay intervlewers ask. Journal gf Personality. 2.90-106. Tweney. R.D., Doherty. M.E., Worner. W.J.. Pliske, D.B., & Mynatt. C.R. (1980). Strategies of rule discovery in a n inference task. Quarterlv J ournal of 109-123. Ex~erimentalPsvchology, 2, Wason, P.C. (1960).On the failure to eliminate hypotheses in a conceptual task. l m , 12, 129-140. . Wason, P.C.. & Green D.W. (1984). ReasonLng and mental representation. Quarterly Journal of ExDerlmental Psvcholo-, 597-610.

a,

a,

m,

Cognitive Biases J. -P. C a v e r n i , J. -M. F a b r e and M . Gonzalez (Eds.) 0 E l s e v i e r S c i e n c e P u b l i s h e r s B. V. (North-Holland),

1990

A STUDY OF TWO BIASES IN PROaABILISTIC JUDGMENTS: REPRESENTATIVENESS AND EQUIPRORABILITY Marie-Paule Lecoutre, Jean-Luc Durand, and Jean Cordier Mathematics and Psychology Group, Rene D e s c a r t e s U n i v e r s i t y ,

C.N.R.S.

and

Paris, F r a n c e

Abstract. From an experimental study of two biases in probabilistic judgments - the representativeness bias in situations of inductive inference, and the equiprobability bias in "purely random" situations - it is shown that it is possible to characterize two types of cognitive aids which differ by their impact point: the object

to which the probabilistic judgment pertains (for the representativeness bias), or the activated representation (for the equiprobability bias).

INTRODUCTION

In close connection with the statistical work on Bayesian inference done by the "Mathematics and Psychology Group" in Pans over the last twelve years, an experimental project was conducted on the intuitive probabilistic judgments of adult subjects in various (1) uncertainty situations, and especially the following two. Situations of inductive inference in which probabilistic judgments are built upon statistical data. (2) "Purely random" situations such as games of chance (dice, coins, etc.), or drawings from a jar, in which probabilistic judgments are essentially built upon considerations of symmetry. After what Gigerenzer and Murray (1987) called the "inference revolution", that is to say when the theory of probability entered into the theories of reasoning, one of the initial questions which has been raised in many experimental projects can be formulated as follows: "For estimating probabilities, do people spontaneously behave as Bayesian subjects?" In other words, do people give probabilistic evaluations close to those expected when calculated in the Bayesian framework? I t appeared that in most cases, the observed behaviours notably diverged from "ideal Bayesian" behaviour; for example, people are "conservative" by giving a n undue weight to initial

564

M.P.

Lecoutre, J . L . D u r a n d ,

&

J. Cordier

probabilities with respect to new information (cf. Rouanet, 1961; Edwards, 1962; Phillips & Edwards, 1966; Peterson & Ducharme, 1967, etc.); they develop various heuristics to facilitate estimation tasks which involve some distortions, or using the English terminology, some biases in probabilistic judgments. Without giving an exhaustive review of all the listed biases (for recent accounts, see especially Nisbett & Ross 1981; Kahneman, Slovic, & Tversky, 1982: Oakes. 1986). we shall mention three main biases. (1) The "representativeness bias" to which we are going to come back in detail later on. (2) The "availability heuristic": a person is said to employ this heuristic whenever he estimates frequency or probability by the ease with which instances or associations can be brought to mind. This heuristic uses strength of association as a basis for the judgment of probability and consequently leads to systematic biases, and especially to the "illusory correlation" which involves the overestimation of the probability with which two events go together when there is a link between them (for a detailed account see Chapman & Chapman, 1969). (3)The "anchoring bias" according to which proceeding from different initial values, people give final probabilistic judgments biased in the direction of the initial values. During our research on inductive inference situations, we again found the representativeness bias, which we shall focus on in the first part of this text. It is from purely random situations that we observed a new bias, a very stable one, that we called "equiprobability bias" (for a detailed analysis see Lecoutre, 1984, 1985; Lecoutre & Durand, 1988); the second part of this text will focus on this bias. Faced with such biases in probabilistic judgments, one of the possible centres of interest for research carried out in this field consists of characterizing efficient cognitive aids likely to lead to a notable reduction in these biases. We can connect probability estimation situations with problem solving situations. The subject's task is to give a response (a probability evaluation) pertaining to a n object (such as the result of a throw of dice for example) by activating some representation (called the "activated representation") which will lead to choosing a response. We can make the following diagram (cf. Figure 1). On the basis of our experimental research, we chose here to illustrate the two biases mentioned above because they allow u s to define two types of cognitive aids which differ by their impact point:

Representativeness and equiprobability

565

the object and the activated representation. The representativeness bias will give us an example of a cognitive aid related to the object: the equiprobability bias that will provide an example of a cognitive aid related to the activated representation. Problem

Object

Activated representation

Response (Probabilistic judgment)

Figure 1. Probabilistic judgment situation.

THE REPRESENTATIVENESS BIAS Illustration

In situations of statistical inference, the representativeness bias reflects undue confidence in the fact that a sample is highly representative of the population. This leads to the estimation that two samples from a same population have to resemble each other in their essential characteristics, and such a representation appears especially through the overestimation of the replicability of experimental results. This was found by Tversky and Kahneman (1971) in an experiment that can be summarized as follows. Researchers were asked to evaluate the probability, given a first significant result, of finding another significant result for an additional group of subjects. The probability values given as responses were found to be markedly higher than reference Bayesian probability values. This result is stable: it was found later in several experiments and again recently by Oakes (cf. Oakes. 1986). We have also obtained a similar result from an experiment whose main purpose was to compare the predictive judgments about a test statistic (namely, a Student's t-test) and about an elementary descriptive statistic (namely, an observed difference in mean). Each

566

M.P.

L e c o u t r e , J.L. Durand,

& J.

Cordier

of these two statistics is a prototype of two main kinds of welldissociated statistics, The difference in mean pertains to statistics that we call "natural statistics", that is to say. procedures for which there is an immediate link between data and outcome: most descriptive elementary statistics such as relative frequency, average, etc. may in this sense be called natural statistics. On the contrary, a significance test is the prototype of statistics that we call "sophisticated statistics" since they are based on a non-intuitive argument that relates data and outcome, and their interpretation is sometimes quite counter-intuitive. I t is well known that the interpretation of significance test results are often biased: above all, there is a cognitive tendency that we have often observed in our experiments (Lecoutre, 1983) to "take a position" and to "rank" every obtained result in a "cognitive closet" only from the test's outcome: if significant, the result is ranked in "there is an effect", and if non-significant, in "there is no effect". Such an interpretation can be compared to the one proposed by Oakes (1986) and called the "significance hypothesis" according to which the outcome of a significance test is a dichotomy: either an effect presumably "exists" (significant outcome), or "does not exist'' (non-significant outcome). In our experiment, 48 experimental psychologists working in various laboratories were requested to give spontaneous responses which were gathered individually via semi-directive interviews. There were three types of results for the first experiment presented: strong effect (situation l), small effect (situation 2),and no effect (situation 3). Taking into account these results, researchers were asked predictive questions about an experiment to come, described as a replication of the first one. For each of the three situations, there were two predictive questions: the first one (question A) about the observed difference in mean ("What is, for you, the probability that the observed difference d' in the second experiment has the same sign as d, that is, is positive?"), and a second one (question B) about Student's t-test (for example, in the first situation, "What is, for you, the probability that in the second experiment, the observed difference d' has the same sign as d, and the result of Student's t-test is at least as significant as in the first experiment?"). In accordance with the representativeness bias, a n overestimation of the replicability of the result obtained first should be found not only for the test statistic (the only case considered by

56 1

Representativeness and equiprobability

Tversky and Kahneman and their followers), but also for the natural statistic. TWO main results were obtained. 1) Table 1 shows the reference Bayesian probability values (the Standard Predictive Probability, S.P.P.1 ) calculated for each of the three situations, for questions A and B. with the response means given by researchers.

Table 1. Standard Predictive Probabilities (S.P.P.) and means of observed response values (in parentheses). Question A Situation 1 Situation 2 Situation 3

.92 (.70) .77 (.56) .57 (.44)

Question B .50(.57) .27 (.51) .07 I.56)

There is no representativeness bias when the predictive questions are about the natural statistic, since in this case, the probability values given as responses are, on the whole, close to the reference Bayesian probability values. Furthermore, when there is a difference, we observe some underestimation of the given responses with respect to Bayesian probabilities: the three means are significantly lower than the reference Bayesian values (p < 0.001). This can be interpreted as reflecting a “prudence attitude”. that is, a reluctance to hastily generalize properties found on a sample. These findings go against the representativeness bias. On the other hand, as expected, this bias is observed for the predictive questions about the significance test: the responses given by researchers in almost all cases are markedly higher than reference Bayesian probability values. The three means are significantly higher than the

1. In situations like the present one, where there is no available information external to the data, the Bayesian methods based on diffuse initial distributions yield probabilities that can be taken as Standard Predictive Probabilities; see B. Lecoutre, 1984.

5 68

M.P.

Lecoutre, J.L.

Durand,

& J.

Cordier

Bayesian values (p c 0.05 in situation 1, and p < 0.001 in situations 2 and 3). 2) About a third of the researchers give exactly the same response for the two predictive questions (about the difference in mean and about the significance test) in each situation. Furthermore, when two different response values are provided, they are often very close to each other. Thus, a number of researchers hardly dissociate -some of them do not dissociate at all -the predictions about the observed difference and the predictions about the significance test. This result strongly suggests that a substantial proportion of researchers lack a specific intuitive representation of the significance test. Thus, the representativeness bias is observed only when the predictive questions are about the significance test. Our findings show that in this case, either subjects have no specific representation and use an available one appropriate for another case (see the subjects who give exactly the same response for the predictions about the significance test as for the natural statistic), or they construct biased representations for interpreting the outcomes of significance tests such as those previously mentioned ("cognitive closet" or "significance hypothesis"). A discussion of these findings can be found in Lecoutre and Rouanet (1989). Cognitive aids

For the situation just presented, it is possible to specify Figure 1 as follows (cf. Figure 2): Natural statistic Predictive situation

4 Sophisticated statistic

Activated representation

t Probabilistic judgment

1 1

Unbiased

Biased

Figure 2. Probabilistic judgment situation: representativeness bias.

Representativeness and equiprobabil i t y

569

Our findings show that the representativeness bias is not linked to the activated representation but to the object. For the characterization of cognitive aids, which will thus have to rest on the significance test, the first idea which occurs consists of trying to let subjects know the correct interpretations of test outcomes, for example, by explicitly warning them about the more usual biases. But such an idea is not a novel one, and many attempts have already been made in this respect. Without systematically reviewing the texts in which we can find warnings against biased interpretations of significance test outcomes, we shall mention two main texts, the Morisson and Henkel symposium (1970). and Barnett (1973). But all these attempts have remained quite vain. In fact, we shall propose here two new ways which seem more promising, since they give a more natural s t a t u s to the interpretation of the test. 1) The first way consists of going below the usual inference by proposing a n interpretation in the set-theoretic inference framework developed these last years by Rouanet and members of the Mathematics and Psychology Group (Rouanet, Bernard, & Lecoutre, 1986). In this type of inference, there is no parent population: the idea is simply to ask if a group of observations is "typical" or not vis-a-vis a reference population. In this procedure, the population is simply used as a reference: no probabilistic judgment is implied, and the problem of representativeness consequently becomes irrelevant. This type of inference shows that the significance test can provide interesting information in a "nonprobabilistic". purely set-theoretic framework. 2) The second way consists, on the contrary, of going beyond the usual inference by recasting the test in a wider probabilistic framework with initial hypotheses, namely the Bayesian framework. Here, the obtained probabilities have a natural character, going from the known (the data) to the unknown (the parameters). I t should be noted that in significance testing the probability is on the contrary "non-natural" because it goes from the unknown (the null hypothesis) to the known (the data). The characterization of effective cognitive aids will have to take into account the results obtained from comparative experimental analysis of the varying degrees of availability of these two types of significance test interpretations (in set-theoretic or Bayesian terms) among researchers and students finishing their

570

M.P.

Lecoutre,

J.L.

Durand,

&

J. C o r d i e r

university education: a n experimental study is in progress at the present time. THE EQUIPROBAEILITY BIAS

Illustration Consider the following problem: two dice are simultaneously thrown, and the following two results are considered. R1: "A 5 and a 6 are obtained", and R2: "A 6 is obtained twice". The question asked is, "Do you think that there is as much chance of obtaining each of these two results? Or is there more chance of obtaining one of these two results, and if so, which, R1 or R2?. Or is it impossible for you to give a n opinion, and if so, why?" Now consider another problem. Three chips are in a jar, two red and one white, and two chips are simultaneously drawn. Asking the same question as before, the result Rl "A red chip and a white chip are obtained" is to be compared to the result FU "Two red chips are obtained." In these two examples, R1 is twice as probable as R2, and so the correct response is "There is more chance of obtaining the result Rl." I t is interesting to systematically study the responses given for this type of problem, which will hereafter be called "problem-type", because they are an example of a large class of situations that bring into play the notion of exchangeability, nowadays recognized in the theory of probability as more fundamental than the notion of independence (De Finetti, 1974)2 . In our experimental research (Lecoutre, 1984; 1985). we observed that nearly 60 percent of the subjects - students of various backgrounds -give an incorrect equiprobability response to the two results under consideration, and that barely 30 percent of the responses are correct. Such a result reflects the existence of a systematic bias that we have called "equiprobability bias". This bias is very resistant against variations in the classification factors of the subjects such as practice at games of chance (see especially Lecoutre, 1984),the nature of their studies -- literary vs scientific -- (see Durand, 1989). or their 2. I n a n e x c h a n g e a b i l i t y s t r u c t u r e , all e v e n t s p l a y t h e same r o l e v i s - a - v i s t h e p r o b a b i l i t y : e v e r y e v e n t " i s e q u i v a l e n t " t o e v e r y o t h e r o n e , e v e r y p a i r of e v e n t s " i s e q u i v a l e a t " t o e v e r y o t h e r o n e , e v e r y t r i p l e t of e v e n t s " i s e q u i v a l e n t " t o e v e r y o t h e r o n e , and so o n .

Representativeness and equiprobability

571

background in the theory of probability. It is interesting to point out here that even a thorough background does not lead to a notable increase in the proportion of correct responses (Lecoutre, 1984). Fischbein et al. (1971)found similar results from research with a didactic purpose, and according to these authors, the high proportion of incorrect responses is essentially due to the fact that most subjects do not perceive the compound character of result R1, that is, the fact that this result can be obtained in two ways, unlike result R2 that can only be obtained in a single way. We have defined various experimental conditions a priori favouring a better understanding of the structure of the "problemtype", and especially of its compound character. A series of experiments carried out with this aim can be found in Durand (1989). In all these experiments, it was a question of defining cognitive aids related to the structure of the problem-type. that is, to the object. For example, we tried to explicitly show the subject the compound character of result R2 through two experimental conditions. In the first condition, two dice of different colours (one red, the other blue) are utilized, and the problem-type is preceded by two problems P and P ' formulated as follows: P': R l "a 5 with the red die and a 6 with the blue die are

obtained" R 2 "a 6 is obtained on each of the dice"

P': R1 "a 6 with the red die and a 5 with the blue die are obtained" R2 "a 6 is obtained on each of the dice" Then, the problem-type is asked as follows: R1 "a 5 and a 6 are obtained" (without any additional information) R2 "a 6 is obtained twice"

In the second condition, successive throws of two dice (here of the same colour) are considered for obtaining a n order in terms of result R1 in each of the two problems, P' and P',preceding the problem- type. About 600 students took part in these experiments from which it principally appears that none of the considered aids have a massive positive effect: indeed, still more than 60 percent of the responses given are equiprobability responses.

572

M.P. Lecoutre, J . L . Durand,

&

J. C o r d i e r

S o , since the aids related to the object were ineffective, we then focused our study on activated representations. We defined a new experimental paradigm for identifying t h e subjects' representations from the responses and the justifications given on a prior sequence of problems. The analysis of about 400 individual protocols shows that the available representations are limited in number: without any ambiguity, we could distinguish seven main representations, including one which is the most frequent and can be summarized as follows: "The two results to compare are equiprobable because it is random." According to this representation, which accounts by itself for more than 65 percent of the equiprobability responses, random events should be equiprobable by nature (for a detailed description of the various available representations, see Lecoutre & Durand. 1988). We can see here that, at least for the subjects who devise s u c h a representation, every aid related to the object remains without any effect: the only aids that can be considered are those dealing with representations. This is what will be developed in the following section.

Cognitive aids

Our purpose was to define situations favourable to the activation of representations, other than the one mentioned above, which must be available but not spontaneously associated with situations in which "chance" is a factor. New situations, isomorphous to the preceding ones, b u t in which the "chance" aspect is concealed, have been characterized. Here is an example. Three cards on which figures are drawn are substituted for the three chips of the problem previously presented: a triangle is drawn on two of the cards (instead of the two red chips), and a square, on the third (instead of the white chip) (cf. Figure 3). A

B

C

Figure 3. The three cards used in the described experiment.

Representativeness and equiprobability

513

The subjects3 are shown that it is possible to construct either a house if either the pair AB or the pair BC is drawn, or a lozenge if AC is drawn: the two results to be compared are: R1 "A house is obtained". and FU "A lozenge is obtained." In this new presentation of the problem, the reference to "chance" has been concealed by attracting the subjects' attention to the "construction" of a geometrical figure in order to aid in the activation of combinatorial or logical representations: a modification of the space in which the solution is searched is thus introduced (Richard, 1984). An experiment in which this problem is first posed (Ql), and then the "classical" problem-type is posed with three chips and the usual formulations (Q2),was carried out with 87 students. The main results are reported in Table 2. Table 2. Distribution of the pairs of answers for questions Q1 and Q2 in Experiment 11.

tR1 +R1

38 ( . 4 4 )

-

+R2

?

20 (.23)

3 (.03)

4 (.05)

-

1 (.01)

18 (.21)

1 (.01)

0 (.OO)

+R2

0 (.OO)

0 (.OO)

2 (.02)

0 (.OO)

?

0 (.OO)

0 (.OO)

0 (.OO)

0 (.OO)

Q1

For the first problem, the proportion of equiprobability responses massively declines to the benefit of the correct response, which is given in 75 percent of the cases, where the proportion of 3 . E x a c t l y t h e same e x p e r i m e n t a l c o n d i t i o n , i n which t h e p o s s i b l e c a s e s a r e shown t o t h e s u b j e c t s b u t w i t h c h i p s , was s t u d i e d i n L e c o u t r e (1985).

574

M.P. L e c o u t r e , J.L. D u r a n d ,

&

J. Cordier

equiprobability responses is very low: a Bayes-fiducial analysis 4 shows that we obtain the Bayes-fiducial guarantee of 0.95 that the parent frequency 0 of equiprobability responses is lower than 0.31 (P(0 < 0.31) = 0.95): for more details about Bayes-fiducial inferences on frequencies specially adapted to the cases of small samples, see Poitevineau and Bernard (1986). Furthermore, t h e correct response is transferred to the subsequent problem-type in about 60 percent of the cases. An analysis of the representations developed in the first problem shows t h a t the representation based on "chance" previously illustrated is now devised by only 3 percent of the subjects: on the other hand, t h e activation of new representations can be observed: combinatorial (with explicit enumeration of the different possible cases) or logical (with hypothetico-deductive reasoning). Most subjects who develop such representations for the first problem, again develop these representations for responding to the subsequent problem-type, which leads to a proportion of correct responses of about 50 percent. Thus, we can see that correct representations are often available, but not spontaneously associated with the considered situations. Such a result should be compared to the one found by Escarabajal and Richard (1986) in situations of arithmetic problem solving, according to which subjects who do not spontaneously develop correct reasoning can still do so when they are prompted in this respect. Generally speaking, when a bias is observed in spontaneously developed representations, we think that correct representations are in most cases available, but experimental tricks are required to activate them: we have just seen an illustration of such an approach.

REFERENCES Barnett, V.D.(1973). ComDarative Statistical Inference. London: Wiley. Chapman, L.J.. & Chapman, J.P. (1969). Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. Journal of Abnom a 1 Psvchology, 74. 27 1280. De Finetti, B. (1974). Theow of Probabilitv (two volumes). New York: Wfley. Durand, J.-L.(1989). Jueements Drobabilistes dan s une situation altatoire: ttude emtrimentale de leur stabilite. Doctoral Thesis. May 1989, Paris V.

4. T h e s e a n a l y s e s were c a r r i e d o u t a t t h e C . I . R . C . E . c o m p u t a t i n g c e n t e r of t h e C.N.R.S., O r s a y ) .

(the

Representativeness and equiprobabilify

515

Edwards, W. (1962). Dynamic decision theory and probabilistic information 59-73. processing. Human FactoR ,4. Escarabajal, M.-C.. & Rtchard. J.-F. (19861. Le transfert analogique de procedures dans l'interpretation et la resolution de probltmes d'inclusion de classes chez des adultes. Archives de Psvcholoeie, 2.39-64. Fischbein, E.. Barbat. I . , & Minzat. I. (1971). Intuitions primaires et intultlons secondaires dans l'initiation aux probabilites. -at ional Studies in Mathematics, 4,264-280. Gfgerenzer, G., & Murray, D.J. 11987). Thinking: from insight to intuitive statistics, In Coenition as intuitive statistics. London: Lawrence Erlbaum Associates. Kahneman, D.. Slovic. P.. & Tversky. A. (1982). Judement under uncertaintv: Heuristics and biases. New York: Cambridge University Press. Lecoutre. B. (1984). L'Analvse Bavesienne des ComDaraisong. Lille: Presses Universitaires de Lille. Lecoutre. M.-P. (1983).La demarche du chercheur en psychologie d a m des situations conflictuelles d'analyse de donnees exPCrimentales. Journal de Psvcholoeie Normale et Patholoeiaue, 3. 275-296. Lecoutre. M.-P. (1984). Jugements probabilistes chez des adultes: pratique des J e w de hasard et formation en theorie des probabilites. Bulletin de Psvcholooie. 891-899. Lecoutre, M.-P. (1985).Effet d'informations de nature combinatoire et de nature frtquentielle sur les jugements probabilistes. Pecherches en Didact iaue des Mathimatiaues. S . 193-213. kcoutre, M.-P., & Durand, J.-L. (1988). Jugements probabilistes et modeles cognltifs: s .19,357etude d'une situation aleatoire. 5 368. Lecoutre. M.-P., & Rouanet, H. (1989). Probabilistic judgments in situations of &atistical Drediction. Submitted for publication. Morrison. D.E.. & Henkel. R.E. (1970).The sienificance test controversy. London: Butterworths. Nisbett. R , & Ross. L. (1981). Human inference: strateeies and shortcomines of socid iudementg. Englewood Cliffs, N J : Prentice Hall. Oakes, M. (1986).Statistical inference: a commentarv for the social and behaviourd sciences. London: John Wiley and Sons. Peterson, C.R. & Ducharme. W.M. (1967).A primacy effect in subjective probability revision. Journal of ExDerimental Psvchology. 73,61-65. Phillips. L.D.. & Edwards, W. (1966). Conservatism in a simple probability inferences task. Journal of Emerimental Pmcholoq, 346-354. Poltevineau. J.. & Bernard. J.-M. (1986). La serie des programmes IBF. Informatiaue et Sciences Humaines, M.135-137. Richard, J.-F.(1984). Mechanisms of problem solving behavior. In G . dYdevalle (Ed.), Copnition. information Drocessing and motivation. Amsterdam: NorthHolland, 425-444. Rouanet, H. (1961).Etude de decisions experimentales et calcul de probabilites. In decision. Colloque International du C.N.RS. Paris: C.N.RS., 33-44. Rouanet, H.. Bernard, J.-M.. & Lecoutre, B. (1986). Non-probabilistic statistical inference: A set theoretic approach. The American Statistician, 40,60-65. Tversky, A.. & Kahneman. D. 11971). Belief in the law of small numbers. Psvcholoeical Bulletin, 105-110.

a,

a,

z,

This Page Intentionally Left Blank